yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1390072] [NEW] Nova HyperV does not rescan ISCSI target for new luns

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Launchpad Bug Tracker <1390072@xxxxxxxxxxxxxxxxxx>
Date: Thu, 07 Jan 2016 15:34:30 -0000
Reply-to: Bug 1390072 <1390072@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
You have been subscribed to a public bug:

I have taken latest devstack code today and activated HyperV compute from msi downloaded from cloudbase.
instance created successfully.
Configure cinder using hp3pariscsi driver, volumes got created.

Attach volume1 to instance1 runnning on hyperV, succeeded, i can see the disk attached to VM running in HyperV manager.
Attach volume2 to instance1 runnning on hyperV, fails, volume comes back to available state.

compute logs shows:
2014-09-03 18:31:49.290 2148 ERROR oslo.messaging.rpc.dispatcher [req-19fc9430-10e5-42a9-bd4b-80b7d04d6a82 ] Exception during message handling: Unable to find a mounted disk for target_iqn: iqn.2000-05.com.3pardata:20210002ac00943e
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last):
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher   File "C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\oslo\messaging\rpc\dispatcher.py", line 137, in _dispatch_and_reply
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher     incoming.message))
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher   File "C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\oslo\messaging\rpc\dispatcher.py", line 180, in _dispatch
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher     return self._do_dispatch(endpoint, method, ctxt, args)
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher   File "C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\oslo\messaging\rpc\dispatcher.py", line 126, in _do_dispatch
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher     result = getattr(endpoint, method)(ctxt, **new_args)
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher   File "C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\nova\compute\manager.py", line 409, in decorated_function
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher   File "C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\nova\exception.py", line 88, in wrapped
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher     payload)
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher   File "C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\oslo\utils\excutils.py", line 82, in __exit__
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher   File "C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\nova\exception.py", line 71, in wrapped
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher     return f(self, context, *args, **kw)
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher   File "C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\nova\compute\manager.py", line 293, in decorated_function
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher     pass
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher   File "C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\oslo\utils\excutils.py", line 82, in __exit__
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher   File "C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\nova\compute\manager.py", line 279, in decorated_function
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher   File "C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\nova\compute\manager.py", line 321, in decorated_function
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher     kwargs['instance'], e, sys.exc_info())
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher   File "C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\oslo\utils\excutils.py", line 82, in __exit__
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher   File "C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\nova\compute\manager.py", line 309, in decorated_function
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher   File "C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\nova\compute\manager.py", line 4537, in attach_volume
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher     do_attach_volume(context, instance, driver_bdm)
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher   File "C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\oslo\concurrency\lockutils.py", line 311, in inner
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher     return f(*args, **kwargs)
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher   File "C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\nova\compute\manager.py", line 4535, in do_attach_volume
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher     bdm.destroy(context)
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher   File "C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\oslo\utils\excutils.py", line 82, in __exit__
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher   File "C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\nova\compute\manager.py", line 4532, in do_attach_volume
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher     return self._attach_volume(context, instance, driver_bdm)
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher   File "C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\nova\compute\manager.py", line 4555, in _attach_volume
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher     self.volume_api.unreserve_volume(context, bdm.volume_id)
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher   File "C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\oslo\utils\excutils.py", line 82, in __exit__
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher   File "C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\nova\compute\manager.py", line 4547, in _attach_volume
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher     do_check_attach=False, do_driver_attach=True)
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher   File "C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\nova\virt\block_device.py", line 47, in wrapped
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher     ret_val = method(obj, context, *args, **kwargs)
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher   File "C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\nova\virt\block_device.py", line 262, in attach
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher     connector)
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher   File "C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\oslo\utils\excutils.py", line 82, in __exit__
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher   File "C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\nova\virt\block_device.py", line 253, in attach
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher     device_type=self['device_type'], encryption=encryption)
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher   File "C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\nova\virt\hyperv\driver.py", line 82, in attach_volume
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher     instance['name'])
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher   File "C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\nova\virt\hyperv\volumeops.py", line 155, in attach_volume
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher     self._volutils.logout_storage_target(target_iqn)
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher   File "C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\oslo\utils\excutils.py", line 82, in __exit__
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher   File "C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\nova\virt\hyperv\volumeops.py", line 124, in attach_volume
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher     self._login_storage_target(connection_info)
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher   File "C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\nova\virt\hyperv\volumeops.py", line 113, in _login_storage_target
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher     self._get_mounted_disk_from_lun(target_iqn, target_lun, True)
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher   File "C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\nova\virt\hyperv\volumeops.py", line 227, in _get_mounted_disk_from_lun
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher     'target_iqn: %s') % target_iqn)
2014-09-03 18:31:49.290 2148 TRACE oslo.messaging.rpc.dispatcher NotFound: Unable to find a mounted disk for target_iqn: iqn.2000-05.com.3pardata:20210002ac00943e


After some troubleshooting i found, that nova compute HyperV virt driver does not rescan for storage, while attaching volume.
 on 3PAR volume is exprted to the Host, but on HyperVhost it does not show.
First volume is attached because,  it connects to the target at once, for second time it uses same iscsi connection to export the disk.

** Affects: nova
     Importance: Low
     Assignee: Lucian Petrut (petrutlucian94)
         Status: Triaged

** Affects: os-win
     Importance: Undecided
         Status: New


** Tags: hyper-v
-- 
Nova HyperV does not rescan ISCSI target for new luns
https://bugs.launchpad.net/bugs/1390072
You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova).
References

[Bug 1390072] [NEW] Nova HyperV does not rescan ISCSI target for new luns
From: Ajay Bajaj, 2014-11-06