← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #13657

[Bug 1308649] [NEW] Attaching Nova volumes fails if open-iscsi daemon is not already running

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

nova-compute traces on first volume attach if open-iscsi is not running:

2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481] Traceback (most recent call last):
2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]   File "/usr/lib64/python2.6/site-packages/nova/compute/manager.py", line 4142, in _attach_volume
2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]     do_check_attach=False, do_driver_attach=True)
2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]   File "/usr/lib64/python2.6/site-packages/nova/virt/block_device.py", line 44, in wrapped
2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]     ret_val = method(obj, context, *args, **kwargs)
2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]   File "/usr/lib64/python2.6/site-packages/nova/virt/block_device.py", line 248, in attach
2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]     connector)
2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]   File "/usr/lib64/python2.6/site-packages/nova/openstack/common/excutils.py", line 68, in __exit__
2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]     six.reraise(self.type_, self.value, self.tb)
2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]   File "/usr/lib64/python2.6/site-packages/nova/virt/block_device.py", line 239, in attach
2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]     device_type=self['device_type'], encryption=encryption)
2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]   File "/usr/lib64/python2.6/site-packages/nova/virt/libvirt/driver.py", line 1224, in attach_volume
2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]     disk_info)
2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]   File "/usr/lib64/python2.6/site-packages/nova/virt/libvirt/driver.py", line 1183, in volume_driver_method
2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]     return method(connection_info, *args, **kwargs)
2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]   File "/usr/lib64/python2.6/site-packages/nova/openstack/common/lockutils.py", line 249, in inner
2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]     return f(*args, **kwargs)
2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]   File "/usr/lib64/python2.6/contextlib.py", line 34, in __exit__
2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]     self.gen.throw(type, value, traceback)
2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]   File "/usr/lib64/python2.6/site-packages/nova/openstack/common/lockutils.py", line 212, in lock
2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]     yield sem
2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]   File "/usr/lib64/python2.6/site-packages/nova/openstack/common/lockutils.py", line 249, in inner
2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]     return f(*args, **kwargs)
2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]   File "/usr/lib64/python2.6/site-packages/nova/virt/libvirt/volume.py", line 285, in connect_volume
2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]     for ip, iqn in self._get_target_portals_from_iscsiadm_output(out):
2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481] ValueError: too many values to unpack
2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481] 


Thats because it runs "iscsiadm -m discovery -t sendtargets -p", which
prints upon first run:

Loading iscsi modules:                                                                                   done
Starting iSCSI initiator service:                                                                        done
Setting up iSCSI targets:                                                                                unused
192.168.204.82:3260,1 iqn.2010-10.org.openstack:volume-f9b12623-6ce3-4dac-a71f-09ad4249bdd3
192.168.204.82:3260,1 iqn.2010-10.org.openstack:volume-f9b12623-6ce3-4dac-a71f-09ad4249bdd3


aka there is some garbage before the actual output that it looks for. this is due to /etc/iscsid.conf having configured to start the daemon: 

from /etc/iscsid.conf:

# Default for upstream open-iscsi scripts (uncomment to activate).
iscsid.startup = /usr/sbin/rcopen-iscsi start

** Affects: nova
     Importance: Undecided
     Assignee: Dirk Mueller (dmllr)
         Status: New

** Changed in: nova
     Assignee: (unassigned) => Dirk Mueller (dmllr)

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1308649

Title:
  Attaching Nova volumes fails if open-iscsi daemon is not already
  running

Status in OpenStack Compute (Nova):
  New

Bug description:
  nova-compute traces on first volume attach if open-iscsi is not
  running:

  2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481] Traceback (most recent call last):
  2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]   File "/usr/lib64/python2.6/site-packages/nova/compute/manager.py", line 4142, in _attach_volume
  2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]     do_check_attach=False, do_driver_attach=True)
  2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]   File "/usr/lib64/python2.6/site-packages/nova/virt/block_device.py", line 44, in wrapped
  2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]     ret_val = method(obj, context, *args, **kwargs)
  2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]   File "/usr/lib64/python2.6/site-packages/nova/virt/block_device.py", line 248, in attach
  2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]     connector)
  2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]   File "/usr/lib64/python2.6/site-packages/nova/openstack/common/excutils.py", line 68, in __exit__
  2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]     six.reraise(self.type_, self.value, self.tb)
  2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]   File "/usr/lib64/python2.6/site-packages/nova/virt/block_device.py", line 239, in attach
  2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]     device_type=self['device_type'], encryption=encryption)
  2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]   File "/usr/lib64/python2.6/site-packages/nova/virt/libvirt/driver.py", line 1224, in attach_volume
  2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]     disk_info)
  2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]   File "/usr/lib64/python2.6/site-packages/nova/virt/libvirt/driver.py", line 1183, in volume_driver_method
  2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]     return method(connection_info, *args, **kwargs)
  2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]   File "/usr/lib64/python2.6/site-packages/nova/openstack/common/lockutils.py", line 249, in inner
  2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]     return f(*args, **kwargs)
  2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]   File "/usr/lib64/python2.6/contextlib.py", line 34, in __exit__
  2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]     self.gen.throw(type, value, traceback)
  2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]   File "/usr/lib64/python2.6/site-packages/nova/openstack/common/lockutils.py", line 212, in lock
  2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]     yield sem
  2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]   File "/usr/lib64/python2.6/site-packages/nova/openstack/common/lockutils.py", line 249, in inner
  2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]     return f(*args, **kwargs)
  2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]   File "/usr/lib64/python2.6/site-packages/nova/virt/libvirt/volume.py", line 285, in connect_volume
  2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481]     for ip, iqn in self._get_target_portals_from_iscsiadm_output(out):
  2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481] ValueError: too many values to unpack
  2014-04-16 13:19:44.359 6079 TRACE nova.compute.manager [instance: ddc07867-67a9-4af7-b7d0-d75dff53b481] 


  
  Thats because it runs "iscsiadm -m discovery -t sendtargets -p", which prints upon first run:

  Loading iscsi modules:                                                                                   done
  Starting iSCSI initiator service:                                                                        done
  Setting up iSCSI targets:                                                                                unused
  192.168.204.82:3260,1 iqn.2010-10.org.openstack:volume-f9b12623-6ce3-4dac-a71f-09ad4249bdd3
  192.168.204.82:3260,1 iqn.2010-10.org.openstack:volume-f9b12623-6ce3-4dac-a71f-09ad4249bdd3

  
  aka there is some garbage before the actual output that it looks for. this is due to /etc/iscsid.conf having configured to start the daemon: 

  from /etc/iscsid.conf:

  # Default for upstream open-iscsi scripts (uncomment to activate).
  iscsid.startup = /usr/sbin/rcopen-iscsi start

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1308649/+subscriptions

Follow ups

References

  Thread Previous • Date Previous • • Thread Next