yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1479291] [NEW] Cannot attach to iscsi volume with multipath

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Vincent Untz <1479291@xxxxxxxxxxxxxxxxxx>
Date: Wed, 29 Jul 2015 10:30:07 -0000
Reply-to: Bug 1479291 <1479291@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

This is Kilo specific (and works in Liberty, thanks to the move to os-
brick).

When attaching a volume, I'm seeing a failure like this:

  Traceback (most recent call last):
    File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 255, in attach
      device_type=self['device_type'], encryption=encryption)
    File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1097, in attach_volume
      self._disconnect_volume(connection_info, disk_dev)
    File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 85, in __exit__
      six.reraise(self.type_, self.value, self.tb)
    File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1086, in attach_volume
      virt_dom.attachDeviceFlags(conf.to_xml(), flags)
    File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 183, in doit
      result = proxy_call(self._autowrap, f, *args, **kwargs)
    File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 141, in proxy_call
      rv = execute(f, *args, **kwargs)
    File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 122, in execute
      six.reraise(c, e, tb)
    File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 80, in tworker
      rv = meth(*args, **kwargs)
    File "/usr/lib64/python2.7/site-packages/libvirt.py", line 521, in attachDeviceFlags
      if ret == -1: raise libvirtError ('virDomainAttachDeviceFlags() failed', dom=self)
  libvirtError: unable to resolve '/dev/mapper/Jul': No such file or directory

It turns out that the output of multipath -ll is this:


2015-07-29 10:06:15.737 9409 DEBUG nova.virt.libvirt.volume [req-b7fd06a9-0b92-4d22-b0b5-fc90155523e4 c2cc2530388045fcb18af40982cd620b 0f06428e3867430cba110ca433bcdfec - - -] multipath ['-ll', '/dev/sda']: stdout=Jul 29 10:06:15 | vda: failed to get udev uid: Invalid arg
ument
Jul 29 10:06:15 | vda: failed to get sysfs uid: Invalid argument
Jul 29 10:06:15 | vda: failed to get sgio uid: No such file or directory
Jul 29 10:06:15 | nbd0: failed to get udev uid: Invalid argument
Jul 29 10:06:15 | nbd0: failed to get sysfs uid: Invalid argument
Jul 29 10:06:15 | nbd0: failed to get sgio uid: Inappropriate ioctl for device
Jul 29 10:06:15 | nbd1: failed to get udev uid: Invalid argument
Jul 29 10:06:15 | nbd1: failed to get sysfs uid: Invalid argument
Jul 29 10:06:15 | nbd1: failed to get sgio uid: Inappropriate ioctl for device
Jul 29 10:06:15 | nbd10: failed to get udev uid: Invalid argument
Jul 29 10:06:15 | nbd10: failed to get sysfs uid: Invalid argument
Jul 29 10:06:15 | nbd10: failed to get sgio uid: Inappropriate ioctl for device
Jul 29 10:06:15 | nbd11: failed to get udev uid: Invalid argument
Jul 29 10:06:15 | nbd11: failed to get sysfs uid: Invalid argument
Jul 29 10:06:15 | nbd11: failed to get sgio uid: Inappropriate ioctl for device
Jul 29 10:06:15 | nbd12: failed to get udev uid: Invalid argument
Jul 29 10:06:15 | nbd12: failed to get sysfs uid: Invalid argument
Jul 29 10:06:15 | nbd12: failed to get sgio uid: Inappropriate ioctl for device
Jul 29 10:06:15 | nbd13: failed to get udev uid: Invalid argument
Jul 29 10:06:15 | nbd13: failed to get sysfs uid: Invalid argument
Jul 29 10:06:15 | nbd13: failed to get sgio uid: Inappropriate ioctl for device
Jul 29 10:06:15 | nbd14: failed to get udev uid: Invalid argument
Jul 29 10:06:15 | nbd14: failed to get sysfs uid: Invalid argument
Jul 29 10:06:15 | nbd14: failed to get sgio uid: Inappropriate ioctl for device
Jul 29 10:06:15 | nbd15: failed to get udev uid: Invalid argument
Jul 29 10:06:15 | nbd15: failed to get sysfs uid: Invalid argument
Jul 29 10:06:15 | nbd15: failed to get sgio uid: Inappropriate ioctl for device
Jul 29 10:06:15 | nbd2: failed to get udev uid: Invalid argument
Jul 29 10:06:15 | nbd2: failed to get sysfs uid: Invalid argument
Jul 29 10:06:15 | nbd2: failed to get sgio uid: Inappropriate ioctl for device
Jul 29 10:06:15 | nbd3: failed to get udev uid: Invalid argument
Jul 29 10:06:15 | nbd3: failed to get sysfs uid: Invalid argument
Jul 29 10:06:15 | nbd3: failed to get sgio uid: Inappropriate ioctl for device
Jul 29 10:06:15 | nbd4: failed to get udev uid: Invalid argument
Jul 29 10:06:15 | nbd4: failed to get sysfs uid: Invalid argument
Jul 29 10:06:15 | nbd4: failed to get sgio uid: Inappropriate ioctl for device
Jul 29 10:06:15 | nbd5: failed to get udev uid: Invalid argument
Jul 29 10:06:15 | nbd5: failed to get sysfs uid: Invalid argument
Jul 29 10:06:15 | nbd5: failed to get sgio uid: Inappropriate ioctl for device
Jul 29 10:06:15 | nbd6: failed to get udev uid: Invalid argument
Jul 29 10:06:15 | nbd6: failed to get sysfs uid: Invalid argument
Jul 29 10:06:15 | nbd6: failed to get sgio uid: Inappropriate ioctl for device
Jul 29 10:06:15 | nbd7: failed to get udev uid: Invalid argument
Jul 29 10:06:15 | nbd7: failed to get sysfs uid: Invalid argument
Jul 29 10:06:15 | nbd7: failed to get sgio uid: Inappropriate ioctl for device
Jul 29 10:06:15 | nbd8: failed to get udev uid: Invalid argument
Jul 29 10:06:15 | nbd8: failed to get sysfs uid: Invalid argument
Jul 29 10:06:15 | nbd8: failed to get sgio uid: Inappropriate ioctl for device
Jul 29 10:06:15 | nbd9: failed to get udev uid: Invalid argument
Jul 29 10:06:15 | nbd9: failed to get sysfs uid: Invalid argument
Jul 29 10:06:15 | nbd9: failed to get sgio uid: Inappropriate ioctl for device
33000000200000001 dm-4 IET,VIRTUAL-DISK
size=1.0G features='0' hwhandler='0' wp=rw
`-+- policy='service-time 0' prio=1 status=enabled
  `- 4:0:0:1 sda   8:0   active ready  running

So we don't filter out correctly all lines; os-brick does this
correctly.

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1479291

Title:
  Cannot attach to iscsi volume with multipath

Status in OpenStack Compute (nova):
  New

Bug description:
  This is Kilo specific (and works in Liberty, thanks to the move to os-
  brick).

  When attaching a volume, I'm seeing a failure like this:

    Traceback (most recent call last):
      File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 255, in attach
        device_type=self['device_type'], encryption=encryption)
      File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1097, in attach_volume
        self._disconnect_volume(connection_info, disk_dev)
      File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 85, in __exit__
        six.reraise(self.type_, self.value, self.tb)
      File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1086, in attach_volume
        virt_dom.attachDeviceFlags(conf.to_xml(), flags)
      File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 183, in doit
        result = proxy_call(self._autowrap, f, *args, **kwargs)
      File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 141, in proxy_call
        rv = execute(f, *args, **kwargs)
      File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 122, in execute
        six.reraise(c, e, tb)
      File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 80, in tworker
        rv = meth(*args, **kwargs)
      File "/usr/lib64/python2.7/site-packages/libvirt.py", line 521, in attachDeviceFlags
        if ret == -1: raise libvirtError ('virDomainAttachDeviceFlags() failed', dom=self)
    libvirtError: unable to resolve '/dev/mapper/Jul': No such file or directory

  It turns out that the output of multipath -ll is this:

  
  2015-07-29 10:06:15.737 9409 DEBUG nova.virt.libvirt.volume [req-b7fd06a9-0b92-4d22-b0b5-fc90155523e4 c2cc2530388045fcb18af40982cd620b 0f06428e3867430cba110ca433bcdfec - - -] multipath ['-ll', '/dev/sda']: stdout=Jul 29 10:06:15 | vda: failed to get udev uid: Invalid arg
  ument
  Jul 29 10:06:15 | vda: failed to get sysfs uid: Invalid argument
  Jul 29 10:06:15 | vda: failed to get sgio uid: No such file or directory
  Jul 29 10:06:15 | nbd0: failed to get udev uid: Invalid argument
  Jul 29 10:06:15 | nbd0: failed to get sysfs uid: Invalid argument
  Jul 29 10:06:15 | nbd0: failed to get sgio uid: Inappropriate ioctl for device
  Jul 29 10:06:15 | nbd1: failed to get udev uid: Invalid argument
  Jul 29 10:06:15 | nbd1: failed to get sysfs uid: Invalid argument
  Jul 29 10:06:15 | nbd1: failed to get sgio uid: Inappropriate ioctl for device
  Jul 29 10:06:15 | nbd10: failed to get udev uid: Invalid argument
  Jul 29 10:06:15 | nbd10: failed to get sysfs uid: Invalid argument
  Jul 29 10:06:15 | nbd10: failed to get sgio uid: Inappropriate ioctl for device
  Jul 29 10:06:15 | nbd11: failed to get udev uid: Invalid argument
  Jul 29 10:06:15 | nbd11: failed to get sysfs uid: Invalid argument
  Jul 29 10:06:15 | nbd11: failed to get sgio uid: Inappropriate ioctl for device
  Jul 29 10:06:15 | nbd12: failed to get udev uid: Invalid argument
  Jul 29 10:06:15 | nbd12: failed to get sysfs uid: Invalid argument
  Jul 29 10:06:15 | nbd12: failed to get sgio uid: Inappropriate ioctl for device
  Jul 29 10:06:15 | nbd13: failed to get udev uid: Invalid argument
  Jul 29 10:06:15 | nbd13: failed to get sysfs uid: Invalid argument
  Jul 29 10:06:15 | nbd13: failed to get sgio uid: Inappropriate ioctl for device
  Jul 29 10:06:15 | nbd14: failed to get udev uid: Invalid argument
  Jul 29 10:06:15 | nbd14: failed to get sysfs uid: Invalid argument
  Jul 29 10:06:15 | nbd14: failed to get sgio uid: Inappropriate ioctl for device
  Jul 29 10:06:15 | nbd15: failed to get udev uid: Invalid argument
  Jul 29 10:06:15 | nbd15: failed to get sysfs uid: Invalid argument
  Jul 29 10:06:15 | nbd15: failed to get sgio uid: Inappropriate ioctl for device
  Jul 29 10:06:15 | nbd2: failed to get udev uid: Invalid argument
  Jul 29 10:06:15 | nbd2: failed to get sysfs uid: Invalid argument
  Jul 29 10:06:15 | nbd2: failed to get sgio uid: Inappropriate ioctl for device
  Jul 29 10:06:15 | nbd3: failed to get udev uid: Invalid argument
  Jul 29 10:06:15 | nbd3: failed to get sysfs uid: Invalid argument
  Jul 29 10:06:15 | nbd3: failed to get sgio uid: Inappropriate ioctl for device
  Jul 29 10:06:15 | nbd4: failed to get udev uid: Invalid argument
  Jul 29 10:06:15 | nbd4: failed to get sysfs uid: Invalid argument
  Jul 29 10:06:15 | nbd4: failed to get sgio uid: Inappropriate ioctl for device
  Jul 29 10:06:15 | nbd5: failed to get udev uid: Invalid argument
  Jul 29 10:06:15 | nbd5: failed to get sysfs uid: Invalid argument
  Jul 29 10:06:15 | nbd5: failed to get sgio uid: Inappropriate ioctl for device
  Jul 29 10:06:15 | nbd6: failed to get udev uid: Invalid argument
  Jul 29 10:06:15 | nbd6: failed to get sysfs uid: Invalid argument
  Jul 29 10:06:15 | nbd6: failed to get sgio uid: Inappropriate ioctl for device
  Jul 29 10:06:15 | nbd7: failed to get udev uid: Invalid argument
  Jul 29 10:06:15 | nbd7: failed to get sysfs uid: Invalid argument
  Jul 29 10:06:15 | nbd7: failed to get sgio uid: Inappropriate ioctl for device
  Jul 29 10:06:15 | nbd8: failed to get udev uid: Invalid argument
  Jul 29 10:06:15 | nbd8: failed to get sysfs uid: Invalid argument
  Jul 29 10:06:15 | nbd8: failed to get sgio uid: Inappropriate ioctl for device
  Jul 29 10:06:15 | nbd9: failed to get udev uid: Invalid argument
  Jul 29 10:06:15 | nbd9: failed to get sysfs uid: Invalid argument
  Jul 29 10:06:15 | nbd9: failed to get sgio uid: Inappropriate ioctl for device
  33000000200000001 dm-4 IET,VIRTUAL-DISK
  size=1.0G features='0' hwhandler='0' wp=rw
  `-+- policy='service-time 0' prio=1 status=enabled
    `- 4:0:0:1 sda   8:0   active ready  running

  So we don't filter out correctly all lines; os-brick does this
  correctly.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1479291/+subscriptions
Follow ups

[Bug 1479291] Re: Cannot attach to iscsi volume with multipath
From: Thierry Carrez, 2015-09-03