yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1288210] Re: Failed to attach volume

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Julio González Gil <julio@xxxxxxxxxxxxxxxx>
Date: Thu, 12 Feb 2015 01:58:13 -0000
Reply-to: Bug 1288210 <1288210@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
I'm able to reproduce this problem with RDO installed with "packstack
--allinone"

OS: CentOS 7.0.1406
Nova: 2014.2.1

2015-02-12 02:26:29.131 4108 ERROR nova.virt.block_device [req-8c1bf3d1-955a-4910-99c8-293395f650df None] [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05] Driver failed to attach volume 910e1aab-fcaa-4b77-be12-a303c6216d04 at /dev/hda
2015-02-12 02:26:29.131 4108 TRACE nova.virt.block_device [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05] Traceback (most recent call last):
2015-02-12 02:26:29.131 4108 TRACE nova.virt.block_device [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05]   File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 252, in attach
2015-02-12 02:26:29.131 4108 TRACE nova.virt.block_device [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05]     device_type=self['device_type'], encryption=encryption)
2015-02-12 02:26:29.131 4108 TRACE nova.virt.block_device [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1380, in attach_volume
2015-02-12 02:26:29.131 4108 TRACE nova.virt.block_device [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05]     conf = self._connect_volume(connection_info, disk_info)
2015-02-12 02:26:29.131 4108 TRACE nova.virt.block_device [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1327, in _connect_volume
2015-02-12 02:26:29.131 4108 TRACE nova.virt.block_device [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05]     return driver.connect_volume(connection_info, disk_info)
2015-02-12 02:26:29.131 4108 TRACE nova.virt.block_device [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05]   File "/usr/lib/python2.7/site-packages/nova/openstack/common/lockutils.py", line 272, in inner
2015-02-12 02:26:29.131 4108 TRACE nova.virt.block_device [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05]     return f(*args, **kwargs)
2015-02-12 02:26:29.131 4108 TRACE nova.virt.block_device [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/volume.py", line 299, in connect_volume
2015-02-12 02:26:29.131 4108 TRACE nova.virt.block_device [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05]     self._run_iscsiadm(iscsi_properties, ("--rescan",))
2015-02-12 02:26:29.131 4108 TRACE nova.virt.block_device [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/volume.py", line 236, in _run_iscsiadm
2015-02-12 02:26:29.131 4108 TRACE nova.virt.block_device [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05]     check_exit_code=check_exit_code)
2015-02-12 02:26:29.131 4108 TRACE nova.virt.block_device [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05]   File "/usr/lib/python2.7/site-packages/nova/utils.py", line 163, in execute
2015-02-12 02:26:29.131 4108 TRACE nova.virt.block_device [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05]     return processutils.execute(*cmd, **kwargs)
2015-02-12 02:26:29.131 4108 TRACE nova.virt.block_device [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05]   File "/usr/lib/python2.7/site-packages/nova/openstack/common/processutils.py", line 203, in execute
2015-02-12 02:26:29.131 4108 TRACE nova.virt.block_device [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05]     cmd=sanitized_cmd)
2015-02-12 02:26:29.131 4108 TRACE nova.virt.block_device [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05] ProcessExecutionError: Unexpected error while running command.
2015-02-12 02:26:29.131 4108 TRACE nova.virt.block_device [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05] Command: sudo nova-rootwrap /etc/nova/rootwrap.conf iscsiadm -m node -T iqn.2010-10.org.openstack:volume-910e1aab-fcaa-4b77-be12-a303c6216d04 -p 192.168.3.150:3260 --rescan
2015-02-12 02:26:29.131 4108 TRACE nova.virt.block_device [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05] Exit code: 21
2015-02-12 02:26:29.131 4108 TRACE nova.virt.block_device [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05] Stdout: u''
2015-02-12 02:26:29.131 4108 TRACE nova.virt.block_device [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05] Stderr: u'iscsiadm: No session found.\n'
2015-02-12 02:26:29.131 4108 TRACE nova.virt.block_device [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05] 


** Changed in: nova
       Status: Invalid => Confirmed

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1288210

Title:
  Failed to attach volume

Status in Fuel: OpenStack installer that works:
  Invalid
Status in OpenStack Compute (Nova):
  Confirmed

Bug description:
  {"build_id": "2014-03-05_07-31-01", "mirantis": "yes", "build_number":
  "235", "nailgun_sha": "f58aad317829112913f364347b14f1f0518ad371",
  "ostf_sha": "dc54d99ddff2f497b131ad1a42362515f2a61afa",
  "fuelmain_sha": "16637e2ea0ae6fe9a773aceb9d76c6e3a75f6c3b",
  "astute_sha": "f15f5615249c59c826ea05d26707f062c88db32a", "release":
  "4.1", "fuellib_sha": "73313007c0914e602246ea41fa5e8ca2dfead9f8"}

  Steps to reproduce:
  1. Create env:
  Centos, multinode, 1Controller, 1 Compute, 1 Cinder, 2 Cephs, Neutron GRE, Cinder LVM, Ceph images
  2. Create Instance(from image)
  3. Create empty volume 
  4. Attach volume to instance

  Actual result:
  Status of volume Available, volume in not attached

  Expected result:
  Status of volumes should be in-use, volume is attached

  
  <0>Mar  5 10:46:09 node-3 <BF><179>nova-nova.compute.manager ERROR: Failed to attach volume 83653211-2ce9-4230-83e8-b6195298b934 at /dev/vdb
  Traceback (most recent call last):
    File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 3687, in _attach_volume
      encryption=encryption)
    File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 1087, in attach_volume
      disk_info)
    File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 1046, in volume_driver_method
      return method(connection_info, *args, **kwargs)
    File "/usr/lib/python2.6/site-packages/nova/openstack/common/lockutils.py", line 246, in inner
      return f(*args, **kwargs)
    File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/volume.py", line 285, in connect_volume
      self._run_iscsiadm(iscsi_properties, ("--rescan",))
    File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/volume.py", line 218, in _run_iscsiadm
      check_exit_code=check_exit_code)
    File "/usr/lib/python2.6/site-packages/nova/utils.py", line 177, in execute
      return processutils.execute(*cmd, **kwargs)
    File "/usr/lib/python2.6/site-packages/nova/openstack/common/processutils.py", line 178, in execute
      cmd=' '.join(cmd))
  ProcessExecutionError: Unexpected error while running command.
  Command: sudo nova-rootwrap /etc/nova/rootwrap.conf iscsiadm -m node -T iqn.2010-10.org.openstack:volume-83653211-2ce9-4230-83e8-b6195298b934 -p 192.168.1.3:3260 --rescan
  Exit code: 21
  Stdout: ''
  Stderr: 'iscsiadm: No session found.\n'
  <0>Mar  5 10:46:09 node-3 <BF><179>nova-nova.openstack.common.rpc.amqp ERROR: Exception during message handling
  Traceback (most recent call last):
    File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 461, in _process_data
      **args)
    File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/dispatcher.py", line 172, in dispatch
      result = getattr(proxyobj, method)(ctxt, **kwargs)
    File "/usr/lib/python2.6/site-packages/nova/exception.py", line 90, in wrapped
      payload)
    File "/usr/lib/python2.6/site-packages/nova/exception.py", line 73, in wrapped
      return f(self, context, *args, **kw)
    File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 243, in decorated_function
      pass
    File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 229, in decorated_function
      return function(self, context, *args, **kwargs)
    File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 271, in decorated_function
      e, sys.exc_info())
    File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 258, in decorated_function
      return function(self, context, *args, **kwargs)
    File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 3655, in attach_volume
      context, instance, mountpoint)
    File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 3650, in attach_volume
      mountpoint, instance)
    File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 3697, in _attach_volume
      connector)
    File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 3687, in _attach_volume
      encryption=encryption)
    File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 1087, in attach_volume
      disk_info)
    File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 1046, in volume_driver_method
      return method(connection_info, *args, **kwargs)
    File "/usr/lib/python2.6/site-packages/nova/openstack/common/lockutils.py", line 246, in inner
      return f(*args, **kwargs)
    File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/volume.py", line 285, in connect_volume
      self._run_iscsiadm(iscsi_properties, ("--rescan",))
    File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/volume.py", line 218, in _run_iscsiadm
      check_exit_code=check_exit_code)
    File "/usr/lib/python2.6/site-packages/nova/utils.py", line 177, in execute
      return processutils.execute(*cmd, **kwargs)
    File "/usr/lib/python2.6/site-packages/nova/openstack/common/processutils.py", line 178, in execute
      cmd=' '.join(cmd))
  ProcessExecutionError: Unexpected error while running command.
  Command: sudo nova-rootwrap /etc/nova/rootwrap.conf iscsiadm -m node -T iqn.2010-10.org.openstack:volume-83653211-2ce9-4230-83e8-b6195298b934 -p 192.168.1.3:3260 --rescan
  Exit code: 21
  Stdout: ''

To manage notifications about this bug go to:
https://bugs.launchpad.net/fuel/+bug/1288210/+subscriptions
References

[Bug 1288210] [NEW] Failed to attach volume
From: Egor Kotko, 2014-03-05