yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1288210] [NEW] Failed to attach volume

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Egor Kotko <ykotko@xxxxxxxxxxxx>
Date: Wed, 05 Mar 2014 11:58:23 -0000
Reply-to: Bug 1288210 <1288210@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

{"build_id": "2014-03-05_07-31-01", "mirantis": "yes", "build_number":
"235", "nailgun_sha": "f58aad317829112913f364347b14f1f0518ad371",
"ostf_sha": "dc54d99ddff2f497b131ad1a42362515f2a61afa", "fuelmain_sha":
"16637e2ea0ae6fe9a773aceb9d76c6e3a75f6c3b", "astute_sha":
"f15f5615249c59c826ea05d26707f062c88db32a", "release": "4.1",
"fuellib_sha": "73313007c0914e602246ea41fa5e8ca2dfead9f8"}

Steps to reproduce:
1. Create env:
Centos, multinode, 1Controller, 1 Compute, 1 Cinder, 2 Cephs, Neutron GRE, Cinder LVM, Ceph images
2. Create Instance(from image)
3. Create empty volume 
4. Attach volume to instance

Actual result:
Status of volume Available, volume in not attached

Expected result:
Status of volumes should be in-use, volume is attached


<0>Mar  5 10:46:09 node-3 <BF><179>nova-nova.compute.manager ERROR: Failed to attach volume 83653211-2ce9-4230-83e8-b6195298b934 at /dev/vdb
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 3687, in _attach_volume
    encryption=encryption)
  File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 1087, in attach_volume
    disk_info)
  File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 1046, in volume_driver_method
    return method(connection_info, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/nova/openstack/common/lockutils.py", line 246, in inner
    return f(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/volume.py", line 285, in connect_volume
    self._run_iscsiadm(iscsi_properties, ("--rescan",))
  File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/volume.py", line 218, in _run_iscsiadm
    check_exit_code=check_exit_code)
  File "/usr/lib/python2.6/site-packages/nova/utils.py", line 177, in execute
    return processutils.execute(*cmd, **kwargs)
  File "/usr/lib/python2.6/site-packages/nova/openstack/common/processutils.py", line 178, in execute
    cmd=' '.join(cmd))
ProcessExecutionError: Unexpected error while running command.
Command: sudo nova-rootwrap /etc/nova/rootwrap.conf iscsiadm -m node -T iqn.2010-10.org.openstack:volume-83653211-2ce9-4230-83e8-b6195298b934 -p 192.168.1.3:3260 --rescan
Exit code: 21
Stdout: ''
Stderr: 'iscsiadm: No session found.\n'
<0>Mar  5 10:46:09 node-3 <BF><179>nova-nova.openstack.common.rpc.amqp ERROR: Exception during message handling
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 461, in _process_data
    **args)
  File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/dispatcher.py", line 172, in dispatch
    result = getattr(proxyobj, method)(ctxt, **kwargs)
  File "/usr/lib/python2.6/site-packages/nova/exception.py", line 90, in wrapped
    payload)
  File "/usr/lib/python2.6/site-packages/nova/exception.py", line 73, in wrapped
    return f(self, context, *args, **kw)
  File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 243, in decorated_function
    pass
  File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 229, in decorated_function
    return function(self, context, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 271, in decorated_function
    e, sys.exc_info())
  File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 258, in decorated_function
    return function(self, context, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 3655, in attach_volume
    context, instance, mountpoint)
  File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 3650, in attach_volume
    mountpoint, instance)
  File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 3697, in _attach_volume
    connector)
  File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 3687, in _attach_volume
    encryption=encryption)
  File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 1087, in attach_volume
    disk_info)
  File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 1046, in volume_driver_method
    return method(connection_info, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/nova/openstack/common/lockutils.py", line 246, in inner
    return f(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/volume.py", line 285, in connect_volume
    self._run_iscsiadm(iscsi_properties, ("--rescan",))
  File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/volume.py", line 218, in _run_iscsiadm
    check_exit_code=check_exit_code)
  File "/usr/lib/python2.6/site-packages/nova/utils.py", line 177, in execute
    return processutils.execute(*cmd, **kwargs)
  File "/usr/lib/python2.6/site-packages/nova/openstack/common/processutils.py", line 178, in execute
    cmd=' '.join(cmd))
ProcessExecutionError: Unexpected error while running command.
Command: sudo nova-rootwrap /etc/nova/rootwrap.conf iscsiadm -m node -T iqn.2010-10.org.openstack:volume-83653211-2ce9-4230-83e8-b6195298b934 -p 192.168.1.3:3260 --rescan
Exit code: 21
Stdout: ''

** Affects: nova
     Importance: Undecided
         Status: New


** Tags: volumes

** Attachment added: "fuel-snapshot-2014-03-05_11-28-05.tgz"
   https://bugs.launchpad.net/bugs/1288210/+attachment/4008165/+files/fuel-snapshot-2014-03-05_11-28-05.tgz

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1288210

Title:
  Failed to attach volume

Status in OpenStack Compute (Nova):
  New

Bug description:
  {"build_id": "2014-03-05_07-31-01", "mirantis": "yes", "build_number":
  "235", "nailgun_sha": "f58aad317829112913f364347b14f1f0518ad371",
  "ostf_sha": "dc54d99ddff2f497b131ad1a42362515f2a61afa",
  "fuelmain_sha": "16637e2ea0ae6fe9a773aceb9d76c6e3a75f6c3b",
  "astute_sha": "f15f5615249c59c826ea05d26707f062c88db32a", "release":
  "4.1", "fuellib_sha": "73313007c0914e602246ea41fa5e8ca2dfead9f8"}

  Steps to reproduce:
  1. Create env:
  Centos, multinode, 1Controller, 1 Compute, 1 Cinder, 2 Cephs, Neutron GRE, Cinder LVM, Ceph images
  2. Create Instance(from image)
  3. Create empty volume 
  4. Attach volume to instance

  Actual result:
  Status of volume Available, volume in not attached

  Expected result:
  Status of volumes should be in-use, volume is attached

  
  <0>Mar  5 10:46:09 node-3 <BF><179>nova-nova.compute.manager ERROR: Failed to attach volume 83653211-2ce9-4230-83e8-b6195298b934 at /dev/vdb
  Traceback (most recent call last):
    File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 3687, in _attach_volume
      encryption=encryption)
    File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 1087, in attach_volume
      disk_info)
    File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 1046, in volume_driver_method
      return method(connection_info, *args, **kwargs)
    File "/usr/lib/python2.6/site-packages/nova/openstack/common/lockutils.py", line 246, in inner
      return f(*args, **kwargs)
    File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/volume.py", line 285, in connect_volume
      self._run_iscsiadm(iscsi_properties, ("--rescan",))
    File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/volume.py", line 218, in _run_iscsiadm
      check_exit_code=check_exit_code)
    File "/usr/lib/python2.6/site-packages/nova/utils.py", line 177, in execute
      return processutils.execute(*cmd, **kwargs)
    File "/usr/lib/python2.6/site-packages/nova/openstack/common/processutils.py", line 178, in execute
      cmd=' '.join(cmd))
  ProcessExecutionError: Unexpected error while running command.
  Command: sudo nova-rootwrap /etc/nova/rootwrap.conf iscsiadm -m node -T iqn.2010-10.org.openstack:volume-83653211-2ce9-4230-83e8-b6195298b934 -p 192.168.1.3:3260 --rescan
  Exit code: 21
  Stdout: ''
  Stderr: 'iscsiadm: No session found.\n'
  <0>Mar  5 10:46:09 node-3 <BF><179>nova-nova.openstack.common.rpc.amqp ERROR: Exception during message handling
  Traceback (most recent call last):
    File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 461, in _process_data
      **args)
    File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/dispatcher.py", line 172, in dispatch
      result = getattr(proxyobj, method)(ctxt, **kwargs)
    File "/usr/lib/python2.6/site-packages/nova/exception.py", line 90, in wrapped
      payload)
    File "/usr/lib/python2.6/site-packages/nova/exception.py", line 73, in wrapped
      return f(self, context, *args, **kw)
    File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 243, in decorated_function
      pass
    File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 229, in decorated_function
      return function(self, context, *args, **kwargs)
    File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 271, in decorated_function
      e, sys.exc_info())
    File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 258, in decorated_function
      return function(self, context, *args, **kwargs)
    File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 3655, in attach_volume
      context, instance, mountpoint)
    File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 3650, in attach_volume
      mountpoint, instance)
    File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 3697, in _attach_volume
      connector)
    File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 3687, in _attach_volume
      encryption=encryption)
    File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 1087, in attach_volume
      disk_info)
    File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 1046, in volume_driver_method
      return method(connection_info, *args, **kwargs)
    File "/usr/lib/python2.6/site-packages/nova/openstack/common/lockutils.py", line 246, in inner
      return f(*args, **kwargs)
    File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/volume.py", line 285, in connect_volume
      self._run_iscsiadm(iscsi_properties, ("--rescan",))
    File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/volume.py", line 218, in _run_iscsiadm
      check_exit_code=check_exit_code)
    File "/usr/lib/python2.6/site-packages/nova/utils.py", line 177, in execute
      return processutils.execute(*cmd, **kwargs)
    File "/usr/lib/python2.6/site-packages/nova/openstack/common/processutils.py", line 178, in execute
      cmd=' '.join(cmd))
  ProcessExecutionError: Unexpected error while running command.
  Command: sudo nova-rootwrap /etc/nova/rootwrap.conf iscsiadm -m node -T iqn.2010-10.org.openstack:volume-83653211-2ce9-4230-83e8-b6195298b934 -p 192.168.1.3:3260 --rescan
  Exit code: 21
  Stdout: ''

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1288210/+subscriptions
Follow ups

[Bug 1288210] Re: Failed to attach volume
From: Markus Zoeller (markus_z), 2016-07-05
[Bug 1288210] Re: Failed to attach volume
From: Julio González Gil, 2015-02-12
[Bug 1288210] Re: Failed to attach volume
From: Joe Gordon, 2014-03-17
[Bug 1288210] Re: Failed to attach volume
From: Ryan Moe, 2014-03-14
[Bug 1288210] Re: Failed to attach volume
From: Miroslav Anashkin, 2014-03-05
[Bug 1288210] [NEW] Failed to attach volume
From: Egor Kotko, 2014-03-05
References

[Bug 1288210] [NEW] Failed to attach volume
From: Egor Kotko, 2014-03-05