yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1429684] Re: Nova and Brick can log each other out of iscsi sessions

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: "Davanum Srinivas \(DIMS\)" <davanum@xxxxxxxxx>
Date: Mon, 09 Mar 2015 02:18:49 -0000
Reply-to: Bug 1429684 <1429684@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Nova does not use brick at the moment, right? What needs to change in
Nova?

** Also affects: cinder
   Importance: Undecided
       Status: New

** Changed in: cinder
       Status: New => Incomplete

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1429684

Title:
  Nova and Brick can log each other out of iscsi sessions

Status in Cinder:
  Incomplete
Status in OpenStack Compute (Nova):
  New

Bug description:
  Brick and nova are not synchronized with the same connect_volume lock.
  This can cause nova or cinder to logout of an iscsi portal when the
  other one is attempting to use it. if nova and cinder are on the same
  node.

  This may seem like a rare situation but commonly occurs in our CI
  system as we perform many operations involving both Nova and Brick
  concurrently. Likely when attaching/detaching to an instance while
  attaching to the node directly for image operations.


  In the below case, cinder logged out of the iscsi session while nova
  was retrying rescans attempting to detect the new LUN.

  Cinder volume logs:

  2015-03-07 17:27:14.288 28940 DEBUG oslo_concurrency.processutils [-]
  Running cmd (subprocess): sudo cinder-rootwrap
  /etc/cinder/rootwrap.conf iscsiadm -m node -T
  iqn.1992-08.com.netapp:sn.60a8eb0cc4bb11e4b041123478563412:vs.40815 -p
  10.250.119.127:3260 --logout execute /usr/local/lib/python2.7/dist-
  packages/oslo_concurrency/processutils.py:199

  2015-03-07 17:27:14.875 28940 DEBUG oslo_concurrency.processutils [-]
  CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf iscsiadm -m node
  -T iqn.1992-08.com.netapp:sn.60a8eb0cc4bb11e4b041123478563412:vs.40815
  -p 10.250.119.127:3260 --logout" returned: 0 in 0.588s execute
  /usr/local/lib/python2.7/dist-
  packages/oslo_concurrency/processutils.py:225

  2015-03-07 17:27:14.876 28940 DEBUG cinder.brick.initiator.connector
  [-] iscsiadm ('--logout',): stdout=Logging out of session [sid: 1,
  target:
  iqn.1992-08.com.netapp:sn.60a8eb0cc4bb11e4b041123478563412:vs.40815,
  portal: 10.250.119.127,3260]

  Logout of [sid: 1, target:
  iqn.1992-08.com.netapp:sn.60a8eb0cc4bb11e4b041123478563412:vs.40815,
  portal: 10.250.119.127,3260] successful.


  Nova compute logs:


  2015-03-07 17:27:12.617 DEBUG nova.virt.libvirt.volume [req-
  55f33c70-ec85-4041-aaf6-205f74abf979
  VolumesV1SnapshotTestJSON-1966398854
  VolumesV1SnapshotTestJSON-1188982339] iscsiadm ('--rescan',):
  stdout=Rescanning session [sid: 1, target:
  iqn.1992-08.com.netapp:sn.60a8eb0cc4bb11e4b041123478563412:vs.40815,
  portal: 10.250.119.127,3260]

   stderr= _run_iscsiadm
  /opt/stack/new/nova/nova/virt/libvirt/volume.py:364

  2015-03-07 17:27:12.617 WARNING nova.virt.libvirt.volume [req-
  55f33c70-ec85-4041-aaf6-205f74abf979
  VolumesV1SnapshotTestJSON-1966398854
  VolumesV1SnapshotTestJSON-1188982339] ISCSI volume not yet found at:
  vdb. Will rescan & retry.  Try number: 0

  2015-03-07 17:27:12.618 DEBUG oslo_concurrency.processutils [req-
  55f33c70-ec85-4041-aaf6-205f74abf979
  VolumesV1SnapshotTestJSON-1966398854
  VolumesV1SnapshotTestJSON-1188982339] Running cmd (subprocess): sudo
  nova-rootwrap /etc/nova/rootwrap.conf iscsiadm -m node -T
  iqn.1992-08.com.netapp:sn.60a8eb0cc4bb11e4b041123478563412:vs.40815 -p
  10.250.119.127:3260 --rescan execute /usr/local/lib/python2.7/dist-
  packages/oslo_concurrency/processutils.py:199

  2015-03-07 17:27:13.503 DEBUG oslo_concurrency.processutils [req-
  55f33c70-ec85-4041-aaf6-205f74abf979
  VolumesV1SnapshotTestJSON-1966398854
  VolumesV1SnapshotTestJSON-1188982339] CMD "sudo nova-rootwrap
  /etc/nova/rootwrap.conf iscsiadm -m node -T
  iqn.1992-08.com.netapp:sn.60a8eb0cc4bb11e4b041123478563412:vs.40815 -p
  10.250.119.127:3260 --rescan" returned: 0 in 0.885s execute
  /usr/local/lib/python2.7/dist-
  packages/oslo_concurrency/processutils.py:225

  2015-03-07 17:27:13.504 DEBUG nova.virt.libvirt.volume [req-
  55f33c70-ec85-4041-aaf6-205f74abf979
  VolumesV1SnapshotTestJSON-1966398854
  VolumesV1SnapshotTestJSON-1188982339] iscsiadm ('--rescan',):
  stdout=Rescanning session [sid: 1, target:
  iqn.1992-08.com.netapp:sn.60a8eb0cc4bb11e4b041123478563412:vs.40815,
  portal: 10.250.119.127,3260]

   stderr= _run_iscsiadm
  /opt/stack/new/nova/nova/virt/libvirt/volume.py:364

  2015-03-07 17:27:14.504 WARNING nova.virt.libvirt.volume [req-
  55f33c70-ec85-4041-aaf6-205f74abf979
  VolumesV1SnapshotTestJSON-1966398854
  VolumesV1SnapshotTestJSON-1188982339] ISCSI volume not yet found at:
  vdb. Will rescan & retry.  Try number: 1

  2015-03-07 17:27:14.505 DEBUG oslo_concurrency.processutils [req-
  55f33c70-ec85-4041-aaf6-205f74abf979
  VolumesV1SnapshotTestJSON-1966398854
  VolumesV1SnapshotTestJSON-1188982339] Running cmd (subprocess): sudo
  nova-rootwrap /etc/nova/rootwrap.conf iscsiadm -m node -T
  iqn.1992-08.com.netapp:sn.60a8eb0cc4bb11e4b041123478563412:vs.40815 -p
  10.250.119.127:3260 --rescan execute /usr/local/lib/python2.7/dist-
  packages/oslo_concurrency/processutils.py:199

  2015-03-07 17:27:14.563 DEBUG oslo_concurrency.processutils [req-
  55f33c70-ec85-4041-aaf6-205f74abf979
  VolumesV1SnapshotTestJSON-1966398854
  VolumesV1SnapshotTestJSON-1188982339] CMD "sudo nova-rootwrap
  /etc/nova/rootwrap.conf iscsiadm -m node -T
  iqn.1992-08.com.netapp:sn.60a8eb0cc4bb11e4b041123478563412:vs.40815 -p
  10.250.119.127:3260 --rescan" returned: 0 in 0.058s execute
  /usr/local/lib/python2.7/dist-
  packages/oslo_concurrency/processutils.py:225

  2015-03-07 17:27:14.564 DEBUG nova.virt.libvirt.volume [req-
  55f33c70-ec85-4041-aaf6-205f74abf979
  VolumesV1SnapshotTestJSON-1966398854
  VolumesV1SnapshotTestJSON-1188982339] iscsiadm ('--rescan',):
  stdout=Rescanning session [sid: 1, target:
  iqn.1992-08.com.netapp:sn.60a8eb0cc4bb11e4b041123478563412:vs.40815,
  portal: 10.250.119.127,3260]

   stderr= _run_iscsiadm
  /opt/stack/new/nova/nova/virt/libvirt/volume.py:364

  2015-03-07 17:27:18.564 WARNING nova.virt.libvirt.volume [req-
  55f33c70-ec85-4041-aaf6-205f74abf979
  VolumesV1SnapshotTestJSON-1966398854
  VolumesV1SnapshotTestJSON-1188982339] ISCSI volume not yet found at:
  vdb. Will rescan & retry.  Try number: 2

  2015-03-07 17:27:18.565 DEBUG oslo_concurrency.processutils [req-
  55f33c70-ec85-4041-aaf6-205f74abf979
  VolumesV1SnapshotTestJSON-1966398854
  VolumesV1SnapshotTestJSON-1188982339] Running cmd (subprocess): sudo
  nova-rootwrap /etc/nova/rootwrap.conf iscsiadm -m node -T
  iqn.1992-08.com.netapp:sn.60a8eb0cc4bb11e4b041123478563412:vs.40815 -p
  10.250.119.127:3260 --rescan execute /usr/local/lib/python2.7/dist-
  packages/oslo_concurrency/processutils.py:199

  2015-03-07 17:27:18.626 DEBUG oslo_concurrency.processutils [req-
  55f33c70-ec85-4041-aaf6-205f74abf979
  VolumesV1SnapshotTestJSON-1966398854
  VolumesV1SnapshotTestJSON-1188982339] CMD "sudo nova-rootwrap
  /etc/nova/rootwrap.conf iscsiadm -m node -T
  iqn.1992-08.com.netapp:sn.60a8eb0cc4bb11e4b041123478563412:vs.40815 -p
  10.250.119.127:3260 --rescan" returned: 21 in 0.061s execute
  /usr/local/lib/python2.7/dist-
  packages/oslo_concurrency/processutils.py:225

  2015-03-07 17:27:18.627 DEBUG oslo_concurrency.processutils [req-
  55f33c70-ec85-4041-aaf6-205f74abf979
  VolumesV1SnapshotTestJSON-1966398854
  VolumesV1SnapshotTestJSON-1188982339] u'sudo nova-rootwrap
  /etc/nova/rootwrap.conf iscsiadm -m node -T
  iqn.1992-08.com.netapp:sn.60a8eb0cc4bb11e4b041123478563412:vs.40815 -p
  10.250.119.127:3260 --rescan' failed. Not Retrying. execute
  /usr/local/lib/python2.7/dist-
  packages/oslo_concurrency/processutils.py:258

  2015-03-07 17:27:18.627 DEBUG oslo_concurrency.lockutils [req-
  55f33c70-ec85-4041-aaf6-205f74abf979
  VolumesV1SnapshotTestJSON-1966398854
  VolumesV1SnapshotTestJSON-1188982339] Lock "connect_volume" released
  by "connect_volume" :: held 6.588s inner /usr/local/lib/python2.7
  /dist-packages/oslo_concurrency/lockutils.py:442

  2015-03-07 17:27:18.627 ERROR nova.virt.block_device [req-
  55f33c70-ec85-4041-aaf6-205f74abf979
  VolumesV1SnapshotTestJSON-1966398854
  VolumesV1SnapshotTestJSON-1188982339] [instance: 10476e16-0f9b-
  45b3-b759-3718c48150e2] Driver failed to attach volume 673c93f7-97c0
  -44da-bed2-2b7e4106ac2d at /dev/vdb

  2015-03-07 17:27:18.627 28765 TRACE nova.virt.block_device [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2] Traceback (most recent call
  last):

  2015-03-07 17:27:18.627 28765 TRACE nova.virt.block_device [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]   File
  "/opt/stack/new/nova/nova/virt/block_device.py", line 249, in attach

  2015-03-07 17:27:18.627 28765 TRACE nova.virt.block_device [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]
  device_type=self['device_type'], encryption=encryption)

  2015-03-07 17:27:18.627 28765 TRACE nova.virt.block_device [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]   File
  "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 1036, in
  attach_volume

  2015-03-07 17:27:18.627 28765 TRACE nova.virt.block_device [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]
  self._connect_volume(connection_info, disk_info)

  2015-03-07 17:27:18.627 28765 TRACE nova.virt.block_device [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]   File
  "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 987, in
  _connect_volume

  2015-03-07 17:27:18.627 28765 TRACE nova.virt.block_device [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]
  driver.connect_volume(connection_info, disk_info)

  2015-03-07 17:27:18.627 28765 TRACE nova.virt.block_device [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]   File "/usr/local/lib/python2.7
  /dist-packages/oslo_concurrency/lockutils.py", line 431, in inner

  2015-03-07 17:27:18.627 28765 TRACE nova.virt.block_device [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]     return f(*args, **kwargs)

  2015-03-07 17:27:18.627 28765 TRACE nova.virt.block_device [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]   File
  "/opt/stack/new/nova/nova/virt/libvirt/volume.py", line 457, in
  connect_volume

  2015-03-07 17:27:18.627 28765 TRACE nova.virt.block_device [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]
  self._run_iscsiadm(iscsi_properties, ("--rescan",))

  2015-03-07 17:27:18.627 28765 TRACE nova.virt.block_device [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]   File
  "/opt/stack/new/nova/nova/virt/libvirt/volume.py", line 359, in
  _run_iscsiadm

  2015-03-07 17:27:18.627 28765 TRACE nova.virt.block_device [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]
  check_exit_code=check_exit_code)

  2015-03-07 17:27:18.627 28765 TRACE nova.virt.block_device [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]   File
  "/opt/stack/new/nova/nova/utils.py", line 205, in execute

  2015-03-07 17:27:18.627 28765 TRACE nova.virt.block_device [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]     return
  processutils.execute(*cmd, **kwargs)

  2015-03-07 17:27:18.627 28765 TRACE nova.virt.block_device [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]   File "/usr/local/lib/python2.7
  /dist-packages/oslo_concurrency/processutils.py", line 233, in execute

  2015-03-07 17:27:18.627 28765 TRACE nova.virt.block_device [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]     cmd=sanitized_cmd)

  2015-03-07 17:27:18.627 28765 TRACE nova.virt.block_device [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2] ProcessExecutionError:
  Unexpected error while running command.

  2015-03-07 17:27:18.627 28765 TRACE nova.virt.block_device [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2] Command: sudo nova-rootwrap
  /etc/nova/rootwrap.conf iscsiadm -m node -T
  iqn.1992-08.com.netapp:sn.60a8eb0cc4bb11e4b041123478563412:vs.40815 -p
  10.250.119.127:3260 --rescan

  2015-03-07 17:27:18.627 28765 TRACE nova.virt.block_device [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2] Exit code: 21

  2015-03-07 17:27:18.627 28765 TRACE nova.virt.block_device [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2] Stdout: u''

  2015-03-07 17:27:18.627 28765 TRACE nova.virt.block_device [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2] Stderr: u'iscsiadm: No session
  found.\n'

  2015-03-07 17:27:18.627 28765 TRACE nova.virt.block_device [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]

  2015-03-07 17:27:18.631 DEBUG keystoneclient.session [req-
  55f33c70-ec85-4041-aaf6-205f74abf979
  VolumesV1SnapshotTestJSON-1966398854
  VolumesV1SnapshotTestJSON-1188982339] REQ: curl -g -i -X POST
  http://127.0.0.1:8776/v2/b275c406f15c4e3c88b9cc2f201430a7/volumes/673c93f7-97c0
  -44da-bed2-2b7e4106ac2d/action -H "User-Agent: python-cinderclient" -H
  "Content-Type: application/json" -H "Accept: application/json" -H "X
  -Auth-Token: {SHA1}62885ded2920dbd143477d615394a9ff6e32ad6e" -d '{"os-
  terminate_connection": {"connector": {"ip": "10.250.121.14", "host":
  "slave-52122-devstack-trusty-cdot-iscsi-rax-iad", "initiator":
  "iqn.1993-08.org.debian:01:e8a2f229da"}}}' _http_log_request
  /usr/local/lib/python2.7/dist-packages/keystoneclient/session.py:195

  2015-03-07 17:27:18.632 DEBUG urllib3.util.retry [req-
  55f33c70-ec85-4041-aaf6-205f74abf979
  VolumesV1SnapshotTestJSON-1966398854
  VolumesV1SnapshotTestJSON-1188982339] Converted retries value: 0 ->
  Retry(total=0, connect=None, read=None, redirect=0) from_int
  /usr/local/lib/python2.7/dist-packages/urllib3/util/retry.py:155

  2015-03-07 17:27:19.109 DEBUG keystoneclient.session [req-
  55f33c70-ec85-4041-aaf6-205f74abf979
  VolumesV1SnapshotTestJSON-1966398854
  VolumesV1SnapshotTestJSON-1188982339] RESP: [202] date: Sat, 07 Mar
  2015 17:27:19 GMT content-length: 0 content-type: text/html;
  charset=UTF-8 x-openstack-request-id: req-e5cef5b1-b0b9-47af-a9fd-
  e462be8862f0 _http_log_response /usr/local/lib/python2.7/dist-
  packages/keystoneclient/session.py:223

  2015-03-07 17:27:19.110 ERROR nova.compute.manager [req-
  55f33c70-ec85-4041-aaf6-205f74abf979
  VolumesV1SnapshotTestJSON-1966398854
  VolumesV1SnapshotTestJSON-1188982339] [instance: 10476e16-0f9b-
  45b3-b759-3718c48150e2] Failed to attach 673c93f7-97c0-44da-
  bed2-2b7e4106ac2d at /dev/vdb

  2015-03-07 17:27:19.110 28765 TRACE nova.compute.manager [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2] Traceback (most recent call
  last):

  2015-03-07 17:27:19.110 28765 TRACE nova.compute.manager [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]   File
  "/opt/stack/new/nova/nova/compute/manager.py", line 4654, in
  _attach_volume

  2015-03-07 17:27:19.110 28765 TRACE nova.compute.manager [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]     do_check_attach=False,
  do_driver_attach=True)

  2015-03-07 17:27:19.110 28765 TRACE nova.compute.manager [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]   File
  "/opt/stack/new/nova/nova/virt/block_device.py", line 48, in wrapped

  2015-03-07 17:27:19.110 28765 TRACE nova.compute.manager [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]     ret_val = method(obj,
  context, *args, **kwargs)

  2015-03-07 17:27:19.110 28765 TRACE nova.compute.manager [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]   File
  "/opt/stack/new/nova/nova/virt/block_device.py", line 258, in attach

  2015-03-07 17:27:19.110 28765 TRACE nova.compute.manager [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]     connector)

  2015-03-07 17:27:19.110 28765 TRACE nova.compute.manager [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]   File "/usr/local/lib/python2.7
  /dist-packages/oslo_utils/excutils.py", line 82, in __exit__

  2015-03-07 17:27:19.110 28765 TRACE nova.compute.manager [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]     six.reraise(self.type_,
  self.value, self.tb)

  2015-03-07 17:27:19.110 28765 TRACE nova.compute.manager [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]   File
  "/opt/stack/new/nova/nova/virt/block_device.py", line 249, in attach

  2015-03-07 17:27:19.110 28765 TRACE nova.compute.manager [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]
  device_type=self['device_type'], encryption=encryption)

  2015-03-07 17:27:19.110 28765 TRACE nova.compute.manager [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]   File
  "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 1036, in
  attach_volume

  2015-03-07 17:27:19.110 28765 TRACE nova.compute.manager [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]
  self._connect_volume(connection_info, disk_info)

  2015-03-07 17:27:19.110 28765 TRACE nova.compute.manager [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]   File
  "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 987, in
  _connect_volume

  2015-03-07 17:27:19.110 28765 TRACE nova.compute.manager [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]
  driver.connect_volume(connection_info, disk_info)

  2015-03-07 17:27:19.110 28765 TRACE nova.compute.manager [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]   File "/usr/local/lib/python2.7
  /dist-packages/oslo_concurrency/lockutils.py", line 431, in inner

  2015-03-07 17:27:19.110 28765 TRACE nova.compute.manager [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]     return f(*args, **kwargs)

  2015-03-07 17:27:19.110 28765 TRACE nova.compute.manager [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]   File
  "/opt/stack/new/nova/nova/virt/libvirt/volume.py", line 457, in
  connect_volume

  2015-03-07 17:27:19.110 28765 TRACE nova.compute.manager [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]
  self._run_iscsiadm(iscsi_properties, ("--rescan",))

  2015-03-07 17:27:19.110 28765 TRACE nova.compute.manager [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]   File
  "/opt/stack/new/nova/nova/virt/libvirt/volume.py", line 359, in
  _run_iscsiadm

  2015-03-07 17:27:19.110 28765 TRACE nova.compute.manager [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]
  check_exit_code=check_exit_code)

  2015-03-07 17:27:19.110 28765 TRACE nova.compute.manager [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]   File
  "/opt/stack/new/nova/nova/utils.py", line 205, in execute

  2015-03-07 17:27:19.110 28765 TRACE nova.compute.manager [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]     return
  processutils.execute(*cmd, **kwargs)

  2015-03-07 17:27:19.110 28765 TRACE nova.compute.manager [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]   File "/usr/local/lib/python2.7
  /dist-packages/oslo_concurrency/processutils.py", line 233, in execute

  2015-03-07 17:27:19.110 28765 TRACE nova.compute.manager [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]     cmd=sanitized_cmd)

  2015-03-07 17:27:19.110 28765 TRACE nova.compute.manager [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2] ProcessExecutionError:
  Unexpected error while running command.

  2015-03-07 17:27:19.110 28765 TRACE nova.compute.manager [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2] Command: sudo nova-rootwrap
  /etc/nova/rootwrap.conf iscsiadm -m node -T
  iqn.1992-08.com.netapp:sn.60a8eb0cc4bb11e4b041123478563412:vs.40815 -p
  10.250.119.127:3260 --rescan

  2015-03-07 17:27:19.110 28765 TRACE nova.compute.manager [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2] Exit code: 21

  2015-03-07 17:27:19.110 28765 TRACE nova.compute.manager [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2] Stdout: u''

  2015-03-07 17:27:19.110 28765 TRACE nova.compute.manager [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2] Stderr: u'iscsiadm: No session
  found.\n'

  2015-03-07 17:27:19.110 28765 TRACE nova.compute.manager [instance:
  10476e16-0f9b-45b3-b759-3718c48150e2]

  2015-03-07 17:27:19.111 DEBUG keystoneclient.session [req-
  55f33c70-ec85-4041-aaf6-205f74abf979
  VolumesV1SnapshotTestJSON-1966398854
  VolumesV1SnapshotTestJSON-1188982339] REQ: curl -g -i -X POST
  http://127.0.0.1:8776/v2/b275c406f15c4e3c88b9cc2f201430a7/volumes/673c93f7-97c0
  -44da-bed2-2b7e4106ac2d/action -H "User-Agent: python-cinderclient" -H
  "Content-Type: application/json" -H "Accept: application/json" -H "X
  -Auth-Token: {SHA1}62885ded2920dbd143477d615394a9ff6e32ad6e" -d '{"os-
  unreserve": null}' _http_log_request /usr/local/lib/python2.7/dist-
  packages/keystoneclient/session.py:195

To manage notifications about this bug go to:
https://bugs.launchpad.net/cinder/+bug/1429684/+subscriptions
References

[Bug 1429684] [NEW] Nova and Brick can log each other out of iscsi sessions
From: Alex Meade, 2015-03-09