yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1253578] Re: Instance Boot from volume fails with iSER

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Sean Dague <sean@xxxxxxxxx>
Date: Wed, 01 Apr 2015 15:21:19 -0000
Reply-to: Bug 1253578 <1253578@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
** Changed in: nova
       Status: Incomplete => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1253578

Title:
  Instance Boot from volume fails with iSER

Status in OpenStack Compute (Nova):
  Invalid

Bug description:
  steps to reproduce:
  1. Create new volume with image source.
  2. Launch a new instance boot from the above volume using iSER transport.

  Nova-Compute side fails with the below logs:
  2013-11-12 16:25:49.508 7157 INFO nova.virt.libvirt.driver [req-233b2283-94b3-4340-b77e-76f4efbcc2c4 2cb6dbc75b0e4df1aed19b1253f5344e 330f12e1871348e09e03ffea2e9523ff] [instance: 19646297-0eec-4182-9c17-d9172faf5663] Creating image 2013-11-12 16:25:56.048 7157 ERROR nova.compute.manager [req-233b2283-94b3-4340-b77e-76f4efbcc2c4 2cb6dbc75b0e4df1aed19b1253f5344e 330f12e1871348e09e03ffea2e9523ff] [instance: 19646297-0eec-4182-9c17-d9172faf5663] Instance failed to spa
  wn 2013-11-12 16:25:56.048 7157 TRACE nova.compute.manager [instance: 19646297-0eec-4182-9c17-d9172faf5663] Traceback (most recent call last):
  2013-11-12 16:25:56.048 7157 TRACE nova.compute.manager [instance: 19646297-0eec-4182-9c17-d9172faf5663] File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1407, in _spawn 2013-11-12 16:25:56.048 7157 TRACE nova.compute.manager [instance: 19646297-0eec-4182-9c17-d9172faf5663] block_device_info)
  2013-11-12 16:25:56.048 7157 TRACE nova.compute.manager [instance: 19646297-0eec-4182-9c17-d9172faf5663] File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 2070, in spawn 2013-11-12 16:25:56.048 7157 TRACE nova.compute.manager [instance: 19646297-0eec-4182-9c17-d9172faf5663] block_device_info, context=context)
  2013-11-12 16:25:56.048 7157 TRACE nova.compute.manager [instance: 19646297-0eec-4182-9c17-d9172faf5663] File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 3184, in _create_domain_and_network 2013-11-12 16:25:56.048 7157 TRACE nova.compute.manager [instance: 19646297-0eec-4182-9c17-d9172faf5663] disk_info)
  2013-11-12 16:25:56.048 7157 TRACE nova.compute.manager [instance: 19646297-0eec-4182-9c17-d9172faf5663] File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 1030, in volume_driver_method 2013-11-12 16:25:56.048 7157 TRACE nova.compute.manager [instance: 19646297-0eec-4182-9c17-d9172faf5663] return method(connection_info, *args, **kwargs)
  2013-11-12 16:25:56.048 7157 TRACE nova.compute.manager [instance: 19646297-0eec-4182-9c17-d9172faf5663] File "/usr/lib/python2.6/site-packages/nova/openstack/common/lockutils.py", line 246, in inner 2013-11-12 16:25:56.048 7157 TRACE nova.compute.manager [instance: 19646297-0eec-4182-9c17-d9172faf5663] return f(*args, **kwargs)
  2013-11-12 16:25:56.048 7157 TRACE nova.compute.manager [instance: 19646297-0eec-4182-9c17-d9172faf5663] File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/volume.py", line 536, in connect_volume 2013-11-12 16:25:56.048 7157 TRACE nova.compute.manager [instance: 19646297-0eec-4182-9c17-d9172faf5663] self._connect_to_iser_portal(iser_properties)
  2013-11-12 16:25:56.048 7157 TRACE nova.compute.manager [instance: 19646297-0eec-4182-9c17-d9172faf5663] File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/volume.py", line 643, in _connect_to_iser_portal 2013-11-12 16:25:56.048 7157 TRACE nova.compute.manager [instance: 19646297-0eec-4182-9c17-d9172faf5663] "iser")
  2013-11-12 16:25:56.048 7157 TRACE nova.compute.manager [instance: 19646297-0eec-4182-9c17-d9172faf5663] File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/volume.py", line 227, in _iscsiadm_update 2013-11-12 16:25:56.048 7157 TRACE nova.compute.manager [instance: 19646297-0eec-4182-9c17-d9172faf5663] return self._run_iscsiadm(iscsi_properties, iscsi_command, **kwargs)
  2013-11-12 16:25:56.048 7157 TRACE nova.compute.manager [instance: 19646297-0eec-4182-9c17-d9172faf5663] File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/volume.py", line 218, in _run_iscsiadm 2013-11-12 16:25:56.048 7157 TRACE nova.compute.manager [instance: 19646297-0eec-4182-9c17-d9172faf5663] check_exit_code=check_exit_code)
  2013-11-12 16:25:56.048 7157 TRACE nova.compute.manager [instance: 19646297-0eec-4182-9c17-d9172faf5663] File "/usr/lib/python2.6/site-packages/nova/utils.py", line 177, in execute 2013-11-12 16:25:56.048 7157 TRACE nova.compute.manager [instance: 19646297-0eec-4182-9c17-d9172faf5663] return processutils.execute(*cmd, **kwargs)
  2013-11-12 16:25:56.048 7157 TRACE nova.compute.manager [instance: 19646297-0eec-4182-9c17-d9172faf5663] File "/usr/lib/python2.6/site-packages/nova/openstack/common/processutils.py", line 178, in execute 2013-11-12 16:25:56.048 7157 TRACE nova.compute.manager [instance: 19646297-0eec-4182-9c17-d9172faf5663] cmd=' '.join(cmd))
  2013-11-12 16:25:56.048 7157 TRACE nova.compute.manager [instance: 19646297-0eec-4182-9c17-d9172faf5663] ProcessExecutionError: Unexpected error while running command. 2013-11-12 16:25:56.048 7157 TRACE nova.compute.manager [instance: 19646297-0eec-4182-9c17-d9172faf5663] Command: sudo nova-rootwrap /etc/nova/rootwrap.conf iscsiadm m node -T iqn.2010-10.org.iser.openstack:volume-55341fa3-cbd5-4c52-87
  53-e3171dd8dcc4 -p 11.131.1.100:3260 --op update -n iface.transport_name -v iser 2013-11-12 16:25:56.048 7157 TRACE nova.compute.manager [instance: 19646297-0eec-4182-9c17-d9172faf5663] Exit code: 15
  2013-11-12 16:25:56.048 7157 TRACE nova.compute.manager [instance: 19646297-0eec-4182-9c17-d9172faf5663] Stdout: '' 2013-11-12 16:25:56.048 7157 TRACE nova.compute.manager [instance: 19646297-0eec-4182-9c17-d9172faf5663] Stderr: 'iscsiadm: Cannot modify node/iface transport name while a session is using it. Log out the session then update record.\n'
  2013-11-12 16:25:56.048 7157 TRACE nova.compute.manager [instance: 19646297-0eec-4182-9c17-d9172faf5663] 2013-11-12 16:25:56.218 7157 AUDIT nova.compute.manager [req-233b2283-94b3-4340-b77e-76f4efbcc2c4 2cb6dbc75b0e4df1aed19b1253f5344e 330f12e1871348e09e03ffea2e9523ff] [instance: 19646297-0eec-4182-9c17-d9172faf5663] Terminating instance
  2013-11-12 16:25:57.157 7157 ERROR nova.virt.libvirt.driver [] [instance: 19646297-0eec-4182-9c17-d9172faf5663] During wait destroy, instance disappeared. 2013-11-12 16:25:57.667 7157 INFO nova.virt.libvirt.driver [req-233b2283-94b3-4340-b77e-76f4efbcc2c4 2cb6dbc75b0e4df1aed19b1253f5344e 330f12e1871348e09e03ffea2e9523ff] [instance: 19646297-0eec-4182-9c17-d9172faf5663] Deleting instance files /var/lib/nova/instances/19646297-0eec-4182-9c17-d9172faf5663
  2013-11-12 16:25:57.668 7157 INFO nova.virt.libvirt.driver [req-233b2283-94b3-4340-b77e-76f4efbcc2c4 2cb6dbc75b0e4df1aed19b1253f5344e 330f12e1871348e09e03ffea2e9523ff] [instance: 19646297-0eec-4182-9c17-d9172faf5663] Deletion of /var/lib/nova/instances/19646297-0eec-4182-9c17-d9172faf5663 complete

  This happens due to attempt  to change iface transport while a session
  is active.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1253578/+subscriptions