yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1199811] Re: nova live-migration of volume-backed VM fails because of image is None

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Thierry Carrez <thierry.carrez+lp@xxxxxxxxx>
Date: Thu, 05 Sep 2013 15:39:03 -0000
Reply-to: Bug 1199811 <1199811@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
** Changed in: nova
       Status: Fix Committed => Fix Released

** Changed in: nova
    Milestone: None => havana-3

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1199811

Title:
  nova live-migration of volume-backed VM fails because of image is None

Status in OpenStack Compute (Nova):
  Fix Released
Status in OpenStack Compute (nova) grizzly series:
  Fix Released

Bug description:
  cinder create --image-id ab24ccbd-4c89-4444-b0b2-a06a79c44306 --display-name my-bootable-vol 3
  nova boot --flavor 1 --block_device_mapping vda=0ea63954-0a9a-4df7-b5a9-08afd0981dcc:::0 volume_backed
  nova live-migration volume_backed

  resultes in migration failed:

  2013-07-10 11:14:06.055 DEBUG nova.api.openstack.wsgi [req-2368adc9-b7b4-4016-82c2-38e815bd0025 584de53b707647ca867b277c6c6d9937 62663061e1774fd18ec35f46a19a1c92] No Content-Type provided in request get_body /usr/lib/python2.6/site-packages/nova/api/openstack/wsgi.py:791
  2013-07-10 11:14:06.056 DEBUG nova.api.openstack.wsgi [req-2368adc9-b7b4-4016-82c2-38e815bd0025 584de53b707647ca867b277c6c6d9937 62663061e1774fd18ec35f46a19a1c92] Calling method <bound method Controller.detail of <nova.api.openstack.compute.servers.Controller object at 0x1cad290>> _process_stack /usr/lib/python2.6/site-packages/nova/api/openstack/wsgi.py:911
  2013-07-10 11:14:06.057 DEBUG nova.compute.api [req-2368adc9-b7b4-4016-82c2-38e815bd0025 584de53b707647ca867b277c6c6d9937 62663061e1774fd18ec35f46a19a1c92] Searching by: {'deleted': False, 'project_id': u'62663061e1774fd18ec35f46a19a1c92'} get_all /usr/lib/python2.6/site-packages/nova/compute/api.py:1372
  2013-07-10 11:14:06.102 INFO nova.osapi_compute.wsgi.server [req-2368adc9-b7b4-4016-82c2-38e815bd0025 584de53b707647ca867b277c6c6d9937 62663061e1774fd18ec35f46a19a1c92] 10.34.69.137 "GET /v2/62663061e1774fd18ec35f46a19a1c92/servers/detail HTTP/1.1" status: 200 len: 1563 time: 0.2836871
   
  2013-07-10 11:14:06.332 DEBUG nova.api.openstack.wsgi [req-f764fd10-e182-43c6-b308-f4eb9fce2495 584de53b707647ca867b277c6c6d9937 62663061e1774fd18ec35f46a19a1c92] Action: 'action', body: {"os-migrateLive": {"disk_over_commit": false, "block_migration": false, "host": null}} _process_stack /usr/lib/python2.6/site-packages/nova/api/openstack/wsgi.py:910
  2013-07-10 11:14:06.332 DEBUG nova.api.openstack.wsgi [req-f764fd10-e182-43c6-b308-f4eb9fce2495 584de53b707647ca867b277c6c6d9937 62663061e1774fd18ec35f46a19a1c92] Calling method <bound method AdminActionsController._migrate_live of <nova.api.openstack.compute.contrib.admin_actions.AdminActionsController object at 0x22e8850>> _process_stack /usr/lib/python2.6/site-packages/nova/api/openstack/wsgi.py:911
  2013-07-10 11:14:06.390 DEBUG nova.compute.api [req-f764fd10-e182-43c6-b308-f4eb9fce2495 584de53b707647ca867b277c6c6d9937 62663061e1774fd18ec35f46a19a1c92] [instance: 32c0284d-ad8d-4f21-b6aa-8a46f1232e98] Going to try to live migrate instance to another host live_migrate /usr/lib/python2.6/site-packages/nova/compute/api.py:2447
  2013-07-10 11:14:06.500 DEBUG nova.openstack.common.rpc.amqp [req-f764fd10-e182-43c6-b308-f4eb9fce2495 584de53b707647ca867b277c6c6d9937 62663061e1774fd18ec35f46a19a1c92] Making synchronous call on scheduler ... multicall /usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py:583
  2013-07-10 11:14:06.501 DEBUG nova.openstack.common.rpc.amqp [req-f764fd10-e182-43c6-b308-f4eb9fce2495 584de53b707647ca867b277c6c6d9937 62663061e1774fd18ec35f46a19a1c92] MSG_ID is c098878314ae435eb57482571ce55115 multicall /usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py:586
  2013-07-10 11:14:06.501 DEBUG nova.openstack.common.rpc.amqp [req-f764fd10-e182-43c6-b308-f4eb9fce2495 584de53b707647ca867b277c6c6d9937 62663061e1774fd18ec35f46a19a1c92] UNIQUE_ID is a68d14e5878e460fb9cb219e5a357180. _add_unique_id /usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py:337
  2013-07-10 11:14:06.511 INFO nova.openstack.common.rpc.impl_qpid [req-f764fd10-e182-43c6-b308-f4eb9fce2495 584de53b707647ca867b277c6c6d9937 62663061e1774fd18ec35f46a19a1c92] Connected to AMQP server on 10.34.69.137:5672
  2013-07-10 11:14:06.518 DEBUG nova.openstack.common.rpc.amqp [req-f764fd10-e182-43c6-b308-f4eb9fce2495 584de53b707647ca867b277c6c6d9937 62663061e1774fd18ec35f46a19a1c92] Pool creating new connection create /usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py:75
  2013-07-10 11:14:06.522 INFO nova.openstack.common.rpc.impl_qpid [req-f764fd10-e182-43c6-b308-f4eb9fce2495 584de53b707647ca867b277c6c6d9937 62663061e1774fd18ec35f46a19a1c92] Connected to AMQP server on 10.34.69.137:5672
  2013-07-10 11:14:06.602 ERROR nova.api.openstack.compute.contrib.admin_actions [req-f764fd10-e182-43c6-b308-f4eb9fce2495 584de53b707647ca867b277c6c6d9937 62663061e1774fd18ec35f46a19a1c92] Live migration of instance 32c0284d-ad8d-4f21-b6aa-8a46f1232e98 to another host failed
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions Traceback (most recent call last):
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions   File "/usr/lib/python2.6/site-packages/nova/api/openstack/compute/contrib/admin_actions.py", line 284, in _migrate_live
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions     disk_over_commit, host)
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions   File "/usr/lib/python2.6/site-packages/nova/compute/api.py", line 134, in inner
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions     return f(self, context, instance, *args, **kw)
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions   File "/usr/lib/python2.6/site-packages/nova/compute/api.py", line 2454, in live_migrate
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions     disk_over_commit, instance, host_name)
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions   File "/usr/lib/python2.6/site-packages/nova/scheduler/rpcapi.py", line 110, in live_migration
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions     dest=dest))
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions   File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/proxy.py", line 80, in call
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions     return rpc.call(context, self._get_topic(topic), msg, timeout)
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions   File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/__init__.py", line 140, in call
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions     return _get_impl().call(CONF, context, topic, msg, timeout)
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions   File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/impl_qpid.py", line 616, in call
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions     rpc_amqp.get_connection_pool(conf, Connection))
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions   File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 612, in call
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions     rv = list(rv)
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions   File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 508, in __iter__
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions     raise result
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions AttributeError: 'NoneType' object has no attribute 'get'
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions Traceback (most recent call last):
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions   File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 430, in _process_data
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions     rval = self.proxy.dispatch(ctxt, version, method, **args)
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions   File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/dispatcher.py", line 133, in dispatch
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions     return getattr(proxyobj, method)(ctxt, **kwargs)
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions   File "/usr/lib/python2.6/site-packages/nova/scheduler/manager.py", line 117, in live_migration
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions     context, ex, {})
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions   File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions     self.gen.next()
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions   File "/usr/lib/python2.6/site-packages/nova/scheduler/manager.py", line 96, in live_migration
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions     block_migration, disk_over_commit)
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions   File "/usr/lib/python2.6/site-packages/nova/scheduler/driver.py", line 196, in schedule_live_migration
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions     ignore_hosts)
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions   File "/usr/lib/python2.6/site-packages/nova/scheduler/driver.py", line 272, in _live_migration_dest_check
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions     filter_properties)[0]
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions   File "/usr/lib/python2.6/site-packages/nova/scheduler/filter_scheduler.py", line 146, in select_hosts
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions     request_spec, filter_properties, instance_uuids)]
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions   File "/usr/lib/python2.6/site-packages/nova/scheduler/filter_scheduler.py", line 336, in _schedule
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions     filter_properties)
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions   File "/usr/lib/python2.6/site-packages/nova/scheduler/host_manager.py", line 342, in get_filtered_hosts
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions     hosts, filter_properties)
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions   File "/usr/lib/python2.6/site-packages/nova/filters.py", line 53, in get_filtered_objects
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions     return list(objs)
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions   File "/usr/lib/python2.6/site-packages/nova/filters.py", line 38, in filter_all
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions     for obj in filter_obj_list:
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions   File "/usr/lib/python2.6/site-packages/nova/filters.py", line 39, in filter_all
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions     if self._filter_one(obj, filter_properties):
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions   File "/usr/lib/python2.6/site-packages/nova/scheduler/filters/__init__.py", line 30, in _filter_one
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions     return self.host_passes(obj, filter_properties)
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions   File "/usr/lib/python2.6/site-packages/nova/scheduler/filters/image_props_filter.py", line 78, in host_passes
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions     image_props = spec.get('image', {}).get('properties', {})
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions AttributeError: 'NoneType' object has no attribute 'get'
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions
  2013-07-10 11:14:06.602 4625 TRACE nova.api.openstack.compute.contrib.admin_actions
  2013-07-10 11:14:06.613 INFO nova.api.openstack.wsgi [req-f764fd10-e182-43c6-b308-f4eb9fce2495 584de53b707647ca867b277c6c6d9937 62663061e1774fd18ec35f46a19a1c92] HTTP exception thrown: Live migration of instance 32c0284d-ad8d-4f21-b6aa-8a46f1232e98 to another host failed
  2013-07-10 11:14:06.613 DEBUG nova.api.openstack.wsgi [req-f764fd10-e182-43c6-b308-f4eb9fce2495 584de53b707647ca867b277c6c6d9937 62663061e1774fd18ec35f46a19a1c92] Returning 400 to user: Live migration of instance 32c0284d-ad8d-4f21-b6aa-8a46f1232e98 to another host failed __call__ /usr/lib/python2.6/site-packages/nova/api/openstack/wsgi.py:1165
  2013-07-10 11:14:06.615 INFO nova.osapi_compute.wsgi.server [req-f764fd10-e182-43c6-b308-f4eb9fce2495 584de53b707647ca867b277c6c6d9937 62663061e1774fd18ec35f46a19a1c92] 10.34.69.137 "POST /v2/62663061e1774fd18ec35f46a19a1c92/servers/32c0284d-ad8d-4f21-b6aa-8a46f1232e98/action HTTP/1.1" status: 400 len: 327 time: 0.5004730

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1199811/+subscriptions