← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #09907

[Bug 1281087] [NEW] Nova block migrate will fail if image or snapshot is not available

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

When you do a block migration nova checks if the image is in the _base directory.
When the machine does not have the image it will try to download it from Glance.
Since some people throw away unused images and especially snapshots the migration will fail (Traceback, below) when it is not available.
This makes block migration for e.g. maintenance a bit of a headache.


There are 2 possible solutions:
1) A feature that will copy the _base file from the source machine to the target migration machine
2) Add a glance "soft-delete" option so just remove it from the view but actually keep it available for live-migration


2014-02-17 07:51:33.170 27273 ERROR nova.openstack.common.rpc.amqp [req-e5d497d8-16a1-4b0b-82b5-7ea1eba0dcc6 a6f038c720424af48ef4dd3ef6ce86c1 2792c7106e98489da6e38a1ad9ae6685] Exception during message handling
2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp Traceback (most recent call last):
2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 461, in _process_data
2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp     **args)
2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/dispatcher.py", line 172, in dispatch
2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp     result = getattr(proxyobj, method)(ctxt, **kwargs)
2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/exception.py", line 90, in wrapped
2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp     payload)
2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/exception.py", line 73, in wrapped
2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp     return f(self, context, *args, **kw)
2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 4067, in live_migration
2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp     block_migration, migrate_data)
2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 4059, in live_migration
2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp     block_migration, disk, dest, migrate_data)
2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/compute/rpcapi.py", line 492, in pre_live_migration
2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp     disk=disk, migrate_data=migrate_data)
2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/rpcclient.py", line 85, in call
2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp     return self._invoke(self.proxy.call, ctxt, method, **kwargs)
2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/rpcclient.py", line 63, in _invoke
2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp     return cast_or_call(ctxt, msg, **self.kwargs)
2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/proxy.py", line 126, in call
2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp     result = rpc.call(context, real_topic, msg, timeout)
2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/__init__.py", line 139, in call
2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp     return _get_impl().call(CONF, context, topic, msg, timeout)
2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/impl_kombu.py", line 816, in call
2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp     rpc_amqp.get_connection_pool(conf, Connection))
2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 574, in call
2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp     rv = list(rv)
2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 539, in __iter__
2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp     raise result
2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp ImageNotFound_Remote: Image 2b72d644-c346-4ac6-a645-8685fe50e06d could not be found.
2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp Traceback (most recent call last):

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1281087

Title:
  Nova block migrate will fail if image or snapshot is not available

Status in OpenStack Compute (Nova):
  New

Bug description:
  When you do a block migration nova checks if the image is in the _base directory.
  When the machine does not have the image it will try to download it from Glance.
  Since some people throw away unused images and especially snapshots the migration will fail (Traceback, below) when it is not available.
  This makes block migration for e.g. maintenance a bit of a headache.

  
  There are 2 possible solutions:
  1) A feature that will copy the _base file from the source machine to the target migration machine
  2) Add a glance "soft-delete" option so just remove it from the view but actually keep it available for live-migration


  
  2014-02-17 07:51:33.170 27273 ERROR nova.openstack.common.rpc.amqp [req-e5d497d8-16a1-4b0b-82b5-7ea1eba0dcc6 a6f038c720424af48ef4dd3ef6ce86c1 2792c7106e98489da6e38a1ad9ae6685] Exception during message handling
  2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp Traceback (most recent call last):
  2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 461, in _process_data
  2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp     **args)
  2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/dispatcher.py", line 172, in dispatch
  2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp     result = getattr(proxyobj, method)(ctxt, **kwargs)
  2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/exception.py", line 90, in wrapped
  2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp     payload)
  2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/exception.py", line 73, in wrapped
  2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp     return f(self, context, *args, **kw)
  2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 4067, in live_migration
  2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp     block_migration, migrate_data)
  2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 4059, in live_migration
  2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp     block_migration, disk, dest, migrate_data)
  2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/compute/rpcapi.py", line 492, in pre_live_migration
  2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp     disk=disk, migrate_data=migrate_data)
  2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/rpcclient.py", line 85, in call
  2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp     return self._invoke(self.proxy.call, ctxt, method, **kwargs)
  2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/rpcclient.py", line 63, in _invoke
  2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp     return cast_or_call(ctxt, msg, **self.kwargs)
  2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/proxy.py", line 126, in call
  2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp     result = rpc.call(context, real_topic, msg, timeout)
  2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/__init__.py", line 139, in call
  2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp     return _get_impl().call(CONF, context, topic, msg, timeout)
  2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/impl_kombu.py", line 816, in call
  2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp     rpc_amqp.get_connection_pool(conf, Connection))
  2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 574, in call
  2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp     rv = list(rv)
  2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 539, in __iter__
  2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp     raise result
  2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp ImageNotFound_Remote: Image 2b72d644-c346-4ac6-a645-8685fe50e06d could not be found.
  2014-02-17 07:51:33.170 27273 TRACE nova.openstack.common.rpc.amqp Traceback (most recent call last):

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1281087/+subscriptions

Follow ups

References

  Thread Previous • Date Previous • • Thread Next