yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #32138
[Bug 1444439] Re: Resource tracker: unable to start nova compute
** Also affects: nova/kilo
Importance: Undecided
Status: New
** Changed in: nova/kilo
Milestone: None => kilo-rc2
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1444439
Title:
Resource tracker: unable to start nova compute
Status in OpenStack Compute (Nova):
Fix Committed
Status in OpenStack Compute (nova) kilo series:
New
Bug description:
After a failure of the resize and a deletion of the instance. I am
unable to restart the nova compute due to the exception below. The
instance was deleted via nova api.
The DB is as follows:
mysql> select * from migrations;
+---------------------+---------------------+------------+----+------------------+------------------+---------------+----------------+--------------------------------------+----------------------+----------------------+------------------+------------------+---------+
| created_at | updated_at | deleted_at | id | source_compute | dest_compute | dest_host | status | instance_uuid | old_instance_type_id | new_instance_type_id | source_node | dest_node | deleted |
+---------------------+---------------------+------------+----+------------------+------------------+---------------+----------------+--------------------------------------+----------------------+----------------------+------------------+------------------+---------+
| 2015-04-15 09:44:02 | 2015-04-15 09:44:08 | NULL | 1 | Ubuntu1404Server | Ubuntu1404Server | 10.160.94.173 | post-migrating | 42264e24-1385-41f1-8dfc-120a1891ab05 | 10 | 11 | domain-c167(DVS) | domain-c167(DVS) | 0 |
| 2015-04-15 09:48:13 | 2015-04-15 10:19:48 | NULL | 2 | Ubuntu1404Server | Ubuntu1404Server | 10.160.94.173 | reverted | fcab4bde-d93e-4d79-ae35-9d1306da10a4 | 10 | 11 | domain-c167(DVS) | domain-c167(DVS) | 0 |
| 2015-04-15 10:23:56 | 2015-04-15 10:24:03 | NULL | 3 | Ubuntu1404Server | Ubuntu1404Server | 10.160.94.173 | post-migrating | d074bbc0-b912-4c85-a02b-aabf56d45f0b | 10 | 11 | domain-c167(DVS) | domain-c167(DVS) | 0 |
| 2015-04-15 10:27:45 | 2015-04-15 10:28:16 | NULL | 4 | Ubuntu1404Server | Ubuntu1404Server | 10.160.94.173 | reverted | 21e59c96-fa2f-45e3-9070-e982a2dafea6 | 10 | 11 | domain-c167(DVS) | domain-c167(DVS) | 0 |
| 2015-04-15 10:28:43 | 2015-04-15 10:29:16 | NULL | 5 | Ubuntu1404Server | Ubuntu1404Server | 10.160.94.173 | confirming | 21e59c96-fa2f-45e3-9070-e982a2dafea6 | 10 | 11 | domain-c167(DVS) | domain-c167(DVS) | 0 |
| 2015-04-15 10:35:15 | 2015-04-15 10:53:16 | NULL | 6 | Ubuntu1404Server | Ubuntu1404Server | 10.160.94.173 | confirmed | 4abd75b5-bb91-4ce7-a928-2a96941ea9cb | 10 | 14 | domain-c167(DVS) | domain-c167(DVS) | 0 |
| 2015-04-15 10:35:39 | 2015-04-15 10:53:17 | NULL | 7 | Ubuntu1404Server | Ubuntu1404Server | 10.160.94.173 | confirmed | 5e01bddb-3978-4f6f-a4d3-6d24ed31afa4 | 14 | 10 | domain-c167(DVS) | domain-c167(DVS) | 0 |
| 2015-04-15 10:55:01 | 2015-04-15 10:55:02 | NULL | 8 | Ubuntu1404Server | Ubuntu1404Server | 10.160.94.173 | migrating | 20017567-5c83-4918-b269-525169009026 | 10 | 15 | domain-c167(DVS) | domain-c167(DVS) | 0 |
+---------------------+---------------------+------------+----+------------------+------------------+---------------+----------------+--------------------------------------+----------------------+----------------------+------------------+------------------+---------+
8 rows in set (0.00 sec)
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup Traceback (most recent call last):
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/openstack/common/threadgroup.py", line 145, in wait
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup x.wait()
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/openstack/common/threadgroup.py", line 47, in wait
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup return self.thread.wait()
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py", line 175, in wait
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup return self._exit_event.wait()
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 121, in wait
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup return hubs.get_hub().switch()
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 294, in switch
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup return self.greenlet.switch()
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py", line 214, in main
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup result = function(*args, **kwargs)
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/openstack/common/service.py", line 497, in run_service
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup service.start()
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/service.py", line 183, in start
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup self.manager.pre_start_hook()
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/compute/manager.py", line 1287, in pre_start_hook
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup self.update_available_resource(nova.context.get_admin_context())
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/compute/manager.py", line 6236, in update_available_resource
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup rt.update_available_resource(context)
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/compute/resource_tracker.py", line 402, in update_available_resource
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup self._update_available_resource(context, resources)
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup File "/usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py", line 445, in inner
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup return f(*args, **kwargs)
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/compute/resource_tracker.py", line 445, in _update_available_resource
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup self._update_usage_from_migrations(context, resources, migrations)
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/compute/resource_tracker.py", line 709, in _update_usage_from_migrations
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup instance = migration.instance
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/objects/migration.py", line 80, in instance
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup return objects.Instance.get_by_uuid(self._context, self.instance_uuid)
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/objects/base.py", line 161, in wrapper
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup args, kwargs)
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/conductor/rpcapi.py", line 325, in object_class_action
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup objver=objver, args=args, kwargs=kwargs)
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 156, in call
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup retry=self.retry)
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 90, in _send
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup timeout=timeout, retry=retry)
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 350, in send
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup retry=retry)
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 341, in _send
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup raise result
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup InstanceNotFound_Remote: Instance 42264e24-1385-41f1-8dfc-120a1891ab05 could not be found.
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup Traceback (most recent call last):
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/conductor/manager.py", line 423, in _object_dispatch
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup return getattr(target, method)(*args, **kwargs)
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/objects/base.py", line 163, in wrapper
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup result = fn(cls, context, *args, **kwargs)
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/objects/instance.py", line 564, in get_by_uuid
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup use_slave=use_slave)
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/db/api.py", line 651, in instance_get_by_uuid
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup columns_to_join, use_slave=use_slave)
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 233, in wrapper
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup return f(*args, **kwargs)
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 1744, in instance_get_by_uuid
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup columns_to_join=columns_to_join, use_slave=use_slave)
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 1756, in _instance_get_by_uuid
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup raise exception.InstanceNotFound(instance_id=uuid)
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup InstanceNotFound: Instance 42264e24-1385-41f1-8dfc-120a1891ab05 could not be found.
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup
2015-04-15 04:47:04.821 TRACE nova.openstack.common.threadgroup
2015-04-15 04:47:04.824 INFO oslo_vmware.api [req-2d7c5d01-438c-494c-a382-b7c55b71d8be None None] Logging out and terminating the current session with ID = f4fcb.
nicira@Ubuntu1404Server:~/devstack$
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1444439/+subscriptions
References
