yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #31538
[Bug 1385246] Re: Catch specific exceptions for _get_instance_nw_info
** Changed in: nova/juno
Status: Fix Committed => Fix Released
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1385246
Title:
Catch specific exceptions for _get_instance_nw_info
Status in OpenStack Compute (Nova):
Fix Released
Status in OpenStack Compute (nova) juno series:
Fix Released
Bug description:
Occasionally see the following logs:
2014-10-19 17:29:54.170 27466 ERROR nova.compute.manager [-] [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] An error occurred while refreshing the network cache.
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] Traceback (most recent call last):
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] File "/opt/stack/nova/nova/compute/manager.py", line 5327, in _heal_instance_info_cache
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] self._get_instance_nw_info(context, instance, use_slave=True)
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] File "/opt/stack/nova/nova/compute/manager.py", line 1233, in _get_instance_nw_info
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] instance)
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] File "/opt/stack/nova/nova/network/api.py", line 48, in wrapped
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] return func(self, context, *args, **kwargs)
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] File "/opt/stack/nova/nova/network/api.py", line 374, in get_instance_nw_info
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] result = self._get_instance_nw_info(context, instance)
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] File "/opt/stack/nova/nova/network/api.py", line 390, in _get_instance_nw_info
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] nw_info = self.network_rpcapi.get_instance_nw_info(context, **args)
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] File "/opt/stack/nova/nova/network/rpcapi.py", line 242, in get_instance_nw_info
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] host=host, project_id=project_id)
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/client.py", line 152, in call
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] retry=self.retry)
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/transport.py", line 90, in _send
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] timeout=timeout, retry=retry)
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/_drivers/amqpdriver.py", line 408, in send
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] retry=retry)
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/_drivers/amqpdriver.py", line 399, in _send
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] raise result
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] InstanceNotFound_Remote: Instance dd50ac93-adf2-4e9d-915f-d8d1527d82e1 could not be found.
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] Traceback (most recent call last):
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1]
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] File "/opt/stack/nova/nova/conductor/manager.py", line 400, in _object_dispatch
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] return getattr(target, method)(context, *args, **kwargs)
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1]
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] File "/opt/stack/nova/nova/objects/base.py", line 155, in wrapper
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] result = fn(cls, context, *args, **kwargs)
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1]
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] File "/opt/stack/nova/nova/objects/virtual_interface.py", line 100, in get_by_instance_uuid
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] use_slave=use_slave)
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1]
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] File "/opt/stack/nova/nova/db/api.py", line 596, in virtual_interface_get_by_instance
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] use_slave=use_slave)
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1]
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 143, in wrapper
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] return f(*args, **kwargs)
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1]
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 155, in wrapper
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] instance_get_by_uuid(context, instance_uuid)
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1]
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 143, in wrapper
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] return f(*args, **kwargs)
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1]
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 1719, in instance_get_by_uuid
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] columns_to_join=columns_to_join, use_slave=use_slave)
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1]
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 1731, in _instance_get_by_uuid
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] raise exception.InstanceNotFound(instance_id=uuid)
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1]
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] InstanceNotFound: Instance dd50ac93-adf2-4e9d-915f-d8d1527d82e1 could not be found.
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1]
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] Traceback (most recent call last):
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1]
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/server.py", line 139, in inner
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] return func(*args, **kwargs)
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1]
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] File "/opt/stack/nova/nova/network/manager.py", line 594, in get_instance_nw_info
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] context, instance_uuid, use_slave=use_slave)
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1]
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] File "/opt/stack/nova/nova/objects/base.py", line 153, in wrapper
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] args, kwargs)
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1]
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] File "/opt/stack/nova/nova/conductor/rpcapi.py", line 346, in object_class_action
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] objver=objver, args=args, kwargs=kwargs)
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1]
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/client.py", line 152, in call
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] retry=self.retry)
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1]
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/transport.py", line 90, in _send
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] timeout=timeout, retry=retry)
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1]
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/_drivers/amqpdriver.py", line 408, in send
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] retry=retry)
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1]
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/_drivers/amqpdriver.py", line 399, in _send
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] raise result
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1]
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] InstanceNotFound_Remote: Instance dd50ac93-adf2-4e9d-915f-d8d1527d82e1 could not be found.
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] Traceback (most recent call last):
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1]
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] File "/opt/stack/nova/nova/conductor/manager.py", line 400, in _object_dispatch
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] return getattr(target, method)(context, *args, **kwargs)
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1]
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] File "/opt/stack/nova/nova/objects/base.py", line 155, in wrapper
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] result = fn(cls, context, *args, **kwargs)
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1]
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] File "/opt/stack/nova/nova/objects/virtual_interface.py", line 100, in get_by_instance_uuid
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] use_slave=use_slave)
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1]
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] File "/opt/stack/nova/nova/db/api.py", line 596, in virtual_interface_get_by_instance
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] use_slave=use_slave)
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1]
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 143, in wrapper
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] return f(*args, **kwargs)
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1]
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 155, in wrapper
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] instance_get_by_uuid(context, instance_uuid)
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1]
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 143, in wrapper
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] return f(*args, **kwargs)
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1]
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 1719, in instance_get_by_uuid
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] columns_to_join=columns_to_join, use_slave=use_slave)
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1]
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 1731, in _instance_get_by_uuid
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] raise exception.InstanceNotFound(instance_id=uuid)
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1]
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1] InstanceNotFound: Instance dd50ac93-adf2-4e9d-915f-d8d1527d82e1 could not be found.
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1]
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1]
2014-10-19 17:29:54.170 27466 TRACE nova.compute.manager [instance: dd50ac93-adf2-4e9d-915f-d8d1527d82e1]
We can catch a specifc exception here and not have the ugly log
traces.
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1385246/+subscriptions
References