yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1257887] [NEW] unable to delete server due to neutron/nvp

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Bhuvan Arumugam <bhuvaneswaran@xxxxxxxxx>
Date: Wed, 04 Dec 2013 19:41:50 -0000
Reply-to: Bug 1257887 <1257887@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

nova delete fail to delete the server. It seem to be due to neutron/nvp out of sync. The conductor has this traceback.
File "/usr/local/csi/share/csi-nova.venv/lib/python2.6/site-packages/nova/network/neutronv2/api.py", line 959, in _nw_info_build_network\n    label=network_name,\n', u'result': u'Error', u'request_id': u'req-1b6a86b8-6485-4a3f-b76e-584ce815fd65', u'message': u"local variable 'network_name' referenced before assignment"

Detailed error log:

2013-12-04 19:26:09,103 (nova.openstack.common.rpc.amqp): DEBUG common
_safe_log received {u'_msg_id': u'59359af3f88249a2ba9cf14981cc03da',
u'_context_quota_class': None, u'_context_request_id': u'req-
1b6a86b8-6485-4a3f-b76e-584ce815fd65', u'_context_service_catalog': [],
u'_context_auth_token': '<SANITIZED>', u'_context_user_id': u'tempest-
admin', u'_reply_q': u'reply_ac6c4ff91f1343628fbbdb615fdf7cd2',
u'namespace': None, u'_context_is_admin': True, u'version': u'1.25',
u'_context_timestamp': u'2013-12-04T19:26:07.626153', u'_context_user':
u'tempest-admin', u'method': u'action_event_finish',
u'_context_remote_address': u'17.115.165.169', u'_context_roles': [u
'csi-tenant-admin', u'csi-role-admin', u'admin'], u'args': {u'values':
{u'instance_uuid': u'a18d9974-1152-461c-bd90-6e54a2d9094b',
u'finish_time': u'2013-12-04T19:26:09.094045', u'traceback': u'  File
"/usr/local/csi/share/csi-nova.venv/lib/python2.6/site-
packages/nova/compute/manager.py", line 302, in decorated_function\n
function(self, context, *args, **kwargs)\n  File "/usr/local/csi/share
/csi-nova.venv/lib/python2.6/site-packages/nova/compute/manager.py",
line 279, in decorated_function\n    e, sys.exc_info())\n  File
"/usr/local/csi/share/csi-nova.venv/lib/python2.6/site-
packages/nova/compute/manager.py", line 266, in decorated_function\n
return function(self, context, *args, **kwargs)\n  File
"/usr/local/csi/share/csi-nova.venv/lib/python2.6/site-
packages/nova/compute/manager.py", line 1995, in terminate_instance\n
do_terminate_instance(instance, bdms)\n  File "/usr/local/csi/share/csi-
nova.venv/lib/python2.6/site-
packages/nova/openstack/common/lockutils.py", line 248, in inner\n
return f(*args, **kwargs)\n  File "/usr/local/csi/share/csi-
nova.venv/lib/python2.6/site-packages/nova/compute/manager.py", line
1987, in do_terminate_instance\n    reservations=reservations)\n  File
"/usr/local/csi/share/csi-nova.venv/lib/python2.6/site-
packages/nova/hooks.py", line 105, in inner\n    rv = f(*args,
**kwargs)\n  File "/usr/local/csi/share/csi-nova.venv/lib/python2.6
/site-packages/nova/compute/manager.py", line 1965, in
_delete_instance\n    user_id=user_id)\n  File "/usr/local/csi/share
/csi-nova.venv/lib/python2.6/site-packages/nova/compute/manager.py",
line 1935, in _delete_instance\n    self._shutdown_instance(context,
db_inst, bdms)\n  File "/usr/local/csi/share/csi-nova.venv/lib/python2.6
/site-packages/nova/compute/manager.py", line 1845, in
_shutdown_instance\n    network_info =
self._get_instance_nw_info(context, instance)\n  File
"/usr/local/csi/share/csi-nova.venv/lib/python2.6/site-
packages/nova/compute/manager.py", line 886, in _get_instance_nw_info\n
instance)\n  File "/usr/local/csi/share/csi-nova.venv/lib/python2.6
/site-packages/nova/network/api.py", line 50, in wrapper\n    res =
f(self, context, *args, **kwargs)\n  File "/usr/local/csi/share/csi-
nova.venv/lib/python2.6/site-packages/nova/network/neutronv2/api.py",
line 451, in get_instance_nw_info\n    result =
self._get_instance_nw_info(context, instance, networks)\n  File
"/usr/local/csi/share/csi-nova.venv/lib/python2.6/site-
packages/nova/network/neutronv2/api.py", line 458, in
_get_instance_nw_info\n    nw_info =
self._build_network_info_model(context, instance, networks)\n  File
"/usr/local/csi/share/csi-nova.venv/lib/python2.6/site-
packages/nova/network/neutronv2/api.py", line 1029, in
_build_network_info_model\n    subnets)\n  File "/usr/local/csi/share
/csi-nova.venv/lib/python2.6/site-
packages/nova/network/neutronv2/api.py", line 959, in
_nw_info_build_network\n    label=network_name,\n', u'result': u'Error',
u'request_id': u'req-1b6a86b8-6485-4a3f-b76e-584ce815fd65', u'message':
u"local variable 'network_name' referenced before assignment", u'event':
u'compute_terminate_instance'}}, u'_unique_id':
u'02521df0a35d4abebf5a038dd136e747', u'_context_project_name': u'csi-
tenant-tempest', u'_context_read_deleted': u'no', u'_context_tenant': u
'csi-tenant-tempest', u'_context_instance_lock_checked': False,
u'_context_project_id': u'csi-tenant-tempest', u'_context_user_name': u
'tempest-admin'}

Refer to this link for following logs:
  1. nova delete --debug
  2. neutron log (no error)
  3. conductor log (above traceback and more)
  
http://paste.openstack.org/show/54451/

** Affects: nova
     Importance: Undecided
         Status: New


** Tags: neutron

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1257887

Title:
  unable to delete server due to neutron/nvp

Status in OpenStack Compute (Nova):
  New

Bug description:
  nova delete fail to delete the server. It seem to be due to neutron/nvp out of sync. The conductor has this traceback.
  File "/usr/local/csi/share/csi-nova.venv/lib/python2.6/site-packages/nova/network/neutronv2/api.py", line 959, in _nw_info_build_network\n    label=network_name,\n', u'result': u'Error', u'request_id': u'req-1b6a86b8-6485-4a3f-b76e-584ce815fd65', u'message': u"local variable 'network_name' referenced before assignment"

  Detailed error log:

  2013-12-04 19:26:09,103 (nova.openstack.common.rpc.amqp): DEBUG common
  _safe_log received {u'_msg_id': u'59359af3f88249a2ba9cf14981cc03da',
  u'_context_quota_class': None, u'_context_request_id': u'req-
  1b6a86b8-6485-4a3f-b76e-584ce815fd65', u'_context_service_catalog':
  [], u'_context_auth_token': '<SANITIZED>', u'_context_user_id': u
  'tempest-admin', u'_reply_q':
  u'reply_ac6c4ff91f1343628fbbdb615fdf7cd2', u'namespace': None,
  u'_context_is_admin': True, u'version': u'1.25',
  u'_context_timestamp': u'2013-12-04T19:26:07.626153',
  u'_context_user': u'tempest-admin', u'method': u'action_event_finish',
  u'_context_remote_address': u'17.115.165.169', u'_context_roles': [u
  'csi-tenant-admin', u'csi-role-admin', u'admin'], u'args': {u'values':
  {u'instance_uuid': u'a18d9974-1152-461c-bd90-6e54a2d9094b',
  u'finish_time': u'2013-12-04T19:26:09.094045', u'traceback': u'  File
  "/usr/local/csi/share/csi-nova.venv/lib/python2.6/site-
  packages/nova/compute/manager.py", line 302, in decorated_function\n
  function(self, context, *args, **kwargs)\n  File "/usr/local/csi/share
  /csi-nova.venv/lib/python2.6/site-packages/nova/compute/manager.py",
  line 279, in decorated_function\n    e, sys.exc_info())\n  File
  "/usr/local/csi/share/csi-nova.venv/lib/python2.6/site-
  packages/nova/compute/manager.py", line 266, in decorated_function\n
  return function(self, context, *args, **kwargs)\n  File
  "/usr/local/csi/share/csi-nova.venv/lib/python2.6/site-
  packages/nova/compute/manager.py", line 1995, in terminate_instance\n
  do_terminate_instance(instance, bdms)\n  File "/usr/local/csi/share
  /csi-nova.venv/lib/python2.6/site-
  packages/nova/openstack/common/lockutils.py", line 248, in inner\n
  return f(*args, **kwargs)\n  File "/usr/local/csi/share/csi-
  nova.venv/lib/python2.6/site-packages/nova/compute/manager.py", line
  1987, in do_terminate_instance\n    reservations=reservations)\n  File
  "/usr/local/csi/share/csi-nova.venv/lib/python2.6/site-
  packages/nova/hooks.py", line 105, in inner\n    rv = f(*args,
  **kwargs)\n  File "/usr/local/csi/share/csi-nova.venv/lib/python2.6
  /site-packages/nova/compute/manager.py", line 1965, in
  _delete_instance\n    user_id=user_id)\n  File "/usr/local/csi/share
  /csi-nova.venv/lib/python2.6/site-packages/nova/compute/manager.py",
  line 1935, in _delete_instance\n    self._shutdown_instance(context,
  db_inst, bdms)\n  File "/usr/local/csi/share/csi-
  nova.venv/lib/python2.6/site-packages/nova/compute/manager.py", line
  1845, in _shutdown_instance\n    network_info =
  self._get_instance_nw_info(context, instance)\n  File
  "/usr/local/csi/share/csi-nova.venv/lib/python2.6/site-
  packages/nova/compute/manager.py", line 886, in
  _get_instance_nw_info\n    instance)\n  File "/usr/local/csi/share
  /csi-nova.venv/lib/python2.6/site-packages/nova/network/api.py", line
  50, in wrapper\n    res = f(self, context, *args, **kwargs)\n  File
  "/usr/local/csi/share/csi-nova.venv/lib/python2.6/site-
  packages/nova/network/neutronv2/api.py", line 451, in
  get_instance_nw_info\n    result = self._get_instance_nw_info(context,
  instance, networks)\n  File "/usr/local/csi/share/csi-
  nova.venv/lib/python2.6/site-packages/nova/network/neutronv2/api.py",
  line 458, in _get_instance_nw_info\n    nw_info =
  self._build_network_info_model(context, instance, networks)\n  File
  "/usr/local/csi/share/csi-nova.venv/lib/python2.6/site-
  packages/nova/network/neutronv2/api.py", line 1029, in
  _build_network_info_model\n    subnets)\n  File "/usr/local/csi/share
  /csi-nova.venv/lib/python2.6/site-
  packages/nova/network/neutronv2/api.py", line 959, in
  _nw_info_build_network\n    label=network_name,\n', u'result':
  u'Error', u'request_id': u'req-1b6a86b8-6485-4a3f-b76e-584ce815fd65',
  u'message': u"local variable 'network_name' referenced before
  assignment", u'event': u'compute_terminate_instance'}}, u'_unique_id':
  u'02521df0a35d4abebf5a038dd136e747', u'_context_project_name': u'csi-
  tenant-tempest', u'_context_read_deleted': u'no', u'_context_tenant':
  u'csi-tenant-tempest', u'_context_instance_lock_checked': False,
  u'_context_project_id': u'csi-tenant-tempest', u'_context_user_name':
  u'tempest-admin'}

  Refer to this link for following logs:
    1. nova delete --debug
    2. neutron log (no error)
    3. conductor log (above traceback and more)
    
  http://paste.openstack.org/show/54451/

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1257887/+subscriptions
Follow ups

[Bug 1257887] [NEW] unable to delete server due to neutron/nvp
From: Bhuvan Arumugam, 2013-12-04
References

[Bug 1257887] [NEW] unable to delete server due to neutron/nvp
From: Bhuvan Arumugam, 2013-12-04