yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1629484] Re: Key error when try to assign a floating ip to a VM

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: OpenStack Infra <1629484@xxxxxxxxxxxxxxxxxx>
Date: Tue, 04 Oct 2016 04:12:39 -0000
Reply-to: Bug 1629484 <1629484@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Reviewed:  https://review.openstack.org/380669
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=19af726e39b30670314b3bf7025b3a682e523953
Submitter: Jenkins
Branch:    master

commit 19af726e39b30670314b3bf7025b3a682e523953
Author: chen, hao <hao.chen@xxxxxxx>
Date:   Fri Sep 30 16:06:38 2016 -0700

    Fix the 'port_id' attribute not exist error
    
    Change-Id: Ie9e42771788d8a20572cb650b15edb95923dd426
    Closes-Bug: #1629484


** Changed in: neutron
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1629484

Title:
  Key error when try to assign a floating ip to a VM

Status in neutron:
  Fix Released

Bug description:
  We have an environment with designate and DNS plugin enabled. When we
  upgrade to newton we got a problem that we can't assign the floating
  ip to the VM because of an key error.

  Here is the error message from neutron server

  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource Traceback (most recent call last):
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource   File "/usr/lib/python2.7/site-packages/neutron/api/v2/resource.py", line 79, in resource
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource     result = method(request=request, **args)
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource   File "/usr/lib/python2.7/site-packages/neutron/api/v2/base.py", line 604, in update
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource     return self._update(request, id, body, **kwargs)
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource   File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 88, in wrapped
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource     setattr(e, '_RETRY_EXCEEDED', True)
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource     self.force_reraise()
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource   File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 84, in wrapped
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource     return f(*args, **kwargs)
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource   File "/usr/lib/python2.7/site-packages/oslo_db/api.py", line 151, in wrapper
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource     ectxt.value = e.inner_exc
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource     self.force_reraise()
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource   File "/usr/lib/python2.7/site-packages/oslo_db/api.py", line 139, in wrapper
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource     return f(*args, **kwargs)
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource   File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 124, in wrapped
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource     traceback.format_exc())
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource     self.force_reraise()
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource   File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 119, in wrapped
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource     return f(*dup_args, **dup_kwargs)
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource   File "/usr/lib/python2.7/site-packages/neutron/api/v2/base.py", line 652, in _update
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource     obj = obj_updater(request.context, id, **kwargs)
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource   File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 159, in wrapped
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource     return method(*args, **kwargs)
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource   File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 88, in wrapped
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource     setattr(e, '_RETRY_EXCEEDED', True)
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource     self.force_reraise()
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource   File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 84, in wrapped
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource     return f(*args, **kwargs)
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource   File "/usr/lib/python2.7/site-packages/oslo_db/api.py", line 151, in wrapper
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource     ectxt.value = e.inner_exc
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource     self.force_reraise()
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource   File "/usr/lib/python2.7/site-packages/oslo_db/api.py", line 139, in wrapper
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource     return f(*args, **kwargs)
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource   File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 124, in wrapped
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource     traceback.format_exc())
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource     self.force_reraise()
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource   File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 119, in wrapped
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource     return f(*dup_args, **dup_kwargs)
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource   File "/usr/lib/python2.7/site-packages/neutron/db/l3_dvr_db.py", line 1015, in update_floatingip
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource     context, id, floatingip)
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource   File "/usr/lib/python2.7/site-packages/neutron/db/l3_db.py", line 1355, in _update_floatingip
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource     context, floatingip_db)
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource   File "/usr/lib/python2.7/site-packages/neutron/db/dns_db.py", line 210, in _process_dns_floatingip_update_precommit
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource     context, floatingip_data))
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource   File "/usr/lib/python2.7/site-packages/neutron/db/dns_db.py", line 313, in _get_requested_state_for_external_dns_service_update
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource     if floatingip_data['port_id']:
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource   File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/models.py", line 54, in __getitem__
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource     return getattr(self, key)
  ./neutron/neutron-server.log:2016-09-30 22:00:36.797 527 ERROR neutron.api.v2.resource AttributeError: 'FloatingIP' object has no attribute 'port_id'

  
  After digging more it seems like there are several places in the code try to get the value for "port_id" which doesn't exist

  https://github.com/openstack/neutron/blob/stable/newton/neutron/db/dns_db.py#L313
  https://github.com/openstack/neutron/blob/stable/newton/neutron/db/dns_db.py#L275
  https://github.com/openstack/neutron/blob/stable/newton/neutron/db/dns_db.py#L281

  from
  https://github.com/openstack/neutron/blob/stable/newton/neutron/db/l3_db.py#L1041
  we can see that port_id is actually the "fix_port_id" attribute in
  floatingip_db object.

  Fix the previous key I mentioned will fix the floating ip assignment.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1629484/+subscriptions
References

[Bug 1629484] [NEW] Key error when try to assign a floating ip to a VM
From: Hao Chen, 2016-09-30