yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1555387] [NEW] ipam driver not using retryrequest correctly

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Kevin Benton <1555387@xxxxxxxxxxxxxxxxxx>
Date: Thu, 10 Mar 2016 00:07:39 -0000
Reply-to: Bug 1555387 <1555387@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

The IPAM driver is not passing a constructed exception into RetryRequest
so the reraise on retry exhaustion fails:


2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource [req-6e0e721f-12df-4132-81ef-f1b96875cdd7 c_rally_3c9bcbbd_m5PryrFI -] create failed
2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource Traceback (most recent call last):
2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource   File "/opt/stack/neutron/neutron/api/v2/resource.py", line 84, in resource
2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource     result = method(request=request, **args)
2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource   File "/opt/stack/neutron/neutron/api/v2/base.py", line 410, in create
2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource     return self._create(request, body, **kwargs)
2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 148, in wrapper
2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource     ectxt.value = e.inner_exc
2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource     self.force_reraise()
2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource   File "<string>", line 2, in reraise
2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource TypeError: exceptions must be old-style classes or derived from BaseException, not type
2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api [req-d357eb93-048c-425b-b6de-f860be1d767a c_rally_3c9bcbbd_IA8UUZUc -] DB exceeded retry limit.
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api Traceback (most recent call last):
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 138, in wrapper
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     return f(*args, **kwargs)
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/api/v2/base.py", line 521, in _create
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     obj = do_create(body)
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/api/v2/base.py", line 503, in do_create
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     request.context, reservation.reservation_id)
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     self.force_reraise()
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     six.reraise(self.type_, self.value, self.tb)
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/api/v2/base.py", line 496, in do_create
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     return obj_creator(request.context, **kwargs)
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/db/l3_hamode_db.py", line 463, in create_router
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     self.delete_router(context, router_dict['id'])
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     self.force_reraise()
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     six.reraise(self.type_, self.value, self.tb)
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/db/l3_hamode_db.py", line 457, in create_router
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     context, router_db)[1]
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/db/l3_hamode_db.py", line 443, in _create_ha_interfaces_and_ensure_network
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     creator, dep_getter, dep_creator, dep_id_attr)
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/common/utils.py", line 707, in create_object_with_dependency
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     ctx.reraise = False
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     self.force_reraise()
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     six.reraise(self.type_, self.value, self.tb)
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/common/utils.py", line 696, in create_object_with_dependency
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     result = creator(dependency)
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/db/l3_hamode_db.py", line 394, in _create_ha_interfaces
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     l3_port_check=False)
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     self.force_reraise()
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     six.reraise(self.type_, self.value, self.tb)
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/db/l3_hamode_db.py", line 388, in _create_ha_interfaces
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     router.tenant_id)
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/db/l3_hamode_db.py", line 375, in add_ha_port
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     deletion, content)
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/db/common_db_mixin.py", line 55, in safe_creation
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     obj = create_fn()
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/plugins/common/utils.py", line 161, in create_port
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     return core_plugin.create_port(context, {'port': port_data})
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/plugins/ml2/plugin.py", line 1123, in create_port
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     result, mech_context = self._create_port_db(context, port)
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/plugins/ml2/plugin.py", line 1094, in _create_port_db
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     result = super(Ml2Plugin, self).create_port(context, port)
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/db/db_base_plugin_v2.py", line 1237, in create_port
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     port_id)
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/db/ipam_pluggable_backend.py", line 179, in allocate_ips_for_port_and_store
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     revert_on_fail=False)
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     self.force_reraise()
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     six.reraise(self.type_, self.value, self.tb)
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/db/ipam_pluggable_backend.py", line 163, in allocate_ips_for_port_and_store
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     ips = self._allocate_ips_for_port(context, port_copy)
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/db/ipam_pluggable_backend.py", line 235, in _allocate_ips_for_port
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     return self._ipam_allocate_ips(context, ipam_driver, p, ips)
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/db/ipam_pluggable_backend.py", line 135, in _ipam_allocate_ips
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     "external system for %s"), addresses)
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     self.force_reraise()
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     six.reraise(self.type_, self.value, self.tb)
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/db/ipam_pluggable_backend.py", line 120, in _ipam_allocate_ips
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     context, ipam_driver, port, ip_list)
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/db/ipam_pluggable_backend.py", line 91, in _ipam_allocate_single_ip
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     port, subnet),
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/db/ipam_pluggable_backend.py", line 80, in _ipam_try_allocate_ip
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     return ipam_subnet.allocate(ip_request)
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/ipam/drivers/neutrondb_ipam/driver.py", line 350, in allocate
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     auto_generated)
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/ipam/drivers/neutrondb_ipam/driver.py", line 219, in _allocate_specific_ip
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     session, db_range, first_ip=first_ip, last_ip=last_ip)
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/ipam/drivers/neutrondb_ipam/db_api.py", line 167, in update_range
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     raise db_exc.RetryRequest(ipam_exc.IPAllocationFailed)
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api RetryRequest
2016-03-09 16:06:10.210 98260 ERROR oslo_db.api
2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource [req-d357eb93-048c-425b-b6de-f860be1d767a c_rally_3c9bcbbd_IA8UUZUc -] create failed
2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource Traceback (most recent call last):
2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource   File "/opt/stack/neutron/neutron/api/v2/resource.py", line 84, in resource
2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource     result = method(request=request, **args)
2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource   File "/opt/stack/neutron/neutron/api/v2/base.py", line 410, in create
2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource     return self._create(request, body, **kwargs)
2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 148, in wrapper
2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource     ectxt.value = e.inner_exc
2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource     self.force_reraise()
2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource   File "<string>", line 2, in reraise
2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource TypeError: exceptions must be old-style classes or derived from BaseException, not type
2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource

** Affects: neutron
     Importance: High
     Assignee: Kevin Benton (kevinbenton)
         Status: New

** Changed in: neutron
    Milestone: None => mitaka-rc1

** Changed in: neutron
   Importance: Undecided => High

** Changed in: neutron
     Assignee: (unassigned) => Kevin Benton (kevinbenton)

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1555387

Title:
  ipam driver not using retryrequest correctly

Status in neutron:
  New

Bug description:
  The IPAM driver is not passing a constructed exception into
  RetryRequest so the reraise on retry exhaustion fails:


  
  2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource [req-6e0e721f-12df-4132-81ef-f1b96875cdd7 c_rally_3c9bcbbd_m5PryrFI -] create failed
  2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource Traceback (most recent call last):
  2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource   File "/opt/stack/neutron/neutron/api/v2/resource.py", line 84, in resource
  2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource     result = method(request=request, **args)
  2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource   File "/opt/stack/neutron/neutron/api/v2/base.py", line 410, in create
  2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource     return self._create(request, body, **kwargs)
  2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 148, in wrapper
  2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource     ectxt.value = e.inner_exc
  2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource     self.force_reraise()
  2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
  2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource   File "<string>", line 2, in reraise
  2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource TypeError: exceptions must be old-style classes or derived from BaseException, not type
  2016-03-09 16:05:41.989 98258 ERROR neutron.api.v2.resource
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api [req-d357eb93-048c-425b-b6de-f860be1d767a c_rally_3c9bcbbd_IA8UUZUc -] DB exceeded retry limit.
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api Traceback (most recent call last):
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 138, in wrapper
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     return f(*args, **kwargs)
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/api/v2/base.py", line 521, in _create
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     obj = do_create(body)
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/api/v2/base.py", line 503, in do_create
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     request.context, reservation.reservation_id)
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     self.force_reraise()
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     six.reraise(self.type_, self.value, self.tb)
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/api/v2/base.py", line 496, in do_create
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     return obj_creator(request.context, **kwargs)
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/db/l3_hamode_db.py", line 463, in create_router
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     self.delete_router(context, router_dict['id'])
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     self.force_reraise()
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     six.reraise(self.type_, self.value, self.tb)
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/db/l3_hamode_db.py", line 457, in create_router
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     context, router_db)[1]
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/db/l3_hamode_db.py", line 443, in _create_ha_interfaces_and_ensure_network
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     creator, dep_getter, dep_creator, dep_id_attr)
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/common/utils.py", line 707, in create_object_with_dependency
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     ctx.reraise = False
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     self.force_reraise()
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     six.reraise(self.type_, self.value, self.tb)
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/common/utils.py", line 696, in create_object_with_dependency
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     result = creator(dependency)
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/db/l3_hamode_db.py", line 394, in _create_ha_interfaces
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     l3_port_check=False)
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     self.force_reraise()
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     six.reraise(self.type_, self.value, self.tb)
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/db/l3_hamode_db.py", line 388, in _create_ha_interfaces
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     router.tenant_id)
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/db/l3_hamode_db.py", line 375, in add_ha_port
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     deletion, content)
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/db/common_db_mixin.py", line 55, in safe_creation
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     obj = create_fn()
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/plugins/common/utils.py", line 161, in create_port
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     return core_plugin.create_port(context, {'port': port_data})
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/plugins/ml2/plugin.py", line 1123, in create_port
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     result, mech_context = self._create_port_db(context, port)
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/plugins/ml2/plugin.py", line 1094, in _create_port_db
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     result = super(Ml2Plugin, self).create_port(context, port)
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/db/db_base_plugin_v2.py", line 1237, in create_port
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     port_id)
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/db/ipam_pluggable_backend.py", line 179, in allocate_ips_for_port_and_store
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     revert_on_fail=False)
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     self.force_reraise()
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     six.reraise(self.type_, self.value, self.tb)
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/db/ipam_pluggable_backend.py", line 163, in allocate_ips_for_port_and_store
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     ips = self._allocate_ips_for_port(context, port_copy)
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/db/ipam_pluggable_backend.py", line 235, in _allocate_ips_for_port
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     return self._ipam_allocate_ips(context, ipam_driver, p, ips)
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/db/ipam_pluggable_backend.py", line 135, in _ipam_allocate_ips
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     "external system for %s"), addresses)
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     self.force_reraise()
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     six.reraise(self.type_, self.value, self.tb)
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/db/ipam_pluggable_backend.py", line 120, in _ipam_allocate_ips
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     context, ipam_driver, port, ip_list)
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/db/ipam_pluggable_backend.py", line 91, in _ipam_allocate_single_ip
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     port, subnet),
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/db/ipam_pluggable_backend.py", line 80, in _ipam_try_allocate_ip
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     return ipam_subnet.allocate(ip_request)
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/ipam/drivers/neutrondb_ipam/driver.py", line 350, in allocate
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     auto_generated)
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/ipam/drivers/neutrondb_ipam/driver.py", line 219, in _allocate_specific_ip
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     session, db_range, first_ip=first_ip, last_ip=last_ip)
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api   File "/opt/stack/neutron/neutron/ipam/drivers/neutrondb_ipam/db_api.py", line 167, in update_range
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api     raise db_exc.RetryRequest(ipam_exc.IPAllocationFailed)
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api RetryRequest
  2016-03-09 16:06:10.210 98260 ERROR oslo_db.api
  2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource [req-d357eb93-048c-425b-b6de-f860be1d767a c_rally_3c9bcbbd_IA8UUZUc -] create failed
  2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource Traceback (most recent call last):
  2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource   File "/opt/stack/neutron/neutron/api/v2/resource.py", line 84, in resource
  2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource     result = method(request=request, **args)
  2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource   File "/opt/stack/neutron/neutron/api/v2/base.py", line 410, in create
  2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource     return self._create(request, body, **kwargs)
  2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 148, in wrapper
  2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource     ectxt.value = e.inner_exc
  2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource     self.force_reraise()
  2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource     six.reraise(self.type_, self.value, self.tb)
  2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource   File "<string>", line 2, in reraise
  2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource TypeError: exceptions must be old-style classes or derived from BaseException, not type
  2016-03-09 16:06:10.219 98260 ERROR neutron.api.v2.resource

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1555387/+subscriptions
Follow ups

[Bug 1555387] Re: ipam driver not using retryrequest correctly
From: OpenStack Infra, 2016-03-11