yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1451558] Re: subnetpool allocation not working with postgresql

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Thierry Carrez <thierry.carrez+lp@xxxxxxxxx>
Date: Wed, 24 Jun 2015 20:13:53 -0000
Reply-to: Bug 1451558 <1451558@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
** Changed in: neutron
       Status: Fix Committed => Fix Released

** Changed in: neutron
    Milestone: None => liberty-1

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1451558

Title:
  subnetpool allocation not working with postgresql

Status in OpenStack Neutron (virtual network service):
  Fix Released

Bug description:
  The following is working with mysql but not with postgresql

  
  #$ neutron subnetpool-create pool --pool-prefix 10.0.0.0/8 --default-prefixlen 24
  #$ neutron net-create net
  #$ neutron subnet-create net --name subnet --subnetpool pool

  
  The last command raises a 501 with postgresql with the stacktrace[2] in neutron-server, because _get_allocated_cidrs[1] performs a SELECT FOR UPDATE with a JOIN on an empty select! (allowed with mysql, not postgresql).



  [1]: https://github.com/openstack/neutron/blob/5962d825a6c98225c51bc6dd304b5c1ac89035ef/neutron/ipam/subnet_alloc.py#L40-L44
    query = session.query(models_v2.Subnet).with_lockmode('update')
    subnets = query.filter_by(subnetpool_id=self._subnetpool['id'])

  
  [2]: neutron-server stacktrace
  2015-05-04 21:47:01.939 ERROR neutron.api.v2.resource [req-a6c14f61-bdb2-4273-a231-df0a85fb33d8 demo b532b7a9302c45b18f06f68b41869ffa] create failed
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource Traceback (most recent call last):
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/api/v2/resource.py", line 83, in resource
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource     result = method(request=request, **args)
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/api/v2/base.py", line 461, in create
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource     obj = obj_creator(request.context, **kwargs)
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/plugins/ml2/plugin.py", line 804, in create_subnet
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource     result, mech_context = self._create_subnet_db(context, subnet)
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/plugins/ml2/plugin.py", line 795, in _create_subnet_db
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource     result = super(Ml2Plugin, self).create_subnet(context, subnet)
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/db/db_base_plugin_v2.py", line 1389, in create_subnet
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource     subnetpool_id)
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 131, in wrapper
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource     return f(*args, **kwargs)
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/db/db_base_plugin_v2.py", line 1283, in _create_subnet_from_pool
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource     ipam_subnet = allocator.allocate_subnet(context.session, req)
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/ipam/subnet_alloc.py", line 141, in allocate_subnet
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource     return self._allocate_any_subnet(session, request)
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/ipam/subnet_alloc.py", line 93, in _allocate_any_subnet
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource     prefix_pool = self._get_available_prefix_list(session)
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/ipam/subnet_alloc.py", line 48, in _get_available_prefix_list
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource     allocations = self._get_allocated_cidrs(session)
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource   File "/opt/stack/neutron/neutron/ipam/subnet_alloc.py", line 44, in _get_allocated_cidrs
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource     return (x.cidr for x in subnets)
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2441, in __iter__
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource     return self._execute_and_instances(context)
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2456, in _execute_and_instances
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource     result = conn.execute(querycontext.statement, self._params)
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 841, in execute
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource     return meth(self, multiparams, params)
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 322, in _execute_on_connection
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource     return connection._execute_clauseelement(self, multiparams, params)
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 938, in _execute_clauseelement
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource     compiled_sql, distilled_params
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1070, in _execute_context
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource     context)
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/oslo_db/sqlalchemy/compat/handle_error.py", line 261, in _handle_dbapi_exception
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource     e, statement, parameters, cursor, context)
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1267, in _handle_dbapi_exception
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource     util.raise_from_cause(newraise, exc_info)
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource     reraise(type(exception), exception, tb=exc_tb)
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1063, in _execute_context
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource     context)
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 442, in do_execute
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource     cursor.execute(statement, parameters)
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource DBError: (NotSupportedError) FOR UPDATE cannot be applied to the nullable side of an outer join
  2015-05-04 21:47:01.939 TRACE neutron.api.v2.resource  'SELECT subnets.tenant_id AS subnets_tenant_id, subnets.id AS subnets_id, subnets.name AS subnets_name, subnets.network_id AS subnets_network_id, subnets.subnetpool_id AS subnets_subnetpool_id, subnets.ip_version AS subnets_ip_version, subnets.cidr AS subnets_cidr, subnets.gateway_ip AS subnets_gateway_ip, subnets.enable_dhcp AS subnets_enable_dhcp, subnets.shared AS subnets_shared, subnets.ipv6_ra_mode AS subnets_ipv6_ra_mode, subnets.ipv6_address_mode AS subnets_ipv6_address_mode, ipallocationpools_1.id AS ipallocationpools_1_id, ipallocationpools_1.subnet_id AS ipallocationpools_1_subnet_id, ipallocationpools_1.first_ip AS ipallocationpools_1_first_ip, ipallocationpools_1.last_ip AS ipallocationpools_1_last_ip, dnsnameservers_1.address AS dnsnameservers_1_address, dnsnameservers_1.subnet_id AS dnsnameservers_1_subnet_id, subnetroutes_1.destination AS subnetroutes_1_destination, subnetroutes_1.nexthop AS subnetroutes_1_nexthop, subnetroutes_1.subnet_id AS subnetroutes_1_subnet_id \nFROM subnets LEFT OUTER JOIN ipallocationpools AS ipallocationpools_1 ON subnets.id = ipallocationpools_1.subnet_id LEFT OUTER JOIN dnsnameservers AS dnsnameservers_1 ON subnets.id = dnsnameservers_1.subnet_id LEFT OUTER JOIN subnetroutes AS subnetroutes_1 ON subnets.id = subnetroutes_1.subnet_id \nWHERE subnets.subnetpool_id = %(subnetpool_id_1)s FOR UPDATE' {'subnetpool_id_1': u'2ee02a0f-863a-41d6-a5cb-7c629395b9da'}

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1451558/+subscriptions
References

[Bug 1451558] [NEW] subnetpool allocation not working with postgresql
From: Cedric Brandily, 2015-05-04