yahoo-eng-team team mailing list archive

[qinhaizhong <1836834@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

When the virtual machines are created in batches, nova will call the
neutron API to create the port concurrently. When an ip allocation
conflict fails to submit a database, a ``DB ERROR`` exception is thrown.
``Create_port`` will catch the above exception and rest after
``retry_interval=0.1`` and re-call ``create_port`` until it exceeds
``max_retries=10``.When it exceeds ``max_retries=10`` times,
"Create_port"  will fail. And bulk create port has similar problem.

The current design using the retry mechanism will cause the neutron
burden to burst rapidly. At this time, the api that calls neutron will
not be able to get the corresponding 60 seconds. The caller will judge
that the neutron api fails to be called, which causes various problems.

Details see https://bugs.launchpad.net/neutron/+bug/1777968.And bulk
port creation has the same problem.

We will implements a new ipam driver by introducing a distributed lock to completely solve the problem of ip address allocation conflict leading to failure. And Distributed locks we will use openstack tooz, which supports many backend 
drivers, such as Zookeeper, Memcached, Redis, Mysql, etc., and it is an openstack native project.

** Affects: neutron
     Importance: Undecided
     Assignee: qinhaizhong (qinhaizhong)
         Status: In Progress


** Tags: rfe

** Changed in: neutron
     Assignee: (unassigned) => qinhaizhong (qinhaizhong)

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1836834

Title:
  [RFE] introduce distributed locks to ipam

Status in neutron:
  In Progress

Bug description:
  When the virtual machines are created in batches, nova will call the
  neutron API to create the port concurrently. When an ip allocation
  conflict fails to submit a database, a ``DB ERROR`` exception is
  thrown. ``Create_port`` will catch the above exception and rest after
  ``retry_interval=0.1`` and re-call ``create_port`` until it exceeds
  ``max_retries=10``.When it exceeds ``max_retries=10`` times,
  "Create_port"  will fail. And bulk create port has similar problem.

  The current design using the retry mechanism will cause the neutron
  burden to burst rapidly. At this time, the api that calls neutron will
  not be able to get the corresponding 60 seconds. The caller will judge
  that the neutron api fails to be called, which causes various
  problems.

  Details see https://bugs.launchpad.net/neutron/+bug/1777968.And bulk
  port creation has the same problem.

  We will implements a new ipam driver by introducing a distributed lock to completely solve the problem of ip address allocation conflict leading to failure. And Distributed locks we will use openstack tooz, which supports many backend 
  drivers, such as Zookeeper, Memcached, Redis, Mysql, etc., and it is an openstack native project.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1836834/+subscriptions