yahoo-eng-team team mailing list archive

[Lujin Luo <luo.lujin@xxxxxxxxxxxxxx>]

Public bug reported:

I have three controller nodes and the Neutron servers on these
controllers are set behind Pacemaker and HAProxy to realize
active/active HA using DevStack. MariaDB Galera cluster is used as my
database backend.  I am using the latest codes.

If I have multiple commands to create floating IPs and associate them to
the same port at the same time, all of the commands would return success
and end up with multiple floating IPs from the same external network
associated to the same port.

How to reproduce:

Step 1: Create a network
$ neutron net-create net1

Step 2: Create a subnet on the network
$ neutron subnet-create --name subnet1 net1 192.168.100.0/24

Step 3: Create a port on the network
$ neutron port-create net1

Step 4: Create a router
$ neutron router-create router-floatingip-test

Step 5: Add the subnet as its interface
$ neutron router-interface-add router-floatingip-test subnet1

Step 5: Create an external network
$ neutron net-create ext-net --router:external True

Step 6: Add a subnet on the external network
$ neutron subnet-create --name ext-subnet ext-net 192.168.122.0/24

Step 7: Set the external network as the router's default gateway
$ neutron router-gateway-set router-floatingip-test ext-net

Step 8: Run the three commands at the same time to create floating IPs
On controller1:
$ neutron floatingip-create ext-net --port-id b53d0826-53c4-427b-81b2-3ab6cb0f4511

On controller2:
$ neutron floatingip-create ext-net --port-id b53d0826-53c4-427b-81b2-3ab6cb0f4511

On controller3:
$ neutron floatingip-create ext-net --port-id b53d0826-53c4-427b-81b2-3ab6cb0f4511

where, port_id b53d0826-53c4-427b-81b2-3ab6cb0f4511 is the port we
created in Step 3.

The result would be three floating IPs associated to the same port, as
shown in http://paste.openstack.org/show/483691/

The expected error message (say, we run the second command after the first one succeeds) would be
Cannot associate floating IP 192.168.122.20 (bd4d47a5-45c1-48e1-a48a-aef08039a955) with port b53d0826-53c4-427b-81b2-3ab6cb0f4511 using fixed IP 192.168.100.3, as that fixed IP already has a floating IP on external network 920ee0f3-3db8-4005-8d29-0be474947186.
Since one port with one fixed_ip is not allowed to have multiple floating IPs from the same external network. 

In the above procedure, I set port_id when creating these three floating
IPs. Same bug occurred when I updated three existing floating IPs to be
associated with the same port at the same time.

I assume this bug happens because multiple APIs are executed
concurrently and the validation check on every API succeeds [1].

[1]
https://github.com/openstack/neutron/blob/master/neutron/db/l3_db.py#L915

** Affects: neutron
     Importance: Undecided
     Assignee: Lujin Luo (luo-lujin)
         Status: New

** Changed in: neutron
     Assignee: (unassigned) => Lujin Luo (luo-lujin)

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1534445

Title:
  Multiple floating IPs from the same external network are associated to
  one port when commands are executed at the same time

Status in neutron:
  New

Bug description:
  I have three controller nodes and the Neutron servers on these
  controllers are set behind Pacemaker and HAProxy to realize
  active/active HA using DevStack. MariaDB Galera cluster is used as my
  database backend.  I am using the latest codes.

  If I have multiple commands to create floating IPs and associate them
  to the same port at the same time, all of the commands would return
  success and end up with multiple floating IPs from the same external
  network associated to the same port.

  How to reproduce:

  Step 1: Create a network
  $ neutron net-create net1

  Step 2: Create a subnet on the network
  $ neutron subnet-create --name subnet1 net1 192.168.100.0/24

  Step 3: Create a port on the network
  $ neutron port-create net1

  Step 4: Create a router
  $ neutron router-create router-floatingip-test

  Step 5: Add the subnet as its interface
  $ neutron router-interface-add router-floatingip-test subnet1

  Step 5: Create an external network
  $ neutron net-create ext-net --router:external True

  Step 6: Add a subnet on the external network
  $ neutron subnet-create --name ext-subnet ext-net 192.168.122.0/24

  Step 7: Set the external network as the router's default gateway
  $ neutron router-gateway-set router-floatingip-test ext-net

  Step 8: Run the three commands at the same time to create floating IPs
  On controller1:
  $ neutron floatingip-create ext-net --port-id b53d0826-53c4-427b-81b2-3ab6cb0f4511

  On controller2:
  $ neutron floatingip-create ext-net --port-id b53d0826-53c4-427b-81b2-3ab6cb0f4511

  On controller3:
  $ neutron floatingip-create ext-net --port-id b53d0826-53c4-427b-81b2-3ab6cb0f4511

  where, port_id b53d0826-53c4-427b-81b2-3ab6cb0f4511 is the port we
  created in Step 3.

  The result would be three floating IPs associated to the same port, as
  shown in http://paste.openstack.org/show/483691/

  The expected error message (say, we run the second command after the first one succeeds) would be
  Cannot associate floating IP 192.168.122.20 (bd4d47a5-45c1-48e1-a48a-aef08039a955) with port b53d0826-53c4-427b-81b2-3ab6cb0f4511 using fixed IP 192.168.100.3, as that fixed IP already has a floating IP on external network 920ee0f3-3db8-4005-8d29-0be474947186.
  Since one port with one fixed_ip is not allowed to have multiple floating IPs from the same external network. 

  In the above procedure, I set port_id when creating these three
  floating IPs. Same bug occurred when I updated three existing floating
  IPs to be associated with the same port at the same time.

  I assume this bug happens because multiple APIs are executed
  concurrently and the validation check on every API succeeds [1].

  [1]
  https://github.com/openstack/neutron/blob/master/neutron/db/l3_db.py#L915

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1534445/+subscriptions