yahoo-eng-team team mailing list archive

[OpenStack Infra <1535549@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.openstack.org/341427
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=17005132d1b17d608077090573c50af591afe670
Submitter: Jenkins
Branch:    master

commit 17005132d1b17d608077090573c50af591afe670
Author: Anh Tran <anhtt@xxxxxxxxxxxxxx>
Date:   Mon Aug 22 17:28:29 2016 +0700

    Rollback port after failed to add it to router
    
    After failed to add port to a router, we cannot re-use and/or delete
    this port.
    
    With concurrent requests occuring, neutron will accept one request
    and the other will be rejected with an 'overlapped CIDR' message.
    Patch [1] fixed the race condition, but neutron raises
    'Port already has an attached device' instead of
    'overlapped CIDR', because neutron didn't cleanup the port when
    the request was retried.
    [1] https://review.openstack.org/#/c/303966/
    
    This patch is needed to fix the bug completely. We will catch any
    exception when adding an interface by port to a router. After that,
    we rollback this port to its original state.
    
    Change-Id: Ib68aee164a3062648fc882012d57b5e381f52196
    Closes-Bug: #1535549


** Changed in: neutron
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1535549

Title:
  Multiple ports which have duplicated CIDRs are added as one router's
  interfaces if commands are executed at the same time

Status in neutron:
  Fix Released

Bug description:
  I have three controller nodes and the Neutron servers on these
  controllers are set behind Pacemaker and HAProxy to realize
  active/active HA using DevStack. MariaDB Galera cluster is used as my
  database backend.I am using the latest codes.

  If one router is going to add two ports as its interface, however these two ports belong to two subnets which have duplicated CIDRs, the expected result would be the later API request would fail, with error message like
  BadRequest: Bad router request: Cidr 192.166.100.0/24 of subnet bee7663c-f0a0-4120-b556-944af7ca40cf overlaps with cidr 192.166.0.0/16 of subnet 697c82cf-82fd-4187-b460-7046c81f13dc.

  But if we run the two commands at the same time, both commands would
  succeed. The router would have two ports, which belong to subnets with
  duplicated CIDRs. I have tested for 30 times and only three times I
  could receive the expected error messages.

  How to reproduce:

  Step 1: Create a router
  $ neutron router-create router-port-test

  Step 2: Create two internal networks
  $ neutron net-create net1
  $ neutron net-create net2

  Step 3: Add one subnet to each of these two networks
  $ neutron subnet-create --name subnet1 net1 192.166.100.0/24
  $ neutron subnet-create --name subnet2 net2 192.166.0.0/16

  Here, we are creating two subnets on different networks with
  DUPLICATED CIDRs.

  Step 4: Create one port on each of these two networks
  $ neutron port-create --name port1 net1
  $ neutron port-create --name port2 net2

  Step 5: Add these two ports as the router's interface at the same time
  On controller1:
  $ neutron router-interface-add router-port-test port=port1
  On controller2:
  $ neutron router-interface-add router-port-test port=port2

  Both commands would work and we can see the ports listed on the router
  as http://paste.openstack.org/show/483839/

  This bug is similar to [1]. We also have _check_for_dup_router_subnet
  method to check if subnets have duplicated CIDRs or not. The problem
  happens multiple API requests arrive at the same time and all the
  checks validate.

  [1] https://bugs.launchpad.net/neutron/+bug/1535226
  [2] https://github.com/openstack/neutron/blob/master/neutron/db/l3_db.py#L535

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1535549/+subscriptions