yahoo-eng-team team mailing list archive

[OpenStack Infra <2060054@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.opendev.org/c/openstack/neutron/+/914900
Committed: https://opendev.org/openstack/neutron/commit/305153883b64c35c60649a4ba7e3f5a5b60a503c
Submitter: "Zuul (22348)"
Branch:    master

commit 305153883b64c35c60649a4ba7e3f5a5b60a503c
Author: Ihtisham ul Haq <ihtisham.ul_haq@mail.schwarz>
Date:   Tue Apr 2 17:43:43 2024 +0200

    Optimize deletion of static routes
    
    Instead of having a separate OVN command for deletion of each
    static route, send out the deletion as a single command.
    
    Which significantly improves the performance. Previously
    deletion of 1000 routes took >1m30s and >6m30s for 2000 routes,
    with this change it takes ~5s and ~8s, respectivily.
    
    Closes-Bug: #2060054
    Change-Id: Iaa5204e2e48795c31c502160041bd128189eef5a


** Changed in: neutron
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2060054

Title:
  Slow deletion of high number of static routes on router with ML2/OVN

Status in neutron:
  Fix Released

Bug description:
  With ML2/OVN if one create about 1000 of static routes on a router,
  the addition takes <5 seconds but deletion of those static routes
  takes about >90 seconds.

  That is because neutron sends a separate deletion command to OVN for
  each of the static route, which is very inefficient, instead of doing
  it in a single transaction.

  Neutron version: latest/main branch
  Deployment tool: devstack

  Steps to reproduce:

  $ openstack network create static_route_test
  $ subnet create static_route_test --subnet-range 10.0.0.0/20 --network static_route_test
  $ openstack router create static_route_test
  $ openstack route add subnet static_route_test static_route_test

  # 1000 static routes
  $ for byte2 in {0..3}; do for byte3 in {0..249}; do echo "{\"destination\": \"192.${byte2}.${byte3}.6/24\", \"nexthop\": \"10.0.0.2\"}"; done; done|jq -s '{"router": {"routes": .}}' > max_routes_body_1000.json

  ## Addition
  $ time curl -g -i -X PUT http://192.168.0.77:9696/networking/v2.0/routers/5d28dfba-b779-4504-bbd0-3e382e9980ac/add_extraroutes -H "Content-Type: application/json" -H "X-Auth-Token: $(openstack token issue -f value -c id)" -d '@max_routes_body_1000.json'
  Response time: 0m4.883s

  ## Deletion
  $ time curl -g -i -X PUT http://192.168.0.77:9696/networking/v2.0/routers/5d28dfba-b779-4504-bbd0-3e382e9980ac -H "Content-Type: application/json" -H "X-Auth-Token: $(openstack token issue -f value -c id)" -d '{"router": {"routes": []}}'
  Response time: 1m36.843s

  # 2000 static routes
  $ for byte2 in {0..7}; do for byte3 in {0..249}; do echo "{\"destination\": \"192.${byte2}.${byte3}.6/24\", \"nexthop\": \"10.0.0.2\"}"; done; done|jq -s '{"router": {"routes": .}}' > max_routes_body_2000.json

  ## Addition
  $ time curl -g -i -X PUT http://192.168.0.77:9696/networking/v2.0/routers/5d28dfba-b779-4504-bbd0-3e382e9980ac/add_extraroutes -H "Content-Type: application/json" -H "X-Auth-Token: $(openstack token issue -f value -c id)" -d '@max_routes_body_2000.json'
  Response time: 0m6.443s

  ## Deletion
  $ time curl -g -i -X PUT http://192.168.0.77:9696/networking/v2.0/routers/5d28dfba-b779-4504-bbd0-3e382e9980ac -H "Content-Type: application/json" -H "X-Auth-Token: $(openstack token issue -f value -c id)" -d '{"router": {"routes": []}}'
  Response time: 6m32.338s

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2060054/+subscriptions