yahoo-eng-team team mailing list archive

[OpenStack Infra <1513765@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.openstack.org/293239
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=d7aeb8dd4b1d122e17eef8687192cd122b79fd6e
Submitter: Jenkins
Branch:    master

commit d7aeb8dd4b1d122e17eef8687192cd122b79fd6e
Author: Kevin Benton <kevin@xxxxxxxxxx>
Date:   Mon Mar 14 13:19:54 2016 -0700

    De-dup conntrack deletions before running them
    
    During a lot of port deletions, the OVS agent will
    build up a lot of remote security group member updates
    for a single device. Once the call to delete all
    of the removed remote IP conntrack state gets issued,
    there will be many duplicated entries for the same
    device in the devices_with_updated_sg_members dicionary
    of lists.
    
    This results in many duplicated calls to remove conntrack
    entries that are just a waste of time. The longer it takes
    to remove conntrack entries, the more of these duplicates
    build up for other pending changes, to the point where there
    can be hundreds of duplicate calls for a single device.
    
    This just adjusts the conntrack manager clearing logic to
    make sure it de-duplicates all of its delete commands before
    it issues them.
    
    In a local test on a single host I have 11 threads create
    11 ports each, plug them into OVS, and then delete them.
    Here are the number of conntrack delete calls issued:
    
    Before this patch - ~232000
    With this patch - ~5200
    
    While the remaining number still seems high, the agent is now
    fast enough to keep up with all of the deletes.
    
    Closes-Bug: #1513765
    Change-Id: Icba88ab47ee17bf5d6ccdfc0f78bec911987ca90


** Changed in: neutron
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1513765

Title:
  bulk delete of ports cost iptables-firewall too much time

Status in neutron:
  Fix Released

Bug description:
  this problem was found in master branch, but I think it also affect liberty.
  reproduce steps:
  1. create 100 VMs in default security group
  2. bulk delete these VMs
  I found the ipset can't be clear as soon as possible, because there were much ip_conntrack need to be clean, so the ovs-agent were doing this work.
  For this problem, what I can think is letting ovs-agent use eventlet.GreenPool to delete ip_conntrack, do other have good idea?

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1513765/+subscriptions