yahoo-eng-team team mailing list archive

[OpenStack Infra <1834257@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.opendev.org/659274
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=76ccdb35d4b09106aa8adc97528b02f0fd8acbcc
Submitter: Zuul
Branch:    master

commit 76ccdb35d4b09106aa8adc97528b02f0fd8acbcc
Author: Sebastian Lohff <sebastian.lohff@xxxxxxx>
Date:   Wed May 15 13:39:55 2019 +0200

    Limit max ports per rpc for dhcp_ready_on_ports()
    
    The Neutron dhcp agents reports all ready ports to the Neutron
    server via the dhcp_ready_on_ports() rpc call. When the dhcp agent
    gets ports ready faster than the server can process them the amount
    of ports per rpc call can grow so high (e.g. 10000 Ports) that the
    neutron server never has a chance of processing the request before
    the rpc timeout kills the request, leading to the dhcp agent
    sending the request again, resulting in an endless loop of
    dhcp_ready_on_ports() calls. This happens especially on agent startup.
    
    To mitigate this problems we now limit the number of ports sent
    per dhcp_ready_on_ports() call.
    
    Closes-bug: #1834257
    Change-Id: I407e126e760ebf6aca4c31b9c3ff58dcfa55107f


** Changed in: neutron
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1834257

Title:
  dhcp-agent can overwhelm neutron server with dhcp_ready_on_ports RPC
  calls

Status in neutron:
  Fix Released

Bug description:
  The Neutron dhcp-agents reports all ready ports to the Neutron server
  via the dhcp_ready_on_ports() RPC call. When the dhcp-agent gets ports
  ready faster than the server can process them, the amount of ports per
  RPC call can grow so high (e.g. 10000 Ports) that the neutron server
  never has a chance of processing the request before the RPC timeout
  kills the request, leading to the dhcp-agent sending the request
  again, possibly with even more ports than before, resulting in an
  endless loop of dhcp_ready_on_ports() calls. This happens especially
  on agent startup.

  We should use either a smaller fixed amount, or use an algorithm to
  reduce the number being sent in the event a message timeout is
  received.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1834257/+subscriptions