yahoo-eng-team team mailing list archive

[Thierry Carrez <thierry.carrez+lp@xxxxxxxxx>]

** Changed in: neutron
       Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1491922

Title:
  ovs agent doesn't configure new ovs-port for an instance

Status in neutron:
  Fix Released

Bug description:
  in case of massive resource deletion (networks, ports) it may take agent quite a lot of time to process.
  Port delete priocessing is happening during ovs agent periodic task. It takes agent ~0.25s to process one port deletion.
  From the attached log we can see that on a certain iteration the agent had to process deletion of 1625 ports.
   1625 * 0.25 = 406 seconds.
  Indeed:

   2015-08-29 09:13:46.004 21292 DEBUG
  neutron.plugins.openvswitch.agent.ovs_neutron_agent [req-
  55e0a577-e03b-4476-9bdd-f5480cfef966 ] Agent rpc_loop -
  iteration:25863 - starting polling. Elapsed:0.047 rpc_loop
  /usr/lib/python2.7/dist-
  packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py:1733

   ... (ports deletion handling)

   2015-08-29 09:20:28.569 21292 DEBUG neutron.plugins.openvswitch.agent.ovs_neutron_agent [req-55e0a577-e03b-4476-9bdd-f5480cfef966 ] Agent rpc_loop - iteration:25863 - port information retrieved. Elapsed:402.612 rpc_loop /usr/lib/python2.7/dist-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py:1748
   ... (from here agent starts processing new ports)

  402 seconds is somewhat not acceptable. Nova waits for 300 seconds by default and then fails with vif plugging timeout.
  From log we can also see that new ovs port appeared during agent was busy with ports deletion stuff: 

  2015-08-29 09:13:52.432 21292 DEBUG neutron.agent.linux.ovsdb_monitor [-] Output received from ovsdb monitor: {"data":[["8fd481a4-1267-445b-bedc-f1f6b3a47898","old",null,["set",[]]],["","new","qvoced59c11-1b",76]],"headings":["row","action","name","ofport"]}
   _read_stdout /usr/lib/python2.7/dist-packages/neutron/agent/linux/ovsdb_monitor.py:44

  Port deletion handling needs to be optimised on agent side.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1491922/+subscriptions