yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #93965
[Bug 2065577] [NEW] Empty binding_levels=[] cause ovs-agent skipped to process port
Public bug reported:
During our production environment we noticed some VM boot failures like this:
1. Create port for nova (port revision_number 0->1)
2. Boot nova with --nic port-id
3. Nova scheduled this VM and plug the port
4. Nova update the port device_owner (port revision_number 1->2)
5. Nova update the port host (port revision_number 2->3)
(Yes, nova will call update_port twice!)
6. Before call real _bind_port_if_needed, neutron-server push port Info cache with binding_levels=[], and revision_number=3
7. Neutron-server try to bind this port
8. Neutron-ovs-agent rpc_loop try to get the port details
9. Neutron-ovs-agent Info cache RPC gets empty binding_levels=[] and skip processing port
10. Neutron-server bind done and send Info cache,
and now the revision_number is still 3, binding_levels=[<entry>] is not empty now.
11. neutron-ovs-agent get the new info cache notification, but the revision_number is not changed, so the cache is not updated.
The port will not be processed anymore.
** Affects: neutron
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2065577
Title:
Empty binding_levels=[] cause ovs-agent skipped to process port
Status in neutron:
New
Bug description:
During our production environment we noticed some VM boot failures like this:
1. Create port for nova (port revision_number 0->1)
2. Boot nova with --nic port-id
3. Nova scheduled this VM and plug the port
4. Nova update the port device_owner (port revision_number 1->2)
5. Nova update the port host (port revision_number 2->3)
(Yes, nova will call update_port twice!)
6. Before call real _bind_port_if_needed, neutron-server push port Info cache with binding_levels=[], and revision_number=3
7. Neutron-server try to bind this port
8. Neutron-ovs-agent rpc_loop try to get the port details
9. Neutron-ovs-agent Info cache RPC gets empty binding_levels=[] and skip processing port
10. Neutron-server bind done and send Info cache,
and now the revision_number is still 3, binding_levels=[<entry>] is not empty now.
11. neutron-ovs-agent get the new info cache notification, but the revision_number is not changed, so the cache is not updated.
The port will not be processed anymore.
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2065577/+subscriptions
Follow ups