yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #26613
[Bug 1407664] [NEW] Race: instance nw_info cache is updated to empty list because of nova/neutron event mechanism
Public bug reported:
This applies only when the nova/neutron event reporting mechanism is
enabled.
Boot instance, like this:
nova boot --image xxx --flavor xxx --nic port-id=xxx test_vm
The booting instance is successful, but instance nw_info cache is empty.
This is a probabilistic problem, not always can be reproduced.
After analysis the booting instance and nova/neutron event mechanism workflow,
I get the reproduce timeline:
1. neutronv2.api.allocate_for_instance when booting instance
2. neutronclient.update_port trigger neutron network_change event
3. nova get the port change event, start to dispose event
4. instance.get_by_uuid in external_instance_event , at this time instance.nw_info_cache is empty,
because nw_info cache hadn't been added into db in booting instance thread.
5. booting instance thread start to save the instance nw_info cache into db.
6. event disposing thread start to update instance nw_info cache to empty.
Face this issue in Juno.
I add some breakpoints in order to reproduce this bug in my devstack.
** Affects: nova
Importance: Undecided
Assignee: Rui Chen (kiwik-chenrui)
Status: New
** Changed in: nova
Assignee: (unassigned) => Rui Chen (kiwik-chenrui)
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1407664
Title:
Race: instance nw_info cache is updated to empty list because of
nova/neutron event mechanism
Status in OpenStack Compute (Nova):
New
Bug description:
This applies only when the nova/neutron event reporting mechanism is
enabled.
Boot instance, like this:
nova boot --image xxx --flavor xxx --nic port-id=xxx test_vm
The booting instance is successful, but instance nw_info cache is empty.
This is a probabilistic problem, not always can be reproduced.
After analysis the booting instance and nova/neutron event mechanism workflow,
I get the reproduce timeline:
1. neutronv2.api.allocate_for_instance when booting instance
2. neutronclient.update_port trigger neutron network_change event
3. nova get the port change event, start to dispose event
4. instance.get_by_uuid in external_instance_event , at this time instance.nw_info_cache is empty,
because nw_info cache hadn't been added into db in booting instance thread.
5. booting instance thread start to save the instance nw_info cache into db.
6. event disposing thread start to update instance nw_info cache to empty.
Face this issue in Juno.
I add some breakpoints in order to reproduce this bug in my devstack.
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1407664/+subscriptions
Follow ups
References