← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #55334

[Bug 1615357] [NEW] VMs failed to get ip in vxlan setup due to keyerror

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

When spawning around 4k VMs in 100 network scenario , around 500 Vms failed to get ip by throwing below keyerror exception.
 And it is getting ip by restarting dhcp (S40network) in the VM.

build used: HOS 4.0 ((build 01-547)
 Attached logs

2016-06-22 12:01:36.112 31012 ERROR networking_vsphere.agent.ovsvapp_agent [-] Failed to handle VM_UPDATED event for VM: 1e2b8347-6303-4417-aab8-e48d238f7902.
 2016-06-22 12:01:36.112 31012 ERROR networking_vsphere.agent.ovsvapp_agent Traceback (most recent call last):
 2016-06-22 12:01:36.112 31012 ERROR networking_vsphere.agent.ovsvapp_agent File "/opt/stack/venv/neutron-20160621T031745Z/lib/python2.7/site-packages/networking_vsphere/agent/ovsvapp_agent.py", line 1087, in _notify_device_updated
 2016-06-22 12:01:36.112 31012 ERROR networking_vsphere.agent.ovsvapp_agent updated_port = self.ports_dict[vnic.port_uuid]
 2016-06-22 12:01:36.112 31012 ERROR networking_vsphere.agent.ovsvapp_agent KeyError: 4db21599-5542-46a3-a994-937986eb230b
 2016-06-22 12:01:36.112 31012 ERROR networking_vsphere.agent.ovsvapp_agent
 2016-06-22 12:01:36.112 31012 ERROR networking_vsphere.agent.ovsvapp_agent [-] This may result in failure of network provisioning for VirtualMachine 1e2b8347-6303-4417-aab8-e48d238f7902.
 2016-06-22 12:01:36.113 31012 ERROR networking_vsphere.agent.ovsvapp_agent [-] Cause of failure: 4db21599-5542-46a3-a994-937986eb230b.
 2016-06-22 12:01:36.113 31012 ERROR networking_vsphere.agent.ovsvapp_agent Traceback (most recent call last):
2016-06-22 12:01:36.113 31012 ERROR networking_vsphere.agent.ovsvapp_agent File "/opt/stack/venv/neutron-20160621T031745Z/lib/python2.7/site-packages/networking_vsphere/agent/ovsvapp_agent.py", line 964, in process_event
 2016-06-22 12:01:36.113 31012 ERROR networking_vsphere.agent.ovsvapp_agent self._notify_device_updated(vm, host, event.host_changed)
 2016-06-22 12:01:36.113 31012 ERROR networking_vsphere.agent.ovsvapp_agent File "/opt/stack/venv/neutron-20160621T031745Z/lib/python2.7/site-packages/networking_vsphere/common/utils.py", line 91, in inner
 2016-06-22 12:01:36.113 31012 ERROR networking_vsphere.agent.ovsvapp_agent return f(obj, *args, **kw)
 2016-06-22 12:01:36.113 31012 ERROR networking_vsphere.agent.ovsvapp_agent File "/opt/stack/venv/neutron-20160621T031745Z/lib/python2.7/site-packages/networking_vsphere/agent/ovsvapp_agent.py", line 1108, in _notify_device_updated
 2016-06-22 12:01:36.113 31012 ERROR networking_vsphere.agent.ovsvapp_agent raise error.OVSvAppNeutronAgentError(e)
 2016-06-22 12:01:36.113 31012 ERROR networking_vsphere.agent.ovsvapp_agent OVSvAppNeutronAgentError: 4db21599-5542-46a3-a994-937986eb230b
 2016-06-22 12:01:36.113 31012 ERROR networking_vsphere.agent.ovsvapp_agent

** Affects: neutron
     Importance: Undecided
     Assignee: Arun Kumar (arooncoomar)
         Status: In Progress

** Changed in: neutron
       Status: New => In Progress

** Changed in: neutron
     Assignee: (unassigned) => Arun Kumar (arooncoomar)

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1615357

Title:
  VMs failed to get ip in vxlan setup due to keyerror

Status in neutron:
  In Progress

Bug description:
  When spawning around 4k VMs in 100 network scenario , around 500 Vms failed to get ip by throwing below keyerror exception.
   And it is getting ip by restarting dhcp (S40network) in the VM.

  build used: HOS 4.0 ((build 01-547)
   Attached logs

  2016-06-22 12:01:36.112 31012 ERROR networking_vsphere.agent.ovsvapp_agent [-] Failed to handle VM_UPDATED event for VM: 1e2b8347-6303-4417-aab8-e48d238f7902.
   2016-06-22 12:01:36.112 31012 ERROR networking_vsphere.agent.ovsvapp_agent Traceback (most recent call last):
   2016-06-22 12:01:36.112 31012 ERROR networking_vsphere.agent.ovsvapp_agent File "/opt/stack/venv/neutron-20160621T031745Z/lib/python2.7/site-packages/networking_vsphere/agent/ovsvapp_agent.py", line 1087, in _notify_device_updated
   2016-06-22 12:01:36.112 31012 ERROR networking_vsphere.agent.ovsvapp_agent updated_port = self.ports_dict[vnic.port_uuid]
   2016-06-22 12:01:36.112 31012 ERROR networking_vsphere.agent.ovsvapp_agent KeyError: 4db21599-5542-46a3-a994-937986eb230b
   2016-06-22 12:01:36.112 31012 ERROR networking_vsphere.agent.ovsvapp_agent
   2016-06-22 12:01:36.112 31012 ERROR networking_vsphere.agent.ovsvapp_agent [-] This may result in failure of network provisioning for VirtualMachine 1e2b8347-6303-4417-aab8-e48d238f7902.
   2016-06-22 12:01:36.113 31012 ERROR networking_vsphere.agent.ovsvapp_agent [-] Cause of failure: 4db21599-5542-46a3-a994-937986eb230b.
   2016-06-22 12:01:36.113 31012 ERROR networking_vsphere.agent.ovsvapp_agent Traceback (most recent call last):
  2016-06-22 12:01:36.113 31012 ERROR networking_vsphere.agent.ovsvapp_agent File "/opt/stack/venv/neutron-20160621T031745Z/lib/python2.7/site-packages/networking_vsphere/agent/ovsvapp_agent.py", line 964, in process_event
   2016-06-22 12:01:36.113 31012 ERROR networking_vsphere.agent.ovsvapp_agent self._notify_device_updated(vm, host, event.host_changed)
   2016-06-22 12:01:36.113 31012 ERROR networking_vsphere.agent.ovsvapp_agent File "/opt/stack/venv/neutron-20160621T031745Z/lib/python2.7/site-packages/networking_vsphere/common/utils.py", line 91, in inner
   2016-06-22 12:01:36.113 31012 ERROR networking_vsphere.agent.ovsvapp_agent return f(obj, *args, **kw)
   2016-06-22 12:01:36.113 31012 ERROR networking_vsphere.agent.ovsvapp_agent File "/opt/stack/venv/neutron-20160621T031745Z/lib/python2.7/site-packages/networking_vsphere/agent/ovsvapp_agent.py", line 1108, in _notify_device_updated
   2016-06-22 12:01:36.113 31012 ERROR networking_vsphere.agent.ovsvapp_agent raise error.OVSvAppNeutronAgentError(e)
   2016-06-22 12:01:36.113 31012 ERROR networking_vsphere.agent.ovsvapp_agent OVSvAppNeutronAgentError: 4db21599-5542-46a3-a994-937986eb230b
   2016-06-22 12:01:36.113 31012 ERROR networking_vsphere.agent.ovsvapp_agent

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1615357/+subscriptions

Follow ups

  Thread Previous • Date Previous • • Thread Next