← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 1403528] [NEW] neutron-server hangs at initializing driver for type vlan

 

Public bug reported:

i used nova boot to launch about 50 vms. some of them were active eventually while the rest were in error state.
I later found that neutron-server was not returning  anything for wget neutron-host:9696
in the neutron-server log:
Dec 17 14:14:57 ci-overcloud-controller1-mqrd73cq4fgs neutron-server: 2014-12-17 14:14:57.099 13651 WARNING neutron.db.l3_agentschedulers_db [-] Time since last L3 agent reschedule check has exceeded the interval between checks. Waiting before check to allow agents to send a heartbeat in case there was a clock adjustment.

That's it. the last log that was printed.

then i tried to restart neutron-server with service neutron-server
restart.However, nothing was printed.

i later enabled debug level for the log and the following logs were
printed:


Dec 17 14:34:42 ci-overcloud-controller1-mqrd73cq4fgs neutron-server: 2014-12-17 14:34:42.852 25846 DEBUG neutron.openstack.common.lockutils [-] Created new semaphore "manager" internal_lock /opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/openstack/common/lockutils.py:206
Dec 17 14:34:42 ci-overcloud-controller1-mqrd73cq4fgs neutron-server: 2014-12-17 14:34:42.852 25846 DEBUG neutron.openstack.common.lockutils [-] Acquired semaphore "manager" lock /opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/openstack/common/lockutils.py:229
Dec 17 14:34:42 ci-overcloud-controller1-mqrd73cq4fgs neutron-server: 2014-12-17 14:34:42.853 25846 DEBUG neutron.openstack.common.lockutils [-] Got semaphore / lock "_create_instance" inner /opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/openstack/common/lockutils.py:271
Dec 17 14:34:42 ci-overcloud-controller1-mqrd73cq4fgs neutron-server: 2014-12-17 14:34:42.853 25846 INFO neutron.manager [-] Loading core plugin: neutron.plugins.ml2.plugin.Ml2Plugin
Dec 17 14:34:42 ci-overcloud-controller1-mqrd73cq4fgs neutron-server: 2014-12-17 14:34:42.992 25846 INFO neutron.plugins.ml2.managers [-] Configured type driver names: ['local', 'flat', 'vlan', 'gre', 'vxlan']
Dec 17 14:34:42 ci-overcloud-controller1-mqrd73cq4fgs neutron-server: 2014-12-17 14:34:42.994 25846 INFO neutron.plugins.ml2.drivers.type_flat [-] Allowable flat physical_network names: ['datacentre']
Dec 17 14:34:42 ci-overcloud-controller1-mqrd73cq4fgs neutron-server: 2014-12-17 14:34:42.998 25846 INFO neutron.plugins.ml2.drivers.type_local [-] ML2 LocalTypeDriver initialization complete
Dec 17 14:34:43 ci-overcloud-controller1-mqrd73cq4fgs neutron-server: 2014-12-17 14:34:43.003 25846 INFO neutron.plugins.ml2.drivers.type_vlan [-] Network VLAN ranges: {'datacentre': []}
Dec 17 14:34:43 ci-overcloud-controller1-mqrd73cq4fgs neutron-server: 2014-12-17 14:34:43.003 25846 INFO neutron.plugins.ml2.managers [-] Loaded type driver names: ['flat', 'gre', 'local', 'vxlan', 'vlan']
Dec 17 14:34:43 ci-overcloud-controller1-mqrd73cq4fgs neutron-server: 2014-12-17 14:34:43.003 25846 INFO neutron.plugins.ml2.managers [-] Registered types: ['vlan', 'flat', 'gre', 'local', 'vxlan']
Dec 17 14:34:43 ci-overcloud-controller1-mqrd73cq4fgs neutron-server: 2014-12-17 14:34:43.004 25846 INFO neutron.plugins.ml2.managers [-] Tenant network_types: ['gre']
Dec 17 14:34:43 ci-overcloud-controller1-mqrd73cq4fgs neutron-server: 2014-12-17 14:34:43.004 25846 INFO neutron.plugins.ml2.managers [-] Configured extension driver names: []
Dec 17 14:34:43 ci-overcloud-controller1-mqrd73cq4fgs neutron-server: 2014-12-17 14:34:43.004 25846 INFO neutron.plugins.ml2.managers [-] Loaded extension driver names: []
Dec 17 14:34:43 ci-overcloud-controller1-mqrd73cq4fgs neutron-server: 2014-12-17 14:34:43.004 25846 INFO neutron.plugins.ml2.managers [-] Registered extension drivers: []
Dec 17 14:34:43 ci-overcloud-controller1-mqrd73cq4fgs neutron-server: 2014-12-17 14:34:43.005 25846 INFO neutron.plugins.ml2.managers [-] Configured mechanism driver names: ['openvswitch']
Dec 17 14:34:43 ci-overcloud-controller1-mqrd73cq4fgs neutron-server: 2014-12-17 14:34:43.006 25846 INFO neutron.plugins.ml2.managers [-] Loaded mechanism driver names: ['openvswitch']
Dec 17 14:34:43 ci-overcloud-controller1-mqrd73cq4fgs neutron-server: 2014-12-17 14:34:43.006 25846 INFO neutron.plugins.ml2.managers [-] Registered mechanism drivers: ['openvswitch']
Dec 17 14:34:43 ci-overcloud-controller1-mqrd73cq4fgs neutron-server: 2014-12-17 14:34:43.019 25846 INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'vlan'

The neutron server seems to be stuck at initializing driver for type
vlan.

** Affects: neutron
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1403528

Title:
  neutron-server hangs at initializing driver for type vlan

Status in OpenStack Neutron (virtual network service):
  New

Bug description:
  i used nova boot to launch about 50 vms. some of them were active eventually while the rest were in error state.
  I later found that neutron-server was not returning  anything for wget neutron-host:9696
  in the neutron-server log:
  Dec 17 14:14:57 ci-overcloud-controller1-mqrd73cq4fgs neutron-server: 2014-12-17 14:14:57.099 13651 WARNING neutron.db.l3_agentschedulers_db [-] Time since last L3 agent reschedule check has exceeded the interval between checks. Waiting before check to allow agents to send a heartbeat in case there was a clock adjustment.

  That's it. the last log that was printed.

  then i tried to restart neutron-server with service neutron-server
  restart.However, nothing was printed.

  i later enabled debug level for the log and the following logs were
  printed:

  
  Dec 17 14:34:42 ci-overcloud-controller1-mqrd73cq4fgs neutron-server: 2014-12-17 14:34:42.852 25846 DEBUG neutron.openstack.common.lockutils [-] Created new semaphore "manager" internal_lock /opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/openstack/common/lockutils.py:206
  Dec 17 14:34:42 ci-overcloud-controller1-mqrd73cq4fgs neutron-server: 2014-12-17 14:34:42.852 25846 DEBUG neutron.openstack.common.lockutils [-] Acquired semaphore "manager" lock /opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/openstack/common/lockutils.py:229
  Dec 17 14:34:42 ci-overcloud-controller1-mqrd73cq4fgs neutron-server: 2014-12-17 14:34:42.853 25846 DEBUG neutron.openstack.common.lockutils [-] Got semaphore / lock "_create_instance" inner /opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/openstack/common/lockutils.py:271
  Dec 17 14:34:42 ci-overcloud-controller1-mqrd73cq4fgs neutron-server: 2014-12-17 14:34:42.853 25846 INFO neutron.manager [-] Loading core plugin: neutron.plugins.ml2.plugin.Ml2Plugin
  Dec 17 14:34:42 ci-overcloud-controller1-mqrd73cq4fgs neutron-server: 2014-12-17 14:34:42.992 25846 INFO neutron.plugins.ml2.managers [-] Configured type driver names: ['local', 'flat', 'vlan', 'gre', 'vxlan']
  Dec 17 14:34:42 ci-overcloud-controller1-mqrd73cq4fgs neutron-server: 2014-12-17 14:34:42.994 25846 INFO neutron.plugins.ml2.drivers.type_flat [-] Allowable flat physical_network names: ['datacentre']
  Dec 17 14:34:42 ci-overcloud-controller1-mqrd73cq4fgs neutron-server: 2014-12-17 14:34:42.998 25846 INFO neutron.plugins.ml2.drivers.type_local [-] ML2 LocalTypeDriver initialization complete
  Dec 17 14:34:43 ci-overcloud-controller1-mqrd73cq4fgs neutron-server: 2014-12-17 14:34:43.003 25846 INFO neutron.plugins.ml2.drivers.type_vlan [-] Network VLAN ranges: {'datacentre': []}
  Dec 17 14:34:43 ci-overcloud-controller1-mqrd73cq4fgs neutron-server: 2014-12-17 14:34:43.003 25846 INFO neutron.plugins.ml2.managers [-] Loaded type driver names: ['flat', 'gre', 'local', 'vxlan', 'vlan']
  Dec 17 14:34:43 ci-overcloud-controller1-mqrd73cq4fgs neutron-server: 2014-12-17 14:34:43.003 25846 INFO neutron.plugins.ml2.managers [-] Registered types: ['vlan', 'flat', 'gre', 'local', 'vxlan']
  Dec 17 14:34:43 ci-overcloud-controller1-mqrd73cq4fgs neutron-server: 2014-12-17 14:34:43.004 25846 INFO neutron.plugins.ml2.managers [-] Tenant network_types: ['gre']
  Dec 17 14:34:43 ci-overcloud-controller1-mqrd73cq4fgs neutron-server: 2014-12-17 14:34:43.004 25846 INFO neutron.plugins.ml2.managers [-] Configured extension driver names: []
  Dec 17 14:34:43 ci-overcloud-controller1-mqrd73cq4fgs neutron-server: 2014-12-17 14:34:43.004 25846 INFO neutron.plugins.ml2.managers [-] Loaded extension driver names: []
  Dec 17 14:34:43 ci-overcloud-controller1-mqrd73cq4fgs neutron-server: 2014-12-17 14:34:43.004 25846 INFO neutron.plugins.ml2.managers [-] Registered extension drivers: []
  Dec 17 14:34:43 ci-overcloud-controller1-mqrd73cq4fgs neutron-server: 2014-12-17 14:34:43.005 25846 INFO neutron.plugins.ml2.managers [-] Configured mechanism driver names: ['openvswitch']
  Dec 17 14:34:43 ci-overcloud-controller1-mqrd73cq4fgs neutron-server: 2014-12-17 14:34:43.006 25846 INFO neutron.plugins.ml2.managers [-] Loaded mechanism driver names: ['openvswitch']
  Dec 17 14:34:43 ci-overcloud-controller1-mqrd73cq4fgs neutron-server: 2014-12-17 14:34:43.006 25846 INFO neutron.plugins.ml2.managers [-] Registered mechanism drivers: ['openvswitch']
  Dec 17 14:34:43 ci-overcloud-controller1-mqrd73cq4fgs neutron-server: 2014-12-17 14:34:43.019 25846 INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'vlan'

  The neutron server seems to be stuck at initializing driver for type
  vlan.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1403528/+subscriptions


Follow ups

References