← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #17854

[Bug 1347854] [NEW] nova-network stops responding to RPC requests

  Thread Previous • Date Previous • • Thread Next 
*** This bug is a duplicate of bug 1343613 ***
    https://bugs.launchpad.net/bugs/1343613

Public bug reported:

Occasioanlly it looks like nova-network stops responding in the check
and gate queues.

query: message:"MessagingTimeout\: Timed out waiting for a reply to
message" AND message:"_add_floating_ip"


2014-07-22 23:37:29.097 DEBUG nova.openstack.common.lockutils [req-db21216b-51b5-4ef8-ba41-19e1fa2490e3 TenantUsagesTestXML-169388111 TenantUsagesTestXML-1815715183] Attempting to grab external lock "lock_bridge" external_lock /opt/stack/new/nova/nova/openstack/common/lockutils.py:230
2014-07-22 23:37:29.106 DEBUG nova.openstack.common.lockutils [req-db21216b-51b5-4ef8-ba41-19e1fa2490e3 TenantUsagesTestXML-169388111 TenantUsagesTestXML-1815715183] Got semaphore / lock "ensure_bridge" inner /opt/stack/new/nova/nova/openstack/common/lockutils.py:324
2014-07-22 23:37:29.106 DEBUG nova.network.linux_net [req-db21216b-51b5-4ef8-ba41-19e1fa2490e3 TenantUsagesTestXML-169388111 TenantUsagesTestXML-1815715183] Adding interface eth0 to bridge br100 ensure_bridge /opt/stack/new/nova/nova/network/linux_net.py:1544
2014-07-22 23:37:29.107 DEBUG nova.openstack.common.processutils [req-db21216b-51b5-4ef8-ba41-19e1fa2490e3 TenantUsagesTestXML-169388111 TenantUsagesTestXML-1815715183] Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf brctl addif br100 eth0 execute /opt/stack/new/nova/nova/openstack/common/processutils.py:160
2014-07-22 23:37:29.122 DEBUG nova.openstack.common.lockutils [req-ea618108-6fa1-437e-b99a-908976dca0bf FloatingIPsTestJSON-1189372331 FloatingIPsTestJSON-210985373] Got semaphore "172.24.4.1" internal_lock /opt/stack/new/nova/nova/openstack/common/lockutils.py:263
2014-07-22 23:37:29.123 DEBUG nova.openstack.common.lockutils [req-ea618108-6fa1-437e-b99a-908976dca0bf FloatingIPsTestJSON-1189372331 FloatingIPsTestJSON-210985373] Got semaphore / lock "do_associate" inner /opt/stack/new/nova/nova/openstack/common/lockutils.py:324
2014-07-22 23:37:29.213 DEBUG nova.openstack.common.processutils [req-db21216b-51b5-4ef8-ba41-19e1fa2490e3 TenantUsagesTestXML-169388111 TenantUsagesTestXML-1815715183] Result was 1 execute /opt/stack/new/nova/nova/openstack/common/processutils.py:194
2014-07-22 23:37:29.214 DEBUG nova.openstack.common.processutils [req-db21216b-51b5-4ef8-ba41-19e1fa2490e3 TenantUsagesTestXML-169388111 TenantUsagesTestXML-1815715183] Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf ip link set eth0 up execute /opt/stack/new/nova/nova/openstack/common/processutils.py:160
2014-07-22 23:37:29.266 DEBUG nova.openstack.common.lockutils [req-ea618108-6fa1-437e-b99a-908976dca0bf FloatingIPsTestJSON-1189372331 FloatingIPsTestJSON-210985373] Got semaphore "iptables" internal_lock /opt/stack/new/nova/nova/openstack/common/lockutils.py:263
2014-07-22 23:37:29.266 DEBUG nova.openstack.common.lockutils [req-ea618108-6fa1-437e-b99a-908976dca0bf FloatingIPsTestJSON-1189372331 FloatingIPsTestJSON-210985373] Attempting to grab external lock "iptables" external_lock /opt/stack/new/nova/nova/openstack/common/lockutils.py:230


http://logs.openstack.org/52/105552/2/check/check-tempest-dsvm-full/4529ad3/logs/screen-n-net.txt.gz#_2014-07-22_23_37_28_969

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1347854

Title:
  nova-network stops responding to RPC requests

Status in OpenStack Compute (Nova):
  New

Bug description:
  Occasioanlly it looks like nova-network stops responding in the check
  and gate queues.

  query: message:"MessagingTimeout\: Timed out waiting for a reply to
  message" AND message:"_add_floating_ip"

  
  2014-07-22 23:37:29.097 DEBUG nova.openstack.common.lockutils [req-db21216b-51b5-4ef8-ba41-19e1fa2490e3 TenantUsagesTestXML-169388111 TenantUsagesTestXML-1815715183] Attempting to grab external lock "lock_bridge" external_lock /opt/stack/new/nova/nova/openstack/common/lockutils.py:230
  2014-07-22 23:37:29.106 DEBUG nova.openstack.common.lockutils [req-db21216b-51b5-4ef8-ba41-19e1fa2490e3 TenantUsagesTestXML-169388111 TenantUsagesTestXML-1815715183] Got semaphore / lock "ensure_bridge" inner /opt/stack/new/nova/nova/openstack/common/lockutils.py:324
  2014-07-22 23:37:29.106 DEBUG nova.network.linux_net [req-db21216b-51b5-4ef8-ba41-19e1fa2490e3 TenantUsagesTestXML-169388111 TenantUsagesTestXML-1815715183] Adding interface eth0 to bridge br100 ensure_bridge /opt/stack/new/nova/nova/network/linux_net.py:1544
  2014-07-22 23:37:29.107 DEBUG nova.openstack.common.processutils [req-db21216b-51b5-4ef8-ba41-19e1fa2490e3 TenantUsagesTestXML-169388111 TenantUsagesTestXML-1815715183] Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf brctl addif br100 eth0 execute /opt/stack/new/nova/nova/openstack/common/processutils.py:160
  2014-07-22 23:37:29.122 DEBUG nova.openstack.common.lockutils [req-ea618108-6fa1-437e-b99a-908976dca0bf FloatingIPsTestJSON-1189372331 FloatingIPsTestJSON-210985373] Got semaphore "172.24.4.1" internal_lock /opt/stack/new/nova/nova/openstack/common/lockutils.py:263
  2014-07-22 23:37:29.123 DEBUG nova.openstack.common.lockutils [req-ea618108-6fa1-437e-b99a-908976dca0bf FloatingIPsTestJSON-1189372331 FloatingIPsTestJSON-210985373] Got semaphore / lock "do_associate" inner /opt/stack/new/nova/nova/openstack/common/lockutils.py:324
  2014-07-22 23:37:29.213 DEBUG nova.openstack.common.processutils [req-db21216b-51b5-4ef8-ba41-19e1fa2490e3 TenantUsagesTestXML-169388111 TenantUsagesTestXML-1815715183] Result was 1 execute /opt/stack/new/nova/nova/openstack/common/processutils.py:194
  2014-07-22 23:37:29.214 DEBUG nova.openstack.common.processutils [req-db21216b-51b5-4ef8-ba41-19e1fa2490e3 TenantUsagesTestXML-169388111 TenantUsagesTestXML-1815715183] Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf ip link set eth0 up execute /opt/stack/new/nova/nova/openstack/common/processutils.py:160
  2014-07-22 23:37:29.266 DEBUG nova.openstack.common.lockutils [req-ea618108-6fa1-437e-b99a-908976dca0bf FloatingIPsTestJSON-1189372331 FloatingIPsTestJSON-210985373] Got semaphore "iptables" internal_lock /opt/stack/new/nova/nova/openstack/common/lockutils.py:263
  2014-07-22 23:37:29.266 DEBUG nova.openstack.common.lockutils [req-ea618108-6fa1-437e-b99a-908976dca0bf FloatingIPsTestJSON-1189372331 FloatingIPsTestJSON-210985373] Attempting to grab external lock "iptables" external_lock /opt/stack/new/nova/nova/openstack/common/lockutils.py:230

  
  http://logs.openstack.org/52/105552/2/check/check-tempest-dsvm-full/4529ad3/logs/screen-n-net.txt.gz#_2014-07-22_23_37_28_969

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1347854/+subscriptions

Follow ups

References

  Thread Previous • Date Previous • • Thread Next