yahoo-eng-team team mailing list archive

Thread
Date

[Bug 1101404] Re: nova syslog logging to /dev/log race condition in python 2.6

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Simeon Monov <sdmonov@xxxxxxxxxx>
Date: Tue, 09 Dec 2014 17:47:04 -0000
Reply-to: Bug 1101404 <1101404@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx

** Also affects: neutron
   Importance: Undecided
       Status: New

** Also affects: glance
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1101404

Title:
  nova syslog logging to /dev/log race condition in python 2.6

Status in OpenStack Image Registry and Delivery Service (Glance):
  New
Status in OpenStack Identity (Keystone):
  Confirmed
Status in OpenStack Neutron (virtual network service):
  New
Status in OpenStack Compute (Nova):
  Confirmed
Status in Logging configuration library for OpenStack:
  Confirmed

Bug description:
  
  running nova-api-ec2
  running rsyslog

  service rsyslog restart ; service nova-api-ec2 restart

  nova-api-ec2 consumes up to 100% of the available CPU (or at least a
  full core) and s not responsive.  /var/log/nova/nova-api-ec2.lgo
  states the socket is already in use.

  strace the process

  sendto(3, "<142>2013-01-18 20:00:22 24882 INFO nova.service [-] Caught
  SIGTERM, exiting\0", 77, 0, NULL, 0) = -1 ENOTCONN (Transport endpoint
  is not connected)

  service nova-api-ec2 restart fails as upstart already thinks the
  process has been terminated.

  The only way to recover is to pkill -9 nova-api-ec2 and then restart
  it with 'service nova-api-ec2 restart'.

  The same behavior has been seen in all nova-api services.

To manage notifications about this bug go to:
https://bugs.launchpad.net/glance/+bug/1101404/+subscriptions