← Back to team overview

touch-packages team mailing list archive

[Bug 1389356] [NEW] rsyslogd stops responding, doesn't log data, and eventually hangs the system

 

Public bug reported:

This is happening on a wide variety of client systems that are all
dissimilar--some are virtual machines, some are physical machines, and
it spans 11.04, 12.04, 13.10, and 14.04.

It started about two months ago.

Approximately once per week, we will start getting calls from all our
clients that services running on linux boxes are unavailable or
extremely slow.

Attempts to access the boxes via SSH will either not work (hang and then
timeout after ~2 minutes) or succeed (after hanging for ~1 minute).
Then the shell prompt takes a while (maybe 30 seconds) to display.

After spending several frustrating hours with one particular box, I noticed the following:
* Very low disk IO (i.e. the box isn't hammering the disk)
* Memory usage was appropriate
* Network IO was appropriate and responsive (ping, traceroute, wget, etc...)
* Logs were all 'empty'.  Last log data was in the log from the previous evening (i.e. /var/log/syslog.1 has a final entry at 8:36 PM PST from the previous night)

Running the command 'restart rsyslogd' immediately returns the box to
normal operation.

After a few more testing sessions, I can see that rsyslogd is running on
all these boxes, it just appears to be unresponsive.

The issue happens fairly regularly--every 7-10 days, and it happens on
multiple disparate systems on different networks at approximately the
same time.

** Affects: rsyslog (Ubuntu)
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to rsyslog in Ubuntu.
https://bugs.launchpad.net/bugs/1389356

Title:
  rsyslogd stops responding, doesn't log data, and eventually hangs the
  system

Status in “rsyslog” package in Ubuntu:
  New

Bug description:
  This is happening on a wide variety of client systems that are all
  dissimilar--some are virtual machines, some are physical machines, and
  it spans 11.04, 12.04, 13.10, and 14.04.

  It started about two months ago.

  Approximately once per week, we will start getting calls from all our
  clients that services running on linux boxes are unavailable or
  extremely slow.

  Attempts to access the boxes via SSH will either not work (hang and
  then timeout after ~2 minutes) or succeed (after hanging for ~1
  minute).  Then the shell prompt takes a while (maybe 30 seconds) to
  display.

  After spending several frustrating hours with one particular box, I noticed the following:
  * Very low disk IO (i.e. the box isn't hammering the disk)
  * Memory usage was appropriate
  * Network IO was appropriate and responsive (ping, traceroute, wget, etc...)
  * Logs were all 'empty'.  Last log data was in the log from the previous evening (i.e. /var/log/syslog.1 has a final entry at 8:36 PM PST from the previous night)

  Running the command 'restart rsyslogd' immediately returns the box to
  normal operation.

  After a few more testing sessions, I can see that rsyslogd is running
  on all these boxes, it just appears to be unresponsive.

  The issue happens fairly regularly--every 7-10 days, and it happens on
  multiple disparate systems on different networks at approximately the
  same time.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/rsyslog/+bug/1389356/+subscriptions


Follow ups

References