kernel-packages team mailing list archive

[Dan Streetman <dan.streetman@xxxxxxxxxxxxx>]

This isn't actually an issue with the serial port driver, it's the
printk code that is failing to schedule while printing out all the msgs.
During printk(), if the console lock is held, the msgs are queued up,
and inside console_unlock() any queued msgs are sent to the console
before unlocking:

void console_unlock(void)
{
...
        bool do_cond_resched, retry;
...
        /*                                                                                          
         * Console drivers are called under logbuf_lock, so                                         
         * @console_may_schedule should be cleared before; however, we may                          
         * end up dumping a lot of lines, for example, if called from                               
         * console registration path, and should invoke cond_resched()                              
         * between lines if allowable.  Not doing so can cause a very long                          
         * scheduling stall on a slow console leading to RCU stall and                              
         * softlockup warnings which exacerbate the issue with more                                 
         * messages practically incapacitating the system.                                          
         */
        do_cond_resched = console_may_schedule;
        console_may_schedule = 0;
...
        for (;;) {
...
                call_console_drivers(level, text, len);
...
        	if (do_cond_resched)
        	        cond_resched();
	}

so what happens here is, console_may_schedule is 0 and therefore
cond_resched() isn't called between printk msgs sent to the serial port
driver; and the serial port takes so long to actually send the msgs,
eventually rcu thinks the cpu has soft-locked up.

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1534216

Title:
  lots of printk to serial console can hang system for long time

Status in linux package in Ubuntu:
  In Progress

Bug description:
  This is a clone from bug 1505564, to track the separate issue of the
  serial port driver failing to schedule itself off its cpu.

  The original bug's problem was caused by the kernel spamming a huge
  number of error messages in a certain situation.  Normally, that would
  not be a problem, but in this case the system is virtualized, and logs
  over its serial port.  When the massive number of kernel messages are
  sent to the serial port driver, it can't keep up, so sending all the
  log messages can take a very long time - minutes or longer - and the
  serial port driver fails to schedule itself off the cpu it's using
  during that time.  That results in other cpus hanging, waiting for the
  serial port driver's cpu to become avaiable.

  I'll  update the bug with more details as I debug.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1534216/+subscriptions