← Back to team overview

ubuntu-phone team mailing list archive

Re: Catching CPU run-aways on Touch

 

On Wed, Sep 04, 2013 at 01:16:07PM -0500, Ted Gould wrote:
> > On Wed, Sep 04, 2013 at 10:22:59AM -0500, Ted Gould wrote:
> > > To give people something to attack more specifically, I'll say this.  We
> > > should add a line to Upstart job configs that looks like this:

> > >         cpu limit [CPU Percentage] [seconds]

> > > Then we can have a small upstart-bridge-like process that watches
> > > upstart for started, stopped and added jobs to ensure that they're on
> > > the naughty/nice list and that they behave within those limits.

> > upstart already supports setting kernel ulimits for jobs; through ulimits
> > you can already set "max CPU for the life of the process" and "max memory
> > per process".  You can also set the realtime priority of a process.  You
> > can't set a max *percentage* of CPU usage for the job, or max memory usage
> > for the set of processes spawned by the job; both of these capabilities will
> > arrive with cgroup support.

> > However, in all of the above cases we're talking about *limiting* the CPU
> > usage, not *measuring* it.  If the desired semantics are to measure the
> > process's CPU usage and report on it / *optionally* kill the process, I
> > don't think that's a reasonable fit for upstart.  It makes sense for upstart
> > to apply cgroups to processes upon request, allowing the kernel to limit the
> > amount of CPU the job gets access to... but then by definition no such
> > process is ever a "runaway" because it's kept on a leash, so you don't
> > actually get any useful information this way about which processes are buggy
> > and should be fixed.  If we care about identifying and fixing misbehaving
> > processes, rather than just limiting the damage, that should be handled
> > outside of upstart.

> I wasn't intending to say "upstart should do it" more that we should put
> it in the Upstart job configuration and use that as our basis.  That's
> what I was trying to say with "upstart-bridge-like" thing in that it'd
> get the information from Upstart (including dynamic job creation,
> addition, etc) but still do the tracking and reaction on its own.  I'd
> expect that then it'd stop/start/restart jobs using Upstart as well.

Six of one, half a dozen of the other. :-)  The purpose of upstart bridges
is to generate various events; in some cases the bridges will parse upstart
job files to decide which events from the set of all possible events they
should actually generate, but they're still about generating events.  I
think putting the limit information in the .conf files and having an upstart
bridge to act on them by generating events is a matter of
everything-looks-like-a-nail, not something that's a natural fit for upstart
or upstart bridges.

-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
Ubuntu Developer                                    http://www.debian.org/
slangasek@xxxxxxxxxx                                     vorlon@xxxxxxxxxx

Attachment: signature.asc
Description: Digital signature


Follow ups

References