yahoo-eng-team team mailing list archive

[Lars Kellogg-Stedman <lars@xxxxxxxxxx>]

Public bug reported:

We ran into a situation with a Pike deployment (upgraded from Ocata) in
which new instances were failing to schedule on a target compute node
that was empty.  Nova-conductor would throw a NoValidHost exception
without any details, and there were no other error messages related to
the failure.  It turns out this was due to a number of stale resource
allocations on the target compute node.

If nova-compute finds stale allocations, it logs a message at DEBUG
priority that says:

  There are allocations remaining against the source host that might
need to be removed:

Since this is logged at DEBUG priority, the message will not typically
be seen by operators in a production environment.  However, stale
allocations can prevent Nova from scheduling new instances on the
compute node, leading to mysterious failures for which there are no
obvious errors.

This message should be logged at WARNING priority so that it is visible.

** Affects: nova
     Importance: Undecided
     Assignee: Lars Kellogg-Stedman (larsks)
         Status: In Progress

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1767553

Title:
  compute resource tracker errors regarding stale allocations should be
  logged at WARNING instead of DEBUG

Status in OpenStack Compute (nova):
  In Progress

Bug description:
  We ran into a situation with a Pike deployment (upgraded from Ocata)
  in which new instances were failing to schedule on a target compute
  node that was empty.  Nova-conductor would throw a NoValidHost
  exception without any details, and there were no other error messages
  related to the failure.  It turns out this was due to a number of
  stale resource allocations on the target compute node.

  If nova-compute finds stale allocations, it logs a message at DEBUG
  priority that says:

    There are allocations remaining against the source host that might
  need to be removed:

  Since this is logged at DEBUG priority, the message will not typically
  be seen by operators in a production environment.  However, stale
  allocations can prevent Nova from scheduling new instances on the
  compute node, leading to mysterious failures for which there are no
  obvious errors.

  This message should be logged at WARNING priority so that it is
  visible.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1767553/+subscriptions