yahoo-eng-team team mailing list archive

[Joe Gordon <985489@xxxxxxxxxxxxxxxxxx>]

** Changed in: nova
       Status: Incomplete => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/985489

Title:
  nova-compute stops processing compute.$HOSTNAME occasionally on
  libvirt

Status in OpenStack Compute (Nova):
  Invalid
Status in nova package in Ubuntu:
  Confirmed

Bug description:
  	root@novamanager:~# /usr/sbin/rabbitmqctl list_queues  | awk '$1 ~ /^compute/ && $2 != 0 { print }'
  	compute.nodexyzzy	12

  Occasionally on canonistack, we find that a compute node simply stops
  processing its rabbit queues.  A check of the logs will show no nova-
  compute.log activity for hours, but a restart of nova-compute will
  cause it to check all the instances and then process all the requests
  in rabbit (usually lots of duplicates from frustrated users trying to
  re-send delete requests and get their quota back for another
  deployment).

  In fact, while I was typing this (having restarted nova-compute on
  nodexyzzy before starting), I re-ran the above command to find it now
  silent.

  This happens often enough (once every couple of days at least) but
  we're not sure of how to debug this.  Is there any information we can
  get you about a nova-compute process that is in this unhappy state?

  For the record, here is the last entry in the example node's nova-
  compute.log when I bounced things around 09:00Z:

          2012-04-19 06:35:35 DEBUG nova.virt.libvirt.connection [-]
  Updating host stats from (pid=3428) update_status /usr/lib/python2.7
  /dist-packages/nova/virt/libvirt/connection.py:2467

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/985489/+subscriptions