yahoo-eng-team team mailing list archive

["ChangBo Guo\(gcb\)" <glongwave@xxxxxxxxx>]

Public bug reported:

1. Libvirt driver can receive libvirt lifecycle events(registered in
https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L1004),
then handle it in
https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L969
, that means  shutdown a domain  will  send out shutdown lifecycle event
and nova compute will try to sync the instance's power_state.

2. When reboot compute service ,  compute service is trying to reboot instance which were running before reboot.
https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L911.  Compute service only checks the power_state in database. the value of power_state can be changed in 3.  That leads out  reboot host, some instances which were running before reboot can't be restarted.

3. When reboot the host,  the code path like  1)libvirt-guests will
shutdown all the domain,   2)then sendout  lifecycle event , 3)nova
compute receive it and 4)save power_state 'shutoff' in db , 5)then try
to stop it.   Compute service may be killed in any step,  In my test
enviroment,  two running instances , only one instance was restarted
succefully. another was set power_state with 'shutoff', task_state  with
'power off' in  step 4) .  So it can't pass the check in
https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L911.
won't be restarted.


Not sure this is a bug ,  wonder if there is solution for this .

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1293480

Title:
  Reboot host  didn't restart instances due to  libvirt lifecycle event
  change instance's power_stat as shutdown

Status in OpenStack Compute (Nova):
  New

Bug description:
  1. Libvirt driver can receive libvirt lifecycle events(registered in
  https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L1004),
  then handle it in
  https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L969
  , that means  shutdown a domain  will  send out shutdown lifecycle
  event and nova compute will try to sync the instance's power_state.

  2. When reboot compute service ,  compute service is trying to reboot instance which were running before reboot.
  https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L911.  Compute service only checks the power_state in database. the value of power_state can be changed in 3.  That leads out  reboot host, some instances which were running before reboot can't be restarted.

  3. When reboot the host,  the code path like  1)libvirt-guests will
  shutdown all the domain,   2)then sendout  lifecycle event , 3)nova
  compute receive it and 4)save power_state 'shutoff' in db , 5)then try
  to stop it.   Compute service may be killed in any step,  In my test
  enviroment,  two running instances , only one instance was restarted
  succefully. another was set power_state with 'shutoff', task_state
  with 'power off' in  step 4) .  So it can't pass the check in
  https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L911.
  won't be restarted.

  
  Not sure this is a bug ,  wonder if there is solution for this .

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1293480/+subscriptions