← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #13260

[Bug 1303615] [NEW] Instance's Task state never change from "scheduling" after sending SIGHUP to nova-compute

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

[Issue]

I tried to reload nova.conf with sending SIGHUP to nova-compute.
In my understanding, nova-compute can reload nova.conf by receiving SIGHUP when it has started as daemon.

Reloading config is succeed.
However, booting new instance doesn't work correctly after sending SIGHUP.
Task State would never change from "scheduling".

$ nova list
+--------------------------------------+------+--------+------------+-------------+----------+
| ID                                   | Name | Status | Task State | Power State | Networks |
+--------------------------------------+------+--------+------------+-------------+----------+
| 9aa89186-28fa-44ee-8e97-84bd0de23f91 | vm   | BUILD  | scheduling | NOSTATE     |          |
+--------------------------------------+------+--------+------------+-------------+----------+

[How to reproduce]

nova's commit id: 33fc957a5aeb9d310cbff3ac22c7a3c97a794f72

1) Start nova-compute as daemon.

$ sudo cat /etc/init/nova-compute.conf
description "nova-compute"
author "openstack"

start on (local-filesystems and net-device-up IFACE!=lo)
stop on runlevel [016]

exec su -s /bin/sh -c "exec /usr/local/bin/nova-compute --config-file
/etc/nova/nova.conf --log-file /home/devstack/log/nova-compute.log >
/dev/null 2>&1" devstack

$ sudo service nova-compute start
nova-compute start/running, process 10521

2) Send SIGHUP to nova-compute's PID.

$ ps aux|grep nova-compute
root     10521  0.0  0.0   4052  1548 ?        Ss   15:42   0:00 su -s /bin/sh -c exec /usr/local/bin/nova-compute --config-file /etc/nova/nova.conf --log-file /home/devstack/log/nova-compute.log > /dev/null 2>&1 devstack
devstack 10523 19.5  1.5 220744 32796 ?        Ssl  15:42   0:00 /usr/bin/python /usr/local/bin/nova-compute --config-file /etc/nova/nova.conf --log-file /home/devstack/log/nova-compute.log
$ sudo kill -SIGHUP 10523

3) Verify nova-compute.log and check reload success.

$ cat /home/devstack/log/nova-compute.log
    :
2014-04-07 15:46:18.791 INFO nova.openstack.common.service [-] Caught SIGHUP, exiting
2014-04-07 15:46:18.811 DEBUG nova.openstack.common.service [-] Full set of CONF: from (pid=10523) _wait_for_exit_or_signal /opt/stack/nova/nova/openstack/common/service.py:167
    :

4) Boot new instance and check Task State of new instance repeatedly.

$ nova boot --flavor 1 --image dee24998-10f7-42a3-8cd7-d46d185281ca vm
+--------------------------------------+----------------------------------------------------------------+
| Property                             | Value                                                          |
+--------------------------------------+----------------------------------------------------------------+
| OS-DCF:diskConfig                    | MANUAL                                                         |
| OS-EXT-AZ:availability_zone          | nova                                                           |
    :

5) Task State would never change from scheduling.

$ nova list
+--------------------------------------+------+--------+------------+-------------+----------+
| ID                                   | Name | Status | Task State | Power State | Networks |
+--------------------------------------+------+--------+------------+-------------+----------+
| 9aa89186-28fa-44ee-8e97-84bd0de23f91 | vm   | BUILD  | scheduling | NOSTATE     |          |
+--------------------------------------+------+--------+------------+-------------+----------+

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1303615

Title:
  Instance's Task state never change from "scheduling" after sending
  SIGHUP to nova-compute

Status in OpenStack Compute (Nova):
  New

Bug description:
  [Issue]

  I tried to reload nova.conf with sending SIGHUP to nova-compute.
  In my understanding, nova-compute can reload nova.conf by receiving SIGHUP when it has started as daemon.

  Reloading config is succeed.
  However, booting new instance doesn't work correctly after sending SIGHUP.
  Task State would never change from "scheduling".

  $ nova list
  +--------------------------------------+------+--------+------------+-------------+----------+
  | ID                                   | Name | Status | Task State | Power State | Networks |
  +--------------------------------------+------+--------+------------+-------------+----------+
  | 9aa89186-28fa-44ee-8e97-84bd0de23f91 | vm   | BUILD  | scheduling | NOSTATE     |          |
  +--------------------------------------+------+--------+------------+-------------+----------+

  [How to reproduce]

  nova's commit id: 33fc957a5aeb9d310cbff3ac22c7a3c97a794f72

  1) Start nova-compute as daemon.

  $ sudo cat /etc/init/nova-compute.conf
  description "nova-compute"
  author "openstack"

  start on (local-filesystems and net-device-up IFACE!=lo)
  stop on runlevel [016]

  exec su -s /bin/sh -c "exec /usr/local/bin/nova-compute --config-file
  /etc/nova/nova.conf --log-file /home/devstack/log/nova-compute.log >
  /dev/null 2>&1" devstack

  $ sudo service nova-compute start
  nova-compute start/running, process 10521

  2) Send SIGHUP to nova-compute's PID.

  $ ps aux|grep nova-compute
  root     10521  0.0  0.0   4052  1548 ?        Ss   15:42   0:00 su -s /bin/sh -c exec /usr/local/bin/nova-compute --config-file /etc/nova/nova.conf --log-file /home/devstack/log/nova-compute.log > /dev/null 2>&1 devstack
  devstack 10523 19.5  1.5 220744 32796 ?        Ssl  15:42   0:00 /usr/bin/python /usr/local/bin/nova-compute --config-file /etc/nova/nova.conf --log-file /home/devstack/log/nova-compute.log
  $ sudo kill -SIGHUP 10523

  3) Verify nova-compute.log and check reload success.

  $ cat /home/devstack/log/nova-compute.log
      :
  2014-04-07 15:46:18.791 INFO nova.openstack.common.service [-] Caught SIGHUP, exiting
  2014-04-07 15:46:18.811 DEBUG nova.openstack.common.service [-] Full set of CONF: from (pid=10523) _wait_for_exit_or_signal /opt/stack/nova/nova/openstack/common/service.py:167
      :

  4) Boot new instance and check Task State of new instance repeatedly.

  $ nova boot --flavor 1 --image dee24998-10f7-42a3-8cd7-d46d185281ca vm
  +--------------------------------------+----------------------------------------------------------------+
  | Property                             | Value                                                          |
  +--------------------------------------+----------------------------------------------------------------+
  | OS-DCF:diskConfig                    | MANUAL                                                         |
  | OS-EXT-AZ:availability_zone          | nova                                                           |
      :

  5) Task State would never change from scheduling.

  $ nova list
  +--------------------------------------+------+--------+------------+-------------+----------+
  | ID                                   | Name | Status | Task State | Power State | Networks |
  +--------------------------------------+------+--------+------------+-------------+----------+
  | 9aa89186-28fa-44ee-8e97-84bd0de23f91 | vm   | BUILD  | scheduling | NOSTATE     |          |
  +--------------------------------------+------+--------+------------+-------------+----------+

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1303615/+subscriptions

Follow ups

References

  Thread Previous • Date Previous • • Thread Next