yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1260618] [NEW] nova-api, nova-cert, nova-network not shown after installation. please guide.

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: yashjit <yashjit.teotia@xxxxxxxxx>
Date: Fri, 13 Dec 2013 07:39:06 -0000
Reply-to: Bug 1260618 <1260618@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

Please help me - Issue description : I install havana ( with nova-network as i am very new to it) 1 controller node and 1 compute node. This topology and script i used to install grizzly which got installed. So very minor changes in script were made. Now for havana few minor changes i made in conf files -
Nova.conf - [database] connection is used rather sql_connection
Cinder - Same as above.

Now when I launch script it runs and on controller service shown using nova-manage service list
Binary                       Host                                 Zone             Status     State     Updated_At
nova-conductor     control                              internal         enabled     :-)        2013-12-13 06:36:41
nova-consoleauth control                               internal         enabled     :-)       2013-12-13 06:36:41
nova-scheduler      control                              internal         enabled     :-)        2013-12-13 06:36:42
root@control:/etc# ps aux | grep nova
nova      1471  0.0  1.6  67772 ?        Ss   Dec12   0:35 /usr/bin/python /usr/bin/nova-consoleauth --config-file=/etc/nova/nova.conf
nova      1472  0.0  1.6  67808 ?        Ss   Dec12   0:35 /usr/bin/python /usr/bin/nova-conductor --config-file=/etc/nova/nova.conf
nova      1474  0.0  1.6  68416 ?        Ss   Dec12   0:40 /usr/bin/python /usr/bin/nova-scheduler --config-file=/etc/nova/nova.conf
nova      1476  0.0  0.7  32276 ?        Ss   Dec12   0:12 /usr/bin/python /usr/bin/nova-novncproxy --config-file=/etc/nova/nova.conf
root      4479  0.0  0.0  13588   912 pts/2    S+   12:08   0:00 grep --color=auto nova

Same way on compute Node script runs well but nova-compute, nova-network,nova-meata-api does not start. When restarted service on controller node result is like below-
root@control:/etc# cd /etc/init.d/; for i in $( ls nova-* ); do sudo service $i restart; done
stop: Unknown instance: 
nova-api start/running, process 4497
stop: Unknown instance: 
nova-cert start/running, process 4508
nova-conductor stop/waiting
nova-conductor start/running, process 4519
nova-consoleauth stop/waiting
nova-consoleauth start/running, process 4530
nova-novncproxy stop/waiting
nova-novncproxy start/running, process 4541
nova-scheduler stop/waiting
nova-scheduler start/running, process 4556
so i find nova-api, nova-cert, nova-network,nova-metadata-api services are not running. Now the Log and Conf files- Log first

1. nova-api -
2013-12-12 16:27:43.068 11954 INFO nova.wsgi [-] osapi_compute listening on 0.0.0.0:8774
2013-12-12 16:27:43.068 11954 INFO nova.openstack.common.service [-] Starting 1 workers
2013-12-12 16:27:43.070 11954 INFO nova.openstack.common.service [-] Started child 12085
2013-12-12 16:27:43.082 11954 INFO nova.network.driver [-] Loading network driver 'nova.network.linux_net'
2013-12-12 16:27:43.088 11954 INFO nova.wsgi [-] metadata listening on 0.0.0.0:8775
2013-12-12 16:27:43.093 11954 INFO nova.openstack.common.service [-] Starting 1 workers
2013-12-12 16:27:43.095 11954 INFO nova.openstack.common.service [-] Started child 12086
2013-12-12 16:27:43.075 12085 INFO nova.osapi_compute.wsgi.server [-] (12085) wsgi starting up on http://0.0.0.0:8774/
2013-12-12 16:27:44.005 12086 INFO nova.metadata.wsgi.server [-] (12086) wsgi starting up on http://0.0.0.0:8775/
2013-12-12 16:29:32.864 12036 INFO nova.openstack.common.service [-] Caught SIGTERM, exiting
2013-12-12 16:29:32.864 12086 INFO nova.openstack.common.service [-] Caught SIGTERM, exiting
2013-12-12 16:29:32.864 12085 INFO nova.openstack.common.service [-] Caught SIGTERM, exiting
2013-12-12 16:29:32.864 12036 INFO nova.wsgi [-] Stopping WSGI server.
2013-12-12 16:29:32.864 12086 INFO nova.wsgi [-] Stopping WSGI server.
2013-12-12 16:29:32.865 12085 INFO nova.wsgi [-] Stopping WSGI server.
2013-12-12 16:29:32.867 11954 INFO nova.openstack.common.service [-] Caught SIGTERM, stopping children
2013-12-12 16:29:32.867 11954 INFO nova.openstack.common.service [-] Waiting on 3 children to exit
2013-12-12 16:29:32.868 11954 INFO nova.openstack.common.service [-] Child 12086 exited with status 1
2013-12-12 16:29:32.868 11954 INFO nova.openstack.common.service [-] Child 12036 exited with status 1
2013-12-12 16:29:32.869 11954 INFO nova.openstack.common.service [-] Child 12085 exited with status 1

2. Nova-cert-
2013-12-12 16:27:42.600 11994 INFO nova.openstack.common.periodic_task [-] Skipping periodic task _periodic_update_dns because its interval is negative
2013-12-12 16:27:42.634 11994 AUDIT nova.service [-] Starting cert node (version 2013.2)
2013-12-12 16:27:43.449 11994 INFO nova.openstack.common.rpc.common [req-cc5378ca-5534-48ac-855b-af1851b84cbd None None] Connected to AMQP server on localhost:5672
2013-12-12 16:29:32.967 11994 INFO nova.openstack.common.service [-] Caught SIGTERM, exiting

3. nova-schduler
2013-12-12 16:27:45.000 12184 INFO nova.openstack.common.periodic_task [-] Skipping periodic task _periodic_update_dns because its interval is negative
2013-12-12 16:27:45.036 12184 AUDIT nova.service [-] Starting scheduler node (version 2013.2)
2013-12-12 16:27:45.558 12184 INFO nova.openstack.common.rpc.common [req-8f23aa08-5980-4d9b-b601-fe7dce98a211 None None] Connected to AMQP server on localhost:5672
2013-12-12 16:29:33.306 12184 INFO nova.openstack.common.service [-] Caught SIGTERM, exiting

4. nova-api-metadata
2013-12-12 16:52:58.093 7748 INFO nova.network.driver [-] Loading network driver 'nova.network.linux_net'
2013-12-12 16:52:58.221 7748 INFO nova.openstack.common.periodic_task [-] Skipping periodic task _periodic_update_dns because its interval is negative
2013-12-12 16:52:58.223 7748 INFO nova.wsgi [-] metadata listening on 0.0.0.0:8775
2013-12-12 16:52:58.331 7748 INFO nova.metadata.wsgi.server [-] (7748) wsgi starting up on http://0.0.0.0:8775/
2013-12-12 16:53:02.995 7748 INFO nova.openstack.common.service [-] Caught SIGTERM, exiting
2013-12-12 16:53:03.045 7748 INFO nova.wsgi [-] WSGI server has stopped.

5. nova-compute
2013-12-12 16:53:01.193 7884 INFO nova.openstack.common.periodic_task [-] Skipping periodic task _periodic_update_dns because its interval is negative
2013-12-12 16:53:01.220 7884 INFO nova.virt.driver [-] Loading compute driver 'libvirt.LibvirtDriver'
2013-12-12 16:53:01.274 7884 ERROR nova.openstack.common.rpc.common [req-349ea2aa-763b-4fef-ab67-a79900bb5cce None None] AMQP server on localhost:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 1 seconds.
2013-12-12 16:53:02.276 7884 INFO nova.openstack.common.rpc.common [req-349ea2aa-763b-4fef-ab67-a79900bb5cce None None] Reconnecting to AMQP server on localhost:5672
2013-12-12 16:53:02.285 7884 ERROR nova.openstack.common.rpc.common [req-349ea2aa-763b-4fef-ab67-a79900bb5cce None None] AMQP server on localhost:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 3 seconds.

6. nova-network
2013-12-12 16:52:58.981 7795 INFO nova.openstack.common.periodic_task [-] Skipping periodic task _periodic_update_dns because its interval is negative
2013-12-12 16:52:58.990 7795 INFO nova.network.driver [-] Loading network driver 'nova.network.linux_net'
2013-12-12 16:52:59.007 7795 AUDIT nova.service [-] Starting network node (version 2013.2)
2013-12-12 16:52:59.701 7795 ERROR nova.openstack.common.rpc.common [-] AMQP server on localhost:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 1 seconds.
2013-12-12 16:53:00.702 7795 INFO nova.openstack.common.rpc.common [-] Reconnecting to AMQP server on localhost:5672
2013-12-12 16:53:00.713 7795 ERROR nova.openstack.common.rpc.common [-] AMQP server on localhost:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 3 seconds.
2013-12-12 16:53:03.144 7795 INFO nova.openstack.common.service [-] Caught SIGTERM, exiting

Now the conf file-
nova.conf
[default]
logdir=/var/log/nova
state_path=/var/lib/nova
lock_path=/run/lock/nova
verbose=true
debug=true
api_paste_config=/etc/nova/api-paste.ini
compute_scheduler_driver=nova.scheduler.simple.SimpleScheduler
rabbit_host=192.168.80.199
rabbit_password=guest
nova_url=http://192.168.80.199:8774/v1.1/

#[database]
#connection=mysql://novauser:novapass@192.168.80.199/nova
root_helper=sudo nova-rootwrap /etc/nova/rootwrap.conf

# auth
use_deprecated_auth=false
auth_strategy=keystone

# imaging service
glance_api_servers=192.168.80.199:9292
image_service=nova.image.glance.GlanceImageService

# vnc configuration
vnc_enabled=true
novncproxy_base_url=http://10.1.1.245:6080/vnc_auto.html
novncproxy_port=6080
vncserver_proxyclient_address=192.168.80.200
vncserver_listen=0.0.0.0

# nova-network
libvirt_use_virtio_for_bridges = true
network_manager=nova.network.manager.FlatDHCPManager
dhcpbridge_flagfile=/etc/nova/nova.conf
dhcpbridge=/usr/bin/nova-dhcpbridge
public_interface=br100
flat_interface=eth1
flat_network_bridge=br100
fixed_range=192.168.80.0/24
flat_network_dhcp_start=192.168.80.0
network_size=256
force_dhcp_release = True
flat_injected=false
use_ipv6=false
multi_host=True

#metadata
metadata_host = 192.168.80.200
metadata_listen = 192.168.80.200
metadata_listen_port = 8775

please suggest me What is wrong and how can be fixed ?

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1260618

Title:
  nova-api,nova-cert,nova-network not shown after installation. please
  guide.

Status in OpenStack Compute (Nova):
  New

Bug description:
  Please help me - Issue description : I install havana ( with nova-network as i am very new to it) 1 controller node and 1 compute node. This topology and script i used to install grizzly which got installed. So very minor changes in script were made. Now for havana few minor changes i made in conf files -
  Nova.conf - [database] connection is used rather sql_connection
  Cinder - Same as above.

  Now when I launch script it runs and on controller service shown using nova-manage service list
  Binary                       Host                                 Zone             Status     State     Updated_At
  nova-conductor     control                              internal         enabled     :-)        2013-12-13 06:36:41
  nova-consoleauth control                               internal         enabled     :-)       2013-12-13 06:36:41
  nova-scheduler      control                              internal         enabled     :-)        2013-12-13 06:36:42
  root@control:/etc# ps aux | grep nova
  nova      1471  0.0  1.6  67772 ?        Ss   Dec12   0:35 /usr/bin/python /usr/bin/nova-consoleauth --config-file=/etc/nova/nova.conf
  nova      1472  0.0  1.6  67808 ?        Ss   Dec12   0:35 /usr/bin/python /usr/bin/nova-conductor --config-file=/etc/nova/nova.conf
  nova      1474  0.0  1.6  68416 ?        Ss   Dec12   0:40 /usr/bin/python /usr/bin/nova-scheduler --config-file=/etc/nova/nova.conf
  nova      1476  0.0  0.7  32276 ?        Ss   Dec12   0:12 /usr/bin/python /usr/bin/nova-novncproxy --config-file=/etc/nova/nova.conf
  root      4479  0.0  0.0  13588   912 pts/2    S+   12:08   0:00 grep --color=auto nova

  Same way on compute Node script runs well but nova-compute, nova-network,nova-meata-api does not start. When restarted service on controller node result is like below-
  root@control:/etc# cd /etc/init.d/; for i in $( ls nova-* ); do sudo service $i restart; done
  stop: Unknown instance: 
  nova-api start/running, process 4497
  stop: Unknown instance: 
  nova-cert start/running, process 4508
  nova-conductor stop/waiting
  nova-conductor start/running, process 4519
  nova-consoleauth stop/waiting
  nova-consoleauth start/running, process 4530
  nova-novncproxy stop/waiting
  nova-novncproxy start/running, process 4541
  nova-scheduler stop/waiting
  nova-scheduler start/running, process 4556
  so i find nova-api, nova-cert, nova-network,nova-metadata-api services are not running. Now the Log and Conf files- Log first

  1. nova-api -
  2013-12-12 16:27:43.068 11954 INFO nova.wsgi [-] osapi_compute listening on 0.0.0.0:8774
  2013-12-12 16:27:43.068 11954 INFO nova.openstack.common.service [-] Starting 1 workers
  2013-12-12 16:27:43.070 11954 INFO nova.openstack.common.service [-] Started child 12085
  2013-12-12 16:27:43.082 11954 INFO nova.network.driver [-] Loading network driver 'nova.network.linux_net'
  2013-12-12 16:27:43.088 11954 INFO nova.wsgi [-] metadata listening on 0.0.0.0:8775
  2013-12-12 16:27:43.093 11954 INFO nova.openstack.common.service [-] Starting 1 workers
  2013-12-12 16:27:43.095 11954 INFO nova.openstack.common.service [-] Started child 12086
  2013-12-12 16:27:43.075 12085 INFO nova.osapi_compute.wsgi.server [-] (12085) wsgi starting up on http://0.0.0.0:8774/
  2013-12-12 16:27:44.005 12086 INFO nova.metadata.wsgi.server [-] (12086) wsgi starting up on http://0.0.0.0:8775/
  2013-12-12 16:29:32.864 12036 INFO nova.openstack.common.service [-] Caught SIGTERM, exiting
  2013-12-12 16:29:32.864 12086 INFO nova.openstack.common.service [-] Caught SIGTERM, exiting
  2013-12-12 16:29:32.864 12085 INFO nova.openstack.common.service [-] Caught SIGTERM, exiting
  2013-12-12 16:29:32.864 12036 INFO nova.wsgi [-] Stopping WSGI server.
  2013-12-12 16:29:32.864 12086 INFO nova.wsgi [-] Stopping WSGI server.
  2013-12-12 16:29:32.865 12085 INFO nova.wsgi [-] Stopping WSGI server.
  2013-12-12 16:29:32.867 11954 INFO nova.openstack.common.service [-] Caught SIGTERM, stopping children
  2013-12-12 16:29:32.867 11954 INFO nova.openstack.common.service [-] Waiting on 3 children to exit
  2013-12-12 16:29:32.868 11954 INFO nova.openstack.common.service [-] Child 12086 exited with status 1
  2013-12-12 16:29:32.868 11954 INFO nova.openstack.common.service [-] Child 12036 exited with status 1
  2013-12-12 16:29:32.869 11954 INFO nova.openstack.common.service [-] Child 12085 exited with status 1

  2. Nova-cert-
  2013-12-12 16:27:42.600 11994 INFO nova.openstack.common.periodic_task [-] Skipping periodic task _periodic_update_dns because its interval is negative
  2013-12-12 16:27:42.634 11994 AUDIT nova.service [-] Starting cert node (version 2013.2)
  2013-12-12 16:27:43.449 11994 INFO nova.openstack.common.rpc.common [req-cc5378ca-5534-48ac-855b-af1851b84cbd None None] Connected to AMQP server on localhost:5672
  2013-12-12 16:29:32.967 11994 INFO nova.openstack.common.service [-] Caught SIGTERM, exiting

  3. nova-schduler
  2013-12-12 16:27:45.000 12184 INFO nova.openstack.common.periodic_task [-] Skipping periodic task _periodic_update_dns because its interval is negative
  2013-12-12 16:27:45.036 12184 AUDIT nova.service [-] Starting scheduler node (version 2013.2)
  2013-12-12 16:27:45.558 12184 INFO nova.openstack.common.rpc.common [req-8f23aa08-5980-4d9b-b601-fe7dce98a211 None None] Connected to AMQP server on localhost:5672
  2013-12-12 16:29:33.306 12184 INFO nova.openstack.common.service [-] Caught SIGTERM, exiting

  4. nova-api-metadata
  2013-12-12 16:52:58.093 7748 INFO nova.network.driver [-] Loading network driver 'nova.network.linux_net'
  2013-12-12 16:52:58.221 7748 INFO nova.openstack.common.periodic_task [-] Skipping periodic task _periodic_update_dns because its interval is negative
  2013-12-12 16:52:58.223 7748 INFO nova.wsgi [-] metadata listening on 0.0.0.0:8775
  2013-12-12 16:52:58.331 7748 INFO nova.metadata.wsgi.server [-] (7748) wsgi starting up on http://0.0.0.0:8775/
  2013-12-12 16:53:02.995 7748 INFO nova.openstack.common.service [-] Caught SIGTERM, exiting
  2013-12-12 16:53:03.045 7748 INFO nova.wsgi [-] WSGI server has stopped.

  5. nova-compute
  2013-12-12 16:53:01.193 7884 INFO nova.openstack.common.periodic_task [-] Skipping periodic task _periodic_update_dns because its interval is negative
  2013-12-12 16:53:01.220 7884 INFO nova.virt.driver [-] Loading compute driver 'libvirt.LibvirtDriver'
  2013-12-12 16:53:01.274 7884 ERROR nova.openstack.common.rpc.common [req-349ea2aa-763b-4fef-ab67-a79900bb5cce None None] AMQP server on localhost:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 1 seconds.
  2013-12-12 16:53:02.276 7884 INFO nova.openstack.common.rpc.common [req-349ea2aa-763b-4fef-ab67-a79900bb5cce None None] Reconnecting to AMQP server on localhost:5672
  2013-12-12 16:53:02.285 7884 ERROR nova.openstack.common.rpc.common [req-349ea2aa-763b-4fef-ab67-a79900bb5cce None None] AMQP server on localhost:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 3 seconds.

  6. nova-network
  2013-12-12 16:52:58.981 7795 INFO nova.openstack.common.periodic_task [-] Skipping periodic task _periodic_update_dns because its interval is negative
  2013-12-12 16:52:58.990 7795 INFO nova.network.driver [-] Loading network driver 'nova.network.linux_net'
  2013-12-12 16:52:59.007 7795 AUDIT nova.service [-] Starting network node (version 2013.2)
  2013-12-12 16:52:59.701 7795 ERROR nova.openstack.common.rpc.common [-] AMQP server on localhost:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 1 seconds.
  2013-12-12 16:53:00.702 7795 INFO nova.openstack.common.rpc.common [-] Reconnecting to AMQP server on localhost:5672
  2013-12-12 16:53:00.713 7795 ERROR nova.openstack.common.rpc.common [-] AMQP server on localhost:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 3 seconds.
  2013-12-12 16:53:03.144 7795 INFO nova.openstack.common.service [-] Caught SIGTERM, exiting

  Now the conf file-
  nova.conf
  [default]
  logdir=/var/log/nova
  state_path=/var/lib/nova
  lock_path=/run/lock/nova
  verbose=true
  debug=true
  api_paste_config=/etc/nova/api-paste.ini
  compute_scheduler_driver=nova.scheduler.simple.SimpleScheduler
  rabbit_host=192.168.80.199
  rabbit_password=guest
  nova_url=http://192.168.80.199:8774/v1.1/

  #[database]
  #connection=mysql://novauser:novapass@192.168.80.199/nova
  root_helper=sudo nova-rootwrap /etc/nova/rootwrap.conf

  # auth
  use_deprecated_auth=false
  auth_strategy=keystone

  # imaging service
  glance_api_servers=192.168.80.199:9292
  image_service=nova.image.glance.GlanceImageService

  # vnc configuration
  vnc_enabled=true
  novncproxy_base_url=http://10.1.1.245:6080/vnc_auto.html
  novncproxy_port=6080
  vncserver_proxyclient_address=192.168.80.200
  vncserver_listen=0.0.0.0

  # nova-network
  libvirt_use_virtio_for_bridges = true
  network_manager=nova.network.manager.FlatDHCPManager
  dhcpbridge_flagfile=/etc/nova/nova.conf
  dhcpbridge=/usr/bin/nova-dhcpbridge
  public_interface=br100
  flat_interface=eth1
  flat_network_bridge=br100
  fixed_range=192.168.80.0/24
  flat_network_dhcp_start=192.168.80.0
  network_size=256
  force_dhcp_release = True
  flat_injected=false
  use_ipv6=false
  multi_host=True

  #metadata
  metadata_host = 192.168.80.200
  metadata_listen = 192.168.80.200
  metadata_listen_port = 8775

  please suggest me What is wrong and how can be fixed ?

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1260618/+subscriptions
Follow ups

[Bug 1260618] Re: nova-api, nova-cert, nova-network not shown after installation. please guide.
From: Abhishek Chanda, 2013-12-13
[Bug 1260618] [NEW] nova-api, nova-cert, nova-network not shown after installation. please guide.
From: yashjit, 2013-12-13
References

[Bug 1260618] [NEW] nova-api, nova-cert, nova-network not shown after installation. please guide.
From: yashjit, 2013-12-13