yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1283788] [NEW] nova service-list raises "Error setting Service.id" with nova-cell installed

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Joe Hakim Rahme <joe.hakim.rahme@xxxxxxxxxxxx>
Date: Sun, 23 Feb 2014 18:32:10 -0000
Reply-to: Bug 1283788 <1283788@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

Installed all-in-one mode by devstack with the following localrc:

      ADMIN_PASSWORD=passw0rd
      MYSQL_PASSWORD=root
      RABBIT_PASSWORD=stackqueue
      SERVICE_PASSWORD=$ADMIN_PASSWORD
      LOGFILE=$DEST/logs/stack.sh.log
      LOGDAYS=2

      CINDER_BRANCH=stable/havana
      GLANCE_BRANCH=stable/havana
      HORIZON_BRANCH=stable/havana
      KEYSTONE_BRANCH=stable/havana
      KEYSTONECLIENT_BRANCH=stable/havana
      NOVA_BRANCH=stable/havana
      NOVACLIENT_BRANCH=stable/havana
      NEUTRON_BRANCH=stable/havana
      SWIFT_BRANCH=stable/havana
     
      SWIFT_HASH=66a3d6b56c1f479c8b4e70ab5c2000f5
      SWIFT_REPLICAS=1

      SWIFT_DATA_DIR=$DEST/data


      FLOATING_RANGE=192.168.0.224/27
      FIXED_RANGE=10.10.1.0/24
      FIXED_NETWORK_SIZE=256
      FLAT_INTERFACE=eth1

      disable_service n-net
      enable_service q-svc
      enable_service q-agt
      enable_service q-dhcp
      enable_service q-l3
      enable_service q-lbaas
      enable_service q-meta
      enable_service q-metering
      enable_service neutron
      # Optional, to enable tempest configuration as part of DevStack
      enable_service tempest
      enable_service heat h-api h-api-cfn h-api-cw h-eng
      enable_service n-cell
      SERVICE_TOKEN=passw0rd

After finished the installation, can use nova boot to boot instance, but if use "nova service-list" or "nova host-list"
show the services and hosts, it will return an error msg:

    2013-12-30 07:56:14.910 ERROR object [req-005a967f-97be-4401-8afb-c0c562402405 admin admin] Error setting Service.id
    2013-12-30 07:56:14.910 TRACE object Traceback (most recent call last):
    2013-12-30 07:56:14.910 TRACE object   File "/opt/stack/nova/nova/objects/base.py", line 70, in setter
    2013-12-30 07:56:14.910 TRACE object     field.coerce(self, name, value))
    2013-12-30 07:56:14.910 TRACE object   File "/opt/stack/nova/nova/objects/fields.py", line 166, in coerce
    2013-12-30 07:56:14.910 TRACE object     return self._type.coerce(obj, attr, value)
    2013-12-30 07:56:14.910 TRACE object   File "/opt/stack/nova/nova/objects/fields.py", line 231, in coerce
    2013-12-30 07:56:14.910 TRACE object     return int(value)
    2013-12-30 07:56:14.910 TRACE object ValueError: invalid literal for int() with base 10: 'region!child@1'
    2013-12-30 07:56:14.910 TRACE object 
    2013-12-30 07:56:14.917 ERROR nova.api.openstack [req-005a967f-97be-4401-8afb-c0c562402405 admin admin] Caught error: invalid literal for int() with base 10: 'region!child@1'
    2013-12-30 07:56:14.917 TRACE nova.api.openstack Traceback (most recent call last):
    2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/opt/stack/nova/nova/api/openstack/__init__.py", line 121, in __call__
    2013-12-30 07:56:14.917 TRACE nova.api.openstack     return req.get_response(self.application)
    2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/request.py", line 1296, in send
    2013-12-30 07:56:14.917 TRACE nova.api.openstack     application, catch_exc_info=False)
    2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/request.py", line 1260, in call_application
    2013-12-30 07:56:14.917 TRACE nova.api.openstack     app_iter = application(self.environ, start_response)
    2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
    2013-12-30 07:56:14.917 TRACE nova.api.openstack     return resp(environ, start_response)
    2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/opt/stack/python-keystoneclient/keystoneclient/middleware/auth_token.py", line 581, in __call__
    2013-12-30 07:56:14.917 TRACE nova.api.openstack     return self.app(env, start_response)
    2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
    2013-12-30 07:56:14.917 TRACE nova.api.openstack     return resp(environ, start_response)
    2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
    2013-12-30 07:56:14.917 TRACE nova.api.openstack     return resp(environ, start_response)
    2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/routes/middleware.py", line 131, in __call__
    2013-12-30 07:56:14.917 TRACE nova.api.openstack     response = self.app(environ, start_response)
    2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
    2013-12-30 07:56:14.917 TRACE nova.api.openstack     return resp(environ, start_response)
    2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__
    2013-12-30 07:56:14.917 TRACE nova.api.openstack     resp = self.call_func(req, *args, **self.kwargs)
    2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func
    2013-12-30 07:56:14.917 TRACE nova.api.openstack     return self.func(req, *args, **kwargs)
    2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 930, in __call__
    2013-12-30 07:56:14.917 TRACE nova.api.openstack     content_type, body, accept)
    2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 992, in _process_stack
    2013-12-30 07:56:14.917 TRACE nova.api.openstack     action_result = self.dispatch(meth, request, action_args)
    2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 1073, in dispatch
    2013-12-30 07:56:14.917 TRACE nova.api.openstack     return method(req=request, **action_args)
    2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/opt/stack/nova/nova/api/openstack/compute/contrib/services.py", line 139, in index
    2013-12-30 07:56:14.917 TRACE nova.api.openstack     services = self._get_services_list(req, detailed)
    2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/opt/stack/nova/nova/api/openstack/compute/contrib/services.py", line 117, in _get_services_list
    2013-12-30 07:56:14.917 TRACE nova.api.openstack     services = self._get_services(req)
    2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/opt/stack/nova/nova/api/openstack/compute/contrib/services.py", line 86, in _get_services
    2013-12-30 07:56:14.917 TRACE nova.api.openstack     context, set_zones=True)
    2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/opt/stack/nova/nova/compute/cells_api.py", line 464, in service_get_all
    2013-12-30 07:56:14.917 TRACE nova.api.openstack     services)
    2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/opt/stack/nova/nova/objects/base.py", line 596, in obj_make_list
    2013-12-30 07:56:14.917 TRACE nova.api.openstack     **extra_args)
    2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/opt/stack/nova/nova/objects/service.py", line 73, in _from_db_object
    2013-12-30 07:56:14.917 TRACE nova.api.openstack     service[key] = db_service[key]
    2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/opt/stack/nova/nova/objects/base.py", line 389, in __setitem__
    2013-12-30 07:56:14.917 TRACE nova.api.openstack     setattr(self, name, value)
    2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/opt/stack/nova/nova/objects/base.py", line 70, in setter
    2013-12-30 07:56:14.917 TRACE nova.api.openstack     field.coerce(self, name, value))
    2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/opt/stack/nova/nova/objects/fields.py", line 166, in coerce
    2013-12-30 07:56:14.917 TRACE nova.api.openstack     return self._type.coerce(obj, attr, value)
    2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/opt/stack/nova/nova/objects/fields.py", line 231, in coerce
    2013-12-30 07:56:14.917 TRACE nova.api.openstack     return int(value)
    2013-12-30 07:56:14.917 TRACE nova.api.openstack ValueError: invalid literal for int() with base 10: 'region!child@1'
    2013-12-30 07:56:14.917 TRACE nova.api.openstack 
       
I debugged the code, found the value of "db_service[key]" is "region!child@1", and it was created by the method add_cell_to_compute_node which is from "nova/cells/utils.py" .
I also tried "nova hypervisor-list", and got the following result:

    +----------------------+--------------------------------+
    | ID                   | Hypervisor hostname  |
    +----------------------+--------------------------------+
    | region!child@1 | devstack                   |
    +----------------------+--------------------------------+

The ID is also not an integer, which is different with not installed
n-cell one.

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1283788

Title:
  nova service-list raises "Error setting Service.id" with nova-cell
  installed

Status in OpenStack Compute (Nova):
  New

Bug description:
  Installed all-in-one mode by devstack with the following localrc:

        ADMIN_PASSWORD=passw0rd
        MYSQL_PASSWORD=root
        RABBIT_PASSWORD=stackqueue
        SERVICE_PASSWORD=$ADMIN_PASSWORD
        LOGFILE=$DEST/logs/stack.sh.log
        LOGDAYS=2

        CINDER_BRANCH=stable/havana
        GLANCE_BRANCH=stable/havana
        HORIZON_BRANCH=stable/havana
        KEYSTONE_BRANCH=stable/havana
        KEYSTONECLIENT_BRANCH=stable/havana
        NOVA_BRANCH=stable/havana
        NOVACLIENT_BRANCH=stable/havana
        NEUTRON_BRANCH=stable/havana
        SWIFT_BRANCH=stable/havana
       
        SWIFT_HASH=66a3d6b56c1f479c8b4e70ab5c2000f5
        SWIFT_REPLICAS=1

        SWIFT_DATA_DIR=$DEST/data

  
        FLOATING_RANGE=192.168.0.224/27
        FIXED_RANGE=10.10.1.0/24
        FIXED_NETWORK_SIZE=256
        FLAT_INTERFACE=eth1

        disable_service n-net
        enable_service q-svc
        enable_service q-agt
        enable_service q-dhcp
        enable_service q-l3
        enable_service q-lbaas
        enable_service q-meta
        enable_service q-metering
        enable_service neutron
        # Optional, to enable tempest configuration as part of DevStack
        enable_service tempest
        enable_service heat h-api h-api-cfn h-api-cw h-eng
        enable_service n-cell
        SERVICE_TOKEN=passw0rd

  After finished the installation, can use nova boot to boot instance, but if use "nova service-list" or "nova host-list"
  show the services and hosts, it will return an error msg:

      2013-12-30 07:56:14.910 ERROR object [req-005a967f-97be-4401-8afb-c0c562402405 admin admin] Error setting Service.id
      2013-12-30 07:56:14.910 TRACE object Traceback (most recent call last):
      2013-12-30 07:56:14.910 TRACE object   File "/opt/stack/nova/nova/objects/base.py", line 70, in setter
      2013-12-30 07:56:14.910 TRACE object     field.coerce(self, name, value))
      2013-12-30 07:56:14.910 TRACE object   File "/opt/stack/nova/nova/objects/fields.py", line 166, in coerce
      2013-12-30 07:56:14.910 TRACE object     return self._type.coerce(obj, attr, value)
      2013-12-30 07:56:14.910 TRACE object   File "/opt/stack/nova/nova/objects/fields.py", line 231, in coerce
      2013-12-30 07:56:14.910 TRACE object     return int(value)
      2013-12-30 07:56:14.910 TRACE object ValueError: invalid literal for int() with base 10: 'region!child@1'
      2013-12-30 07:56:14.910 TRACE object 
      2013-12-30 07:56:14.917 ERROR nova.api.openstack [req-005a967f-97be-4401-8afb-c0c562402405 admin admin] Caught error: invalid literal for int() with base 10: 'region!child@1'
      2013-12-30 07:56:14.917 TRACE nova.api.openstack Traceback (most recent call last):
      2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/opt/stack/nova/nova/api/openstack/__init__.py", line 121, in __call__
      2013-12-30 07:56:14.917 TRACE nova.api.openstack     return req.get_response(self.application)
      2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/request.py", line 1296, in send
      2013-12-30 07:56:14.917 TRACE nova.api.openstack     application, catch_exc_info=False)
      2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/request.py", line 1260, in call_application
      2013-12-30 07:56:14.917 TRACE nova.api.openstack     app_iter = application(self.environ, start_response)
      2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
      2013-12-30 07:56:14.917 TRACE nova.api.openstack     return resp(environ, start_response)
      2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/opt/stack/python-keystoneclient/keystoneclient/middleware/auth_token.py", line 581, in __call__
      2013-12-30 07:56:14.917 TRACE nova.api.openstack     return self.app(env, start_response)
      2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
      2013-12-30 07:56:14.917 TRACE nova.api.openstack     return resp(environ, start_response)
      2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
      2013-12-30 07:56:14.917 TRACE nova.api.openstack     return resp(environ, start_response)
      2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/routes/middleware.py", line 131, in __call__
      2013-12-30 07:56:14.917 TRACE nova.api.openstack     response = self.app(environ, start_response)
      2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
      2013-12-30 07:56:14.917 TRACE nova.api.openstack     return resp(environ, start_response)
      2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__
      2013-12-30 07:56:14.917 TRACE nova.api.openstack     resp = self.call_func(req, *args, **self.kwargs)
      2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func
      2013-12-30 07:56:14.917 TRACE nova.api.openstack     return self.func(req, *args, **kwargs)
      2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 930, in __call__
      2013-12-30 07:56:14.917 TRACE nova.api.openstack     content_type, body, accept)
      2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 992, in _process_stack
      2013-12-30 07:56:14.917 TRACE nova.api.openstack     action_result = self.dispatch(meth, request, action_args)
      2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 1073, in dispatch
      2013-12-30 07:56:14.917 TRACE nova.api.openstack     return method(req=request, **action_args)
      2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/opt/stack/nova/nova/api/openstack/compute/contrib/services.py", line 139, in index
      2013-12-30 07:56:14.917 TRACE nova.api.openstack     services = self._get_services_list(req, detailed)
      2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/opt/stack/nova/nova/api/openstack/compute/contrib/services.py", line 117, in _get_services_list
      2013-12-30 07:56:14.917 TRACE nova.api.openstack     services = self._get_services(req)
      2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/opt/stack/nova/nova/api/openstack/compute/contrib/services.py", line 86, in _get_services
      2013-12-30 07:56:14.917 TRACE nova.api.openstack     context, set_zones=True)
      2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/opt/stack/nova/nova/compute/cells_api.py", line 464, in service_get_all
      2013-12-30 07:56:14.917 TRACE nova.api.openstack     services)
      2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/opt/stack/nova/nova/objects/base.py", line 596, in obj_make_list
      2013-12-30 07:56:14.917 TRACE nova.api.openstack     **extra_args)
      2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/opt/stack/nova/nova/objects/service.py", line 73, in _from_db_object
      2013-12-30 07:56:14.917 TRACE nova.api.openstack     service[key] = db_service[key]
      2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/opt/stack/nova/nova/objects/base.py", line 389, in __setitem__
      2013-12-30 07:56:14.917 TRACE nova.api.openstack     setattr(self, name, value)
      2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/opt/stack/nova/nova/objects/base.py", line 70, in setter
      2013-12-30 07:56:14.917 TRACE nova.api.openstack     field.coerce(self, name, value))
      2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/opt/stack/nova/nova/objects/fields.py", line 166, in coerce
      2013-12-30 07:56:14.917 TRACE nova.api.openstack     return self._type.coerce(obj, attr, value)
      2013-12-30 07:56:14.917 TRACE nova.api.openstack   File "/opt/stack/nova/nova/objects/fields.py", line 231, in coerce
      2013-12-30 07:56:14.917 TRACE nova.api.openstack     return int(value)
      2013-12-30 07:56:14.917 TRACE nova.api.openstack ValueError: invalid literal for int() with base 10: 'region!child@1'
      2013-12-30 07:56:14.917 TRACE nova.api.openstack 
         
  I debugged the code, found the value of "db_service[key]" is "region!child@1", and it was created by the method add_cell_to_compute_node which is from "nova/cells/utils.py" .
  I also tried "nova hypervisor-list", and got the following result:

      +----------------------+--------------------------------+
      | ID                   | Hypervisor hostname  |
      +----------------------+--------------------------------+
      | region!child@1 | devstack                   |
      +----------------------+--------------------------------+

  The ID is also not an integer, which is different with not installed
  n-cell one.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1283788/+subscriptions
Follow ups

[Bug 1283788] [NEW] nova service-list raises "Error setting Service.id" with nova-cell installed
From: Joe Hakim Rahme, 2014-02-23
References

[Bug 1283788] [NEW] nova service-list raises "Error setting Service.id" with nova-cell installed
From: Joe Hakim Rahme, 2014-02-23