← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #36710

[Bug 1483321] [NEW] Kilo nova-compute unable to register with Juno nova-conductor

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

When deploying a compute node running Kilo against a control node
running Juno, nova-compute fails to start with the following exceptions
thrown by nova-conductor:

2015-08-10 16:56:02.236 977 ERROR oslo.messaging.rpc.dispatcher [req-1d9be6ed-7b53-4dc8-bb3a-995a7ca7e359 ] Exception during message handling: Version 1.12 of Service is not supported
2015-08-10 16:56:02.236 977 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last):
2015-08-10 16:56:02.236 977 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 134, in _dispatch_and_reply
2015-08-10 16:56:02.236 977 TRACE oslo.messaging.rpc.dispatcher     incoming.message))
2015-08-10 16:56:02.236 977 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 177, in _dispatch
2015-08-10 16:56:02.236 977 TRACE oslo.messaging.rpc.dispatcher     return self._do_dispatch(endpoint, method, ctxt, args)
2015-08-10 16:56:02.236 977 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 123, in _do_dispatch
2015-08-10 16:56:02.236 977 TRACE oslo.messaging.rpc.dispatcher     result = getattr(endpoint, method)(ctxt, **new_args)
2015-08-10 16:56:02.236 977 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/conductor/manager.py", line 408, in object_class_action
2015-08-10 16:56:02.236 977 TRACE oslo.messaging.rpc.dispatcher     objver)
2015-08-10 16:56:02.236 977 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/objects/base.py", line 288, in obj_class_from_name
2015-08-10 16:56:02.236 977 TRACE oslo.messaging.rpc.dispatcher     supported=latest_ver)
2015-08-10 16:56:02.236 977 TRACE oslo.messaging.rpc.dispatcher IncompatibleObjectVersion: Version 1.12 of Service is not supported
2015-08-10 16:56:02.236 977 TRACE oslo.messaging.rpc.dispatcher 
2015-08-10 16:56:02.237 977 ERROR oslo.messaging._drivers.common [req-1d9be6ed-7b53-4dc8-bb3a-995a7ca7e359 ] Returning exception Version 1.12 of Service is not supported to caller
2015-08-10 16:56:02.237 977 ERROR oslo.messaging._drivers.common [req-1d9be6ed-7b53-4dc8-bb3a-995a7ca7e359 ] ['Traceback (most recent call last):\n', '  File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 134, in _dispatch_and_reply\n    incoming.message))\n', '  File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 177, in _dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 123, in _do_dispatch\n    result = getattr(endpoint, method)(ctxt, **new_args)\n', '  File "/usr/lib/python2.7/dist-packages/nova/conductor/manager.py", line 408, in object_class_action\n    objver)\n', '  File "/usr/lib/python2.7/dist-packages/nova/objects/base.py", line 288, in obj_class_from_name\n    supported=latest_ver)\n', 'IncompatibleObjectVersion: Version 1.12 of Service is not supported\n']
2015-08-10 16:56:04.212 976 WARNING nova.context [-] Arguments dropped when creating context: {u'read_only': False, u'domain': None, u'show_deleted': False, u'user_identity': u'- - - - -', u'project_domain': None, u'resource_uuid': None, u'user_domain': None}

2015-08-10 16:56:04.244 972 ERROR oslo.messaging.rpc.dispatcher [req-eb961fea-3b0a-4c34-b774-7c8f4312467f ] Exception during message handling: Version 1.16 of InstanceList is not supported
2015-08-10 16:56:04.244 972 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last):
2015-08-10 16:56:04.244 972 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 134, in _dispatch_and_reply
2015-08-10 16:56:04.244 972 TRACE oslo.messaging.rpc.dispatcher     incoming.message))
2015-08-10 16:56:04.244 972 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 177, in _dispatch
2015-08-10 16:56:04.244 972 TRACE oslo.messaging.rpc.dispatcher     return self._do_dispatch(endpoint, method, ctxt, args)
2015-08-10 16:56:04.244 972 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 123, in _do_dispatch
2015-08-10 16:56:04.244 972 TRACE oslo.messaging.rpc.dispatcher     result = getattr(endpoint, method)(ctxt, **new_args)
2015-08-10 16:56:04.244 972 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/conductor/manager.py", line 408, in object_class_action
2015-08-10 16:56:04.244 972 TRACE oslo.messaging.rpc.dispatcher     objver)
2015-08-10 16:56:04.244 972 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/objects/base.py", line 288, in obj_class_from_name
2015-08-10 16:56:04.244 972 TRACE oslo.messaging.rpc.dispatcher     supported=latest_ver)
2015-08-10 16:56:04.244 972 TRACE oslo.messaging.rpc.dispatcher IncompatibleObjectVersion: Version 1.16 of InstanceList is not supported
2015-08-10 16:56:04.244 972 TRACE oslo.messaging.rpc.dispatcher 
2015-08-10 16:56:04.246 972 ERROR oslo.messaging._drivers.common [req-eb961fea-3b0a-4c34-b774-7c8f4312467f ] Returning exception Version 1.16 of InstanceList is not supported to caller
2015-08-10 16:56:04.246 972 ERROR oslo.messaging._drivers.common [req-eb961fea-3b0a-4c34-b774-7c8f4312467f ] ['Traceback (most recent call last):\n', '  File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 134, in _dispatch_and_reply\n    incoming.message))\n', '  File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 177, in _dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 123, in _do_dispatch\n    result = getattr(endpoint, method)(ctxt, **new_args)\n', '  File "/usr/lib/python2.7/dist-packages/nova/conductor/manager.py", line 408, in object_class_action\n    objver)\n', '  File "/usr/lib/python2.7/dist-packages/nova/objects/base.py", line 288, in obj_class_from_name\n    supported=latest_ver)\n', 'IncompatibleObjectVersion: Version 1.16 of InstanceList is not supported\n']
2015-08-10 16:56:08.205 976 WARNING nova.context [-] Arguments dropped when creating context: {u'read_only': False, u'domain': None, u'show_deleted': False, u'user_identity': u'- - - - -', u'project_domain': None, u'resource_uuid': None, u'user_domain': None}

upgrade_levels is set to Juno on both the compute node and the control
node:

[upgrade_levels]
compute = juno
conductor=juno
scheduler=juno
cert=juno
network=juno
consoleauth=juno
console=juno

The situation is further complicated by the fact that the new compute
node is running CentOS 7.1, while the control node is running Ubuntu
14.04.  However, I believe I should be able to make this combination
work especially with the compatibility / upgrade_levels options set.

On the controller:

# dpkg -l | grep -i nova
ii  nova-api                             1:2014.2.2-0ubuntu1~cloud0            all          OpenStack Compute - API frontend
ii  nova-cert                            1:2014.2.2-0ubuntu1~cloud0            all          OpenStack Compute - certificate management
ii  nova-common                          1:2014.2.2-0ubuntu1~cloud0            all          OpenStack Compute - common files
ii  nova-conductor                       1:2014.2.2-0ubuntu1~cloud0            all          OpenStack Compute - conductor service
ii  nova-consoleauth                     1:2014.2.2-0ubuntu1~cloud0            all          OpenStack Compute - Console Authenticator
ii  nova-novncproxy                      1:2014.2.2-0ubuntu1~cloud0            all          OpenStack Compute - NoVNC proxy
ii  nova-scheduler                       1:2014.2.2-0ubuntu1~cloud0            all          OpenStack Compute - virtual machine scheduler
ii  python-nova                          1:2014.2.2-0ubuntu1~cloud0            all          OpenStack Compute Python libraries
ii  python-novaclient                    1:2.19.0-0ubuntu1~cloud0              all          client library for OpenStack Compute API

On the compute node:

# rpm -qa | grep -i nova
openstack-nova-common-2015.1.0-3.el7.noarch
openstack-nova-compute-2015.1.0-3.el7.noarch
python-novaclient-2.23.0-1.el7.noarch
python-nova-2015.1.0-3.el7.noarch

Expected result is for nova-compute to start and register itself via
nova-conductor.

Actual result is that nova-compute fails to start.

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1483321

Title:
  Kilo nova-compute unable to register with Juno nova-conductor

Status in OpenStack Compute (nova):
  New

Bug description:
  When deploying a compute node running Kilo against a control node
  running Juno, nova-compute fails to start with the following
  exceptions thrown by nova-conductor:

  2015-08-10 16:56:02.236 977 ERROR oslo.messaging.rpc.dispatcher [req-1d9be6ed-7b53-4dc8-bb3a-995a7ca7e359 ] Exception during message handling: Version 1.12 of Service is not supported
  2015-08-10 16:56:02.236 977 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last):
  2015-08-10 16:56:02.236 977 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 134, in _dispatch_and_reply
  2015-08-10 16:56:02.236 977 TRACE oslo.messaging.rpc.dispatcher     incoming.message))
  2015-08-10 16:56:02.236 977 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 177, in _dispatch
  2015-08-10 16:56:02.236 977 TRACE oslo.messaging.rpc.dispatcher     return self._do_dispatch(endpoint, method, ctxt, args)
  2015-08-10 16:56:02.236 977 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 123, in _do_dispatch
  2015-08-10 16:56:02.236 977 TRACE oslo.messaging.rpc.dispatcher     result = getattr(endpoint, method)(ctxt, **new_args)
  2015-08-10 16:56:02.236 977 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/conductor/manager.py", line 408, in object_class_action
  2015-08-10 16:56:02.236 977 TRACE oslo.messaging.rpc.dispatcher     objver)
  2015-08-10 16:56:02.236 977 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/objects/base.py", line 288, in obj_class_from_name
  2015-08-10 16:56:02.236 977 TRACE oslo.messaging.rpc.dispatcher     supported=latest_ver)
  2015-08-10 16:56:02.236 977 TRACE oslo.messaging.rpc.dispatcher IncompatibleObjectVersion: Version 1.12 of Service is not supported
  2015-08-10 16:56:02.236 977 TRACE oslo.messaging.rpc.dispatcher 
  2015-08-10 16:56:02.237 977 ERROR oslo.messaging._drivers.common [req-1d9be6ed-7b53-4dc8-bb3a-995a7ca7e359 ] Returning exception Version 1.12 of Service is not supported to caller
  2015-08-10 16:56:02.237 977 ERROR oslo.messaging._drivers.common [req-1d9be6ed-7b53-4dc8-bb3a-995a7ca7e359 ] ['Traceback (most recent call last):\n', '  File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 134, in _dispatch_and_reply\n    incoming.message))\n', '  File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 177, in _dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 123, in _do_dispatch\n    result = getattr(endpoint, method)(ctxt, **new_args)\n', '  File "/usr/lib/python2.7/dist-packages/nova/conductor/manager.py", line 408, in object_class_action\n    objver)\n', '  File "/usr/lib/python2.7/dist-packages/nova/objects/base.py", line 288, in obj_class_from_name\n    supported=latest_ver)\n', 'IncompatibleObjectVersion: Version 1.12 of Service is not supported\n']
  2015-08-10 16:56:04.212 976 WARNING nova.context [-] Arguments dropped when creating context: {u'read_only': False, u'domain': None, u'show_deleted': False, u'user_identity': u'- - - - -', u'project_domain': None, u'resource_uuid': None, u'user_domain': None}

  2015-08-10 16:56:04.244 972 ERROR oslo.messaging.rpc.dispatcher [req-eb961fea-3b0a-4c34-b774-7c8f4312467f ] Exception during message handling: Version 1.16 of InstanceList is not supported
  2015-08-10 16:56:04.244 972 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last):
  2015-08-10 16:56:04.244 972 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 134, in _dispatch_and_reply
  2015-08-10 16:56:04.244 972 TRACE oslo.messaging.rpc.dispatcher     incoming.message))
  2015-08-10 16:56:04.244 972 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 177, in _dispatch
  2015-08-10 16:56:04.244 972 TRACE oslo.messaging.rpc.dispatcher     return self._do_dispatch(endpoint, method, ctxt, args)
  2015-08-10 16:56:04.244 972 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 123, in _do_dispatch
  2015-08-10 16:56:04.244 972 TRACE oslo.messaging.rpc.dispatcher     result = getattr(endpoint, method)(ctxt, **new_args)
  2015-08-10 16:56:04.244 972 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/conductor/manager.py", line 408, in object_class_action
  2015-08-10 16:56:04.244 972 TRACE oslo.messaging.rpc.dispatcher     objver)
  2015-08-10 16:56:04.244 972 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/objects/base.py", line 288, in obj_class_from_name
  2015-08-10 16:56:04.244 972 TRACE oslo.messaging.rpc.dispatcher     supported=latest_ver)
  2015-08-10 16:56:04.244 972 TRACE oslo.messaging.rpc.dispatcher IncompatibleObjectVersion: Version 1.16 of InstanceList is not supported
  2015-08-10 16:56:04.244 972 TRACE oslo.messaging.rpc.dispatcher 
  2015-08-10 16:56:04.246 972 ERROR oslo.messaging._drivers.common [req-eb961fea-3b0a-4c34-b774-7c8f4312467f ] Returning exception Version 1.16 of InstanceList is not supported to caller
  2015-08-10 16:56:04.246 972 ERROR oslo.messaging._drivers.common [req-eb961fea-3b0a-4c34-b774-7c8f4312467f ] ['Traceback (most recent call last):\n', '  File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 134, in _dispatch_and_reply\n    incoming.message))\n', '  File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 177, in _dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 123, in _do_dispatch\n    result = getattr(endpoint, method)(ctxt, **new_args)\n', '  File "/usr/lib/python2.7/dist-packages/nova/conductor/manager.py", line 408, in object_class_action\n    objver)\n', '  File "/usr/lib/python2.7/dist-packages/nova/objects/base.py", line 288, in obj_class_from_name\n    supported=latest_ver)\n', 'IncompatibleObjectVersion: Version 1.16 of InstanceList is not supported\n']
  2015-08-10 16:56:08.205 976 WARNING nova.context [-] Arguments dropped when creating context: {u'read_only': False, u'domain': None, u'show_deleted': False, u'user_identity': u'- - - - -', u'project_domain': None, u'resource_uuid': None, u'user_domain': None}

  upgrade_levels is set to Juno on both the compute node and the control
  node:

  [upgrade_levels]
  compute = juno
  conductor=juno
  scheduler=juno
  cert=juno
  network=juno
  consoleauth=juno
  console=juno

  The situation is further complicated by the fact that the new compute
  node is running CentOS 7.1, while the control node is running Ubuntu
  14.04.  However, I believe I should be able to make this combination
  work especially with the compatibility / upgrade_levels options set.

  On the controller:

  # dpkg -l | grep -i nova
  ii  nova-api                             1:2014.2.2-0ubuntu1~cloud0            all          OpenStack Compute - API frontend
  ii  nova-cert                            1:2014.2.2-0ubuntu1~cloud0            all          OpenStack Compute - certificate management
  ii  nova-common                          1:2014.2.2-0ubuntu1~cloud0            all          OpenStack Compute - common files
  ii  nova-conductor                       1:2014.2.2-0ubuntu1~cloud0            all          OpenStack Compute - conductor service
  ii  nova-consoleauth                     1:2014.2.2-0ubuntu1~cloud0            all          OpenStack Compute - Console Authenticator
  ii  nova-novncproxy                      1:2014.2.2-0ubuntu1~cloud0            all          OpenStack Compute - NoVNC proxy
  ii  nova-scheduler                       1:2014.2.2-0ubuntu1~cloud0            all          OpenStack Compute - virtual machine scheduler
  ii  python-nova                          1:2014.2.2-0ubuntu1~cloud0            all          OpenStack Compute Python libraries
  ii  python-novaclient                    1:2.19.0-0ubuntu1~cloud0              all          client library for OpenStack Compute API

  On the compute node:

  # rpm -qa | grep -i nova
  openstack-nova-common-2015.1.0-3.el7.noarch
  openstack-nova-compute-2015.1.0-3.el7.noarch
  python-novaclient-2.23.0-1.el7.noarch
  python-nova-2015.1.0-3.el7.noarch

  Expected result is for nova-compute to start and register itself via
  nova-conductor.

  Actual result is that nova-compute fails to start.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1483321/+subscriptions

Follow ups

  Thread Previous • Date Previous • • Thread Next