yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #59011
[Bug 1644186] [NEW] nova service performs wrong update from version 15 to 9
Public bug reported:
We are running openstack mitaka out of clean installation (no upgrade).
All was functioning correctly until recently nova started to perform updates like this
2016-11-23 10:48:09.065 17498 INFO nova.service [req-c91ec40f-2dc2 - - -
- -] Updating service version for nova-conductor on nova-tlc01.specs.de
from 15 to 9
The full error log from /var/log/nova/nova-conductor.log is looks like
2016-11-23 10:48:08.967 17499 INFO nova.service [-] Starting conductor node (version 13.1.2)
2016-11-23 10:48:09.065 17498 INFO nova.service [req-c91ec40f-2dc2- - - - - -] Updating service version for nova-conductor on nova-tlc01.specs.de from 15 to 9
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service [req-c91ec40f-2dc2-4ab0-ad61-1ab1f9d7bbc9 - - - - -] Error starting thread.
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service Traceback (most recent call last):
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/oslo_service/service.py", line 680, in run_service
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service service.start()
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/nova/service.py", line 195, in start
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service _update_service_ref(self.service_ref)
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/nova/service.py", line 151, in _update_service_ref
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service service.save()
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 223, in wrapper
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service return fn(self, *args, **kwargs)
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/nova/objects/service.py", line 291, in save
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service self._check_minimum_version()
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/nova/objects/service.py", line 275, in _check_minimum_version
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service minver=minver)
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service ServiceTooOld: This service is older (v9) than the minimum (v15) version of the rest of the deployment. Unable to continue.
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service
I checked the changes in recent versions of nova/service.py
https://fossies.org/diffs/nova/13.1.1_vs_13.1.2/nova/service.py-diff.html
Not sure, but looks like the problem is in this function under
nova/service.py
def _update_service_ref(service):
if service.version != service_obj.SERVICE_VERSION:
LOG.info(_LI('Updating service version for %(binary)s on '
'%(host)s from %(old)i to %(new)i'),
{'binary': service.binary,
'host': service.host,
'old': service.version,
'new': service_obj.SERVICE_VERSION})
service.version = service_obj.SERVICE_VERSION
service.save()
** Affects: nova
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1644186
Title:
nova service performs wrong update from version 15 to 9
Status in OpenStack Compute (nova):
New
Bug description:
We are running openstack mitaka out of clean installation (no upgrade).
All was functioning correctly until recently nova started to perform updates like this
2016-11-23 10:48:09.065 17498 INFO nova.service [req-c91ec40f-2dc2 - -
- - -] Updating service version for nova-conductor on nova-
tlc01.specs.de from 15 to 9
The full error log from /var/log/nova/nova-conductor.log is looks like
2016-11-23 10:48:08.967 17499 INFO nova.service [-] Starting conductor node (version 13.1.2)
2016-11-23 10:48:09.065 17498 INFO nova.service [req-c91ec40f-2dc2- - - - - -] Updating service version for nova-conductor on nova-tlc01.specs.de from 15 to 9
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service [req-c91ec40f-2dc2-4ab0-ad61-1ab1f9d7bbc9 - - - - -] Error starting thread.
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service Traceback (most recent call last):
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/oslo_service/service.py", line 680, in run_service
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service service.start()
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/nova/service.py", line 195, in start
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service _update_service_ref(self.service_ref)
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/nova/service.py", line 151, in _update_service_ref
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service service.save()
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 223, in wrapper
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service return fn(self, *args, **kwargs)
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/nova/objects/service.py", line 291, in save
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service self._check_minimum_version()
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/nova/objects/service.py", line 275, in _check_minimum_version
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service minver=minver)
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service ServiceTooOld: This service is older (v9) than the minimum (v15) version of the rest of the deployment. Unable to continue.
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service
I checked the changes in recent versions of nova/service.py
https://fossies.org/diffs/nova/13.1.1_vs_13.1.2/nova/service.py-diff.html
Not sure, but looks like the problem is in this function under
nova/service.py
def _update_service_ref(service):
if service.version != service_obj.SERVICE_VERSION:
LOG.info(_LI('Updating service version for %(binary)s on '
'%(host)s from %(old)i to %(new)i'),
{'binary': service.binary,
'host': service.host,
'old': service.version,
'new': service_obj.SERVICE_VERSION})
service.version = service_obj.SERVICE_VERSION
service.save()
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1644186/+subscriptions
Follow ups