yahoo-eng-team team mailing list archive

[Apoorva Deshpande <apps.desh@xxxxxxxxx>]

The issue was a breaking change in Cinder. Symptoms of the failure were
different than what are mentioned earlier. Marking it as invalid

** Changed in: nova
       Status: Incomplete => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1703049

Title:
  Cannot update service status on host<host> since it is not
  registered.: ComputeHostNotFound_Remote: Compute host<host> could not
  be found.

Status in OpenStack Compute (nova):
  Invalid

Bug description:
  Cinder CI is failing.

  http://openstack-ci.tintri.com/tintri/refs-
  changes-35-481735-1/logs/screen-n-cpu.log.txt

  >>>

  
  2017-07-07 14:04:32.357 12955 DEBUG nova.virt.libvirt.host [-] Registering for connection events: <nova.virt.libvirt.host.Host object at 0x7fd7000d3910> _get_new_connection /opt/stack/nova/nova/virt/libvirt/host.py:382
  2017-07-07 14:04:32.358 12955 INFO nova.virt.libvirt.driver [-] Connection event '1' reason 'None'
  2017-07-07 14:04:32.390 12955 WARNING nova.virt.libvirt.driver [req-5d828b59-c72b-4aaf-b1de-23c7bca74818 - -] Cannot update service status on host "303-openstack-test2" since it is not registered.: ComputeHostNotFound_Remote: Compute host 303-openstack-test2 could not be found.
  Traceback (most recent call last):

    File "/opt/stack/nova/nova/conductor/manager.py", line 123, in _object_dispatch
      return getattr(target, method)(*args, **kwargs)

    File "/usr/local/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper
      result = fn(cls, context, *args, **kwargs)

    File "/opt/stack/nova/nova/objects/service.py", line 304, in get_by_compute_host
      use_slave=use_slave)

    File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 235, in wrapper
      return f(*args, **kwargs)

    File "/opt/stack/nova/nova/objects/service.py", line 299, in _db_service_get_by_compute_host
      return db.service_get_by_compute_host(context, host)

    File "/opt/stack/nova/nova/db/api.py", line 176, in service_get_by_compute_host
      return IMPL.service_get_by_compute_host(context, host)

    File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 280, in wrapped
      return f(context, *args, **kwargs)

    File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 578, in service_get_by_compute_host
      raise exception.ComputeHostNotFound(host=host)

  ComputeHostNotFound: Compute host 303-openstack-test2 could not be found.
  2017-07-07 14:04:32.392 12955 DEBUG nova.virt.libvirt.volume.mount [req-5d828b59-c72b-4aaf-b1de-23c7bca74818 - -] Initialising _HostMountState generation 0 host_up /opt/stack/nova/nova/virt/libvirt/volume/mount.py:131


  >>>>


  2017-07-07 14:08:34.699 12955 ERROR nova.compute.manager [req-5b556b60-93bd-41d6-9fe9-099c26c2a372 - -] Error updating resources for node 303-openstack-test2.: InvalidDiskInfo: Disk info file is invalid: qemu-img failed to execute on /opt/stack/data/nova/instances/b8f65737-a0cb-4fcf-83a1-2005232fef0a/disk : Unexpected error while running command.
  Command: /usr/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=8 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b8f65737-a0cb-4fcf-83a1-2005232fef0a/disk
  Exit code: 1
  Stdout: u''
  Stderr: u"qemu-img: Could not open '/opt/stack/data/nova/instances/b8f65737-a0cb-4fcf-83a1-2005232fef0a/disk': Could not open '/opt/stack/data/nova/instances/b8f65737-a0cb-4fcf-83a1-2005232fef0a/disk': No such file or directory\n"
  2017-07-07 14:08:34.699 12955 ERROR nova.compute.manager Traceback (most recent call last):
  2017-07-07 14:08:34.699 12955 ERROR nova.compute.manager   File "/opt/stack/nova/nova/compute/manager.py", line 6610, in update_available_resource_for_node
  2017-07-07 14:08:34.699 12955 ERROR nova.compute.manager     rt.update_available_resource(context, nodename)
  2017-07-07 14:08:34.699 12955 ERROR nova.compute.manager   File "/opt/stack/nova/nova/compute/resource_tracker.py", line 610, in update_available_resource
  2017-07-07 14:08:34.699 12955 ERROR nova.compute.manager     resources = self.driver.get_available_resource(nodename)
  2017-07-07 14:08:34.699 12955 ERROR nova.compute.manager   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 5765, in get_available_resource
  2017-07-07 14:08:34.699 12955 ERROR nova.compute.manager     disk_over_committed = self._get_disk_over_committed_size_total()
  2017-07-07 14:08:34.699 12955 ERROR nova.compute.manager   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 7264, in _get_disk_over_committed_size_total
  2017-07-07 14:08:34.699 12955 ERROR nova.compute.manager     config, block_device_info)
  2017-07-07 14:08:34.699 12955 ERROR nova.compute.manager   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 7182, in _get_instance_disk_info_from_config
  2017-07-07 14:08:34.699 12955 ERROR nova.compute.manager     backing_file = libvirt_utils.get_disk_backing_file(path)
  2017-07-07 14:08:34.699 12955 ERROR nova.compute.manager   File "/opt/stack/nova/nova/virt/libvirt/utils.py", line 200, in get_disk_backing_file
  2017-07-07 14:08:34.699 12955 ERROR nova.compute.manager     backing_file = images.qemu_img_info(path, format).backing_file
  2017-07-07 14:08:34.699 12955 ERROR nova.compute.manager   File "/opt/stack/nova/nova/virt/images.py", line 72, in qemu_img_info
  2017-07-07 14:08:34.699 12955 ERROR nova.compute.manager     raise exception.InvalidDiskInfo(reason=msg)
  2017-07-07 14:08:34.699 12955 ERROR nova.compute.manager InvalidDiskInfo: Disk info file is invalid: qemu-img failed to execute on /opt/stack/data/nova/instances/b8f65737-a0cb-4fcf-83a1-2005232fef0a/disk : Unexpected error while running command.
  2017-07-07 14:08:34.699 12955 ERROR nova.compute.manager Command: /usr/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=8 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/b8f65737-a0cb-4fcf-83a1-2005232fef0a/disk
  2017-07-07 14:08:34.699 12955 ERROR nova.compute.manager Exit code: 1
  2017-07-07 14:08:34.699 12955 ERROR nova.compute.manager Stdout: u''
  2017-07-07 14:08:34.699 12955 ERROR nova.compute.manager Stderr: u"qemu-img: Could not open '/opt/stack/data/nova/instances/b8f65737-a0cb-4fcf-83a1-2005232fef0a/disk': Could not open '/opt/stack/data/nova/instances/b8f65737-a0cb-4fcf-83a1-2005232fef0a/disk': No such file or directory\n"
  2017-07-07 14:08:34.699 12955 ERROR nova.compute.manager

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1703049/+subscriptions