yahoo-eng-team team mailing list archive

[Matt Riedemann <mriedem.os@xxxxxxxxx>]

Public bug reported:

Resize tests are intermittently failing in the gate:

http://logs.openstack.org/96/516396/1/check/legacy-tempest-dsvm-
py35/ecb9db4/logs/screen-n-super-
cond.txt.gz?level=TRACE#_Oct_30_18_01_18_003148

Oct 30 18:01:18.003148 ubuntu-xenial-inap-mtl01-0000586035 nova-
conductor[22452]: ERROR nova.conductor.tasks.migrate [None req-
2818e7b7-6881-4cfb-ae79-1816cb948748 tempest-
ListImageFiltersTestJSON-1403553182 tempest-
ListImageFiltersTestJSON-1403553182] [instance:
f5aec132-8a62-47a5-a967-8e5d18a9c6f8] Unable to replace resource claim
on source host ubuntu-xenial-inap-mtl01-0000586035 node ubuntu-xenial-
inap-mtl01-0000586035 for instance

The request in the placement logs starts here:

http://logs.openstack.org/96/516396/1/check/legacy-tempest-dsvm-
py35/ecb9db4/logs/screen-placement-api.txt.gz#_Oct_30_18_01_16_940644

Oct 30 18:01:17.993287 ubuntu-xenial-inap-mtl01-0000586035 devstack@placement-api.service[15936]: DEBUG nova.api.openstack.placement.wsgi_wrapper [None req-7eec8dd2-f65c-43fa-b3df-cdf7a236aa03 service placement] Placement API returning an error response: Inventory changed while attempting to allocate: Another thread concurrently updated the data. Please retry your update {{(pid=15938) call_func /opt/stack/new/nova/nova/api/openstack/placement/wsgi_wrapper.py:31}}
Oct 30 18:01:17.994558 ubuntu-xenial-inap-mtl01-0000586035 devstack@placement-api.service[15936]: INFO nova.api.openstack.placement.requestlog [None req-7eec8dd2-f65c-43fa-b3df-cdf7a236aa03 service placement] 198.72.124.85 "PUT /placement/allocations/52b215a6-0d60-4fcc-8389-2645ffb22562" status: 409 len: 305 microversion: 1.8

The error from placement is a bit misleading. It's probably not that
inventory has changed, but allocations have changed in the meantime
since this is a single-node environment, so capacity changd and
conductor needs to retry, just like the scheduler does.

** Affects: nova
     Importance: High
         Status: Triaged


** Tags: conductor placement resize

** Changed in: nova
       Status: New => Triaged

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1728722

Title:
  Resize test fails in conductor during migration/instance allocation
  swap: "Unable to replace resource claim on source host"

Status in OpenStack Compute (nova):
  Triaged

Bug description:
  Resize tests are intermittently failing in the gate:

  http://logs.openstack.org/96/516396/1/check/legacy-tempest-dsvm-
  py35/ecb9db4/logs/screen-n-super-
  cond.txt.gz?level=TRACE#_Oct_30_18_01_18_003148

  Oct 30 18:01:18.003148 ubuntu-xenial-inap-mtl01-0000586035 nova-
  conductor[22452]: ERROR nova.conductor.tasks.migrate [None req-
  2818e7b7-6881-4cfb-ae79-1816cb948748 tempest-
  ListImageFiltersTestJSON-1403553182 tempest-
  ListImageFiltersTestJSON-1403553182] [instance:
  f5aec132-8a62-47a5-a967-8e5d18a9c6f8] Unable to replace resource claim
  on source host ubuntu-xenial-inap-mtl01-0000586035 node ubuntu-xenial-
  inap-mtl01-0000586035 for instance

  The request in the placement logs starts here:

  http://logs.openstack.org/96/516396/1/check/legacy-tempest-dsvm-
  py35/ecb9db4/logs/screen-placement-api.txt.gz#_Oct_30_18_01_16_940644

  Oct 30 18:01:17.993287 ubuntu-xenial-inap-mtl01-0000586035 devstack@placement-api.service[15936]: DEBUG nova.api.openstack.placement.wsgi_wrapper [None req-7eec8dd2-f65c-43fa-b3df-cdf7a236aa03 service placement] Placement API returning an error response: Inventory changed while attempting to allocate: Another thread concurrently updated the data. Please retry your update {{(pid=15938) call_func /opt/stack/new/nova/nova/api/openstack/placement/wsgi_wrapper.py:31}}
  Oct 30 18:01:17.994558 ubuntu-xenial-inap-mtl01-0000586035 devstack@placement-api.service[15936]: INFO nova.api.openstack.placement.requestlog [None req-7eec8dd2-f65c-43fa-b3df-cdf7a236aa03 service placement] 198.72.124.85 "PUT /placement/allocations/52b215a6-0d60-4fcc-8389-2645ffb22562" status: 409 len: 305 microversion: 1.8

  The error from placement is a bit misleading. It's probably not that
  inventory has changed, but allocations have changed in the meantime
  since this is a single-node environment, so capacity changd and
  conductor needs to retry, just like the scheduler does.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1728722/+subscriptions