yahoo-eng-team team mailing list archive

[OpenStack Infra <1728722@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.openstack.org/516708
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=62d35009755a45c39a0b1cdc4c69791f469e469e
Submitter: Zuul
Branch:    master

commit 62d35009755a45c39a0b1cdc4c69791f469e469e
Author: Dan Smith <dansmith@xxxxxxxxxx>
Date:   Tue Oct 31 07:46:36 2017 -0700

    Make put_allocations() retry on concurrent update
    
    This adds a retries decorator to the scheduler report client
    and modifies put_allocations() so that it will detect a concurrent
    update, raising the Retry exception to trigger the decorator.
    
    This should be usable by other methods in the client easily, but
    this patch only modifies put_allocations() to fix the bug.
    
    Change-Id: Ic32a54678dd413668f02e77d5e6c4195664ac24c
    Closes-Bug: #1728722


** Changed in: nova
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1728722

Title:
  Resize test fails in conductor during migration/instance allocation
  swap: "Unable to replace resource claim on source host"

Status in OpenStack Compute (nova):
  Fix Released

Bug description:
  Resize tests are intermittently failing in the gate:

  http://logs.openstack.org/96/516396/1/check/legacy-tempest-dsvm-
  py35/ecb9db4/logs/screen-n-super-
  cond.txt.gz?level=TRACE#_Oct_30_18_01_18_003148

  Oct 30 18:01:18.003148 ubuntu-xenial-inap-mtl01-0000586035 nova-
  conductor[22452]: ERROR nova.conductor.tasks.migrate [None req-
  2818e7b7-6881-4cfb-ae79-1816cb948748 tempest-
  ListImageFiltersTestJSON-1403553182 tempest-
  ListImageFiltersTestJSON-1403553182] [instance:
  f5aec132-8a62-47a5-a967-8e5d18a9c6f8] Unable to replace resource claim
  on source host ubuntu-xenial-inap-mtl01-0000586035 node ubuntu-xenial-
  inap-mtl01-0000586035 for instance

  The request in the placement logs starts here:

  http://logs.openstack.org/96/516396/1/check/legacy-tempest-dsvm-
  py35/ecb9db4/logs/screen-placement-api.txt.gz#_Oct_30_18_01_16_940644

  Oct 30 18:01:17.993287 ubuntu-xenial-inap-mtl01-0000586035 devstack@placement-api.service[15936]: DEBUG nova.api.openstack.placement.wsgi_wrapper [None req-7eec8dd2-f65c-43fa-b3df-cdf7a236aa03 service placement] Placement API returning an error response: Inventory changed while attempting to allocate: Another thread concurrently updated the data. Please retry your update {{(pid=15938) call_func /opt/stack/new/nova/nova/api/openstack/placement/wsgi_wrapper.py:31}}
  Oct 30 18:01:17.994558 ubuntu-xenial-inap-mtl01-0000586035 devstack@placement-api.service[15936]: INFO nova.api.openstack.placement.requestlog [None req-7eec8dd2-f65c-43fa-b3df-cdf7a236aa03 service placement] 198.72.124.85 "PUT /placement/allocations/52b215a6-0d60-4fcc-8389-2645ffb22562" status: 409 len: 305 microversion: 1.8

  The error from placement is a bit misleading. It's probably not that
  inventory has changed, but allocations have changed in the meantime
  since this is a single-node environment, so capacity changd and
  conductor needs to retry, just like the scheduler does.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1728722/+subscriptions