← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 1844296] [NEW] Stale BDM records remain in the DB after n-api to n-cpu RPC timeouts during reserve_block_device_name

 

Public bug reported:

Description
===========
When attaching block devices n-api will call out to the n-cpu service hosting an instance and attempt to reserve a block device name via reserve_block_device_name. This call also creates the initial BDM record within the database. RPC timeouts, seen by n-api as exceptions can then result in stale BDM records persisting in the database.

Steps to reproduce
==================

1. Attach a volume to an instance, ensuring any call to
reserve_block_device_name takes longer than the configured RPC timeout
within the environment.

Expected result
===============
The RPC timeout is hit and any BDM records created by the n-cpu service are removed by n-api.

Actual result
=============
The RPC timeout is hit but the BDM records persist.

Environment
===========
1. Exact version of OpenStack you are running. See the following
  list for all releases: http://docs.openstack.org/releases/

   Master.

2. Which hypervisor did you use?
   (For example: Libvirt + KVM, Libvirt + XEN, Hyper-V, PowerKVM, ...)
   What's the version of that?

   N/A

2. Which storage type did you use?
   (For example: Ceph, LVM, GPFS, ...)
   What's the version of that?

   N/A

3. Which networking type did you use?
   (For example: nova-network, Neutron with OpenVSwitch, ...)

   N/A

Logs & Configs
==============

Invalid bdm record remains when reserve_block_device_name rpc call times out
https://bugzilla.redhat.com/show_bug.cgi?id=1752734

** Affects: nova
     Importance: Undecided
     Assignee: Lee Yarwood (lyarwood)
         Status: In Progress


** Tags: compute volumes

** Tags added: compute

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1844296

Title:
  Stale BDM records remain in the DB after n-api to n-cpu RPC timeouts
  during reserve_block_device_name

Status in OpenStack Compute (nova):
  In Progress

Bug description:
  Description
  ===========
  When attaching block devices n-api will call out to the n-cpu service hosting an instance and attempt to reserve a block device name via reserve_block_device_name. This call also creates the initial BDM record within the database. RPC timeouts, seen by n-api as exceptions can then result in stale BDM records persisting in the database.

  Steps to reproduce
  ==================

  1. Attach a volume to an instance, ensuring any call to
  reserve_block_device_name takes longer than the configured RPC timeout
  within the environment.

  Expected result
  ===============
  The RPC timeout is hit and any BDM records created by the n-cpu service are removed by n-api.

  Actual result
  =============
  The RPC timeout is hit but the BDM records persist.

  Environment
  ===========
  1. Exact version of OpenStack you are running. See the following
    list for all releases: http://docs.openstack.org/releases/

     Master.

  2. Which hypervisor did you use?
     (For example: Libvirt + KVM, Libvirt + XEN, Hyper-V, PowerKVM, ...)
     What's the version of that?

     N/A

  2. Which storage type did you use?
     (For example: Ceph, LVM, GPFS, ...)
     What's the version of that?

     N/A

  3. Which networking type did you use?
     (For example: nova-network, Neutron with OpenVSwitch, ...)

     N/A

  Logs & Configs
  ==============

  Invalid bdm record remains when reserve_block_device_name rpc call times out
  https://bugzilla.redhat.com/show_bug.cgi?id=1752734

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1844296/+subscriptions