yahoo-eng-team team mailing list archive

[Sylvain Bauza <1921804@xxxxxxxxxxxxxxxxxx>]

While I understand your concern, the nova community has a consensus
about telling that nova services shouldn't verify the status of Rabbit
MQs and those should expect that MQs are working.

Given there are workarounds for removing the attachment in case you had
a failure, I'll move this bug to Wontfix.

Unfortunately, this consensus isn't captured in
https://docs.openstack.org/nova/latest/contributor/project-scope.html
but I'll propose a patch for writing it clearly.

** Changed in: nova
       Status: New => Won't Fix

** Tags added: volumes

** Tags added: oslo

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1921804

Title:
  leftover bdm when rabbitmq unstable

Status in OpenStack Compute (nova):
  Won't Fix

Bug description:
  Description
  ===========

  When rabbitMQ unstable, there might be a chance that method
  https://github.com/openstack/nova/blob/7a1222a8654684262a8e589d91e67f2b9a9da336/nova/compute/api.py#L4741
  will timeout but bdm is successfully created.

  Under such cases, volume will be shown in server show, but cannot be
  detached. and volume status is available.

  Steps to reproduce
  ==================
  there might be no way to safely reproduce this failure, because when rabbitmq is
  unstable, many other services will also show unusual behavior.

  Expected result
  ===============
  We should be able to remove such attachment from api without manually fixing db...

  ```console
  root@mgt02:~# openstack server show 4e5c3c7d-6b4c-4841-9e6e-9a3374036a3e
  +-------------------------------------+-------------------------------------------------------------------------------------------+
  | Field                               | Value                                                                                     |
  +-------------------------------------+-------------------------------------------------------------------------------------------+
  | OS-DCF:diskConfig                   | MANUAL                                                                                    |
  | OS-EXT-AZ:availability_zone         | cn-north-3a                                                                               |
  | OS-EXT-SRV-ATTR:host                | compute01                                                                                 |
  | OS-EXT-SRV-ATTR:hypervisor_hostname | compute01                                                                                 |
  | OS-EXT-SRV-ATTR:instance_name       | instance-0000ce4c                                                                         |
  | OS-EXT-STS:power_state              | Running                                                                                   |
  | OS-EXT-STS:task_state               | None                                                                                      |
  | OS-EXT-STS:vm_state                 | active                                                                                    |
  | OS-SRV-USG:launched_at              | 2021-03-29T09:06:38.000000                                                                |
  | OS-SRV-USG:terminated_at            | None                                                                                      |
  | accessIPv4                          |                                                                                           |
  | accessIPv6                          |                                                                                           |
  | addresses                           | newsql-net=192.168.1.217; service_mgt=100.114.3.41                                        |
  | config_drive                        | True                                                                                      |
  | created                             | 2021-03-29T09:05:19Z                                                                      |
  | flavor                              | newsql_2C8G40G_general (51db3192-cece-4b9a-9969-7916b4543beb)                             |
  | hostId                              | cf1f3937a3286677b3020d817541ac33d7c8f1ca74be49b26f128093                                  |
  | id                                  | 4e5c3c7d-6b4c-4841-9e6e-9a3374036a3e                                                      |
  | image                               | newsql-bini2.0.0alpha-ubuntu18.04-x64-20210112-pub (4531e3bf-0433-40c6-816b-6763f9d02c7a) |
  | key_name                            | None                                                                                      |
  | name                                | NewSQL-1abc5b28-b9e6-45cd-893d-5bb3a7732a43-3                                             |
  | progress                            | 0                                                                                         |
  | project_id                          | acfcc87fc1db430880f0cb1cce410906                                                          |
  | properties                          | productTag='NewSQL'                                                                       |
  | security_groups                     | name='default'                                                                            |
  |                                     | name='csf-NewSQL-cluster-security-group'                                                  |
  | status                              | ACTIVE                                                                                    |
  | updated                             | 2021-03-29T09:06:39Z                                                                      |
  | user_id                             | a38ef24677cc4a45a143a31c5fb59ee9                                                          |
  | volumes_attached                    | id='1351f861-55a7-4dc5-bf5c-31963657876d'                                                 |
  +-------------------------------------+-------------------------------------------------------------------------------------------+

  root@mgt02:~# openstack server remove volume 4e5c3c7d-6b4c-4841-9e6e-9a3374036a3e 1351f861-55a7-4dc5-bf5c-31963657876d
  Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2ca68e00-760d-47c2-913e-b0fe54e86296) (HTTP 400) (Request-ID: req-264aea62-a970-4241-ab14-ea7aad74cd6e)
  ```

  Actual result
  =============
  openstack server remove volume failed

  Environment
  ===========
  1. stable/victoria in k8s

  2. qemu+kvm

  2. ceph rbd

  3. neutron ml2 openvswitch

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1921804/+subscriptions