yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1764125] [NEW] Re-attaching an encrypted(Barbican) Cinder (RBD) volume to an instance fails

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Tzach Shefi <tshefi@xxxxxxxxxx>
Date: Sun, 15 Apr 2018 13:52:09 -0000
Reply-to: Bug 1764125 <1764125@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

Description of problem: 
An attached encrypted (Barbican) RBD Cinder volume was attached to instance, write data to it. 
Then volume was detached, when trying to reattach the volume to same instance volume fails to attach. Odd errors on attached nova-compute.log 

2018-04-15 13:14:06.274 1 ERROR nova.compute.manager [instance: 923c5318-8502-4f85-a215-78afc4fd641b]     uuid=managed_object_id)
2018-04-15 13:14:06.274 1 ERROR nova.compute.manager [instance: 923c5318-8502-4f85-a215-78afc4fd641b] ManagedObjectNotFoundError: Key not found, uuid: 7912eac8-2652-4c92-b53f-3db4ecca7bc7

2018-04-15 13:14:06.523 1 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinderclient/client.py", line 177, in request
2018-04-15 13:14:06.523 1 ERROR oslo_messaging.rpc.server     raise exceptions.from_response(resp, body)
2018-04-15 13:14:06.523 1 ERROR oslo_messaging.rpc.server VolumeAttachmentNotFound: Volume attachment c17e2b89-5a36-4e7e-8c71-b975f2f5ccb3 could not be found.
2018-04-15 13:14:06.523 1 ERROR oslo_messaging.rpc.server 


How reproducible:
Unsure looks like every time I try to re-attach. 

Steps to Reproduce:
1. Boot an instance
2. Create an encrypted(Barbican) backed Cinder(RBD) volume, attach to instance write data.  
3. Detach volume from instance
4. Try to reattach same volume to same instance. 

$nova volume-attach 923c5318-8502-4f85-a215-78afc4fd641b
16584072-ef78-4a80-91ab-cbd47e9bc70d auto

5. Volume fails to attach
No error volume remains unattached
cinder list 
+--------------------------------------+-----------+-------------+------+----------------------------+----------+--------------------------------------+
| ID                                   | Status    | Name        | Size | Volume Type                | Bootable | Attached to                          |
+--------------------------------------+-----------+-------------+------+----------------------------+----------+--------------------------------------+
| 16584072-ef78-4a80-91ab-cbd47e9bc70d | available | 2-Encrypted | 1    | LuksEncryptor-Template-256 | false    |                                      |


Actual results:
Volume fails to attach. 


Expected results:
Volume should successfully reattach. 


Environment / Version-Release number of selected component (if applicable):
rhel7.5 
openstack-nova-conductor-17.0.2-0.20180323024604.0390d5f.el7ost.noarch
python-nova-17.0.2-0.20180323024604.0390d5f.el7ost.noarch
python-novaclient-9.1.1-1.el7ost.noarch
openstack-cinder-12.0.1-0.20180326201852.46c4ec1.el7ost.noarch
openstack-nova-scheduler-17.0.2-0.20180323024604.0390d5f.el7ost.noarch
openstack-nova-console-17.0.2-0.20180323024604.0390d5f.el7ost.noarch
puppet-cinder-12.3.1-0.20180222074326.18152ac.el7ost.noarch
openstack-nova-compute-17.0.2-0.20180323024604.0390d5f.el7ost.noarch
python2-cinderclient-3.5.0-1.el7ost.noarch
python-cinder-12.0.1-0.20180326201852.46c4ec1.el7ost.noarch
openstack-nova-api-17.0.2-0.20180323024604.0390d5f.el7ost.noarch
openstack-nova-novncproxy-17.0.2-0.20180323024604.0390d5f.el7ost.noarch
puppet-nova-12.3.1-0.20180319062741.9db79a6.el7ost.noarch
openstack-nova-common-17.0.2-0.20180323024604.0390d5f.el7ost.noarch
openstack-nova-migration-17.0.2-0.20180323024604.0390d5f.el7ost.noarch
openstack-nova-placement-api-17.0.2-0.20180323024604.0390d5f.el7ost.noarch

Libvirt + KVM
Neutron networking
Cinder volume is RBD backed and encrypted via Barbican.

** Affects: nova
     Importance: Undecided
         Status: New

** Attachment added: "Nova compute log"
   https://bugs.launchpad.net/bugs/1764125/+attachment/5116630/+files/nova-compute.log

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1764125

Title:
  Re-attaching an encrypted(Barbican) Cinder (RBD) volume to an instance
  fails

Status in OpenStack Compute (nova):
  New

Bug description:
  Description of problem: 
  An attached encrypted (Barbican) RBD Cinder volume was attached to instance, write data to it. 
  Then volume was detached, when trying to reattach the volume to same instance volume fails to attach. Odd errors on attached nova-compute.log 

  2018-04-15 13:14:06.274 1 ERROR nova.compute.manager [instance: 923c5318-8502-4f85-a215-78afc4fd641b]     uuid=managed_object_id)
  2018-04-15 13:14:06.274 1 ERROR nova.compute.manager [instance: 923c5318-8502-4f85-a215-78afc4fd641b] ManagedObjectNotFoundError: Key not found, uuid: 7912eac8-2652-4c92-b53f-3db4ecca7bc7

  2018-04-15 13:14:06.523 1 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinderclient/client.py", line 177, in request
  2018-04-15 13:14:06.523 1 ERROR oslo_messaging.rpc.server     raise exceptions.from_response(resp, body)
  2018-04-15 13:14:06.523 1 ERROR oslo_messaging.rpc.server VolumeAttachmentNotFound: Volume attachment c17e2b89-5a36-4e7e-8c71-b975f2f5ccb3 could not be found.
  2018-04-15 13:14:06.523 1 ERROR oslo_messaging.rpc.server 

  
  How reproducible:
  Unsure looks like every time I try to re-attach. 

  Steps to Reproduce:
  1. Boot an instance
  2. Create an encrypted(Barbican) backed Cinder(RBD) volume, attach to instance write data.  
  3. Detach volume from instance
  4. Try to reattach same volume to same instance. 

  $nova volume-attach 923c5318-8502-4f85-a215-78afc4fd641b
  16584072-ef78-4a80-91ab-cbd47e9bc70d auto

  5. Volume fails to attach
  No error volume remains unattached
  cinder list 
  +--------------------------------------+-----------+-------------+------+----------------------------+----------+--------------------------------------+
  | ID                                   | Status    | Name        | Size | Volume Type                | Bootable | Attached to                          |
  +--------------------------------------+-----------+-------------+------+----------------------------+----------+--------------------------------------+
  | 16584072-ef78-4a80-91ab-cbd47e9bc70d | available | 2-Encrypted | 1    | LuksEncryptor-Template-256 | false    |                                      |

  
  Actual results:
  Volume fails to attach. 

  
  Expected results:
  Volume should successfully reattach. 

  
  Environment / Version-Release number of selected component (if applicable):
  rhel7.5 
  openstack-nova-conductor-17.0.2-0.20180323024604.0390d5f.el7ost.noarch
  python-nova-17.0.2-0.20180323024604.0390d5f.el7ost.noarch
  python-novaclient-9.1.1-1.el7ost.noarch
  openstack-cinder-12.0.1-0.20180326201852.46c4ec1.el7ost.noarch
  openstack-nova-scheduler-17.0.2-0.20180323024604.0390d5f.el7ost.noarch
  openstack-nova-console-17.0.2-0.20180323024604.0390d5f.el7ost.noarch
  puppet-cinder-12.3.1-0.20180222074326.18152ac.el7ost.noarch
  openstack-nova-compute-17.0.2-0.20180323024604.0390d5f.el7ost.noarch
  python2-cinderclient-3.5.0-1.el7ost.noarch
  python-cinder-12.0.1-0.20180326201852.46c4ec1.el7ost.noarch
  openstack-nova-api-17.0.2-0.20180323024604.0390d5f.el7ost.noarch
  openstack-nova-novncproxy-17.0.2-0.20180323024604.0390d5f.el7ost.noarch
  puppet-nova-12.3.1-0.20180319062741.9db79a6.el7ost.noarch
  openstack-nova-common-17.0.2-0.20180323024604.0390d5f.el7ost.noarch
  openstack-nova-migration-17.0.2-0.20180323024604.0390d5f.el7ost.noarch
  openstack-nova-placement-api-17.0.2-0.20180323024604.0390d5f.el7ost.noarch

  Libvirt + KVM
  Neutron networking
  Cinder volume is RBD backed and encrypted via Barbican.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1764125/+subscriptions
Follow ups

[Bug 1764125] Re: Re-attaching an encrypted(Barbican) Cinder (RBD) volume to an instance fails
From: Lee Yarwood, 2018-05-17