yahoo-eng-team team mailing list archive

[Matthew Booth <mbooth@xxxxxxxxxx>]

Public bug reported:

We currently permit the following:

Create multiattach volumes a and b
Create servers 1 and 2
Attach volume a to servers 1 and 2
swap_volume(server 1, volume a, volume b)

In fact, we have a tempest test which tests exactly this sequence:
api.compute.admin.test_volume_swap.TestMultiAttachVolumeSwap.test_volume_swap_with_multiattach

The problem is that writes from server 2 during the copy operation on
server 1 will continue to hit the underlying storage, but as server 1
doesn't know about them they won't be reflected on the copy on volume b.
This will lead to an inconsistent copy, and therefore data corruption on
volume b.

Also, this whole flow makes no sense for a multiattached volume because
even if we managed a consistent copy all we've achieved is forking our
data between the 2 volumes. The purpose of this call is to allow the
operator to move volumes. We need a fundamentally different approach for
multiattached volumes.

In the short term we should at least prevent data corruption by
preventing swap volume of a multiattached volume. This would also cause
the above tempest test to fail, but as I don't believe it's possible to
implement the test safely this would be correct.

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1775418

Title:
  Swap volume of multiattached volume will corrupt data

Status in OpenStack Compute (nova):
  New

Bug description:
  We currently permit the following:

  Create multiattach volumes a and b
  Create servers 1 and 2
  Attach volume a to servers 1 and 2
  swap_volume(server 1, volume a, volume b)

  In fact, we have a tempest test which tests exactly this sequence:
  api.compute.admin.test_volume_swap.TestMultiAttachVolumeSwap.test_volume_swap_with_multiattach

  The problem is that writes from server 2 during the copy operation on
  server 1 will continue to hit the underlying storage, but as server 1
  doesn't know about them they won't be reflected on the copy on volume
  b. This will lead to an inconsistent copy, and therefore data
  corruption on volume b.

  Also, this whole flow makes no sense for a multiattached volume
  because even if we managed a consistent copy all we've achieved is
  forking our data between the 2 volumes. The purpose of this call is to
  allow the operator to move volumes. We need a fundamentally different
  approach for multiattached volumes.

  In the short term we should at least prevent data corruption by
  preventing swap volume of a multiattached volume. This would also
  cause the above tempest test to fail, but as I don't believe it's
  possible to implement the test safely this would be correct.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1775418/+subscriptions