yahoo-eng-team team mailing list archive

[Nicolas Simonds <1297451@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

When uploading to the filesystem store, the image is uploaded,
checksummed, and added to the database, with appropriate rollback
operations in place in case of failures.  This is all good, but there
are various corner-cases where partially-uploaded files can be left
lying around in the filesystem_store_datadir.

Mostly this kind of thing interferes with operational processes we have
that are external to Glance and not Glance itself.  e.g., we have a
monitoring process to reconcile the list/checksums of glance images
against what's on-disk, and this occasionally goes funny on us.
Additionally, we have a side-band backup/replication process that ends
up occasionally transferring partial files for no reason.

Using a temporary upload space and moving the file into place on the
filesystem store would cause a lot of our co-processes to work more
cleanly, as well as reducing the risk of the _delete_partial job
deleting the wrong thing should it go sideways.

** Affects: glance
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to Glance.
https://bugs.launchpad.net/bugs/1297451

Title:
  Image uploads to the filesystem driver are not fully atomic

Status in OpenStack Image Registry and Delivery Service (Glance):
  New

Bug description:
  When uploading to the filesystem store, the image is uploaded,
  checksummed, and added to the database, with appropriate rollback
  operations in place in case of failures.  This is all good, but there
  are various corner-cases where partially-uploaded files can be left
  lying around in the filesystem_store_datadir.

  Mostly this kind of thing interferes with operational processes we
  have that are external to Glance and not Glance itself.  e.g., we have
  a monitoring process to reconcile the list/checksums of glance images
  against what's on-disk, and this occasionally goes funny on us.
  Additionally, we have a side-band backup/replication process that ends
  up occasionally transferring partial files for no reason.

  Using a temporary upload space and moving the file into place on the
  filesystem store would cause a lot of our co-processes to work more
  cleanly, as well as reducing the risk of the _delete_partial job
  deleting the wrong thing should it go sideways.

To manage notifications about this bug go to:
https://bugs.launchpad.net/glance/+bug/1297451/+subscriptions