yahoo-eng-team team mailing list archive

[nikhil komawar <nikhil.komawar@xxxxxxxxxxxxx>]

** Also affects: glance-store
   Importance: Undecided
       Status: New

** Changed in: glance-store
       Status: New => In Progress

** Changed in: glance-store
     Assignee: (unassigned) => Stuart McLaren (stuart-mclaren)

** Changed in: glance-store
    Milestone: None => v0.1.11

** Changed in: glance-store
   Importance: Undecided => Critical

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to Glance.
https://bugs.launchpad.net/bugs/1412802

Title:
  copy-from broken for large files and swift

Status in OpenStack Image Registry and Delivery Service (Glance):
  Fix Released
Status in Glance icehouse series:
  In Progress
Status in Glance juno series:
  In Progress
Status in OpenStack Glance backend store-drivers library (glance_store):
  In Progress

Bug description:
  Glance may loose some image data while transferring it to the backing
  store thus corrupting the image when ALL the following conditions are
  met:

  - Image is being created by copying data from remote source (--copy-from CLI parameter or appropriate API call)
  - Backing store is Swift
  - Image size is larger then configured "swift_store_large_object_size"

  In such scenarios the last chunk stored in Swift will have the size
  significantly less then expected. An attempt to download the image
  will result in a checksum verification error, however the checksum
  stored in Glance (with image metadata) is correct, and so is the size.

  This is easily reproducible even on devstack (if the devstack is
  configured to run Swift as Glance backend). Just decrease
  'swift_store_large_object_size' to some reasonably low value (i.e. 200
  Mb) and try to copy-from any image which is larger then that value.
  After the upload is successful, check the object size in swift (by
  either summing the sizes of all the chunks or by looking to the size
  of virtual large object) - they will be lower then expected:

  
  glance image-create --name tst --disk-format qcow2 --container-format bare --copy-from http://192.168.56.1:8000/F18-x86_64-cfntools.qcow2

  ...

  glance image-list
  +--------------------------------------+---------------------------------+-------------+------------------+-----------+--------+
  | ID                                   | Name                            | Disk Format | Container Format | Size      | Status |
  +--------------------------------------+---------------------------------+-------------+------------------+-----------+--------+
  | fc34ec49-4bd3-40dd-918f-44d3254f2ac9 | tst                             | qcow2       | bare             | 536412160 | active |
  +--------------------------------------+---------------------------------+-------------+------------------+-----------+--------+

  ...

  swift stat glance fc34ec49-4bd3-40dd-918f-44d3254f2ac9 --os-tenant-name service --os-username admin
         Account: AUTH_cce6e9c12fa34c63b64ef29e84861554
       Container: glance
          Object: fc34ec49-4bd3-40dd-918f-44d3254f2ac9
    Content Type: application/octet-stream
  Content Length: 509804544                         <---- see, the size is different!
   Last Modified: Mon, 19 Jan 2015 15:52:18 GMT
            ETag: "6d0612f82db9a531b34d25823a45073d"
        Manifest: glance/fc34ec49-4bd3-40dd-918f-44d3254f2ac9-
   Accept-Ranges: bytes
     X-Timestamp: 1421682737.01148
      X-Trans-Id: tx01a19f7476a541808c9a1-0054bd28e1

  ....

  glance image-download tst --file out.qcow2
  [Errno 32] Corrupt image download. Checksum was 0eeddae1007f01b0029136d28518f538 expected 3ecddfe0787a392960d230c87a421c6a

To manage notifications about this bug go to:
https://bugs.launchpad.net/glance/+bug/1412802/+subscriptions