yahoo-eng-team team mailing list archive

[Abhishek Kekane <2127798@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

When EC2 credentials used for S3 backend authentication are
rotated/deleted in Keystone, Glance images with S3 locations become
inaccessible because the image metadata still references the old
credentials in the location URLs. There is no automatic mechanism to
update image location metadata when S3 credentials change.

Steps to Reproduce:
1. Setup S3 backend with EC2 credentials:

   # Create EC2 credentials
   openstack ec2 credential create
   # Configure Glance with S3 backend using these credentials

2. Create an image with S3 backend:

   openstack image create --file cirros.img --disk-format qcow2 test-
image

3. Verify image has S3 location with current credentials:

   openstack image show test-image --format json | grep -A 5 locations
   # Shows: s3://ACCESS_KEY:SECRET_KEY@host/path

4. Rotate EC2 credentials:

   # Delete old credentials
   openstack ec2 credential delete OLD_ACCESS_KEY
   # Create new credentials  
   openstack ec2 credential create
   # Update Glance configuration with new credentials
   # Restart Glance service

5. Attempt to download the image:

   openstack image save --file downloaded.img test-image


Expected Behavior

- Glance should automatically update image location metadata when S3 credentials change
- Images should remain accessible after credential rotation
- No manual intervention should be required


Actual Behavior

- Image becomes inaccessible with "Image has no data" error
- Old credential URLs remain in metadata even after credential rotation
- Manual location management required to restore image access
- No automatic credential update mechanism exists


Workaround, Currently requires manual steps:

Add new location with new credentials:

   glance location-add --url "s3://NEW_ACCESS_KEY:NEW_SECRET_KEY@host/path" \
     --metadata '{"store": "s3_fast"}' IMAGE_ID

With this workaround you will get access to your data but image will
have two locations out of which one will be forever not accessible and
needs to be deleted from the database only.

** Affects: glance
     Importance: High
     Assignee: Abhishek Kekane (abhishek-kekane)
         Status: Triaged

** Changed in: glance
   Importance: Undecided => High

** Changed in: glance
     Assignee: (unassigned) => Abhishek Kekane (abhishek-kekane)

** Changed in: glance
       Status: New => Triaged

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to Glance.
https://bugs.launchpad.net/bugs/2127798

Title:
  Glance S3 credential update requires manual metadata management when
  EC2 credentials are rotated

Status in Glance:
  Triaged

Bug description:
  When EC2 credentials used for S3 backend authentication are
  rotated/deleted in Keystone, Glance images with S3 locations become
  inaccessible because the image metadata still references the old
  credentials in the location URLs. There is no automatic mechanism to
  update image location metadata when S3 credentials change.

  Steps to Reproduce:
  1. Setup S3 backend with EC2 credentials:

     # Create EC2 credentials
     openstack ec2 credential create
     # Configure Glance with S3 backend using these credentials

  2. Create an image with S3 backend:

     openstack image create --file cirros.img --disk-format qcow2 test-
  image

  3. Verify image has S3 location with current credentials:

     openstack image show test-image --format json | grep -A 5 locations
     # Shows: s3://ACCESS_KEY:SECRET_KEY@host/path

  4. Rotate EC2 credentials:

     # Delete old credentials
     openstack ec2 credential delete OLD_ACCESS_KEY
     # Create new credentials  
     openstack ec2 credential create
     # Update Glance configuration with new credentials
     # Restart Glance service

  5. Attempt to download the image:

     openstack image save --file downloaded.img test-image

  
  Expected Behavior

  - Glance should automatically update image location metadata when S3 credentials change
  - Images should remain accessible after credential rotation
  - No manual intervention should be required

  
  Actual Behavior

  - Image becomes inaccessible with "Image has no data" error
  - Old credential URLs remain in metadata even after credential rotation
  - Manual location management required to restore image access
  - No automatic credential update mechanism exists

  
  Workaround, Currently requires manual steps:

  Add new location with new credentials:

     glance location-add --url "s3://NEW_ACCESS_KEY:NEW_SECRET_KEY@host/path" \
       --metadata '{"store": "s3_fast"}' IMAGE_ID

  With this workaround you will get access to your data but image will
  have two locations out of which one will be forever not accessible and
  needs to be deleted from the database only.

To manage notifications about this bug go to:
https://bugs.launchpad.net/glance/+bug/2127798/+subscriptions