openstack team mailing list archive

[Jay Pipes <jaypipes@xxxxxxxxx>]

On Tue, Nov 15, 2011 at 4:51 PM, Ewan Mellor <Ewan.Mellor@xxxxxxxxxxxxx> wrote:
>> -----Original Message-----
>> From: Jay Pipes [mailto:jaypipes@xxxxxxxxx]
>> Sent: Tuesday, November 15, 2011 1:36 PM
>> To: Ewan Mellor
>> Cc: openstack@xxxxxxxxxxxxxxxxxxx
>> Subject: Re: [Openstack] Glance API semantics when image sizes aren't
>> known
>>
>> On Tue, Nov 15, 2011 at 4:28 PM, Ewan Mellor
>> <Ewan.Mellor@xxxxxxxxxxxxx> wrote:
>> >> -----Original Message-----
>> >> From: Jay Pipes [mailto:jaypipes@xxxxxxxxx]
>> >> Sent: Tuesday, November 15, 2011 12:45 PM
>> >> To: Ewan Mellor
>> >> Cc: openstack@xxxxxxxxxxxxxxxxxxx
>> >> Subject: Re: [Openstack] Glance API semantics when image sizes
>> aren't
>> >> known
>> >>
>> >> On Tue, Nov 15, 2011 at 3:37 PM, Ewan Mellor
>> >> <Ewan.Mellor@xxxxxxxxxxxxx> wrote:
>> >> >> From: Jay Pipes [mailto:jaypipes@xxxxxxxxx]
>> >> >>
>> >> >> The else: block is ONLY met when you are not using the Python
>> glance
>> >> >> client, the glance CLI tool, and are not setting either the
>> >> >> Content-Length or X-Image-Meta-Size header. If you use the Python
>> >> >> glance client or CLI tool, the image you are feeding to the
>> client
>> >> >> automatically does a seek/tell to determine the size of the image
>> >> you
>> >> >> are uploading.
>> >> >
>> >> > Or if you're using something that can't seek/tell, like a stream.
>>  In
>> >> > that case, you get neither a Content-Length nor an X-Image-Meta-
>> Size.
>> >> > Hence my question.
>> >>
>> >> And hence my answer ;)
>> >
>> > I don't think you answered the question.  My question was, what is
>> the intended behavior when given a request that doesn't have either X-
>> Image-Meta-Size or Content-Length set?
>>
>> It depends. If you are just reserving/registering image metadata,
>> Glance will happily store 0, Content-Length or X-Image-Meta-Size.
>>
>> On upload, once an image file is successfully uploaded to the backend
>> store, Glance will set the image's size to the number of bytes that
>> the backend store reported that it wrote.
>>
>> On uploading an image when Swift is the backing store, Glance does its
>> best to upload the image file if no Content-Length or
>> X-Image-Meta-Size is specified, but on the event of a failed Swift
>> upload, the image size does not get set.
>
> It also does not set the image size if the upload is successful.
> image_size is returned unchanged from glance.store.swift.Store.add, so
> 0 is passed in, and passed out again.
>
> So the question is whether this is a bug.  Are you expecting the
> backend to return a correct image size if the upload is successful?

Absolutely. If it's not, that's a bug.

>> > Because the docs imply that the size will be inferred from the body
>> of the upload request, but that's not what is happening in the Swift
>> backend.
>>
>> Would it be possible to paste the code you are using to do the upload?
>
> ~ # cat test_glance.py
> import sys
> import glance.client
>
> client = glance.client.Client('localhost', 9292, auth_tok="999888777666")
> print client.add_image({}, sys.stdin)
> ~ # echo a | python26 ./test_glance.py
> {u'status': u'active', u'name': None, u'deleted': False, u'container_format': None, u'created_at': u'2011-11-15T21:44:21', u'disk_format': None, u'updated_at': u'2011-11-15T21:44:22', u'id': 6, u'owner': u'Administrator', u'location': u'swift+http://root:password@localhost:5000/v1.0/glance/6', u'min_ram': 0, u'checksum': u'60b725f10c9c85c70d97880dfe8191b3', u'min_disk': 0, u'is_public': False, u'deleted_at': None, u'properties': {}, u'size': 0}
>
> Note that size is returned as 0, not 1.

Yes, indeed. That is because the client is not designed to be used
that way, hence this in the glance.client code:

        if hasattr(image_data, 'seek') and hasattr(image_data, 'tell'):
            try:
                image_data.seek(0, os.SEEK_END)
                image_size = image_data.tell()
                image_data.seek(0)
                return image_size
            except IOError, e:
                if e.errno == errno.ESPIPE:
                    # Illegal seek. This means the user is trying
                    # to pipe image data to the client, e.g.
                    # echo testdata | bin/glance add blah..., or
                    # that stdin is empty
                    return None
                else:
                    raise