openstack team mailing list archive

[Stephen Gran <stephen.gran@xxxxxxxxxxxxxx>]

Hi,

I think user_data is probably reasonably common - most people who use,
eg, cloud-init will use it (we do).

As the 64k limit is a MySQL limitation, and not a nova limitation, why
not just say, "if you want more storage, use postgres" (or similar)?  I
have no issue with making the size guarded in the application, with a
configurable limit, but the particular problem that started this off is
an implementation issue rather than a code issue.

Storing the user_data in some place like the database is fairly
important for making things like launch configs for autoscale groups
work.  I'd like to not make that harder to implement.

Cheers,

On Mon, 2012-08-13 at 09:12 -0400, Dan Prince wrote:
> 
> ----- Original Message -----
> > From: "Michael Still" <michael.still@xxxxxxxxxxxxx>
> > To: openstack@xxxxxxxxxxxxxxxxxxx, openstack-operators@xxxxxxxxxxxxxxxxxxx
> > Sent: Saturday, August 11, 2012 5:12:22 AM
> > Subject: [Openstack] [Nova] How common is user_data for instances?
> > 
> > Greetings.
> > 
> > I'm seeking information about how common user_data is for instances
> > in
> > nova. Specifically for large deployments (rackspace and HP, here's
> > looking at you). What sort of costs would be associated with changing
> > the data type of the user_data column in the nova database?
> > 
> > Bug 1035055 [1] requests that we allow user_data of more than 65,535
> > bytes per instance. Note that this size is a base64 encoded version
> > of
> > the data, so that's only a bit under 50k of data. This is because the
> > data is a sqlalchemy Text column.
> > 
> > We could convert to a LongText column, which allows 2^32 worth of
> > data,
> > but I want to understand the cost to operators of that change some
> > more.
> > Is user_data really common? Do you think people would start uploading
> > much bigger user_data? Do you care?
> 
> Nova has configurable quotas on most things so if we do increase the size of the DB column we should probably guard it in a configurable manner with quotas as well.
> 
> My preference would actually be that we go the other way though and not have to store user_data in the database at all. That unfortunately may not be possible since some images obtain user_data via the metadata service which needs a way to look it up. Other methods of injecting metadata via disk injection, agents and/or config drive however might not need it to be store in the database right?
> 
> As a simpler solution:
> 
> Would setting a reasonable limit (hopefully smaller) and returning a HTTP 400 bad request if incoming requests exceed that limit be good enough to resolve this ticket? That way we don't have to increase the DB column at all and end users would be notified up front that user_data is too large (not silently truncated). They way I see it user_data is really for bootstrapping instances... we probably don't need it to be large enough to write an entire application, etc.
> 
> 
> > 
> > Mikal
> > 
> > 1: https://bugs.launchpad.net/nova/+bug/1035055
> > 
> > _______________________________________________
> > Mailing list: https://launchpad.net/~openstack
> > Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> > Unsubscribe : https://launchpad.net/~openstack
> > More help   : https://help.launchpad.net/ListHelp
> > 
> 
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp

-- 
Stephen Gran
Senior Systems Integrator - guardian.co.uk

Please consider the environment before printing this email.
------------------------------------------------------------------
Visit guardian.co.uk - newspaper of the year

www.guardian.co.uk    www.observer.co.uk     www.guardiannews.com 

On your mobile, visit m.guardian.co.uk or download the Guardian
iPhone app www.guardian.co.uk/iphone and iPad edition www.guardian.co.uk/iPad 
 
Save up to 37% by subscribing to the Guardian and Observer - choose the papers you want and get full digital access. 
Visit guardian.co.uk/subscribe 

---------------------------------------------------------------------
This e-mail and all attachments are confidential and may also
be privileged. If you are not the named recipient, please notify
the sender and delete the e-mail and all attachments immediately.
Do not disclose the contents to another person. You may not use
the information for any purpose, or store, or copy, it in any way.
 
Guardian News & Media Limited is not liable for any computer
viruses or other material transmitted with or as part of this
e-mail. You should employ virus checking software.

Guardian News & Media Limited

A member of Guardian Media Group plc
Registered Office
PO Box 68164
Kings Place
90 York Way
London
N1P 2AP

Registered in England Number 908396