openstack team mailing list archive

[Vishvananda Ishaya <vishvananda@xxxxxxxxx>]

Inline

On Nov 29, 2011, at 11:16 AM, Soren Hansen wrote:

> 2011/11/29 Vishvananda Ishaya <vishvananda@xxxxxxxxx>:
>> e) is the right solution imho.  The only reason joinedloads slipped in is for efficiency reasons.
>> 
>> In an ideal world the solution would be:
>> 
>> 1) (explicitness) Every object or list of related objects is retrieved with an explicit call:
>>  instance = db.instance_get(id)
>>  ifaces = db.interfaces_get_by_instance(id)
>>  for iface in ifaces:
>>     ip = db.fixed_ip_get_by_interface(iface['id'])
>> 2) (efficiency) Queries are perfectly efficient and all joins that will be used are made at once.
>>  So the above would be a single db query that joins all instances ifaces and ips.
> 
> The way I'd attack these expensive-if-done-one-at-a-time-but-dirt-cheap-
> if-done-as-one-big-query is to have a method in the generic layer that
> is taylored for this use case. E.g.
> 
> def instances_get_all_for_network_with_fixed_ip_addresses():
>    retval =  []
>    for inst in instance_get_all_by_network():
>        x = inst.copy()
>        x['fixed_ip_addresses'] = []
>        for ip in fixed_ip_get_by_instance(inst['id']):
>            x['fixed_ip_addresses'].append(ip['address'])
> 		retval.append(x)
>    return x

I think there are a couple of issues with this approach:

 1) combinatorial explosion of queries.
    Every time we need an additional joined field we will be adding a new method. This could get out of hand
    if we aren't careful.
 2) interface clarity.
    Sometimes an instance dict contains extra fields and other times it doesn't. This is especially annoying
    if the resulting object is passed through a queue or another method.  Does the instance object have
    an address or not?  Do I need to explicitly request it or is it already embedded?

These are probably surmountable obstacles if we want to go this route.  I just want to point out that it has its
own drawbacks.

> 
> And then, in the sqlalchemy driver, I could override that method with
> one that issues a query with joinedloads and all the rest of it. The
> intent is explicit, drivers that have no speedier way to achieve this
> get a free implementation made up of the more primitive methods.
> 
> fixed_ip_get_by_instace might also have a default implementation that
> issues a fixed_ip_get_all() and then filters the results. This way, a
> new driver would be quick to add, and then we could optimize each query
> as we move along.
> 
> -- 
> Soren Hansen        | http://linux2go.dk/
> Ubuntu Developer    | http://www.ubuntu.com/
> OpenStack Developer | http://www.openstack.org/