openstack team mailing list archive

[Soren Hansen <soren@xxxxxxxxxxx>]

2011/11/29 Vishvananda Ishaya <vishvananda@xxxxxxxxx>:
> On Nov 29, 2011, at 11:16 AM, Soren Hansen wrote:
>> The way I'd attack these expensive-if-done-one-at-a-time-but-dirt-cheap-
>> if-done-as-one-big-query is to have a method in the generic layer that
>> is taylored for this use case. E.g.
>>
>> def instances_get_all_for_network_with_fixed_ip_addresses():
>>    retval =  []
>>    for inst in instance_get_all_by_network():
>>        x = inst.copy()
>>        x['fixed_ip_addresses'] = []
>>        for ip in fixed_ip_get_by_instance(inst['id']):
>>            x['fixed_ip_addresses'].append(ip['address'])
>>               retval.append(x)
>>    return x
>
> I think there are a couple of issues with this approach:
>
>  1) combinatorial explosion of queries.     Every time we need an
> additional joined field we will be adding a new method. This could get
> out of hand if we aren't careful.

This is true. However, the code needs to live somewhere, so to speak.
We can either have separate methods for separate use cases or the logic
to support the separate use cases will just need to exist as
conditionals in more generic methods.

I'm just concerned we'll refactor this and end up with something else
that is just as SQLAlchemy specific as the old implementation, only with
a different API style. It's hard (for me, at least) to predict how other
drivers will be able to achieve good performance, so I'd be wary of
going down a particular path just because I can envision how I'd
implement it with the SQLAlchemy backend.

> 2) interface clarity.

>  Sometimes an instance dict contains extra fields and other times it
> doesn't. This is especially annoying    if the resulting object is
> passed through a queue or another method.  Does the instance object
> have    an address or not?  Do I need to explicitly request it or is
> it already embedded?

Indeed. We have this problem already, though. If I do an instance_get, a
bunch of things will already be joined in and available for me to use.

However, if I get hold of an Instance object from somewhere else (say,
from a FloatingIp object: floating_ip_ref['fixed_ip']['instance']) if I
attempt to access something that hasn't been eagerly joined, I'll get
the aforementioned, infamous DetachedInstanceError.

If we switch to this other style instead, I think we'll fairly quickly
get used to the idea that we shouldn't expect anything but the bare
essentials to be available (e.g. only the id's of related object, not
the objects themselves (for the one-to-many use case), and to get
related objects (in the many-to-one use case), you have to call a
different method altogether, etc.), and we'll automatically be more
careful about our use of the various objects. Or so the hypothesis goes.

> These are probably surmountable obstacles if we want to go this route.
>  I just want to point out that it has its own drawbacks.

Agreed.

-- 
Soren Hansen        | http://linux2go.dk/
Ubuntu Developer    | http://www.ubuntu.com/
OpenStack Developer | http://www.openstack.org/