maas-devel team mailing list archive

[Gavin Panella <gavin.panella@xxxxxxxxxxxxx>]

On 14 April 2014 17:01, Graham Binns <graham.binns@xxxxxxxxxxxxx> wrote:
> Hi all,
>
> Julian's asked me to take charge of the work to rid us of the need to
> edit bootresources.yaml on the cluster controllers (or indeed get rid of
> the file altogether).
>
> Here are my initial thoughts; I'd appreciate your feedback so that we
> can get cracking. All questions, comments and thoughts welcome.
>
> bootresources.yaml
> ------------------
>
> 1. bootresources.yaml's contents can be reasonably easily held in the
>    database.  A standard bootresources.yaml looks like this:
>
>     boot:
>       storage: "/var/lib/maas/boot-resources/"
>       sources:
>         - path: "http://maas.ubuntu.com/images/ephemeral-v2/daily/";
>           keyring: "/usr/share/keyrings/ubuntu-cloudimage-keyring.gpg"
>           selections:
>             - release: "trusty"
>               arches: ["i386", "amd64"]
>               subarches: ["generic"]
>               labels: ["daily"]
>
> - The `storage` variable can easily be made a default.

Agreed.

In my mind, the goal is to have only two pieces of configuration on the
cluster:

- URL of the region's web API.

- Certificate for the RPC mesh/thing.

This will allow the cluster to bootstrap securely, for the region to
uniquely identify the cluster (the existing cluster UUID could be
embedded in the certificate for continuity).

However, settings like on-disk paths shouldn't be something that the
region cares about. We should have a default which it's possible to
override, intended chiefly for development.

> - The `sources` section is the meat of the file. ISTM that we can easily
>   turn what lives in this section into a Django model so that we can
>   have this configuration managed through the web UI and stored in the
>   database.
>
> Keeping bootresources.yaml around
> ---------------------------------
>
> We have a couple of options for how to do away with editing
> bootresources.yaml. We can:
>
> 1. Get rid of the file altogether; maas-import-pxe-files would either
>    given explicit configuration when run by celery, or would request it
>    from the region on startup.
> 2. Keep the file – it's useful for being able to re-run
>    maas-import-pxe-files manually should the need arise, and have it
>    updated by a celery job or using the RPC mesh.
>
> I favour option 2, but I'm open to suggestions.

I'm in favour of option 1, but Tycho's point about being able to run
maas-import-pxe-files manually makes me think again. I don't mind
keeping it, but it ought to move into /var/lib/maas if we do, and we
make no promises about preserving manual edits.

As for writing it, I suggest we send the sources list to the m-i-p-f
job, and it can write out a new bootresources.yaml file just before
running m-i-p-f.

Also, we ought to rename m-i-pxe-f to, say, maas-import-boot-resources.

>
> Configuration UI
> ----------------
>
> The configuration UI for this should be fairly straightforward. The only
> real question is whether we pre-populate the form with a set of useable
> options or give a more generic form that gives users more chance to
> shoot themselves in the foot by getting things wrong.
>
> I've attached two mockups:
>
> 1. Form pre-filled with useable options.
> 2. Generic form that requires more knowledge.
>
> Obviously we don't want to give people more potential foot-guns than
> necessary (though I don't think this one is by any means
> unrecoverable-from).
>
> If we did go with option #1, we have a further problem: pre-populating
> the UI.
>
> This is a bit of a chicken-and-egg. We already populate some UI elements
> (node architecture and power type fields) based on what a cluster knows
> it can support, but that requires the cluster to have downloaded stuff
> from simplestreams first (in the case of the architectures, anyway).
>
> So how do we know what we can ask a cluster to download before the
> cluster downloads anything?
>
> My first thought is that the region controller could introspect
> simplestreams somehow to find out what's availble, but that means that
> we'd have to do it on the fly every time someone puts a new
> simplestreams source into the form, which is impractical.

It's not impractical. Think of the cluster as just another data source,
like the database. This is not going to be a high-traffic part of the
MAAS UI, so we do it simply at first, i.e.

- First page: enter the stream URL, click next.

- Second page: the region downloads the stream and creates the UI around
  it. The user can then select those parts he/she wants.

Bear in mind that Mark wants us to download *everything* by default, so
the second page could be omitted in the first pass, and only available
by clicking "configure" later on.

Also, the stream is highly cacheable. We can ensure that it's downloaded
via a caching proxy, making the download-and-introspect approach even
more palatable, nay, toothsome.

>
> Any other thoughts on this are welcome, but for a first iteration I
> think that option #2 – simpler form with an accepted higher risk of user
> error – is fine. We can also make reporting problems due to user error
> less critical by having well-thought-out error messages (e.g. "You
> configured the cluster to download images for the following
> architectures: AlphaAXP, NASA-AGC; only the following architectures are
> available from this source: amd64, i386.")
>
> We could add an API to give maas-import-pxe-files a way to register such
> errors with the region.

We need better error reporting regardless, but I think the introspection
route is better from the start. User error is expensive to users, costs
the reputation of MAAS, and costs us dear in terms of support. Let's
prevent errors where we can.

>
> Finally, we need a listing page for the sources for a particular
> cluster. That's relatively easy to do, and in the case of the generic
> form above, could even benefit from some slick in-line UI to make things
> even easier to use.

Slick is stricly secondary to working though ;)