maas-devel team mailing list archive

[Gavin Panella <gavin.panella@xxxxxxxxxxxxx>]

On 23 May 2014 07:59, Mark Shuttleworth <mark@xxxxxxxxxx> wrote:
> On 22/05/14 16:16, Gavin Panella wrote:
>> On the maas-import-pxe-files side (i.e. the import script), it needs
>> that config to drive it. It doesn't know what to do otherwise. That
>> config has a nested structure which would be awkward at best to
>> translate to the command-line.
>
> It has to decide what to download. If it downloads "everything" then the
> "config" can be hardcoded - it's everything that we, Canonical, publish
> for this purpose. That way one MAAS region is the same as the next MAAS
> region. They have all the bits we publish.

Right, this is the outcome, but without an absolutely set-in-stone
hard-coded configuration. We generate the config on the region, with
everything in, and send it to clusters for processing.

>
>> On the region side, MAAS will generate that config with everything in
>> it by default, as agreed, and pass it over to the import script to run
>> on the cluster. However, we need a way to narrow that selection down
>> for development; it takes too long to download the ~400MB per
>> arch+subarch per release, especially when trying to iterate. For
>> amd64, i386, arm, and power, that's 1.6GB (7.2GB on disk once
>> uncompressed) before even considering different subarchitectures.
>> That's not cool for someone on DSL, even with caching. The
>> certification team also wants to be able to narrow down the selection
>> of boot resources.
>
> U... you are adding a lot of complexity - and configurability - for
> this? Seems like you could have a much simpler outcome that just focuses
> on a single version / architecture then lets the rest go on in the
> background.

We could change it to do that, but it already works as I've described,
and has done since Trusty was first released. It's extra work to do as
you describe.

>
>> They'll also need to be able to use custom resources. I imagine this
>> is true for the Hardware Enablement team too. The UI was mocked-up
>> before Austin, but has not been implemented. We need to talk to the UX
>> team about it; my guess is that they might recommend putting it behind
>> a developer-only flag, or leaving it as API-only, but that's an
>> engineer's guess :)
>
> So you want us to do a lot of design and implementation then hide it
> away? PLEASE STOP.

Okay.

We've done only minimal design work so far; we could also see that it's
not worth dedicating much to. It would have come up as part of a wider
discussion about usability with the UX team; it would not have had time
dedicated to it.

>
>> To summarise: - We need configuration to drive the import script.
>
> No we don't. We need to stop, think carefully, and come up with a
> simpler approach that does not invite:
>
>  * bastard complex stupid config ("I want the arm-hf and my own custom
> hackery over there please"). Our goal is to deliver Ubuntu, CentOS,
> Windows, period. The rest are custom uploads.

I think you're imagining that it's more complex than it is. The import
code that works from a provided config is already done and was released
with Trusty. I only said that the configuration was not amenable to
being expressed well with command-line arguments, not that it was a
baroque monstrosity.

You have asked repeatedly for two things:

1. Configuration and state in the region (and not as config files).

2. Download all boot images.

We are taking what was already there - which was built to satisfy the
use-cases as were understood them at the time - and are reforming it to
fulfill the points above.

>  * unnecessary code

We are not gratuitously adding code. With the code we'll be able to
excise, we'll probably end up with no net change to the size of the
codebase.

>  * design work that gets hidden away

Okay.

>
>> The region therefore has ultimate control over what is imported; there
>> are no defaults baked into the cluster. - We *could* change the import
>> script to know more, and thus run without configuration. However, we
>> want to get rid of the script as a directly runnable entity, so this
>> would be short-lived work. - We need a way to narrow down the boot
>> resources we import for the sake of developers, both core and casual,
>> and for the certification team.
>
> The certification team can use a custom image for testing purposes but
> remember they WILL NOT CERTIFY anything that requires something special,
> so you don't really have to argue for hackery on their behalf, they are
> just normal Ubuntu users. There is a requirement to be able to use
> pre-production versions of Ubuntu kernels, yes, but that's a specific
> use case.

Okay, perhaps the needs of the certification and hardware enablement
teams have been overstated. With the cessation of UI work on this we are
now into QA; no additional development is needed.

>
>> - We already do not recommend that regular users run
>> maas-import-pxe-files directly; they should use the UI or the API.
>> That there is no good feedback on success/failure/progress for this is
>> a problem we're working on this cycle.
>
> Thank you, but hiding the complexity from users is not the same as
> eliminating complexity. Think more carefully about this please, you're
> just moving the complexity, not addressing it directly.

There is complexity in managing configuration, hard-coded or not. Users
and developers are going to change this configuration however hard we
try to prevent it. This already happened with power types.

By putting this configuration in the database we've made it easier for
us to manage - via database migrations - and avoid breaking user's
systems when upgrading.

Gavin.