cloud-init-dev team mailing list archive

[Scott Moser <smoser@xxxxxxxxxx>]

On Tue, 8 Oct 2013, Haefliger, Juerg wrote:

> I just realized I never replied to this email. Sorry.
>
> > > All,
> > >
> > > OpenStack supports a rescue feature that allows a user to try to fix a
> > > broken instance. The way it works is that if you engage it, a new
> > > instance (the rescue instance) is booted using the same Glance image
> > > as the 'broken' instance and the root disk of the broken instance is
> > > attached as a second disk to the rescue instance. The problem is that
> > > both disks have the same filesystem UUIDs (since they're both based
> > > off the same Glance image) which seems to confuse the guest kernel.
> > > Sometimes things work as expected and sometimes the two disks are
> > > swapped.
> >
> > This generally sounds like broken design.  How could it possibly be expected
> > to work outside of hard coded 'root=/dev/vda1' on the kernel command line,
> > which would be busted so many other places.
>
> Not sure which design you're referring to. The fact that the same disk
> image is used for the rescue operation? Or the fact that both disk
> images have the same UUID?

I'm referring to the design that assumed if you attached 2 identical disks
that the bootloader (or initramfs or kernel or any part of the system)
would somehow just "know" which one it was supposed to boot off of.

I have to think about this more.  I don't think the solution that is in
openstack is a good one, and before changing guests to accommodate a bad
design decision, I'd like to see if I can come up with a better design.

Are you going to Openstack Summit? Maybe we could at least get a
unconference session time to talk about this ?
Scott

>
>
> > I wonder if we could fix that broken design.  I haven't spent time thinking
> > about that, but we shouldn't assume that we have to fix instances to
> > function according to someones shortsighted design.
>
> Well, it's a generic problem that we have a single image with a hard-coded UUID (or filesystem label) that can be used multiple times. Renders the UUID pretty much very non-unique. I don't see how we can fix that without letting the cloud environment modify the image before booting it and that's a broken design as well.
>
>
> > > I'm wondering if we should extend cloud-init to change the UUIDs on
> > > first boot. Obviously we'd have to also change it in all the places
> > > where it's being used (/etc/fstab, /boot/grub/grub.conf, ...) or the
> > > instance becomes unbootable (which makes me just a little bit nervous).
> >
> > This first popped up in bug 665235 (http://pad.lv/665235).
> >
> > Your suggestion of changing the uuid (and using uuid for boot) doesn't seem
> > terribly unreasonable, and would solve some cases.
> >
> > It doesn't solve another case that I've run into where I actually want to attach
> > the disk twice on first boot.  Once as the boot image, once as just a data disk.
> > The use case is for customizing the second disk (similar to libguestfs would
> > do).  I know this is an odd case, but I'm just throwing it out there.
>
> Yeah, you're right. Modifying the UUID/label wouldn't solve this problem.
>
>
> > Which device to use as root device is a decision made at initramfs time,
> > combined with kernel command line.
> >
> >
> > > Comments? Suggestions?
> >
> > Sorry I dont have easy answers.  I just want to think this trough and i'm not
> > interested in doing something now that hackily solves a specific problem.
>
> Sure. That's why we're having this discussion.
>
> ...Juerg
>
> > Scott
>
>