ubuntu-phone team mailing list archive

[Ricardo Salveti de Araujo <ricardo.salveti@xxxxxxxxxxxxx>]

On Wed, Mar 12, 2014 at 8:05 PM, Colin Watson <cjwatson@xxxxxxxxxx> wrote:
> On Wed, Mar 12, 2014 at 06:30:04PM -0300, Ricardo Salveti de Araujo wrote:
>> On Wed, Mar 12, 2014 at 5:20 PM, Sergio Schvezov
>> <sergio.schvezov@xxxxxxxxxxxxx> wrote:
>> > So instead of 'uninstalling/unregistering' them from the default
>> > image; it would just not be installed.
>> >
>> > cjwatson mention that this is really not necessary; but I still wanted
>> > to explore the alternative.
>>
>> But are they removed completely from the disk if you unregister them?
>> We'd still have some additional logic during first boot that is not
>> necessarily useful for all the cases.
>
> That's not really how it works.  While maybe we don't want to ship
> exactly the set of core apps we ship today, I think removing all apps is
> entirely unnecessary, not to mention somewhat unhelpful for me in that
> it makes it more troublesome to test preinstalled cases as part of our
> normal CI processes, and it makes our stock images less useful.  I'll
> try to explain how it actually works and then hopefully this will make
> more sense to people.

The idea wasn't necessarily to remove all the click packages, but at
least a subset. But I agree that this complicate things as well,
specially when testing the image.

> == Technical details (you can skip this if you don't care) ==
>
> Click has multiple databases where packages may be unpacked: by default
> we have the "core" database for core apps
> (/usr/share/click/preinstalled/), the "custom" database for carrier/OEM
> customisations (/custom/), and the "default" database for user-installed
> applications (/opt/click.ubuntu.com/), although these are configurable in
> /etc/click/databases/.  Each database may have multiple unpacked
> versions of any given package.
>
> Each database may also have user registrations, which live in
> .click/users/ relative to the database root.  Each user has a
> subdirectory of that, which contains symlinks to the versions of each
> package they have registered.  This means that on a tablet, say, I can
> install an app without it also showing up on my children's accounts;
> they'd need to install it separately, although the disk space for the
> unpacked copy of the app would be shared.
>
> There was an idea early on that we'd deal with preinstalled apps by
> going round and registering them all for all active users on first boot.
> This made it onto a week or so's worth of images in the saucy cycle
> because I was on holiday at the time.  This has lots of problems for the
> packaging system, though, and I killed it as quickly as I could.  Most
> notably, doing it that way makes it hard for a user to remove an app and
> make it stick, because it would tend to reappear on system updates.  You
> can probably fudge your way around this somehow, but it gets very fiddly
> and easy to get wrong.
>
> What we do instead is: we have an "@all" pseudo-user which you can
> register packages for, typically in the core database ("click register
> --root=/usr/share/click/preinstalled --all-users").  If a user wants to
> remove a package, we do this by creating a deliberately broken symlink
> pointing to "@hidden" in their user registration area in
> /opt/click.ubuntu.com/.click/users/$USERNAME/.  When click is asked to
> list the set of packages for a given user, it walks its way down the
> list of databases from top (default) to bottom (core).  For each
> database, it checks registrations for that user, followed by
> registrations for "@all".  It takes the first registration for any given
> package name that it finds.  If that registration is "@hidden", then it
> ignores the package, otherwise it must be a link to the unpacked copy of
> the appropriate version of the package.
>
> There are still some things that can't be done just with static files in
> the image and instead have to be done at boot time and on session
> startup: we have to make sure the right AppArmor profiles are loaded, do
> things to the user's home directory like creating .desktop files, and
> that kind of thing.  We run "click hook run-system" at boot time and
> "click hook run-user" on session startup, and these deal with running
> hooks for whatever packages are visible in context, according to the
> rules above.
>
> All this is very well-established logic in Click; it has been in place
> since September without needing significant modifications.  (I did some
> work on it this week in response to corner cases that showed up after
> not having touched it for ages, but that was just refinement.)
>
> == TL;DR ==
>
> The effect of all this is that we can hide a core app for a carrier by
> doing this as root when preparing their custom overlay image:
>
>   click unregister --root=/custom --all-users PACKAGE-NAME
>
> This will create a symlink /custom/.click/users/@all/PACKAGE-NAME
> pointing to "@hidden".  Unless a user explicitly installs the app in
> question, the effect of this will be that it's as if the app just isn't
> there.  It shouldn't incur any more than a negligible cost at startup
> (basically just a readlink call); at the moment I think we might still
> create an AppArmor profile for it, which isn't free, but that can be
> fixed easily enough.
>
> Obviously this doesn't free the disk space used by the unpacked copy of
> the app in the base image; we can't, since it's read-only.  I would not
> expect the preinstalled apps to be particularly large; if there are
> large ones that are commonly hidden by carriers, or maybe ones that are
> frivolous enough that they don't add much value to our stock images,
> then I can see the logic in removing those entirely; but that shouldn't
> be the first thing we reach for.
>
> I think this unregister-for-all-users mechanism is the best way to deal
> with this kind of thing, and I designed the multiple-database system in
> Click with exactly this use case as one of those I had in mind.

I agree, this is enough to cover our current use case. We still don't
know if the carrier will indeed ask us for the default apps to not be
installed at all, but that's hart do predict.

Thanks a lot for the detailed explanation, really useful (which brings
a question if this is documented already somewhere).

Cheers,
-- 
Ricardo Salveti de Araujo