sts-sponsors team mailing list archive

[Mauricio Oliveira <mauricio.oliveira@xxxxxxxxxxxxx>]

Hi Matthew,

Long story short, the debian-installer rebuild on PPA fails on Bionic.

The fix for this should apparently involve changes to kmod and systemd,
at least systemd, and I think this may get push back from release team
at this time before the point release (however small the change is.)

If the fix is needed only in the network-installer, that can be an SRU later.

Let me explain, and hopefully you can start addressing it with Lukasƶ
on your Monday, before I join.

The d-i build on PPA fail because 'udev-udeb' cannot pull 'libkmod2'.

    Building dependency tree...
      Installing libslang2-udeb as Depends of cdebconf-newt-udeb
      udev-udeb:amd64 Depends on libkmod2:amd64 < none @un H > can't
be satisfied!

    ...

    The following packages have unmet dependencies:
     udev-udeb : Depends: libkmod2 but it is not installable
    E: Unable to correct problems, you have held broken packages.
    Makefile:620: recipe for target
'stamps/get_udebs-cdrom_isolinux-stamp' failed
    make[7]: *** [stamps/get_udebs-cdrom_isolinux-stamp] Error 100

This is correct, because the d-i build does configure sources only for udeb
packages, and libkmod2 is a regular deb.

Previously it worked because udev-udeb pulled libkmod2-udeb (note suffix.)

This is a regression in the dependencies of udev-udeb in 237-3ubuntu10.40:

    https://launchpad.net/ubuntu/bionic/amd64/udev-udeb/237-3ubuntu10.39
        Depends on: libkmod2-udeb

    https://launchpad.net/ubuntu/bionic/amd64/udev-udeb/237-3ubuntu10.40
        Depends on: libkmod2

This happened because the build dependency kmod (which you're familiar with :)
changed versions in between, 24-1ubuntu3.2 to 3.3:

    On u10.39: Get:45 http://ftpmaster.internal/ubuntu
bionic-updates/main amd64 libkmod2 amd64 24-1ubuntu3.2 [40.0 kB]
    On u10.40: Get:45 http://ftpmaster.internal/ubuntu
bionic-updates/main amd64 libkmod2 amd64 24-1ubuntu3.3 [40.1 kB]

And this change caused this:

    kmod (24-1ubuntu3.3) bionic; urgency=medium
    ...
      * Removed --add-udeb from dh_makeshlibs, since libkmod2-udeb does not
        actually contain a library. (Closes: #939779)
    ...

This still builds/ships the libkmod2-udeb package, but...

The lack of dh_makeshlibs with --add-udeb causes the libkmod2
(non-udeb)'s "shlibs"
file to no longer indicate that udeb packages which depend on that
library package
should depend on the -udeb package instead (exactly what happens here.)

>From dh_makeshlibs(1):

       --add-udeb=udeb
           Create an additional line for udebs in the shlibs file and
use udeb as the package name for udebs to depend on instead of the
regular library package.

This can be confirmed with:

    $ pull-lp-debs -a amd64 libkmod2 bionic 24-1ubuntu3.2
    $ pull-lp-debs -a amd64 libkmod2 bionic 24-1ubuntu3.3

    $ dpkg-deb --control libkmod2_24-1ubuntu3.2_amd64.deb libkmod2-32-control
    $ dpkg-deb --control libkmod2_24-1ubuntu3.3_amd64.deb libkmod2-33-control

    $ cat libkmod2-32-control/shlibs
    libkmod 2 libkmod2
    udeb: libkmod 2 libkmod2-udeb

    $ cat libkmod2-33-control/shlibs
    libkmod 2 libkmod2

And according to systemd/debian/control file's entry for udev-udeb:

    Package: udev-udeb
    ...
    Depends: ${shlibs:Depends},
         ${misc:Depends},
         util-linux-udeb

That dependency comes from ${shlibs:Depends}, handled by dh_shlibdeps(1):

    https://www.debian.org/doc/manuals/maint-guide/dreq.en.html
    """
    dh_shlibdeps(1) calculates shared library dependencies for binary packages.
    It generates a list of ELF executables and shared libraries it has
found for each binary package.
    This list is used for substituting ${shlibs:Depends}.
    """

So,

Since d-i needs both udev-udeb and libkmod2-udeb (for udev,
load/remove modules, etc)
fixing the udev-udeb dependency would require *at least* a
change/rebuild to systemd
to update the Depends: field of udev-udeb.

Say, even if we came up with a hack/solution that doesn't need a
change/fix to libkmod2.

This is where I think the SRU/release team might prefer to hold things
for release.
(systemd.)

At this point I think that *if* the case/matter is only about the
network installer,
not the ISO (which is indeed critical with 18.04.5), then it can be
fixed via SRU,
without any concerns.

Would you please evaluate this w/ customer/Lukasƶ on Monday?


A bit on libkmod2 change:

The change/regression of --add-udeb to libkmod2 is not incorrect on
its own, since
libkmod2-udeb doesn't ship a shared library.

I discussed this a bit with Rafael Tinoco (which introduced the
change), and perhaps
the change can be reverted (I'd have to check with a rebuild of kmod
w/ it reverted.)

(then rebuild kmod, then systemd, then d-i -- after debootstrap w/
your fix, of course.
See it's a chain that looks big for this time before the release. :)


Some tech details:

udev-udeb depends on libkmod2 shared library because of udevadm, for
the kmod built-in.

You may notice that even before the --add-udeb flag removal,
libkmod2-udeb *did not*
ship that shared library!

I wondered how the installer worked. It DOES have that library in
/lib/libkmod.so.2.
And I have no idea where it pulled it from.

I checked MANIFEST files, d-i build log, even download all udebs in
bionic/bionic-updates,
extracted, but nothing shipped it.

This might be some d-i build magic, which I didn't look/find yet.


Finally, I'm a bit tired to think now, so I'll give you the ball for now to play
a bit on considerations of 1) should this really be fixed before
release for ISO,
or just network installer is OK?   and more fun in 2) how to properly
fix this :)

Thanks and have a great weekend!

P.S.: I'm attaching my hacky notes about this today, in case it might help.

On Fri, Jul 24, 2020 at 7:27 PM Mauricio Oliveira
<mauricio.oliveira@xxxxxxxxxxxxx> wrote:
>
> Matthew,
>
> By the way, per the SF case, the customer uses the network install method.
> This can be fixed post-release, as SRUs to debian-installer will
> produce new netboot images.
>
> Hopefully they (and other people) don't plan to try to use HTTPS on
> the _ISO_ installer?
> The ISO image is what is more strictly tied to the release date.
> I'm not sure how much that scenario makes sense.
> (there may be the case of trying to use HTTPS for the ISO installer to
> download updates though?)
>
> I'm saying this because this is now (of course!) heading into systemd :-)
> specifically the dependencies of the udev-udeb package.  And that may
> be just an adventure
> to try to fix / SRU before release date. :)
>
> I'll give more details later.
>
> cheers,
>
>
> On Fri, Jul 24, 2020 at 6:30 PM Mauricio Oliveira
> <mauricio.oliveira@xxxxxxxxxxxxx> wrote:
> >
> > Definitely looking at it right now. :)
> >
> > The debootstrap portion is OK. However,
> >
> > there's a PPA build error with the required rebuild of d-i in Bionic
> > (not on Focal -- there's always fun on the releases we actually need!)
> > due to udev-udeb changing dependencies from libkmod2-udeb to libkmod2
> > (non-udeb), which I just tracked down to kmod 24-1ubuntu3.3.
> >
> > I'll provide a write up on the LP bug (and mention internally in SF
> > case) and hopefully it should be relatively simple to fix.
> >
> > As mentioned, debootstrap looks good; so I'll move that one soon.
> >
> > cheers,
> >
> > On Fri, Jul 24, 2020 at 6:25 PM Dan Streetman
> > <dan.streetman@xxxxxxxxxxxxx> wrote:
> > >
> > > Hi Matthew,
> > >
> > > As @mfo has done work in this (installer image) area before (LP:
> > > #1807023) I asked him to take a look; I'm cc'ing our sts sponsors
> > > email list (which includes Mauricio) also.
> > >
> > > I think he's handling the sponsoring, so hopefully it'll make it for
> > > the point release!
> > >
> > > Thanks!
> > >
> > > On Fri, Jul 24, 2020 at 1:14 AM Matthew Ruffell
> > > <matthew.ruffell@xxxxxxxxxxxxx> wrote:
> > > >
> > > > Hi Dan and Eric,
> > > >
> > > > [URGENT] Please review and sponsor LP1772556. This needs to make 18.04.5 LTS.
> > > >
> > > > https://bugs.launchpad.net/ubuntu/+source/debootstrap/+bug/1772556
> > > >
> > > > I have written to Lukasz Zemczak and made him aware of this bug, and I have also
> > > > asked him to review and sponsor the debdiffs, since he was the last to modify
> > > > debootstrap in Bionic, and is also in charge of producing 18.04.5 LTS images.
> > > >
> > > > I haven't heard back from Lukasz though. I emailed him on my Wednesday, and
> > > > on my Thursday.
> > > >
> > > > Could you please maybe ping Lukasz during your day to make sure he got my email?
> > > >
> > > > The debdiffs in the LP bug are kinda weird. debootstrap uses "3.0 (native)"
> > > > instead of "3.0 (quilt)", and the debdiff seems to take diffs of all the ubuntu
> > > > releases, even though they are all symlinks to a single file that got changed,
> > > > which is "gutsy".
> > > >
> > > > Anyway, please nudge Lukasz, or sponsor my debdiffs. Konica Minolta filed a
> > > > L2 about this in SF289200, and it needs to make 18.04.5 LTS or it will never be
> > > > fixed.
> > > >
> > > > Thanks!
> > > > Matthew
> > >
> > > --
> > > Mailing list: https://launchpad.net/~sts-sponsors
> > > Post to     : sts-sponsors@xxxxxxxxxxxxxxxxxxx
> > > Unsubscribe : https://launchpad.net/~sts-sponsors
> > > More help   : https://help.launchpad.net/ListHelp
> >
> >
> >
> > --
> > Mauricio Faria de Oliveira
>
>
>
> --
> Mauricio Faria de Oliveira



-- 
Mauricio Faria de Oliveira

Attachment: 0724
Description: Binary data