sts-sponsors team mailing list archive

[Matthew Ruffell <matthew.ruffell@xxxxxxxxxxxxx>]

Hi everyone,

Firstly, I deeply apologise for causing the regression.

Even with three separate people testing the test packages and the packages in
-proposed, the failure still went unnoticed. I should have considered
the impacts
of changing the default behaviour of adcli a little more deeply than treating it
like a normal SRU.

Here are the facts:

The failure is limited to adcli, version 0.8.2-1ubuntu1 on Bionic. At the time
of writing, it is still in the archive. To archive admins, this needs
to be pulled.

adcli versions 0.9.0-1ubuntu0.20.04.1 in Focal, 0.9.0-1ubuntu1.2 in Groovy and
0.9.0-1ubuntu2 in Hirsute are not affected.

sssd 1.16.1-1ubuntu1.7 in Bionic, and 2.2.3-3ubuntu0.1 in Focal are
not affected.

Bug Reports:

There are two launchpad bugs open:

LP #1906627 "adcli fails, can't contact LDAP server"
https://bugs.launchpad.net/ubuntu/+source/adcli/+bug/1906627

LP #1906673 "Realm join hangs"
https://bugs.launchpad.net/ubuntu/+source/sssd/+bug/1906673

Customer Cases:

SF 00298839 "Ubuntu Client Not Joining the Nasdaq AD Domain"
https://canonical.my.salesforce.com/5004K000003u9EW

SF 00299039 "Regression Issue due to
https://bugs.launchpad.net/ubuntu/+source/sssd/+bug/1906673";
https://canonical.my.salesforce.com/5004K000003uAkL

Root Cause:

The recent SRU in LP #1868703 "Support "ad_use_ldaps" flag for new AD
requirements (ADV190023)"
https://bugs.launchpad.net/ubuntu/+source/sssd/+bug/1868703

introduced two changes for adcli on Bionic. The first, was to change from
GSS-API to GSS-SPNEGO, and the second was to implement support for the flag
--use-ldaps.

I built a upstream master of adcli, and it still fails on Ubuntu. This indicates
that the failure is not actually in the adcli package. adcli does not implement
GSS-SPNEGO, it is linked in from the libsasl2-modules-gssapi-mit package,
which is a part of cyrus-sasl2.

I built the source of cyrus-sasl2 2.1.27+dfsg-2 from Focal on Bionic, and it
works with the problematic adcli package.

The root cause is that the implementation of GSS-SPNEGO in cyrus-sasl2 on
Bionic is broken, and has never worked.

There is more details about commits which the cyrus-sasl2 package in Bionic is
missing in comment #5 in LP #1906627.

https://bugs.launchpad.net/ubuntu/+source/adcli/+bug/1906627/comments/5

Steps taken yesterday:

I added regression-update to LP #1906627, and I pinged ubuntu-archive in
#ubuntu-release with these details, but they seem to have been lost in the
noise.

Located root cause to cryus-sasl2 on Bionic.

Next steps:

We don't need to revert any changes for adcli or sssd on Focal onward.

We don't need to revert any changes on sssd on Bionic.

We need to push a new adcli into Bionic with the recent patches reverted.

We need to fix the GSS-SPNEGO implementation in cyrus-sasl2 in Bionic.

We need to re-release all the SRUs from LP #1868703 after some very thorough
testing and validation.

Again, I am deeply sorry for causing this regression. I will fix it, starting
with getting adcli removed from the Bionic archive.

Thanks,
Matthew

On Sat, Dec 5, 2020 at 10:37 AM Jamie Strandboge <jamie@xxxxxxxxxxxxx> wrote:
>
> Looping in security@
> On Fri, 04 Dec 2020, Sergio Durigan Junior wrote:
>
> > Hi Matthew,
> >
> > How are things?  I'm writing to you because the last upload to
> > sssd/adcli introduced a regression that is causing "realm join" to
> > hang.  The bug in question is this one:
> >
> >   https://bugs.launchpad.net/ubuntu/+source/sssd/+bug/1906673
> >
> > There is also a SalesForce case opened from AWS:
> >
> >   https://canonical.my.salesforce.com/5004K000003uAkLQAU
> >
> > (I don't have access to it, but cnewcomer said it's basically the same
> > issue, but that AWS is actually reporting it against adcli).
> >
> > I am not entirely sure whether this bug affects both sssd and adcli, or
> > just one of them.  It is possible that this is just affecting adcli,
> > based on input from Tobias Karnat, but we have to investigate this
> > further.
> >
> > This regression was introduced because of the work done here:
> >
> >   https://bugs.launchpad.net/ubuntu/+source/sssd/+bug/1868703
> >
> > Lukasz (sil2100) has already pulled the sssd package from the
> > -security/-update pockets.  I've asked him to also pull the adcli
> > package.  At the time of this writing, he hasn't done that yet (he had
> > to go AFK), but he told me he would.  In any case, this is not going to
> > help much because by now most systems probably have the updates already
> > because of unattended-upgrades.
> >
> > Having said all that, would it be possible for you to handle this issue?
> > I can offer any help you need, of course, but I feel like you already
> > have all the context in your head and would be able to make progress
> > much faster.
> >
> > Thanks in advance,
> >
> > --
> > Sergio
> > GPG key ID: E92F D0B3 6B14 F1F4 D8E0  EB2F 106D A1C8 C3CB BF14
>
>
> --
> Jamie Strandboge             | http://www.canonical.com