enterprise-support team mailing list archive

[Andreas Hasenack <andreas@xxxxxxxxxxxxx>]

** Description changed:

  [Impact]
  When using syncrepl replication with openldap, the consumer needs to authenticate to the provider in order to perform the searches and fetch the data. When this authentication is a simple bind, a simple username/password pair is used and that can be easily supplied in a configuration file.
  
  When one wants to use a stronger authentication mechanism, like gssapi
  (kerberos), the authentication is based on keytab files and tickets. The
  consumer needs to obtain a ticket from the KDC, and use that ticket to
  authenticate itself with the provider.
  
  For users, it's a simple matter of running kinit(1) and providing a
  password. For services, the solution is to extract the key from the KDC
  and store it in a local keytab file, which is then used by the service
  to obtain the TGT.
  
  Problem is this TGT expires, and needs to be renewed periodically.
  Solutions have popped up for that issue, the most famous one being
  kstart (https://www.eyrie.org/~eagle/software/kstart/), but since the
  MIT kerberos 1.11 release, there is a simpler way.
  
  Via the default_client_keytab_name krb5.conf(5) option, one can specify
  the default location of a keytab file per local user. The kerberos
  library will then automatically use that file to obtain the TGT, and
  proceed as usual from there.
  
  The default value of that setting is /etc/krb5/user/<uid>/client.keytab.
  
  Turns out the openldap slapd apparmor profile doesn't account for that,
  and blocks attempts to read that file. It also blocks reading the TGT
  that is obtained and stored in /tmp/krb5cc_<uid>, resulting in such
  DENIED errors in the logs:
  
  apparmor="DENIED" operation="open" profile="/usr/sbin/slapd"
  name="/etc/krb5/user/389/client.keytab" pid=19080 comm="slapd"
  requested_mask="r" denied_mask="r" fsuid=389 ouid=389
  
  apparmor="DENIED" operation="file_lock" profile="/usr/sbin/slapd"
  name="/tmp/krb5cc_389" pid=19080 comm="slapd" requested_mask="k"
  denied_mask="k" fsuid=389 ouid=389
  
  Since the slapd apparmor is enabled by default, this blocks the usage of
  this very helpful feature. Also considering that kerberos/gssapi errors
  are usually hard to debug, it might take a while for an admin to figure
  out what is going on.
  
  The fix is to update the apparmor profile and allow access to those
  files. Unfortunately there are no rich globbing rules for paths in an
  apparmor profile, nothing like @{uid} of the current process yet, or a
  regexp rule like [0-9]+, so the rules have to be a bit accomodating. For
  this bug, I came up with these two new lines:
  
    /etc/krb5/user/*/client.keytab kr,
    owner /tmp/krb5cc_* rwk,
  
  One could relax the first one perhaps into something like /etc/krb5/**,
  but the above works with the default settings.
  
  [Test Case]
  Setting up openldap replication via gssapi can be complicated, so I wrote some scripts to help.
  - setup-kdc.sh: sets up the KDC server
  - setup-provider.sh: sets up the openldap provider
  - setup-consumer.sh: sets up the openldap consumer
  
  The scripts expect LXD containers to be used, that have a working DNS
  setup for a ".lxd" domain. In other words, if you do:
  
  lxc launch ubuntu-daily:bionic bionic-provider
  
  The script expects the container to be reachable via "bionic-
  provider.lxd". That is the default behavior of LXD, and changing the
  scripts to work in a more generic case was deemed not worth it.
  
  So here we go. Here are the steps for a cosmic test, just replace
  "cosmic" with the name of the ubuntu release you are testing.
  
  = KDC =
  
  * launch the KDC container and copy the setup-kdc.sh script into it:
  lxc launch ubuntu-daily:cosmic cosmic-kdc
  lxc file push ./setup-kdc.sh cosmic-kdc/root/
  
  * Enter the container and run the setup-kdc.sh script:
  lxc exec cosmic-kdc bash
  bash ./setup-kdc.sh
  
  * The script will show two prompts: one from debconf, with just an "ok"
  option, so hit ENTER there. The second prompt will be for a password.
  Use any one you like, it won't be needed again.
  
  * The KDC is setup, you can exit the container.
  
  = PROVIDER =
  * launch the provider and copy the setup-provider.sh script into it:
  lxc launch ubuntu-daily:cosmic cosmic-provider
  lxc file push ./setup-provider.sh cosmic-provider/root/
  
  * Enter the container and run the setup-provider.sh script:
  lxc exec cosmic-provider bash
  bash ./setup-provider.sh
  
  * Leave a tail on the slapd logs, we will come back to this later:
  tail -f /var/log/syslog|grep slapd
  
- 
  = CONSUMER =
  * launch the consumer and copy the setup-consumer.sh script into it:
  lxc launch ubuntu-daily:cosmic cosmic-consumer
  lxc file push ./setup-consumer.sh cosmic-consumer/root/
  
  * Enter the container and run the setup-consumer.sh script:
  lxc exec cosmic-consumer bash
  bash ./setup-consumer.sh
  
  * On the provider's log tail, you should soon see a connection and immediate disconnection like this:
  Oct 23 17:37:18 cosmic-provider slapd[2396]: conn=1004 fd=12 ACCEPT from IP=10.0.100.93:35276 (IP=0.0.0.0:389)
  Oct 23 17:37:18 cosmic-provider slapd[2396]: conn=1004 op=0 UNBIND
  Oct 23 17:37:18 cosmic-provider slapd[2396]: conn=1004 fd=12 closed
  
  * On this consumer log, you should see an error like this:
  tail /var/log/syslog |grep slapd
  ...
  Oct 23 17:37:18 cosmic-consumer slapd[2408]: do_syncrepl: rid=001 rc -1 retrying
  
  * On the host's log, dmesg will show an apparmor DENIED message like this:
  [104159.081883] audit: type=1400 audit(1540316238.330:1881): apparmor="DENIED" operation="open" namespace="root//lxd-cosmic-consumer_<var-lib-lxd>" profile="/usr/sbin/slapd" name="/etc/krb5/user/110/client.keytab" pid=22385 comm="slapd" requested_mask="r" denied_mask="r" fsuid=165646 ouid=165536
  
  This confirms the bug.
  
  The replication request from the consumer to the provider will be
  repeated every 60s, so these messages will continue until the fixed
  package is installed.
  
  To verify the fix, install the updated openldap packages on the consumer. Right after, the provider should log what is now a successful replication:
  Oct 23 17:41:34 cosmic-provider slapd[2396]: conn=1009 fd=12 ACCEPT from IP=10.0.100.93:35332 (IP=0.0.0.0:389)
  Oct 23 17:41:34 cosmic-provider slapd[2396]: conn=1009 op=0 BIND dn="" method=163
- Oct 23 17:41:39 cosmic-provider slapd[2396]: conn=1009 op=0 RESULT tag=97 err=14 text=SASL(0): successful result: 
+ Oct 23 17:41:39 cosmic-provider slapd[2396]: conn=1009 op=0 RESULT tag=97 err=14 text=SASL(0): successful result:
  Oct 23 17:41:39 cosmic-provider slapd[2396]: conn=1009 op=1 BIND dn="" method=163
- Oct 23 17:41:39 cosmic-provider slapd[2396]: conn=1009 op=1 RESULT tag=97 err=14 text=SASL(0): successful result: 
+ Oct 23 17:41:39 cosmic-provider slapd[2396]: conn=1009 op=1 RESULT tag=97 err=14 text=SASL(0): successful result:
  Oct 23 17:41:39 cosmic-provider slapd[2396]: conn=1009 op=2 BIND dn="" method=163
  Oct 23 17:41:39 cosmic-provider slapd[2396]: conn=1009 op=2 BIND authcid="consumer" authzid="consumer"
  Oct 23 17:41:39 cosmic-provider slapd[2396]: conn=1009 op=2 BIND dn="uid=consumer,cn=gssapi,cn=auth" mech=GSSAPI sasl_ssf=56 ssf=56
  Oct 23 17:41:39 cosmic-provider slapd[2396]: conn=1009 op=2 RESULT tag=97 err=0 text=
  Oct 23 17:41:39 cosmic-provider slapd[2396]: conn=1009 op=3 SRCH base="dc=lxd" scope=2 deref=0 filter="(objectClass=*)"
  Oct 23 17:41:39 cosmic-provider slapd[2396]: conn=1009 op=3 SRCH attr=* +
  
  The consumer should also have a TGT file in /tmp, with the uid number of
  the slapd process:
  
  root@cosmic-consumer:~# ll /tmp/krb5cc*
  -rw------- 1 openldap openldap 1903 Oct 23 17:41 /tmp/krb5cc_110
  root@cosmic-consumer:~# id -u openldap
  110
  
+ [Regression Potential]
+ The fix opens up reading of a file in a subdirectory of /etc/krb5. To prevent slapd from reading other files in that directory, or other directories, the sysadmin has to use normal filesystem permissions.
+ In /tmp we open up writing of TGT files with a globbing pattern (krb5cc_*). The somewhat generous globbing is counteracted by the "owner" restriction we use in that rule, so that should be fine.
  
- [Regression Potential]
- 
-  * discussion of how regressions are most likely to manifest as a result
- of this change.
- 
-  * It is assumed that any SRU candidate patch is well-tested before
-    upload and has a low overall risk of regression, but it's important
-    to make the effort to think about what ''could'' happen in the
-    event of a regression.
- 
-  * This both shows the SRU team that the risks have been considered,
-    and provides guidance to testers in regression-testing the SRU.
  
  [Other Info]
+ Setting the test scenario up manually is complicated even for admins familiar with the technologies involved, so a set of scripts was used. To prevent the scripts from becoming super complicated, a specific test scenario with LXD is targeted, and error conditions are not thoroughly checked. I welcome feedback if I missed some obvious case, but the scripts won't be changed into generic tools to setup slapd gssapi replication. That is a task for a blog post or manual :)
  
-  * Anything else you think is useful to include
-  * Anticipate questions from users, SRU, +1 maintenance, security teams and the Technical Board
-  * and address these questions in advance
+ This will be submitted to Debian as well, as reviews happen here first.
  
  [Original Description]
  
  Can we get /etc/krb5/** and /tmp/krb5cc_* added with the appropriate
  permissions to the slapd apparmor profile? I'm getting the following
  kinds of errors:
  
  apparmor="DENIED" operation="open" profile="/usr/sbin/slapd"
  name="/etc/krb5/user/389/client.keytab" pid=19080 comm="slapd"
  requested_mask="r" denied_mask="r" fsuid=389 ouid=389
  
  apparmor="DENIED" operation="file_lock" profile="/usr/sbin/slapd"
  name="/tmp/krb5cc_389" pid=19080 comm="slapd" requested_mask="k"
  denied_mask="k" fsuid=389 ouid=389

** Also affects: openldap (Ubuntu Trusty)
   Importance: Undecided
       Status: New

** Also affects: openldap (Ubuntu Cosmic)
   Importance: Medium
     Assignee: Andreas Hasenack (ahasenack)
       Status: In Progress

** Also affects: openldap (Ubuntu Xenial)
   Importance: Undecided
       Status: New

** Also affects: openldap (Ubuntu Bionic)
   Importance: Undecided
       Status: New

** Changed in: openldap (Ubuntu Bionic)
       Status: New => In Progress

** Changed in: openldap (Ubuntu Xenial)
       Status: New => In Progress

** Changed in: openldap (Ubuntu Trusty)
       Status: New => In Progress

** Changed in: openldap (Ubuntu Trusty)
   Importance: Undecided => Medium

** Changed in: openldap (Ubuntu Xenial)
   Importance: Undecided => Medium

** Changed in: openldap (Ubuntu Bionic)
   Importance: Undecided => Medium

** Changed in: openldap (Ubuntu Bionic)
     Assignee: (unassigned) => Andreas Hasenack (ahasenack)

** Changed in: openldap (Ubuntu Xenial)
     Assignee: (unassigned) => Andreas Hasenack (ahasenack)

** Changed in: openldap (Ubuntu Trusty)
     Assignee: (unassigned) => Andreas Hasenack (ahasenack)

** Attachment added: "setup-kdc.sh"
   https://bugs.launchpad.net/ubuntu/+source/openldap/+bug/1783183/+attachment/5204629/+files/setup-kdc.sh

-- 
You received this bug notification because you are a member of Ubuntu
Server/Client Support Team, which is subscribed to openldap in Ubuntu.
Matching subscriptions: Ubuntu Server/Client Support Team
https://bugs.launchpad.net/bugs/1783183

Title:
  apparmor profile denied for kerberos client keytab and credential
  cache files

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/openldap/+bug/1783183/+subscriptions