yahoo-eng-team team mailing list archive

[James Falcon <1911680@xxxxxxxxxxxxxxxxxx>]

This was released in the 20.4.1 hotfix

** Changed in: cloud-init
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1911680

Title:
  Wrong access permissions of authorized keys file and parent directory
  when using absolute AuthorizedKeysFile

Status in cloud-init:
  Fix Released

Bug description:
  [Impact]

  For users who provide more than one unique SSH key to cloud-init and
  have a shared AuthorizedKeysFile configured in sshd_config, cloud-init
  20.4 started writing all of these keys to such a file, granting all
  such keys SSH access as root.  Previously, cloud-init did not handle
  such sshd_config configuration, and so would have written users' keys
  to the hardcoded `~/.ssh/authorized_keys` unconditionally.

  [Test Case]

  This test case has been written for use with the cloud-init
  integration testing framework:

  ```
  ABSOLUTE_AUTH_KEYS_PATH = "/etc/ssh/authorized_keys"
  BACKDOOR_KEYS = "/etc/backdoor_keys"
  USER_DATA_TMPL = """\
  #cloud-config
  bootcmd:
  - sed -i "s,#AuthorizedKeysFile.*,AuthorizedKeysFile {0} {1}," /etc/ssh/sshd_config
  runcmd:
  - |
      cat << EOF > {1}
      {{}}
      EOF
  """.format(ABSOLUTE_AUTH_KEYS_PATH, BACKDOOR_KEYS)

  
  # This test checks that we do not write keys into an absolute
  # AuthorizedKeysFile by default, because doing so presents a security issue.
  #
  # Because we're intentionally testing that we do not write authorized keys to
  # the location configured for sshd, we cannot SSH into the system using keys in
  # that path.  We use runcmd to populate a second configured path to grant us
  # access; runcmd will install the key after all the SSH key determination has
  # been completed, therefore not affecting it.

  
  def test_test(session_cloud, setup_image):
      # We shouldn't write out absolute files until we've figured out how to do
      # it right
      user_data = USER_DATA_TMPL.format(
          session_cloud.cloud_instance.key_pair.public_key_content
      )
      with session_cloud.launch(user_data=user_data) as client:
          assert client.execute(
              "test -f {}".format(ABSOLUTE_AUTH_KEYS_PATH)
          ).failed
  ```

  [Regression Potential]

  This update touches only cloud-init's SSH handling, by reverting a
  commit which modified its behaviour.  It has been present in Ubuntu
  images since the 6th of January.

  Users who have started relying on the new behaviour (cloud-init
  writing keys to non-standard AuthorizedKeysFile locations configured
  in sshd_config) in the past ~2 weeks will see this behaviour stop
  working.

  [Original Report]

  Starting on the 6th January 2021 we started observing SSH
  authentication issues in AWS AMI builds.

  We have SSH configured with an absolute (i.e. rather than per-user)
  authorised keys file, e.g.

     AuthorizedKeysFile  /etc/ssh/authorized_keys

  We observed that the file and parent folder permissions had been
  modified, to:

     /etc/ssh - 0700
     /etc/ssh/authorized_key - 0600

  These permissions would be fine if the authorised keys file were in a
  users home directory, but not for a centrally owned absolute file.

  We investigated and identified that between the 4th and 6th January
  2021, cloud-init on Ubuntu 16.04 was upgraded from 20.3-2-g371b392c-
  0ubuntu1~16.04.1 to 20.4-0ubuntu1~16.04.1. The newer version included
  the following fix, that led to the problem, although prior commits set
  the scene for this commit to cause us trouble.

  https://github.com/canonical/cloud-
  init/commit/b0e73814db4027dba0b7dc0282e295b7f653325c

  While trying workarounds (e.g. oneshot service to revert permissions),
  we then ran into another change that appended exit(142) to the command
  option:

  https://github.com/canonical/cloud-
  init/commit/e161059a18173e2b61c54dba9eab774401fb5f1f

  which then meant, as root is disabled, that SSH would not work using
  the authorised key pair for any user. This is because cloud-init first
  writes the key for the user (e.g. ubuntu) and in our case writing the
  key to /etc/ssh/authorized_keys, then writes the key for the disabled
  root user to the same location, overwriting the previous write.

  There are similarities to https://bugs.launchpad.net/cloud-
  init/+bug/1839061, but this is a different issue.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1911680/+subscriptions