group.of.nepali.translators team mailing list archive

[Launchpad Bug Tracker <1942929@xxxxxxxxxxxxxxxxxx>]

This bug was fixed in the package ubuntu-advantage-tools - 27.3~21.10.1

---------------
ubuntu-advantage-tools (27.3~21.10.1) impish; urgency=medium

  * d/tools.postinst:
    - consider cloud to be "none" on any cloud-id error
    - purge old ua-messaging.timer/service files
    - keep ua-timer.timer disabled if ua-messaging.timer was disabled by
      the user
    - properly configure both ubuntu-advantage-timer and
      ubuntu-advantage-licence-check logs
  * d/tools.postrm:
    - remove ubuntu-advantage-timer and ubuntu-advantage-license-check logs
      during purge
  * systemd:
    - remove ua-messaging.timer/service
    - add new ua-timer.timer that runs every 6 hours
    - add new ua-license_check.timer that runs every 5 minutes only if
      activated by ua-license-check.path
  * New upstream release 27.3 (LP: #1942929)
    - ros:
      + add beta support to enable ros and ros-updates
      + add support for "required services" so that esm-infra and esm-apps
        get auto-enabled when enabling ros or ros-updates
      + add support for "dependent services" so that user gets prompted to
        disable ros/ros-updates if they disable esm-infra/esm-apps
    - fips:
      + allow fips on GCP bionic now that optimized kernel is ready
      + disallow enabling fips on focal on clouds until cloud-optimized focal
        fips-certified kernel is ready (LP: #1939449, LP: #1939932)
      + print warning about generic fips kernel if cloud-id fails
    - cloud:
      + rely only on cloud-id to determine cloud type (LP: #1940131)
      + catch errors when determining cloud type
        (LP: #1938207, LP: #1944676) (GH: #1541)
    - azure:
      + bump IMDS API version to support Azure published images
    - cli:
      + collect-logs command that creates a tar file with debug-relevant logs
        and status info (GH: #463)
      + clean locks on exceptions more thoroughly to avoid false "Operation in
        progress" status messages
      + retain past service state after detach
      + shows better error message when a port value in a proxy is invalid
    - non-unicode locale support:
      + remove unicode-only characters from help file
      + don't print unicode-only characters in ua fix if non-utf8 locale
        (GH: #1463)
    - logrotate:
      + add logrotate functionality for ubuntu-advantage-timer.log.
      + Fix root:root logrotate permissions.
    - ua-timer.timer:
      + introduce a single systemd timer to handle ua recurring jobs
      + timer runs every 2 hours to support most frequent timer job
      + recurring job intervals are configurable in uaclient.conf
      + individual jobs are disabled if their interval is set to 0
      - status job:
        + update ua status every 12 hours
      - messaging job:
        + update APT/MOTD ESM messaging every 6 hours
      - metering job:
        + disabled until infrastructure is ready
        + for attached machines only, periodically update contract server with
          status information for proper contract metering
    - ua-license-check.timer:
      + only runs on LTS GCP instances that are not attached
      + runs every 5 minutes to check if gcp instance has license required to
        auto-attach
    - logs:
      + fixes duplicate logging (GH: #553)
    - tests and support:
      + remove groovy integration tests
      + various improvements to integration tests

 -- Grant Orndorff <grant.orndorff@xxxxxxxxxxxxx>  Tue, 21 Sep 2021
09:02:06 -0400

** Changed in: ubuntu-advantage-tools (Ubuntu Impish)
       Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of नेपाली
भाषा समायोजकहरुको समूह, which is subscribed to Xenial.
Matching subscriptions: Ubuntu 16.04 Bugs
https://bugs.launchpad.net/bugs/1942929

Title:
  [SRU] ubuntu-advantage-tools (27.2.2 -> 27.3) Xenial, Bionic, Focal,
  Hirsute

Status in ubuntu-advantage-tools package in Ubuntu:
  Fix Committed
Status in ubuntu-advantage-tools source package in Xenial:
  Fix Released
Status in ubuntu-advantage-tools source package in Bionic:
  Fix Released
Status in ubuntu-advantage-tools source package in Focal:
  Fix Released
Status in ubuntu-advantage-tools source package in Hirsute:
  Fix Released
Status in ubuntu-advantage-tools source package in Impish:
  Fix Released

Bug description:
  [Impact]
  This release sports both bug-fixes and new features and we would like to
  make sure all of our supported customers have access to these
  improvements. The notable ones are:

  * more robust error handling when determining the cloud we're on LP: #1940131 LP: #1938207 LP: #1944676
  * disallows fips on focal aws/azure LP: #1939449 LP: #1939932
  * adds/changes to ua-related recurring jobs:
    - change in frequency to existing job: updates the apt and motd esm update messaging: every 6 hours
    - new job: updates the contract details and status: every 12 hours
    - new job: ONLY ON GCP (implemented as separate timer that is only activated on GCP LTS when not attached): checks for license changes and auto-attaches if a pro license was added: every 5 minutes
  * adds support for ros/ros-updates entitlements with --beta flag

  With this change, the ua-message timer is renamed to ua-timer, as it
  has a more generic functionality: it triggers sub jobs which need to
  be executed periodically. One of those is exactly the job which
  updates the messaging - which has its interval reduced to 6h, as well
  as the timer itself. There is also a job to update the client status
  every 12h, and a third one to collect metrics, which is disabled for
  this release.

  See the changelog entry below for a full list of changes and bugs.

  [Test Case]
  The following development and SRU process was followed:
  https://wiki.ubuntu.com/UbuntuAdvantageToolsUpdates

  The ubuntu-advantage-tools team will be in charge of attaching the artifacts and
  console output of the appropriate run to the bug.  ubuntu-advantage-tools team
  members will not mark ‘verification-done’ until this has happened.

  <TODO After released to -proposed: attach integration test artifacts>

  In addition to the automated integration testing, here are manual test
  instructions to verify that the packaging changes are functional. All
  of these tests use the ua-client/staging PPA, which has the 27.3
  release build in it.

  Manual Test 1:
  Here we verify that the cloud-id command changes in postinst work correctly such that the installation succeeds even if the cloud-id command fails.
  ```
  lxc launch ubuntu-daily:impish dev-i
  lxc exec dev-i -- /bin/sh -c "printf 'exit 1' > /usr/bin/cloud-id"
  lxc exec dev-i -- chmod +x /usr/bin/cloud-id
  lxc exec dev-i -- cloud-id
  # CHECK: successfully faking cloud-init error. command should've exited 1
  lxc exec dev-i -- add-apt-repository -yu ppa:ua-client/staging
  lxc exec dev-i -- apt install -y ubuntu-advantage-tools
  # CHECK: installation should've succeeded despite failing cloud-init
  lxc delete dev-i --force
  ```

  Manual Test 2:
  Here we verify that the new license check timer only runs on LTS instances that are identified as running on GCP.
  ```
  # should not be enabled when not gcp, on LTS (focal)
  lxc launch ubuntu-daily:focal dev-f
  lxc exec dev-f -- add-apt-repository -yu ppa:ua-client/staging
  lxc exec dev-f -- apt install -y ubuntu-advantage-tools
  lxc exec dev-f -- systemctl list-timers --all
  lxc exec dev-f -- systemctl status ua-license-check.timer
  # CHECK: should not be running
  lxc exec dev-f -- reboot
  lxc exec dev-f -- systemctl list-timers --all
  lxc exec dev-f -- systemctl status ua-license-check.timer
  # CHECK: still should not be running
  lxc delete dev-f --force

  # fake gcp by overwriting cloud-id, still should not be enabled because not on LTS (impish)
  lxc launch ubuntu-daily:impish dev-i
  lxc exec dev-i -- /bin/sh -c "printf 'echo gce' > /usr/bin/cloud-id"
  lxc exec dev-i -- chmod +x /usr/bin/cloud-id
  lxc exec dev-i -- cloud-id
  # CHECK: successfully faking gcp. output should be "gce" (with an "e")
  lxc exec dev-i -- add-apt-repository -yu ppa:ua-client/staging
  lxc exec dev-i -- apt install -y ubuntu-advantage-tools
  lxc exec dev-i -- systemctl list-timers --all
  lxc exec dev-i -- systemctl status ua-license-check.timer
  # CHECK: should not be running
  lxc delete dev-i --force

  # fake gcp by overwriting cloud-id, on LTS (focal), should be enabled
  lxc launch ubuntu-daily:focal dev-f
  lxc exec dev-f -- /bin/sh -c "printf 'echo gce' > /usr/bin/cloud-id"
  lxc exec dev-f -- chmod +x /usr/bin/cloud-id
  lxc exec dev-f -- cloud-id
  # CHECK: successfully faking gcp. output should be "gce" (with an "e")
  lxc exec dev-f -- add-apt-repository -yu ppa:ua-client/staging
  lxc exec dev-f -- apt install -y ubuntu-advantage-tools
  lxc exec dev-f -- systemctl list-timers --all
  lxc exec dev-f -- systemctl status ua-license-check.timer
  # CHECK: should be enabled
  lxc delete dev-f --force
  ```

  Manual Test 3:
  Here we verify that the old ua-messaging.{timer,service} artifacts are properly removed on upgrade.
  ```
  lxc launch ubuntu-daily:impish dev-i
  lxc exec dev-i -- /bin/sh -c "ls -1 /etc/systemd/system/timers.target.wants /usr/lib/systemd/system /usr/lib/systemd/system /var/lib/systemd/deb-systemd-helper-enabled/timers.target.wants /var/lib/systemd/deb-systemd-helper-enabled/ | grep ua-messaging"
  # CHECK: verify several ua-messaging artifacts
  lxc exec dev-i -- add-apt-repository -yu ppa:ua-client/staging
  lxc exec dev-i -- apt install -y ubuntu-advantage-tools
  lxc exec dev-i -- /bin/sh -c "ls -1 /etc/systemd/system/timers.target.wants /usr/lib/systemd/system /usr/lib/systemd/system /var/lib/systemd/deb-systemd-helper-enabled/timers.target.wants /var/lib/systemd/deb-systemd-helper-enabled/ | grep ua-messaging"
  # CHECK: verify ua-messaging artifacts are not left behind
  lxc delete dev-i --force
  ```

  Manual Test 4:
  Here we verify that if the user had disabled the old ua-messaging timer, then we will carry that preference forward to the new ua-timer timer.
  ```
  lxc launch ubuntu-daily:impish dev-i
  lxc exec dev-i -- systemctl list-timers --all
  lxc exec dev-i -- systemctl status ua-messaging.timer
  # CHECK: verify ua-messaging.timer is enabled
  lxc exec dev-i -- systemctl stop ua-messaging.timer
  lxc exec dev-i -- systemctl disable ua-messaging.timer
  lxc exec dev-i -- systemctl list-timers --all
  lxc exec dev-i -- systemctl status ua-messaging.timer
  # CHECK: verify ua-messaging.timer is disabled
  lxc exec dev-i -- add-apt-repository -yu ppa:ua-client/staging
  lxc exec dev-i -- apt install -y ubuntu-advantage-tools
  lxc exec dev-i -- systemctl list-timers --all
  lxc exec dev-i -- systemctl status ua-timer.timer
  # CHECK: verify ua-timer.timer is disabled
  lxc delete dev-i --force
  ```

  Manual Test 5:
  Here we verify that the new log files are appropriately created on install, rotated by logrotate, and deleted on purge.
  ```
  lxc launch ubuntu-daily:impish dev-i
  lxc exec dev-i -- add-apt-repository -yu ppa:ua-client/staging
  lxc exec dev-i -- apt install -y ubuntu-advantage-tools
  lxc exec dev-i -- /bin/sh -c 'ls -l /var/log/ubuntu-advantage*'
  # CHECK: verify that three ua log files were created, are owned by root, and have 600 permissions
  lxc exec dev-i -- /bin/sh -c "printf testcontent > /var/log/ubuntu-advantage.log"
  lxc exec dev-i -- /bin/sh -c "printf testcontent > /var/log/ubuntu-advantage-timer.log"
  lxc exec dev-i -- /bin/sh -c "printf testcontent > /var/log/ubuntu-advantage-license-check.log"
  lxc exec dev-i -- logrotate --force /etc/logrotate.d/ubuntu-advantage-tools
  lxc exec dev-i -- /bin/sh -c 'ls -l /var/log/ubuntu-advantage*'
  # CHECK: verify all 3 logs were rotated
  lxc exec dev-i -- /bin/sh -c "printf testcontent > /var/log/ubuntu-advantage.log"
  lxc exec dev-i -- /bin/sh -c "printf testcontent > /var/log/ubuntu-advantage-timer.log"
  lxc exec dev-i -- /bin/sh -c "printf testcontent > /var/log/ubuntu-advantage-license-check.log"
  lxc exec dev-i -- /bin/sh -c 'ls /var/log/ubuntu-advantage*'
  # CHECK: verify all ua log files exist including rotated versions
  lxc exec dev-i -- apt purge -y ubuntu-advantage-tools
  lxc exec dev-i -- /bin/sh -c "ls /var/log/"
  # CHECK: verify that all ua log files are removed
  lxc delete dev-i --force
  ```

  [Regression Potential]
  In order to mitigate the regression potential, the results of the
  aforementioned integration tests are attached to this bug.

  We moved the trigger of the apt and motd messaging updates from a
  dedicated systemd timer to a shared timer that conditionally calls the
  messaging updates in our python code. This adds complexity. If we made
  a mistake, then either the job won't get called frequently enough or
  will get called too frequently. If the former, then some esm updates
  related messaging will be out of date in apt and motd. If the latter,
  then cpu cycles will be wasted in needlessly updating messages.

  We touched postinst to handle cloud-id failures more robustly.
  Touching postinst is always scary because it is the most likely way
  for us to break upgrades. In theory this change made upgrades less
  likely to fail, but if we made a mistake, it could cause new
  unexpected failures.

  We added more recurring jobs in the service of new features. This
  increases complexity and potential for mistakes. In particular, we
  have strived to avoid excessive logging from these jobs. If we made a
  mistake in our logging, we could inadvertently fill up disks with
  useless logs. Additional recurring jobs will also use more cpu over
  time than previous versions. This is at least partially addressed
  below.

  We instrumented a high frequency timer to only run on GCP, but if we
  made a mistake, this could be accidentally activated on non-GCP
  machines, which would be a waste. (See below for additional high
  frequency timer discussion).

  We check if the ua-messaging timer was disabled prior to this update,
  and if so we also disable the new ua-timer timer in systemd. Failing
  to do so would keep enabled a service that the user had explicitly
  disabled in the past, resulting in a unwanted behavior. Our migration
  of this user configuration only covers the case where the user ran
  `systemctl disable` to disable the old timer. If they disabled the
  timer in a different way, then their configuration will not be carried
  forward. Furthermore, this is a somewhat complicated postinst
  addition, and carries all the normal risks of editing postinst.

  [Discussion]
  Our timer on GCP runs every 5 minutes. This is necessary to support timely upgrades of gcp instances from standard ubuntu to ubuntu pro. We need to poll the metadata endpoint frequently to catch the license change in a timely manner. We exit as early as possible if there is nothing to be done for any given timer trigger. From our testing, this has minimal overall system performance impact. <TODO @chad.smith insert details and link to spreadsheet>

  [Changelog]

    * d/tools.postinst:
      - consider cloud to be "none" on any cloud-id error
      - purge old ua-messaging.timer/service files
    * systemd:
      - remove ua-messaging.timer/service
      - add new ua-timer.timer that runs every 2 hours
      - add new ua-license_check.timer that runs every 5 minutes only if
        activated by ua-license-check.path
    * New upstream release 27.3
      - ros:
        + add beta support to enable ros and ros-updates
        + add support for "required services" so that esm-infra and esm-apps
          get auto-enabled when enabling ros or ros-updates
        + add support for "dependent services" so that user gets prompted to
          disable ros/ros-updates if they disable esm-infra/esm-apps
      - fips:
        + allow fips on GCP bionic now that optimized kernel is ready
        + disallow enabling fips on focal on clouds until cloud-optimized focal
          fips-certified kernel is ready (LP: #1939449, LP: #1939932)
        + print warning about generic fips kernel if cloud-id fails
      - cloud:
        + rely only on cloud-id to determine cloud type (LP: #1940131)
        + catch errors when determining cloud type (LP: #1938207) (GH: #1541)
      - azure:
        + bump IMDS API version to support Azure published images
      - cli:
        + collect-logs command that creates a tar file with debug-relevant logs
          and status info (GH: #463)
        + clean locks on exceptions more thoroughly to avoid false "Operation in
          progress" status messages
        + retain past service state after detach
        + shows better error message when a port value in a proxy is invalid
      - non-unicode locale support:
        + remove unicode-only characters from help file
        + don't print unicode-only characters in ua fix if non-utf8 locale
          (GH: #1463)
      - ua-timer.timer:
        + introduce a single systemd timer to handle ua recurring jobs
        + timer runs every 2 hours to support most frequent timer job
        + recurring job intervals are configurable in uaclient.conf
        + individual jobs are disabled if their interval is set to 0
        - status job:
          + update ua status every 12 hours
        - messaging job:
          + update APT/MOTD ESM messaging every 6 hours
        - metering job:
          + disabled until infrastructure is ready
          + for attached machines only, periodically update contract server with
            status information for proper contract metering
      - ua-license-check.timer:
        + only runs on LTS GCP instances that are not attached
        + runs every 5 minutes to check if gcp instance has license required to
          auto-attach
      - logs:
        + fixes duplicate logging (GH: #553)
      - tests and support:
        + remove groovy integration tests
        + various improvements to integration tests

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/ubuntu-advantage-tools/+bug/1942929/+subscriptions