ubuntu-bugcontrol team mailing list archive

[Ian Constantin <mp+488429@xxxxxxxxxxxxxxxxxx>]

Ian Constantin has proposed merging ~iconstantin/ubuntu-qa-tools:esm-private-ppa-legacy-support into ubuntu-qa-tools:master.

Commit message:
Adding support to unembargo from esm private ppas to esm legacy ppas

Requested reviews:
  Marc Deslauriers (mdeslaur)
  Ubuntu Security Team (ubuntu-security)

For more details, see:
https://code.launchpad.net/~iconstantin/ubuntu-qa-tools/+git/ubuntu-qa-tools/+merge/488429

Updates prepared in esm private ppas may need to go either to regular esm (infra/apps) prod destination ppas or to legacy esm ppas. The scope of this change is to add support to unembargo from an esm private ppa and to have the correct destination ppa automatically selected/used.

This requires for the setting of opt.ppa and opt.destination to be separated and put into functions (set_src_ppa() and set_dest_ppa()). The reason being:

- set_src_ppa() needs to be called before lpl_common.get_archive() gets called to get the value of origin_archive

- origin_archive needs to continue to be defined where it currently is so that it's value can be used to get the published sources for the packages we want to unembargo.

- set_dest_ppa() on the other hand needs to be called before the call(s) to lpl_common.get_archive() to get 
dest_archive

- the defining of dest_archive has to be moved further down, to after we have all the series_names/releases, as the release name (checked against cve_lib.esm_[apps|infra]_legacy_releases) will be used to determine which dest ppa (opt.destination) needs to be set when the source ppa is an esm private ppa

---

This change avoid unnecessary additional calls to lpl_common.get_archive() when defining dest_archive. We will call lpl_common.get_archive() the first time we are defining dest_archive (at which point it will be equal to None) and thereafter will only call lpl_common.get_archive() if opt.destination is different than it's previous value (which will be saved in prev_destination)

---


Adding support for esm-apps-legacy to have full coverage even though it is not yet being used.

---

Defined is_esm_update to avoid having to use ```if opt.esm or opt.esm_apps or opt.esm_apps_private or opt.esm_apps_legacy  or opt.esm_infra or opt.esm_infra_private or opt.esm_infra_legacy``` throughout the script.



-- 
Your team Ubuntu Bug Control is subscribed to branch ubuntu-qa-tools:master.

diff --git a/security-tools/unembargo b/security-tools/unembargo
index c92bb7e..199bb9e 100755
--- a/security-tools/unembargo
+++ b/security-tools/unembargo
@@ -39,6 +39,9 @@ ESM_APPS_PRIVATE_PPA = "ubuntu-esm/private-esm-apps-security"
 ESM_APPS_STAGING_PPA = "ubuntu-esm/esm-apps-security-staging"
 ESM_APPS_PRODUCTION_PPA = "ubuntu-esm/esm-apps-security"
 
+ESM_APPS_LEGACY_STAGING_PPA = "ubuntu-esm/esm-apps-legacy-security-staging"
+ESM_APPS_LEGACY_PRODUCTION_PPA = "ubuntu-esm/esm-apps-legacy-security"
+
 ESM_INFRA_PRIVATE_PPA = "ubuntu-esm/private-esm-infrastructure-security"
 ESM_INFRA_STAGING_PPA = "ubuntu-esm/esm-infra-security-staging"
 ESM_INFRA_PRODUCTION_PPA = "ubuntu-esm/esm-infra-security"
@@ -70,6 +73,10 @@ parser.add_argument("--esm-apps-private", help="Copy completed ESM packages from
                     "Private ESM Apps PPA to the ESM Apps production PPA "
                     "(default: None)", action='store_true',
                     default=None)
+parser.add_argument("--esm-apps-legacy", help="Copy completed ESM packages from the "
+                    "ESM Apps Legacy staging PPA to the ESM Apps Legacy production PPA "
+                    "(default: None)", action='store_true',
+                    default=None)
 parser.add_argument("--esm-infra", help="Copy completed ESM packages from the "
                     "ESM Infra staging PPA to the ESM Infra production PPA "
                     "(default: None)", action='store_true',
@@ -110,43 +117,69 @@ if len(args) < 1:
     print('Usage: %s [OPTIONS] PKG [PKG...]' % sys.argv[0], file=sys.stderr)
     sys.exit(1)
 
-# If --esm, --esm-apps, --esm-infra, or --esm-infra-legacy option is provided,
-# the script will move from the respective ESM staging PPA to the respective
-# ESM PROD PPA. So the user should not indicate such values.
-if opt.esm or opt.esm_apps or opt.esm_infra or opt.esm_infra_private or opt.esm_infra_legacy:
+is_esm_update = False
+
+# Checking if this is an ESM update
+if opt.esm or opt.esm_apps or opt.esm_apps_private or opt.esm_apps_legacy  or opt.esm_infra or opt.esm_infra_private or opt.esm_infra_legacy:
+    is_esm_update = True
+
+    # For ESM updates this script will move packages from the respective ESM staging
+    # PPA to the respective ESM prod PPA. So the user should not indicate such values.
     if opt.destination != UBUNTU_ARCHIVE or opt.ppa is not None or opt.pocket is not None:
         parser.error("If an esm option is provided, do not specify values for "
                      "package origin, destination, and pocket as this "
                      "script will do that for you.")
         sys.exit(2)
-    elif opt.esm_apps:
-        opt.destination = ESM_APPS_PRODUCTION_PPA
-        opt.ppa = ESM_APPS_STAGING_PPA
-    elif opt.esm_infra:
-        opt.destination = ESM_INFRA_PRODUCTION_PPA
-        opt.ppa = ESM_INFRA_STAGING_PPA
-    elif opt.esm_infra_private:
-        opt.destination = ESM_INFRA_PRODUCTION_PPA
-        opt.ppa = ESM_INFRA_PRIVATE_PPA
-    elif opt.esm_apps_private:
-        opt.destination = ESM_APPS_PRODUCTION_PPA
-        opt.ppa = ESM_APPS_PRIVATE_PPA
-    elif opt.esm_infra_legacy:
-        opt.destination = ESM_INFRA_LEGACY_PRODUCTION_PPA
-        opt.ppa = ESM_INFRA_LEGACY_STAGING_PPA
-    else:
-        opt.destination = ESM_PRODUCTION_PPA
-        opt.ppa = ESM_STAGING_PPA
-else:
-    opt.ppa = UBUNTU_SECURITY_PPA if opt.ppa is None else opt.ppa
 
-# Default to Security pocket, unless we're releasing to a PPA (in which case,
-# default to Release pocket)
+# Default to Security pocket, unless we're doing an ESM update and
+# are releasing to a PPA (in which case, default to Release pocket)
 if opt.pocket is None:
-    if opt.destination == UBUNTU_ARCHIVE:
+    if is_esm_update:
+        opt.pocket = RELEASE_POCKET
+    else: # destination will be UBUNTU_ARCHIVE
         opt.pocket = SECURITY_POCKET
+
+def set_src_ppa():
+    ''' Sets the source PPA if one is not provided '''
+
+    if is_esm_update:
+        if opt.esm_apps:
+            opt.ppa = ESM_APPS_STAGING_PPA
+        elif opt.esm_infra:
+            opt.ppa = ESM_INFRA_STAGING_PPA
+        elif opt.esm_infra_private:
+            opt.ppa = ESM_INFRA_PRIVATE_PPA
+        elif opt.esm_apps_private:
+            opt.ppa = ESM_APPS_PRIVATE_PPA
+        elif opt.esm_infra_legacy:
+            opt.ppa = ESM_INFRA_LEGACY_STAGING_PPA
+        else:
+            opt.ppa = ESM_STAGING_PPA
     else:
-        opt.pocket = RELEASE_POCKET
+        opt.ppa = UBUNTU_SECURITY_PPA if opt.ppa is None else opt.ppa
+
+def set_dest_ppa(release):
+    ''' Sets the appropriate destination PPA when performing an ESM update '''
+
+    if is_esm_update:
+        if opt.esm_apps:
+            opt.destination = ESM_APPS_PRODUCTION_PPA
+        elif opt.esm_infra:
+            opt.destination = ESM_INFRA_PRODUCTION_PPA
+        elif opt.esm_infra_private:
+            if release in cve_lib.esm_infra_legacy_releases:
+                opt.destination = ESM_INFRA_LEGACY_PRODUCTION_PPA
+            else:
+                opt.destination = ESM_INFRA_PRODUCTION_PPA
+        elif opt.esm_apps_private:
+            if release in cve_lib.esm_apps_legacy_releases:
+                opt.destination = ESM_APPS_LEGACY_PRODUCTION_PPA
+            else:
+                opt.destination = ESM_APPS_PRODUCTION_PPA
+        elif opt.esm_infra_legacy:
+            opt.destination = ESM_INFRA_LEGACY_PRODUCTION_PPA
+        else:
+            opt.destination = ESM_PRODUCTION_PPA
 
 # given an archive + release name, find any milestones that are upcoming.
 # return either the first milestone within the window or None.
@@ -238,9 +271,7 @@ if potential_embargo:
 
 
 # don't release security updates on Fridays
-if (opt.pocket == SECURITY_POCKET or \
-   opt.esm or opt.esm_apps or opt.esm_apps_private or \
-   opt.esm_infra or opt.esm_infra_private or opt.esm_infra_legacy) and \
+if (opt.pocket == SECURITY_POCKET or is_esm_update) and \
    datetime.datetime.today().weekday() == 4:
     print("WARNING: The security team has a policy of not releasing security updates "
           "on Fridays.")
@@ -259,7 +290,8 @@ ubuntu = lp.distributions['ubuntu']
 src_distro = lp.distributions[opt.source_distribution]
 dst_distro = lp.distributions[opt.destination_distribution]
 
-dest_archive, dest_group, dest_ppa = lpl_common.get_archive(opt.destination, lp, verbose=True, distribution=dst_distro)
+# Setting opt.ppa
+set_src_ppa()
 
 origin_archive, origin_group, origin_ppa = lpl_common.get_archive(opt.ppa, lp, verbose=True, distribution=src_distro)
 
@@ -283,11 +315,13 @@ for pkg_name in args:
     # Filter for the series we want
     for source_item in sources:
         series_name = source_item.distro_series.name
+
         if len(series) > 0:
             if not series_name in series:
                 continue
         elif not opt.include_devel and series_name == cve_lib.devel_release:
             continue
+
         if opt.source_pocket is not None and opt.source_pocket.lower() != source_item.pocket.lower():
             continue
         if series_name not in seen:
@@ -316,12 +350,23 @@ for pkg_name in args:
                 print("NOTE: To override this check and publish anyway please use the --force.")
                 print("NOTE: unembargo for %s will be skipped." % (series_name))
 
+    # Initializing dest_archive to None in order to ensure we call lpl_common.get_archive() at least once below
+    dest_archive = None
 
     # Publish
     for source_item in unembargo:
         name = source_item.source_package_name
         version = source_item.source_package_version
         series_name = source_item.distro_series.name
+
+        # opt.destination will be set to the appropriate ESM prod PPA if this is an ESM update
+        set_dest_ppa(series_name)
+
+        # Only querying for dest_archive if it is the first time or if opt.destination has changed since the previous query
+        if not dest_archive or opt.destination != prev_destination:
+            dest_archive, dest_group, dest_ppa = lpl_common.get_archive(opt.destination, lp, verbose=True, distribution=dst_distro)
+            prev_destination = opt.destination
+
         if opt.dry_run:
             action = "Want to publish"
         else:
@@ -331,6 +376,7 @@ for pkg_name in args:
 
         if not opt.dry_run:
             retries = opt.retries
+
             while retries > 0:
                 start = time.time()
                 try: