launchpad-reviewers team mailing list archive

[Thomas Bechtold <mp+408287@xxxxxxxxxxxxxxxxxx>]

Thomas Bechtold has proposed merging ~toabctl/launchpad-buildd:vcs-git-shallow-clone into launchpad-buildd:master.

Commit message:
Add git_shallow_clone option to vcs_fetch() and use it for OCI build
Currently when cloning a git repository, the whole history is
cloned. That makes the cloning process slow (in case the repo is
large) and in some cases, the cloning fails due to its size (that's an
infrastructure problem but currently happens for OCI base images).
Just cloning the latest commit (via the "--depth 1" parameter)
improves the situation where large repos need to be cloned.
https://bugs.launchpad.net/launchpad-buildd/+bug/1939392



Requested reviews:
  Launchpad code reviewers (launchpad-reviewers)

For more details, see:
https://code.launchpad.net/~toabctl/launchpad-buildd/+git/launchpad-buildd/+merge/408287
-- 
Your team Launchpad code reviewers is requested to review the proposed merge of ~toabctl/launchpad-buildd:vcs-git-shallow-clone into launchpad-buildd:master.

diff --git a/lpbuildd/target/build_oci.py b/lpbuildd/target/build_oci.py
index 437932c..7f2a9ed 100644
--- a/lpbuildd/target/build_oci.py
+++ b/lpbuildd/target/build_oci.py
@@ -106,7 +106,8 @@ class BuildOCI(BuilderProxyOperationMixin, VCSOperationMixin,
         """Collect git or bzr branch."""
         logger.info("Running repo phase...")
         env = self.build_proxy_environment(proxy_url=self.args.proxy_url)
-        self.vcs_fetch(self.args.name, cwd="/home/buildd", env=env)
+        self.vcs_fetch(self.args.name, cwd="/home/buildd", env=env,
+                       git_shallow_clone=True)
 
     def build(self):
         logger.info("Running build phase...")
diff --git a/lpbuildd/target/tests/test_build_oci.py b/lpbuildd/target/tests/test_build_oci.py
index 4b95221..7689bb3 100644
--- a/lpbuildd/target/tests/test_build_oci.py
+++ b/lpbuildd/target/tests/test_build_oci.py
@@ -219,7 +219,8 @@ class TestBuildOCI(TestCase):
         build_oci.repo()
         self.assertThat(build_oci.backend.run.calls, MatchesListwise([
             RanBuildCommand(
-                ["git", "clone", "lp:foo", "test-image"], cwd="/home/buildd"),
+                ["git", "clone", "--depth", "1", "lp:foo", "test-image"],
+                cwd="/home/buildd"),
             RanBuildCommand(
                 ["git", "submodule", "update", "--init", "--recursive"],
                 cwd="/home/buildd/test-image"),
diff --git a/lpbuildd/target/vcs.py b/lpbuildd/target/vcs.py
index d7c80c3..4ddd9e5 100644
--- a/lpbuildd/target/vcs.py
+++ b/lpbuildd/target/vcs.py
@@ -57,7 +57,7 @@ class VCSOperationMixin:
         else:
             return ["git"]
 
-    def vcs_fetch(self, name, cwd, env=None, quiet=False):
+    def vcs_fetch(self, name, cwd, env=None, quiet=False, git_shallow_clone=False):
         full_env = OrderedDict()
         full_env["LANG"] = "C.UTF-8"
         full_env["SHELL"] = "/bin/sh"
@@ -75,6 +75,8 @@ class VCSOperationMixin:
             cmd = ["git", "clone"]
             if quiet:
                 cmd.append("-q")
+            if git_shallow_clone:
+                cmd.extend(["--depth", "1"])
             if self.args.git_path is not None:
                 git_path = self.args.git_path
                 # "git clone -b" is a bit odd: it takes either branches or