launchpad-reviewers team mailing list archive

[Colin Watson <mp+433719@xxxxxxxxxxxxxxxxxx>]

Colin Watson has proposed merging ~cjwatson/launchpad:celery-rpc-backend into launchpad:master.

Commit message:
Switch to Celery's RPC result backend

Requested reviews:
  Launchpad code reviewers (launchpad-reviewers)

For more details, see:
https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/433719

Using Celery's AMQP result backend produces the following deprecation warning:

  The AMQP result backend is scheduled for deprecation in version 4.0 and removal in version v5.0.  Please use RPC backend or a persistent backend.

The RPC backend scales better due to using a results queue per client rather than a results queue per task call, but it only works if the process that initiates the task is always the process to retrieve the result.  This is fine for Launchpad, though: we generally persist the results of jobs in our database instead.  Our job runner ignores Celery task results unless `BaseRunnableJob.celery_responses` is set, and that's only the case in tests where the process initiating the task is indeed also the one that retrieves the result.
-- 
Your team Launchpad code reviewers is requested to review the proposed merge of ~cjwatson/launchpad:celery-rpc-backend into launchpad:master.

diff --git a/lib/lp/services/job/celeryconfig.py b/lib/lp/services/job/celeryconfig.py
index 7048ca8..6d7946e 100644
--- a/lib/lp/services/job/celeryconfig.py
+++ b/lib/lp/services/job/celeryconfig.py
@@ -109,7 +109,7 @@ def configure(argv):
     }
     result["enable_utc"] = True
     result["imports"] = ("lp.services.job.celeryjob",)
-    result["result_backend"] = "amqp"
+    result["result_backend"] = "rpc"
     result["task_acks_late"] = True
     result["task_create_missing_queues"] = False
     result["task_default_exchange"] = "job"