launchpad-reviewers team mailing list archive

[Colin Watson <mp+444283@xxxxxxxxxxxxxxxxxx>]

Colin Watson has proposed merging ~cjwatson/launchpad:charm-appserver-http-interface into launchpad:master.

Commit message:
charm/launchpad-appserver: Add http interface support

Requested reviews:
  Launchpad code reviewers (launchpad-reviewers)

For more details, see:
https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/444283

This allows relating `launchpad-appserver` to a deployment of the `haproxy` charm to have it load-balance the various workers and expose them on the ports expected by other parts of Launchpad.

The code is based on the `ols-http` layer, but I had to write custom code for it because the appserver exposes two ports with different purposes (the main application, and XML-RPC); and the current production `haproxy` configuration uses slightly different options for those two, with special error page handling for the main application.

While this doesn't result in the exact same `haproxy` configuration that we have on production, I think it's equivalent.
-- 
Your team Launchpad code reviewers is requested to review the proposed merge of ~cjwatson/launchpad:charm-appserver-http-interface into launchpad:master.

diff --git a/charm/launchpad-appserver/README.md b/charm/launchpad-appserver/README.md
index 2222972..26a3afa 100644
--- a/charm/launchpad-appserver/README.md
+++ b/charm/launchpad-appserver/README.md
@@ -9,5 +9,9 @@ You will need the following relations:
     juju relate launchpad-appserver memcached
     juju relate launchpad-appserver rabbitmq-server
 
+You can also relate it to a load balancer:
+
+    juju relate launchpad-appserver:loadbalancer haproxy:reverseproxy
+
 By default the main application server runs on port 8085, and the XML-RPC
 application server runs on port 8087.
diff --git a/charm/launchpad-appserver/config.yaml b/charm/launchpad-appserver/config.yaml
index f016031..2ff6762 100644
--- a/charm/launchpad-appserver/config.yaml
+++ b/charm/launchpad-appserver/config.yaml
@@ -24,6 +24,30 @@ options:
       Secret key for Git access tokens issued to Launchpad users.  Any
       random string of a reasonable size (64 characters) is ok.
     default:
+  haproxy_server_options:
+    type: string
+    description: Options to add to HAProxy "server" lines.
+    default: check inter 5000 rise 2 fall 5 maxconn 16
+  haproxy_service_options_main:
+    type: string
+    description: HAProxy options for the main port.
+    default: |
+      - mode http
+      - option httplog
+      - option httpchk HEAD /_status/ping
+      - option http-server-close
+      - http-check disable-on-404
+      - balance roundrobin
+  haproxy_service_options_xmlrpc:
+    type: string
+    description: HAProxy options for the XML-RPC port.
+    default: |
+      - mode http
+      - option httplog
+      - option httpchk HEAD /_status/ping
+      - option http-server-close
+      - http-check disable-on-404
+      - balance roundrobin
   internal_macaroon_secret_key:
     type: string
     description: >
diff --git a/charm/launchpad-appserver/layer.yaml b/charm/launchpad-appserver/layer.yaml
index 1401723..921b8a8 100644
--- a/charm/launchpad-appserver/layer.yaml
+++ b/charm/launchpad-appserver/layer.yaml
@@ -1,6 +1,7 @@
 includes:
   - layer:launchpad-db
   - layer:coordinator
+  - interface:http
   - interface:memcache
 repo: https://git.launchpad.net/launchpad
 options:
diff --git a/charm/launchpad-appserver/metadata.yaml b/charm/launchpad-appserver/metadata.yaml
index b56f0d6..8fcad01 100644
--- a/charm/launchpad-appserver/metadata.yaml
+++ b/charm/launchpad-appserver/metadata.yaml
@@ -18,3 +18,6 @@ requires:
     interface: pgsql
   memcache:
     interface: memcache
+provides:
+  loadbalancer:
+    interface: http
diff --git a/charm/launchpad-appserver/reactive/launchpad-appserver.py b/charm/launchpad-appserver/reactive/launchpad-appserver.py
index ad4b439..1ceb82c 100644
--- a/charm/launchpad-appserver/reactive/launchpad-appserver.py
+++ b/charm/launchpad-appserver/reactive/launchpad-appserver.py
@@ -5,6 +5,7 @@ import shlex
 import subprocess
 from multiprocessing import cpu_count
 
+import yaml
 from charmhelpers.core import hookenv, host, templating
 from charms.coordinator import acquire
 from charms.launchpad.base import configure_email, get_service_config
@@ -206,3 +207,76 @@ def nrpe_available():
 @when_not("nrpe-external-master.available")
 def nrpe_unavailable():
     clear_flag("launchpad.appserver.nrpe-external-master.published")
+
+
+@when("loadbalancer.available", "service.configured")
+@when_not("launchpad.loadbalancer.configured")
+def configure_loadbalancer():
+    config = hookenv.config()
+
+    try:
+        service_options_main = yaml.safe_load(
+            config["haproxy_service_options_main"]
+        )
+    except Exception:
+        hookenv.log("Could not parse haproxy_service_options_main YAML")
+        hookenv.status_set(
+            "blocked", "Bad haproxy_service_options_main YAML configuration"
+        )
+        return
+    try:
+        service_options_xmlrpc = yaml.safe_load(
+            config["haproxy_service_options_xmlrpc"]
+        )
+    except Exception:
+        hookenv.log("Could not parse haproxy_service_options_xmlrpc YAML")
+        hookenv.status_set(
+            "blocked", "Bad haproxy_service_options_xmlrpc YAML configuration"
+        )
+        return
+    server_options = config["haproxy_server_options"]
+
+    unit_name = hookenv.local_unit().replace("/", "-")
+    unit_ip = hookenv.unit_private_ip()
+    services = [
+        {
+            "service_name": "appserver-main",
+            "service_port": config["port_main"],
+            "service_host": "0.0.0.0",
+            "service_options": list(service_options_main),
+            "servers": [
+                [
+                    f"main_{unit_name}",
+                    unit_ip,
+                    config["port_main"],
+                    server_options,
+                ]
+            ],
+        },
+        {
+            "service_name": "appserver-xmlrpc",
+            "service_port": config["port_xmlrpc"],
+            "service_host": "0.0.0.0",
+            "service_options": list(service_options_xmlrpc),
+            "servers": [
+                [
+                    f"xmlrpc_{unit_name}",
+                    unit_ip,
+                    config["port_xmlrpc"],
+                    server_options,
+                ]
+            ],
+        },
+    ]
+    services_yaml = yaml.dump(services)
+
+    for rel in hookenv.relations_of_type("loadbalancer"):
+        hookenv.relation_set(rel["__relid__"], services=services_yaml)
+
+    set_state("launchpad.loadbalancer.configured")
+
+
+@when("launchpad.loadbalancer.configured")
+@when_not_all("loadbalancer.available", "service.configured")
+def deconfigure_loadbalancer():
+    remove_state("launchpad.loadbalancer.configured")