canonical-ubuntu-qa team mailing list archive

[Skia <mp+468688@xxxxxxxxxxxxxxxxxx>]

Skia has proposed merging ~hyask/autopkgtest-cloud:skia/doc_blackholing into autopkgtest-cloud:master.

Requested reviews:
  Canonical's Ubuntu QA (canonical-ubuntu-qa)

For more details, see:
https://code.launchpad.net/~hyask/autopkgtest-cloud/+git/autopkgtest-cloud/+merge/468688

Documentation after the small DoS we had last week and how to handle such situation.
-- 
Your team Canonical's Ubuntu QA is requested to review the proposed merge of ~hyask/autopkgtest-cloud:skia/doc_blackholing into autopkgtest-cloud:master.

diff --git a/docs/administration.rst b/docs/administration.rst
index c0c0b7e..4b38ee3 100644
--- a/docs/administration.rst
+++ b/docs/administration.rst
@@ -553,3 +553,44 @@ Then setup your browser to go through that proxy (plenty of online doc for this)
 **NOTE**: the access is only authorized from the corresponding bastion in each
 cloud. That means that for a VM in PS5, you need to setup your SOCKS proxy to a
 PS5 bastion, and for a VM in PS6, you need a proxy to a PS6 bastion.
+
+
+Blackhole harmful IP ranges
+---------------------------
+
+As everything exposed to the Internet, the infra might be subject to probing by
+some bots.
+They can raise the load pretty high, leading to some DoS, but this is easily
+prevented by looking at the HAProxy logs and blackholing the harmful IP address
+range.
+
+Example of harmful requests:
+
+.. code-block::
+
+  GET /packages/a/ableton-link/oracular/armhf/portal/attachment_getAttList.action?bean.RecId=1')+AND+EXTRACTVALUE(534543,CONCAT(0x5c,md5(999999999),0x5c))+AND+('n72Yk'='n72Yk&bean.TabName=1
+  GET /index.php?lang=../../../../../../../../usr/local/lib/php/pearcmd&+config-create+/&/<?echo(md5(#22hi#22));?>+/tmp/index1.php
+  GET /<IMG%20SRC=#22javascript:alert(cross_site_scripting.nasl);#22>.jsp
+  GET /packages/a/abseil/oracular/amd64/seeyon/webmail.do?method=doDownloadAtt&filename=index.jsp&filePath=../conf/datasourceCtp.properties
+
+The situation can be handled quickly with the following:
+
+.. code-block:: bash
+
+  # On the HAProxy unit
+  cd /var/log
+  # Change `CONCAT` here by other pattern, like `\.php` or `\.jsp`
+  zgrep 'CONCAT' haproxy.log*.gz > /tmp/harmful.log
+  # Manually inspect the harmful logs if you want
+  less /tmp/harmful.log
+  # Get the list of IP addresses sorted with the most harmful at the bottom
+  grep -o '[0-9]*\.[0-9]*\.[0-9]*\.[0-9]*' /tmp/harmful.log | sort | uniq -c  | sort -n
+  # Get the network range of that IP, very useful is you see a lot of similar but different IPs in the list
+  # Run this on another machine, don't install the tools on the unit
+  whois <ip address> | grep NetRange
+  ipcalc-ng -d <first IP>-<last IP>
+  # Back on the HAProxy unit
+  # Blackhole the whole range
+  sudo ip route add blackhole 123.123.123.123/12
+  # Show the currently blackholed ranges
+  sudo ip route show type blackhole