Re: Malone UI changes

[Ian Jackson <ian@xxxxxxxxxxxxxxxxxxxxxxxx>]

Christian Robottom Reis writes ("Re: Malone UI changes"):
> Well, I want the HTML-parsing to stop, and that's why I'm asking these
> questions :-)

I have a screenscraper too:

My autopkgtest setup files bugs automatically.  It has a separate gpg
key and launchpad account so that I don't have to give it access to my
own launchpad account.  It files the bugs by email.

So that it can avoid filing duplicates, it fetches a list of the bugs
that it has submitted and which are still open, sedderies the packages
out of the bug URLs, and derives a list of packages for which it
should not file another bug.  (This is not exactly correct, since it
can currently do two kinds of test for each package and so might
reasonably have two bugs open at once; however, it's BALGE.)

I cut and pasted the URL to fetch from my browser after performing an
advanced search.  I worry that it'll break if the list of open bugs
starts to be paged.

I would very much like to replace this hideous setup with something
better.

Ian.


#!/bin/sh

cd ${0%/*}

url='https://launchpad.net/ubuntu/+bugs?field.searchtext=&orderby=-importance&field.status%3Alist=New&field.status%3Alist=Incomplete&field.status%3Alist=Confirmed&field.status%3Alist=Triaged&field.status%3Alist=In+Progress&field.status%3Alist=Fix+Committed&assignee_option=any&field.assignee=&field.bug_reporter=ian%2Bubuntu-autopkgtest&field.bug_contact=&field.bug_commenter=&field.subscriber=&field.status_upstream-empty-marker=1&field.omit_dupes.used=&field.has_patch.used=&field.tag=&field.has_cve.used=&search=Search'

echo 'fetching'
curl -s -S -k -o webpage "$url"

echo 'grepping'
perl -ne '
    print "$1\n" or die $! if
     m,"https://bugs.launchpad.net/ubuntu/\+source/([-+.0-9a-z]+)/\+bug/\d+",
' <webpage >suppressions.new

nl -ba suppressions.new

if ! test -s suppressions.new;
then
	echo >&2 'NO SUPPRESSIONS - PROBABLY WENT WRONG

webpage:'
	cat webpage
fi

echo 'uploading'

mv suppressions.new suppressions
RSYNC_RSH=ssh rsync suppressions cadmium.buildd:adt-play/.

echo 'done.'