yellow team mailing list archive

[Gary Poster <gary.poster@xxxxxxxxxxxxx>]

On 04/16/12 12:33, Gary Poster wrote:
> On 04/13/12 17:09, Gary Poster wrote:
> > Benji is out on Monday, so people can take over on his work. From his
> > work/notes, this is what remains:
> >
> > - add test in subunit for TestProtocolClient.tags method
> > - add .tags method to TestProtocolClient (see patch below)
>
> These were added in revno 158 of subunit (unreleased). We just need to
> make our own release (after the work on subunit-filter, below).
>
> > - add test in subunit for subunit-filter command to filter by tags
> > - add filtering by tags
> > - release subunit in PPA
>
> Brad and I will be working on these this afternoon.

--with actually is a multiline Python regex match on the entire subunit 
output for each test. It should work fine.  No work needed here.

> > - Verify that jml's three branches have landed for testtools
> > https://code.launchpad.net/~jml/testtools/tagger/+merge/101904
> >
> > https://code.launchpad.net/~jml/testtools/wrap-result-in-concurrent-suite/+merge/101902
> >
> >
> > https://code.launchpad.net/~jml/testtools/tsfr-fixup/+merge/101898
>
> jml says he will release these no later than tomorrow.
>
> > - verify/release new testtools in PPA
>
> There is a testtools PPA IIRC so jml will probably release there. We
> will need to copy that over, maybe just over to the LP PPA.
>
>
>  > - Apply this patch to testrepository (see patch below)

Proposed as is for comments at 
https://code.launchpad.net/~yellow/testrepository/integrate_worker_tagging/+merge/102165 
.

> Maybe this needs a test too?

I suspect so, but let's wait for Robert's reply.

> > - release testr in PPA

There are more steps, and outlining them should be helpful for everyone 
who cares about this, including me.

- we need testr bug/feature 949950 ("testr should have an option to send 
out (collated) subunit output, including successes") fixed.  Brad and I 
are working on this.

- we need to hook all of this together into buildbot.  That should look 
something like this.
  * we use testrepository, subunit, and testtools from the PPA.  That 
should just be a matter of getting everything in one of the desired PPAs.
  * buildbot's testr call in master.cfg should get the --subunit 
argument from the work on bug 949950
  * we add multiple stdout parsers in the buildbot output for the testr 
call.
    > One will simply be the stdout.
    > We will have one for each of the test processes piped through 
subunit-filter to give us the output per worker (e.g. subunit-filter 
--with="^tags: worker-0" -error --failure --success), and then piped 
through "subunit-ls --no-passthrough"
    > We will have one that will show us only the failures, perhaps in 
non-subunit format.  This one might also update the status of the 
waterfall for on the fly test counts.
  * for the buildbot piping to work, we *might* need to clean up the 
output of our testr command (i.e., removing lines 1-44 of 
http://paste.ubuntu.com/932523/).  it might be robust enough without it.

> > It might be nice to subdivide these into smaller cards...or it might
> > not. :-P Your call, if you take this. This would be a good task for
> > pairing, I think. Someone with PPA/packaging knowledge would be good to
> > have.
> >
> > Have a great weekend.
> >
> > Gary
> >
> >
> >
> > === modified file 'testrepository/repository/file.py'
> > --- testrepository/repository/file.py 2011-11-02 18:32:38 +0000
> > +++ testrepository/repository/file.py 2012-04-13 20:21:54 +0000
> > @@ -242,6 +242,10 @@
> > self._time = timestamp
> > return result
> >
> > + def tags(self, new, gone):
> > + TestProtocolClient.tags(self, new, gone)
> > + self._stream.write('tags: %s\n' % ' '.join(self.current_tags))
> > +
> >
> > class _FailingInserter(_SafeInserter):
> > """Insert a stream into the 'failing' file."""
> > @@ -270,7 +274,7 @@
> > failing = self._repository.get_failing()
> > failing.get_test().run(inserter)
> > inserter.stopTestRun()
> > - inserter= repo.get_inserter(partial=True)
> > + inserter = repo.get_inserter(partial=True)
> > inserter.startTestRun()
> > run = self._repository.get_test_run(run_id)
> > run.get_test().run(inserter)
> >
> >
> >
> >
> > === modified file 'testrepository/commands/load.py'
> > --- testrepository/commands/load.py 2011-11-03 15:09:02 +0000
> > +++ testrepository/commands/load.py 2012-04-13 18:10:23 +0000
> > @@ -17,13 +17,22 @@
> > import optparse
> >
> > import subunit
> > -from testtools import ConcurrentTestSuite, MultiTestResult
> > +from testtools import ConcurrentTestSuite, MultiTestResult, Tagger
> >
> > from testrepository.commands import Command
> > from testrepository.repository import RepositoryNotFound
> > from testrepository.results import TestResultFilter
> >
> >
> > +class TaggingConcurrentTestSuite(ConcurrentTestSuite):
> > +
> > + def _wrap_result(self, result, thread_number):
> > + worker_id = 'worker-%s' % thread_number
> > + tags_to_add = set([worker_id])
> > + tags_to_remove = set()
> > + return Tagger(result, tags_to_add, tags_to_remove)
> > +
> > +
> > class load(Command):
> > """Load a subunit stream into a repository.
> >
> > @@ -62,7 +71,7 @@
> > streams = list(suite)[0]
> > for stream in streams():
> > yield subunit.ProtocolTestCase(stream)
> > - case = ConcurrentTestSuite(cases, make_tests)
> > + case = TaggingConcurrentTestSuite(cases, make_tests)
> > inserter = repo.get_inserter(partial=self.ui.options.partial)
> > try:
> > previous_run = repo.get_latest_run()