subunit-dev team mailing list archive

[Robert Collins <robertc@xxxxxxxxxxxxxxxxx>]

So, I was chatting with Jelmer about this, and put up a strawman of
using a push/pop idiom to allow local/global knowledge split in streams.
E.g. a runner running 5 suites might emit:
progress: 5 push
<invoke the suite>
progress: pop push
<invoke the suite>
progress: pop push
<invoke the suite>
progress: pop push
<invoke the suite>
progress: pop push
<invoke the suite>
progress: pop

Each suite would then output as much progress data as they have; and it
layers well, with arbitrary depths. A pop is equivalent to a test for
advancing progress bars.

Now, I was thinking about this, and I wonder if it won't have negative
consequences. At the moment, and with the progress proposal I made that
doesn't use a push/pop idiom, one can combine streams running
concurrently by:
- buffering up to the bracket of a single test
- summing absolute progress markers in source streams (and turning 
  subsequent absolute markers into a relative adjustment)
- maintaining state on tags in each source and doing appropriate set
  adjustments for the joined stream

However, with a push-pop idiom, it seems harder to me for the state to
be preserved with such minimal overhead/buffering. If you consider 2
streams, one that runs 3 tests in 3 suites, and one that runs 3 tests in
one suite:
progress: 3 push
progress: 1 push
test: foo
success: foo
progress: pop
progress: pop push
progress: 1 push
test: foo
success: foo
progress: pop
progress: pop push
progress: 1 push
test: foo
success: foo
progress: pop
progress: pop

and

progress: 3 push
test: foo
success: foo
test: foo
success: foo
test: foo
success: foo
progress: pop

Then when progress events are overlapped it becomes unclear how to both
preserve the structure and interlace things. If we just sum the counts,
we will be able to see odd things like - 4 tests at depth 2, which would
lead to depth 1 only getting 50% complete. One possibility that seems
promising to me would be to track the height of each source, and when
outputting progress related events: push/pop/counts and tests, normalise
the height for the joined stream: - if its deeper, emit pushes, if its
higher emit (pop +1) (the +1 prevents the pop counting towards
progress).

Anyhow, I'd appreciate someone checking to see if they can poke holes in
that approach. I'd like really good progress, but not at the cost of
concurrency.

-Rob

Attachment: signature.asc
Description: This is a digitally signed message part