launchpad-dev team mailing list archive

[Jonathan Lange <jml@xxxxxxxxxxxxx>]

On Wed, Oct 6, 2010 at 12:41 PM, Jonathan Lange <jml@xxxxxxxxxxxxx> wrote:
> On Wed, Oct 6, 2010 at 12:13 PM, Jonathan Lange <jml@xxxxxxxxxxxxx> wrote:
>> On Wed, Oct 6, 2010 at 9:49 AM, Michael Nelson
>> <michael.nelson@xxxxxxxxxxxxx> wrote:
>>> I just caused a test-fix using ec2 land with a windmill test that
>>> *did* fail, but the run was reported as successful (details below).
>>>
> ...

Forgot to mention that there's a third bug here.

ec2test/remote.py has code that will fail the test run if there are no
test failures *and* the test subprocess returns a non-zero exit code.
It's there precisely to catch stuff like this. This means that either:
  a) the test subprocess has a 0 retcode, even though it's had many
many failures and critical layer teardown issues, or
  b) ec2test/remote.py has a bug.

I strongly suspect a), since the make output in Michael's post doesn't
mention anything about exit codes. If so, it's likely another issue
with zope.testrunner's layers-in-subprocess thingummy.

jml