openstack team mailing list archive

[Dan Prince <dprince@xxxxxxxxxx>]

Hey Jim,

I actually turned off SmokeStack earlier today for potentially the same reason. I was out for a couple days last week and haven't quite put my finger on all the things that are wrong. I'm seeing about half of the functional test runs fail.

This issue seems to be the cause of most of my failures:

https://bugs.launchpad.net/nova/+bug/1010291

I'm also dealing with the fact that Nova now requires libvirt 0.9.7 or later (I think) due to some of the refactoring. This is fine but it does mean I can't fully test things on Fedora 16 like I used to (reboot is out for example).

That is what I've seen.

Dan

----- Original Message -----
> From: "James E. Blair" <corvus@xxxxxxxxxxxx>
> To: "OpenStack Mailing List" <openstack@xxxxxxxxxxxxxxxxxxx>
> Sent: Tuesday, June 12, 2012 7:25:01 PM
> Subject: [Openstack] Intermittent devstack-gate failures
> 
> Hi,
> 
> It looks like there are intermittent, but frequent, failures in the
> devstack-gate.  This suggests a non-deterministic bug has crept into
> some piece of OpenStack software.
> 
> In this kind of situation, certainly could keep re-approving changes
> in
> the hope that they will pass the test and merge, but it would be
> better
> to fix the underlying problem.  Simply re-approving is mostly just
> going
> to make the queue longer.
> 
> Note that the output from Jenkins has changed recently.  I've seen
> some
> people misconstrue some normal parts of the test process as errors.
>  In
> particular, this message from Jenkins is not an error:
> 
>   Looks like the node went offline during the build. Check the slave
>   log
>   for the details.
> 
> That's a normal part of the way the devstack-gate tests run, where we
> add a machine to Jenkins as a slave, run the tests, and remove it
> from
> the list of slaves before it's done.  This is to accommodate the
> one-shot nature of devstack based testing.  It doesn't interfere with
> the results.
> 
> To find out why a test failed, you should scroll up a bit to the
> devstack exercise output, which normally looks like this:
> 
> *********************************************************************
> SUCCESS: End DevStack Exercise: ./exercises/volumes.sh
> *********************************************************************
> =====================================================================
> SKIP boot_from_volume
> SKIP client-env
> SKIP quantum
> SKIP swift
> PASS aggregates
> PASS bundle
> PASS client-args
> PASS euca
> PASS floating_ips
> PASS sec_groups
> PASS volumes
> =====================================================================
> 
> Everything after that point is test running boilerplate.  I'll add
> some
> echo statements to that effect in the future.
> 
> Finally, it may be a little difficult to pinpoint when this started.
>  A
> number of devstack-gate tests have passed recently without actually
> running any tests, due to an issue with one of our OpenStack based
> node
> providers.  We are eating our own dogfood.
> 
> -Jim
> 
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>