openstack team mailing list archive

[Gabe Westmaas <gabe.westmaas@xxxxxxxxxxxxx>]

Sure!  Sorry about that :)

Torpedo is just the set of ruby tests that we have been using for a while
to smoketest the API, separated into a project outside of VPC - the
project that was running for a while on the jenkins server.  Source is
here: https://github.com/dprince/torpedo  When one is hit by torpedo, that
means the OS API functional tests failed in some way.  In this particular
case we got an error when connecting to glance, (searching for traceback
in the paste is a good way to find the errors)  Dan, maybe you can provide
some insight into why this particular one is "hit by torpedo" when there
is no test output?  I suspect this was caused by an invalid glance paste
config issue that was addressed later, but I'm not 100% sure.  I kicked
this particular one off again to see if it was addressed since that issue
was fixed.

Typically its a bit more clear which test failed and why.  For example,
take a look at this paste from another test run, and search for "Error":
http://paste.openstack.org/show/3583/ You can see that an instance failed
to come up after this change was merged.  The system dumps logs from
several servers including installation info from chef as well as the nova
logs, so it can get a bit long, but in the interest of getting something
out quickly that was useful, that route was chosen over trying to focus in
on the specific issues.

It will run tempest (new name for openstack-integration-tests) in addition
since everyone is familiar with that project, just haven't had a chance
yet.

Chef client timeout means there was a problem with deploying which is
often caused by code changes that have an associated required config
change, and the like.  New required middleware could cause that, anything
that keeps services from starting or anything like that.  It means that
there was a failure that prevented chef from finishing installation.  In
this case though, it appears to be because we were unable to get an
additional cloud server for nova1, and this prevented chef from installing
since it had no where to install to.  This is the exception rather than
the rule, but there are already some ideas on how to prevent this from
happening.  Again, just haven't had time to address that yet.

In the end, when I see all green, I feel confident that something passes
basic smoke tests for both ec2 and openstack api. When I see red on
libvirt or xen server, it definitely takes some investigation and my first
instinct was to wonder about the installation when I see chef client
timeout messages, but more often than not there is an api-paste change or
something similar that is causing that issue.  At the same time, I
recognize it can be frustrating to not know why something is red and until
you see it for yourself, there is lower confidence in those being issues
with code..

I'm sure Dan would accept patches that made things a bit more accessible
if anyone has interest in that and is familiar with how to read the
various logs including chef logs!

Gabe

On 12/1/11 3:59 AM, "Soren Hansen" <soren@xxxxxxxxxxx> wrote:

>Can you explain what some of the errors mean? For instance, for this
>merge proposal:
>
>   https://review.openstack.org/#change,1950
>
>the unit tests passed, but the libvirt tests were "Hit by Torpedo" and
>XenServer tests had a "Chef client timeout". Am I right to assume that
>at least the chef client timeout problem is an issue in the test setup
>rather than indicative of a problem with the merge proposal? I have no
>idea what "Hit by Torpedo" means :)
>
>-- 
>Soren Hansen        | http://linux2go.dk/
>Ubuntu Developer    | http://www.ubuntu.com/
>OpenStack Developer | http://www.openstack.org/