openstack team mailing list archive

[Trey Morris <trey.morris@xxxxxxxxxxxxx>]

Add me as a +1 to ensuring that unittests are actually unittests instead of
a mix'n'match of different styles of test. This would make writing and
updating tests much more straightforward and actually catch problems in the
correct layer of tests, for example a change in network should not under
any circumstance cause an api unittest to fail. I think we all understand
the nightmare of trying to fix tests in unrelated areas of the code that
have failed for no immediately apparent reason. How can I help make this
happen?

-tr3buchet

On Tue, Nov 22, 2011 at 8:45 AM, Sandy Walsh <sandy.walsh@xxxxxxxxxxxxx>wrote:

> Excellent!
>
> I wrote a few blog posts recently, mostly based on my experience with
> openstack automated tests:
>
>
> http://www.sandywalsh.com/2011/06/effective-units-tests-and-integration.html
> http://www.sandywalsh.com/2011/08/pain-of-unit-tests-and-dynamically.html
>
> Would love to see some of those changes make it in.
>
> -Sandy
>
> ________________________________________
> From: openstack-bounces+sandy.walsh=rackspace.com@xxxxxxxxxxxxxxxxxxx[openstack-bounces+sandy.walsh=
> rackspace.com@xxxxxxxxxxxxxxxxxxx] on behalf of Soren Hansen [
> soren@xxxxxxxxxxx]
> Sent: Monday, November 21, 2011 8:24 AM
> To: openstack@xxxxxxxxxxxxxxxxxxx
> Subject: [Openstack] [nova-testing] Efforts for Essex
>
> Hi, guys.
>
> We're scattered across enough different timezones to make real-time
> communication really awkward, so let's see if we can get by using e-mail
> instead.
>
> A good test suite will let you keep up the pace of development. It will
> offer confidence that your change didn't break any expectations, and
> will help you understand how things are supposed to work, etc. A bad
> test suite, on the other hand, will actually do the opposite, so the
> quality of the unit test suite is incredibly important to the overall
> health of the project. Integration tests are important as well, but unit
> tests all we can expect people to run on a regular basis.
>
> I'd like to start a bit of discussion around efforts around unit testing
> for Essex. Think of it as brainstorming. Input can be anything from
> small, actionable items, to broad ideas, to measurable goals, random
> thoughts etc. Anything goes. At some point, we can distill this input to
> a set of common themes, set goals, define action items, etc.
>
> A few things from the back of my mind to get the discussion going:
>
> = Speed up the test suite =
>
> A slow test suite gets run much less frequently than a fast one.
> Currently, when wrapped in eatmydata, a complete test run takes more
> than 6 minutes.
>
> Goal: We should get that down to less than one minute.
>
> = Review of existing tests =
> Our current tests have a lot of problems:
>
>  * They overlap (multiple tests effectively testing the same code),
>  * They're hard to understand. Not only are their intent not always
>   clear, but it's often hard to tell how they're doing it.
>  * They're slooooow.
>  * They're interdependent. The failure of one test often cascades and
>   makes others fail, too.
>  * They're riddled with duplicated code.
>
> I think it would be great if we could come up with some guidelines for
> good tests and then go through the existing tests and highlight where
> they violate these guidelines.
>
> = Test coverage =
> We should increase test coverage.
>
> Adding tests to legacy code is hard. Generally, it's much easier to
> write tests for new production code as you go along. The two primary
> reasons are:
>
>  * If you're writing tests and production code at the same time (or
>   perhaps even writing the tests first), the code will almost
>   automatically designed to be easily tested.
>  * You (hopefully1) know how the code is supposed to work. This is not
>   always obvious for existing code (often written by someone else).
>
> Therefore, the most approachable strategy for increasing test coverage
> is simply to ensure that any new code added is accompanied by tests, but
> of course new tests for currently untested code is fantastically
> welcome.
>
> --
> Soren Hansen        | http://linux2go.dk/
> Ubuntu Developer    | http://www.ubuntu.com/
> OpenStack Developer | http://www.openstack.org/
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>