launchpad-dev team mailing list archive

[Robert Collins <robertc@xxxxxxxxxxxxxxxxx>]

So I meant to look at +filebug timeouts yesterday but I got distracted
into test infrastructure (because things were slow).

Everyone else may know this already, but layers are a pretty poor
interface, at least the way they are used in LP's test suite.

Specifically, rather than a regular object, they are pseudo stateless
- classes are used, and helpers that contain some magic code get used.

For instance, this gem:

class FunctionalTestSetup(object):
    """Keeps shared state across several functional test cases."""

    __shared_state = { '_init': False }

    def __init__(self, config_file=None, database_names=None,
                 product_config=None):
        """Initializes Zope 3 framework.

        Creates a volatile memory storage.  Parses Zope3 configuration files.
        """
        self.__dict__ = self.__shared_state


Anyhow, layers have a 'feature' where they can refuse to be torn down.
When this happens, due to a combination of bugs and design features,
we end up with multiple librarians, memcached helpers, and so on
running. This is undesirable and causes problems (for instance, the
log file of the librarian in all but the first test runners process is
not written to because the first test runners librarian keeps running,
has the ports open, and writes to a deleted log file [because the
entire working dir gets deleted]).

Anyhow, how does this relate to performance?

Several ways, firstly layers, by supporting setup-only fixtures add
about 30-40 seconds per test run for every layer that we have that
can't be torn down. Secondly, because layers are so fragile, our test
fixtures do redundant work on setup - they have code to shutdown
failed-to-shutdown instances that can spin for up to 5 seconds (by
default).

And of course, our generally slow startup ties in as well because
we're performing that every time we bring up the librarian (and we
bring it up more often than needed because of layers that can't be
torn down triggering a full new test runner.

Anyhow, sorry that I didn't have a lot of interesting perf details
stuff today, but I had hit my tolerance threshold for this cruft, so
I'm probably going to keep tugging on it for a bit.

-Rob