maria-developers team mailing list archive

[Kristian Nielsen <knielsen@xxxxxxxxxxxxxxx>]

Hi,

When I added my 10.0-mdev26 tree to buildbot, I got some strange errors on the
kvm-fulltest2 builder. I finally tracked it down to be caused by the OOM
killer killing random processes.

The root issue is Cassandra. At some point it seems cassandra was enabled in
the VM used by the fulltest2 builder. However, just starting up Cassandra
creates a jsvc process that uses 3.4 _gigabyte_ of memory, 1.3 GB of which is
resident.

That is too much...

I also noticed that cassandra tests are not even run, plugins.cassandra is
skipped due to cassandra not running. Possibly this is because the OOM killer
has killed the process ;-).

In any case, for now I have disabled cassandra on the -fulltest VMs:

    sudo update-rc.d cassandra disable

If someone wants to re-enable it, feel free, but please first make sure that
it does not cause unrelated test failures due to excessive memory usage! I
would suggest not to run cassandra by default, instead add a separate buildbot
step that explicitly starts up cassandra when needed.

----

In general, I am disappointed about the state of our trees. This problem seems
to have been there for more than a month without anyone doing anything, and
there are tons of other test failures in 10.0 and 10.0-base.

We need to tighten up our processes here. Remember, the rules are that
anything major should be green in a feature tree before pushing to a main
tree. Clearly this is not what is happening at the moment.

It's ridiculous to constantly hear people pressing for release of 10.0.2-beta
when our trees are not even alpha state ...

 - Kristian.