maria-developers team mailing list archive

[Sergey Vojtovich <svoj@xxxxxxxxxxx>]

Hi Elene,

In another thread you mentioned that there's alternative solution used by
Otto to maintain this list. I'm generally fine with either solution.

This patch as such looks Ok.

How do we qualify tests unstable? Failed since the beginning of the year,
modified in last version, failed in Debian?

Who shall maintain this list?

What is the procedure of removing test case from unstable?

I'm not completely happy that 10.0 is GA, TokuDB is "stable", but (almost) all
TokuDB are disabled. This is rather a bad message to our users.

Regards,
Sergey



On Mon, Aug 22, 2016 at 03:04:26PM +0300, Elena Stepanova wrote:
> Hi Sergey (Vojtovich),
> 
> Please review the patch that we've discussed at length.
> Serg, FYI.
> 
> The length of the "unstable" list shouldn't scare anyone. At the end, this
> task should cause no change in our current procedure, the tests will still
> be run in buildbot and hopefully on developers' local machines (and every
> external user who wants to run them can do it easily). The change should
> mainly help those who run tests as a part of their build process.
> 
> The lists can be tuned later, now they have been created generously, to
> solve the most urgent issue in shortest time possible.
> 
> Please note that it's still not a 100% guarantee that nothing will fail
> sporadically, I've already got an absolutely new random failure after
> disabling all known ones, so it will still require some iterations.
> 
> Regards,
> Elena
> 
> -------- Forwarded Message --------
> Subject: [Commits] 5ac5050: MDEV-10604 Disable unstable MTR tests by default
> Date: Mon, 22 Aug 2016 14:44:30 +0300 (EEST)
> From: elenst@xxxxxxxxxxxxxxxx
> Reply-To: maria-developers@xxxxxxxxxxxxxxxxxxx
> To: commits@xxxxxxxxxxx
> 
> revision-id: 5ac505064a75508213d4dbce388ff5ee62d1ad56
> (mariadb-10.0.26-92-g5ac5050)
> parent(s): 05f61ba46046ca835071a73b9255e787dcce9255
> author: Elena Stepanova
> committer: Elena Stepanova
> timestamp: 2016-08-22 14:38:27 +0300
> message:
> 
> MDEV-10604 Disable unstable MTR tests by default
> 
> - add new syntax to disabled.def in MTR to distinguish "unstable"
>   tests from disabled tests;
> - allow wildcards in test names in disabled.def;
> - add a new option enable-unstable and env option MTR_ENABLE_DISABLED;
> - mark "unstable" tests which have failed since the beginning
>   of the year in buildbot on main 10.0 branch;
> - mark "unstable" tests which fail in Debian builds;
> - mark "unstable" tests which have been changed during the last
>   two releases
...skip...

Regards,
Sergey