maria-developers team mailing list archive

[Elena Stepanova <elenst@xxxxxxxxx>]

Hi Rich,

On 6/25/2012 4:49 PM, Rich Prohaska wrote:
> hello,
>
> when i run "./mysql-test-run.pl --suite=engines/funcs,engines/iuds
> --force --mysqld=--partition --nowarnings --max-test-fail=0", i see
> lots of test failures on mariadb-5.5.25 (and other distributions).
> either (1) i am not running the tests correctly, or (2) the test
> and/or test result files are broken, or (3) there are real problems
> being detected. i suspect most of the failures are caused by (2)
> above.

You are right, the result files for this test suite have been taken "as 
is" from the MySQL test suite, and it causes the failures you mentioned, 
as there are some differences in the code, e.g.

- MariaDB processes some comparisons with NULL in a more conventional way,
- SQL states and error messages displayed by the tests can be different 
(like HY000 in MySQL vs 22007 in MariaDB);
- MariaDB produces a warning when it truncates a value, while MySQL 
sometimes does it silently,
- there is an extra collation in MariaDB,
and some more.

We do not use this test suite in our regular tests, so the mismatches 
slipped in unnoticed; but you are right, as long as it is included into 
our packages, it should be consistent with the code. Thank you for 
pointing it out.

If you are looking for a test suite to test an engine, you might want to 
try out the engine independent suite that we have developed for this 
very purpose and are going to publish in the next couple of weeks. It is 
described in detail in the JIRA task 
https://mariadb.atlassian.net/browse/MDEV-11, and is based on recently 
developed extensions to MTR functionality, already released in MariaDB 
5.5 (https://mariadb.atlassian.net/browse/MDEV-30). The idea is to 
provide a suite easily tunable for any new engine without having to copy 
over and modify test files. Currently it works for MyISAM, Aria, XtraDB, 
Archive, InnoDB, Memory, Merge, CSV, Federated.

Regards,
Elena Stepanova

> anyway, here are the failures:
>
> Completed: Failed 37/323 tests, 88.54% were successful.
>
> Failing test(s): engines/funcs.an_calendar
> engines/funcs.datetime_function engines/funcs.db_alter_collate_ascii
> engines/funcs.db_alter_collate_utf8 engines/funcs.db_use_error
> engines/funcs.de_calendar_range
> engines/funcs.in_calendar_2_unique_constraints_duplicate_update
> engines/funcs.in_calendar_pk_constraint_duplicate_update
> engines/funcs.in_calendar_pk_constraint_error
> engines/funcs.in_calendar_pk_constraint_ignore
> engines/funcs.in_calendar_unique_constraint_duplicate_update
> engines/funcs.in_calendar_unique_constraint_error
> engines/funcs.in_calendar_unique_constraint_ignore
> engines/funcs.in_multicolumn_calendar_pk_constraint_duplicate_update
> engines/funcs.in_multicolumn_calendar_pk_constraint_error
> engines/funcs.in_multicolumn_calendar_pk_constraint_ignore
> engines/funcs.in_multicolumn_calendar_unique_constraint_duplicate_update
> engines/funcs.in_multicolumn_calendar_unique_constraint_error
> engines/funcs.in_multicolumn_calendar_unique_constraint_ignore
> engines/funcs.in_multicolumn_number_pk_constraint_duplicate_update
> engines/funcs.in_number_boundary_error
> engines/funcs.in_number_decimal_boundary_error
> engines/funcs.ld_all_number_string_calendar_types
> engines/funcs.se_join_left engines/funcs.se_join_left_outer
> engines/funcs.se_join_natural_left
> engines/funcs.se_join_natural_left_outer engines/funcs.sq_all
> engines/funcs.sq_error engines/funcs.up_calendar_range
> engines/iuds.delete_time engines/iuds.insert_decimal
> engines/iuds.insert_number engines/iuds.insert_time
> engines/iuds.type_bit_iuds engines/iuds.update_delete_number
> engines/iuds.update_time
>
> thanks
>
> rich prohaska
>
> _______________________________________________
> Mailing list: https://launchpad.net/~maria-developers
> Post to     : maria-developers@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~maria-developers
> More help   : https://help.launchpad.net/ListHelp