← Back to team overview

randgen team mailing list archive

Making the RQG more user-friendly



I will be spending some time trying to make the RQG more user friendly and less prone to silent failures that render entire tests useless without much warning. To this end, here is what is on my mind:

- adopt Bernt's MTR --start-and-exit replacement and remove all use of MTR altogether forever and ever; - make the RQG print out textual descriptions for all error codes (already implemented); - properly report errors during the data loading phase. In particular, properly report crashes and any errors that prevented tables from being created and data from being inserted; - properly report errors that caused the test to become meaningless, such as zero valid queries being generated in the entire test;
- report better statistics for the test while it is running:
* report grammar rules that are broken and produced no valid queries (already implemented); * report final table count and table status (a distilled version of SHOW TABLE STATUS showing row count and data size) * a concise report for the % of successfull queries, % of queries that modified rows and % that returned 1 or more rows, to provide a single-line information on the health of the grammar
- enable easy creation of lcov reports (already implemented)

The goal would be to have a framework that is not significantly more "dangerous" to use than MTR, that is, to have no increased risk of creating a meaningless test that reports a false negative sucess for every run.

I have not reviewed Matthias' suggestions from the past, but if you have previously suggested an improvement in the direction of usability that has been forgotten in the meantime, please remind me.

Thank you.

Philip Stoev