← Back to team overview

maria-developers team mailing list archive

Re: [GSoC] Student interested in "Optimize mysql-test runs" or adding SQL features


Hello Sergei, Elena and all,
I just answered the question that Elena asked on my proposal for GSoC. I'm
sorry I was late.
Please let me know if I should answer anything else to be on time before
the deadline.

Regards and Thank you!


On Fri, Mar 21, 2014 at 4:50 PM, Pablo Estrada <polecito.em@xxxxxxxxx>wrote:

> Hello Sergei and all,
> I have made a last update to my proposal, and submitted it. I added more
> specifics regarding the scope of the project (running as part of the
> buildbot), as well as data to leverage for the project and potential
> learning algorithms to use depending on the data that the project will have
> access to in the end (only test result history, or also code change
> history, etc.).
> If anyone has any changes to suggest before the deadline, let me know.
> I'll be glad to look into them.
> All the best, all : )
> Pablo
> On Tue, Mar 11, 2014 at 6:53 PM, Sergei Golubchik <serg@xxxxxxxxxxx>wrote:
>> Hi, Pablo!
>> First: please always cc: maria-developers@xxxxxxxxxxxxxxxxxxx,
>> don't send emails to me only.
>> Now:
>> On Mar 11, Pablo Estrada wrote:
>> > Hello!
>> > I am getting ready to submit my proposal. Is there any feedback for my
>> > application document? Anything that would be good to add, remove,
>> rephrase,
>> > etc?
>> Go on and submit it. There will be a comment form under your proposal,
>> if I will want you to change something, I will leave my comments there.
>> Please mention what tools/languages you think you'll be using.
>> > Also, if I could see the data available (the history of test results), I
>> > could write more in detail : )
>> Hm. Okay. Attached is the preprocessed table that I used two years ago.
>> It lists test failures in buildbot in a given period of time.
>> Columns are (as far as I remember):
>> * time of the test run start
>> * code branch that was tested
>> * revision id that was tested
>>  - it's fake, our buildbot tables don't store revision id. I've
>>    constructed it from the set of changed files, revision number, etc.
>>    hopefully it's as unique as a real revision id
>> * build number
>> * platform where the tests were run
>> * test type
>>  - basically, the set of tests to run and what protocol to use. one test
>>    may be run with different protocols
>> * test name
>> * test variant
>>  - or combination. certain tests may be run in many combinations
>> Using tis file I tried to rearrange test execution order for every
>> specific build number (using failure data from previous builds in the
>> same table) to detect test failures as early as possible. I managed to
>> have 90% of all test failures happening within first 10% of the tests.
>> The goal of this project is to have something we can put back into
>> buildbot, so that it could actually run only 10% of the tests, still
>> having 90% probability of discovering a failure (or 20%, or 95%,
>> whaveter).
>> See also http://buildbot.askmonty.org/buildbot/
>> you can click on a link for any branch, for example,
>> http://buildbot.askmonty.org/buildbot/grid?branch=10.0
>> then on any particular build, for example,
>> http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest/builds/2297
>> and on a stdio log for a particular test run:
>> http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest/builds/2297/steps/test_6/logs/stdio
>> on these pages you'll see what a platform (builder) is, build number, test
>> types, combinations, etc.
>> Regards,
>> Sergei