← Back to team overview

maria-developers team mailing list archive

Re: [GSoC] Optimize mysql-test-runs - Setback

 

I have pushed my latest version of the code, and here is a test run that
ran on this version of the code. It is quite different from the original
expectation; so I'm taking a close look at the code for bugs, and will run
another simulation ASAP  (I'll use less data to make it faster).


On Thu, Jun 19, 2014 at 5:16 PM, Elena Stepanova <elenst@xxxxxxxxxxxxxxxx>
wrote:

> Hi Pablo,
>
> I'll send a more detailed reply later, just a couple of quick
> comments/questions now.
>
> To your question
>
>
>
>> I'm just not quite sure what you mean with this example:
>> mysql-test/plugin/example/mtr/t
>>
>> In this example, what is the test name? And what is exactly the path?
>> (./mysql-test/...) or (./something/mysql-test/...)? I tried to look at
>> some
>> of the test result files but I couldn't find one certain example of this
>> pattern (Meaning that I'm not sure what would be a real instance of it).
>> Can you be more specific please?
>>
>>
> I meant that if you look into the folder <tree>/mysql-test/suite/mtr/t/ ,
> you'll see an example of what I described as "The result file can live not
> only in /r dir, but also in /t dir, together with the test file":
>
> ls mysql-test/suite/mtr/t/
> combs.combinations
> combs.inc
> inc.inc
> newcomb.result
> newcomb.test
> proxy.inc
> self.result
> self.test
> simple,c2,s1.rdiff
> simple.combinations
> simple.result
> simple,s2,c2.rdiff
> simple,s2.result
> simple.test
> single.result
> single.test
> source.result
> source.test
> test2.result
> test2.test
> testsh.result
> testsh.test
>
> As far as I remember, your matching algorithm didn't cover that.
>
>
>
>  Here are the results. They are both a bit counterintuitive, and a bit
>> strange
>>
>
> Have you already done anything regarding (not) populating the queue
> completely? I did expect that with the current logic, after adding full
> cleanup between simulations, the more restrictive configuration would have
> lower recall, because it generally runs much fewer tests.
>
> It would be interesting to somehow indicate in the results how many tests
> were *actually* run. But if you don't have this information, please don't
> re-run the full set just for the sake of it, maybe run only one running set
> for standard/platform/branch/mixed, and let us see the results. No need
> to spend time on graphs for that, a text form will be ok.
>
> Either way, please push the current code, I'd like to see it before I come
> up with any suggestions about the next big moves.
>
> Regards,
> Elena
>

Attachment: randomized_2.png
Description: PNG image


Follow ups

References