randgen team mailing list archive

["Philip Stoev" <pstoev@xxxxxxxxxxxx>]

Hello,

I personally no longer use this feature, as I grep the log files manually 
after the test is run. So you can fix it as you see fit.

The reason this is very CPU intensive is the grep -i option, which forces a 
case-insensitive search. If you remove that option and make sure the strings 
being matched have the right case, you will observe that CPU usage is no 
longer a problem.

Alternatively, you can replace the multiple greps with a single egrep 
command.

Philip Stoev

----- Original Message ----- 
From: "Roel Van de Paar" <roel.van.de.paar@xxxxxxxxxx>
To: <randgen@xxxxxxxxxxxxxxxxxxx>
Sent: Tuesday, November 08, 2011 11:34 AM
Subject: [Randgen] Proposal: removing greps in combinations.pl


> Hi Phillip, all,
>
> We would like to remove the following greps from combinations.pl:
>
>     foreach my $string ('text=', 'bugcheck', 'Error: assertion',
> 'mysqld got signal', 'Received signal', 'exception') {
>         system("grep -i '$string' $workdir/trial*log");
>     }
>
> There are two problems with these greps:
>
> #1 They consume a huge amount of CPU power for large log files (when for
> instance --sqltrace was used in a combinations.pl run):
>
>     %CPU COMMAND
>     93.2 grep -i Received signal ...
>     87.9 grep -i mysqld got signal ...
>     87.9 grep -i Received signal ...
>     84.9 grep -i Received signal ...
>
>     While it is easy enough to grep through results after a 
> combinations.pl run (and there are other interesting strings).
>
>     Something that works well is for instance;
>     grep "exited with exit" *.log | grep -vi "STATUS_OK"
>
> #2 At the moment, these greps are maybe in the wrong place in the code 
> (could be our fault) since logs seem to be scanned multiple times:
>    the same trials logs are showing multiple times for the same string. 
> This would be relatively easy to fix, but we'd like to
>    just remove the greps altogether instead.
>
> Any objections?
>
> Thanks!
>
>
> --
> Kind regards,
> God Bless,
>
> Oracle <http://www.oracle.com>
> Roel Van de Paar | Senior QA Engineer
> Mobile: +61 0400 225 827 <tel:+61%200400%20225%20827>
> Oracle MySQL Server QA
> Oracle Australia | NSW 2440
> Green Oracle <http://www.oracle.com/commitment> Oracle is committed to
> developing practices and products that help protect the environment


--------------------------------------------------------------------------------


> Hi Phillip, all,
>
> We would like to remove the following greps from combinations.pl:
>
>    foreach my $string ('text=', 'bugcheck', 'Error: assertion', 'mysqld 
> got signal', 'Received signal', 'exception') {
>        system("grep -i '$string' $workdir/trial*log");
>    }
>
> There are two problems with these greps:
>
> #1 They consume a huge amount of CPU power for large log files (when for 
> instance --sqltrace was used in a combinations.pl run):
>
>    %CPU COMMAND
>    93.2 grep -i Received signal ...
>    87.9 grep -i mysqld got signal ...
>    87.9 grep -i Received signal ...
>    84.9 grep -i Received signal ...
>
>    While it is easy enough to grep through results after a combinations.pl 
> run (and there are other interesting strings).
>
>    Something that works well is for instance;
>    grep "exited with exit" *.log | grep -vi "STATUS_OK"
>
> #2 At the moment, these greps are maybe in the wrong place in the code 
> (could be our fault) since logs seem to be scanned multiple times:
>   the same trials logs are showing multiple times for the same string. 
> This would be relatively easy to fix, but we'd like to
>   just remove the greps altogether instead.
>
> Any objections?
>
> Thanks!
>
>
> --
> Kind regards,
> God Bless,
>
> Roel Van de Paar | Senior QA Engineer
> Mobile: +61 0400 225 827
> Oracle MySQL Server QA
> Oracle Australia | NSW 2440
> Oracle is committed to developing practices and products that help protect 
> the environment


--------------------------------------------------------------------------------


> _______________________________________________
> Mailing list: https://launchpad.net/~randgen
> Post to     : randgen@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~randgen
> More help   : https://help.launchpad.net/ListHelp