firmware-testing-team team mailing list archive

[Brendan Donegan <brendan.donegan@xxxxxxxxxxxxx>]

On 22/11/12 15:17, Colin Ian King wrote:
> Just following this up.
>
> So, I *think* I can provide a solution using the following mechanism.  
> I am proposing to add two extra command line options:
>
> --filter-error-discard
> --filter-error-keep
>
> these are mutually exclusive options, so you can't use both at the 
> same time.
>
> --filter-error-discard is supplied a comma separated list of fwts 
> error labels that you don't want reported, e.g.
>
> fwts s3 --filter-error-discard=DevConfigDiffAfterS3,ShortSuspend
I think this sounds flexible enough for our means - I still need to 
start actually figuring out which false positives are striking us 
though, but thanks for implementing this!
> ..and fwts will run the test but not report these as errors. It will 
> effectively silently discard these.  So it is an "opt-out" mechanism.
>
> Alternatively, one can specify the errors you just want reported, e.g.
>
> fwts s3 --filter-error-keep=BadWakeAlarmS3,DevConfigDiffAfterS3
>
> ..so fwts will run the test and *only* report these errors.
>
> I *could* also make an option where one can list the errors in a 
> filter file, but for now, lets see if these two options solve your 
> issues.
>
> What do you think Brendan?
>
> If I don't get any response by end of year, I will drop this feature 
> for Raring and it will be carried over to Raring+1.
>
> Colin
>
> On 06/11/12 16:32, Colin Ian King wrote:
> > Hi Brendan,
> >
> > Apologies that I couldn't make it to UDS and talk to you face-to-face. I
> > recall at UDS you said that the main issue was with with the
> > suspend/resume tests producing false positives for you.
> >
> > Can you remind me of the kind of issues you saw so I can see if I can
> > implement a workable solution?
> >
> > Thanks
> >
> > Colin