ubuntu-phone team mailing list archive

[Brendan Donegan <brendan.donegan@xxxxxxxxxxxxx>]

On 27/06/14 19:39, Sergio Schvezov wrote:
> On viernes 27 de junio de 2014 14h'45:47 ART, Łukasz 'sil2100' Zemczak
> wrote:
>> Hi,
>>
>> So, in overall I would also opt for getting the issue fixed instead of
>> just disabling the test, but I can understand the rationale for the
>> disablement itself; as this is an issue that's coming from a separate
>> component - and that the it's actually blocking landing of a serious
>> regression fix.
> 
> Always a difficult spot to be in :-)
> 
>> I agree with what the QA representatives said: let's disable the test
>> but only if we have a bug reported and at least having someone working
>> on it actively, making sure it gets actually fixed as soon as possible.
>> I would also encapsulate the actual test disablement in a big FIXME with
>> the bug number mentioned in the comments.
> 
> Some test runners support an ExpectFail annotation, wouldn't that be
> prefered? Once it starts working the test would fail and you'd get rid
> of your workaround.

Even if they don't it should be possible to invert the logic - so here
the test expects the result to have 2 elements, but instead we are
getting 0 I guess - if we change the logic to expect 0 and put a comment
there - then when it starts working again it will fail and someone will
have to look at it, to be greeted by the comment, immediately telling
them what they need to change. Even though it feels wrong, it seems more
robust than any other solution. Of course if the test runner does
support ExpectFail, then that's way cleaner and better to use. In this
case the test is GTest based so I'd be surprised if it doesn't have such
a construct.

> 
> Cheers
> Sergio
>