2011/6/26 Søren Heldgaard Olesen
<heldgaardolesen@xxxxxxxxx>
It is a bit of a balancing act. On the one hand we want to make
sure that the participants have a good experience using Ubuntu. We will
be handing out live CD's to the participants hoping that they might try
it out on their own computer if they like it. So picking a known
usability problem might spoil their experience.
On the other hand we want to get interesting and useful results from the
tests as well. I think that if we do a proper briefing and debriefing of the
participants it might work out even if we do the tests on a known issue. If we tell the participants that this is a known usability problem; your response is helping
fix it, they might have a good overall experience anyway. We might want
to do tell them as a debriefing in order to ensure that they do not go
into the "experiment" knowing there is a usability issue.
You correctly point out that ensuring they have a "good experience using Ubuntu" is largely if not completely contradictory to the idea of usability testing. Sort of like trying to see which flavors of candy people like, but then hiding the ones that you think they won't like.
How about just pointing out "there are known usability issues, and likely some unknown ones, and we'd like you to help us figure out what's noticable", and then seeing which ones actually both people. If you out-and-out tell them which ones you think are bad, they're likely to focus on that instead of finding things you're not aware of.