unity-design team mailing list archive

[Charline <charline.poirier@xxxxxxxxxxxxx>]

On 26/06/11 15:17, Søren Olesen wrote:
> Hi there,
>
> The community around Ubuntu in Aarhus, Denmark, has been invited to a
> technology conference in late August. The theme of the conference is
> user driven innovation and learning from your own mistakes. The audience
> at the conference will be both techies and non-techies, though mostly
> the non-techies. We have accepted the invitation and we will be at the
> conference two days.
>
> Our aim at the conference is twofold. First, we would like to present
> Ubuntu to a wider audience. Second, we would like to show how users can
> take part in making Ubuntu better in order to fit into the bigger theme
> of user driven innovation. We have thought about ways to combine the two
> and have come up with the idea of doing usability testing of Natty
> Narwhal. Usability testing introduces people to Ubuntu and at the same
> time gives information on how to make Ubuntu better. Our point of
> departure is the usability test report from April on natty, the early
> test of Unity and the usability test of Thunderbird. We are thinking
> about doing short usability studies like  "install X program", "post an
> update on Facebook" or "find Y file using a lens". The tests have to be
> shorter than the ones in aforementioned test because we can't keep the
> participants for too long.
I take it this would be at the conference?  This is a great idea!
> In order to make the results we get from the usability results as valid
> as possible we would like some input from on three issues. First, do any
> of you have programs or features that you would like us to test? If you
> do we would be happy to do so. Please keep in mind that the participants
> are non-techies. Second, what kind of background information should we
> get on the participants? I am thinking age, primary OS, experience using
> their OS, how many hours they spend using a computer every day etc.
> Third how should we conduct the test to ensure valid results?
What do you intend to do with the results? How reliable do the findings 
need to be? If you intend this as a demonstration to non-tech users of 
how they can contribute, maybe your most important goals should be to 1) 
have them experienced a usability problem and make them realise that the 
problem is with the technology and not with them; and 2) give them a 
tool to make the experience better.  You could build your study in the 
following way:

   * Select a task where you know there is definitely a usability problem
   * Choose that usability problem one that you know normal users can
     formulate.  Remember that end users are not usability
     professionals or designers nor should they be.  But there are some
     usability problems users can figure out - a very good example is
     lack of feedback ("I don't know what it's doing.  I don't know
     what to do!  The system should be telling me something")
   * Have them write a bug report about that problem - this would
     support the user-driven innovation part of the conference, would
     be empowering and drive the end user implication in Ubuntu.

At the end of the conference you could present a very nice summary of 
what you found, not only about usability problems (if you've read the 
reports, you know that we are already familiar  with them) but also you 
could speak about what end users  about said the issues they encountered 
and how they wrote bug reports.  You could even speak about how 
developers will use these bug reports to make Ubuntu better.  That 
should be interesting to your audience.

As for validity, keep in mind that valid results will come from:
1) how you select your participants, you might not have much choice there;
2) how you set up and formulate the task.  Make sure you formulate your 
tasks in a context and scenario that make sense to your participants. 
Basically, you need to formulate your tasks from a user point of view:  
For example, normal users don't install programmes, when they do, it's 
because there is something else they want to accomplish, the user goal 
is not to install a programme but it might be to edit their photos.  
Make the task the user goal even better something they do often, like 
erase red eyes from a picture. and;
 3) how you will approach the analysis.  Above, you mentioned age, OS, 
and experience using OS.  You could compare all these different 
variables in your analysis. Generally, I find that Windows and Mac users 
have different expectations and consequently, different usability 
problems with Ubuntu.  Similarly with age.  As for 'expience using OS', 
you would probably be comparing, in the example of editing photos, 
people who have a lot of experience to people who don't (the latters 
would be 'learning' basically what to do,  which is different from using 
a programme).  But remember that the more variables you want to compare, 
the more participants you will need.
> Would
> desktop recorder or observation do or should we try to get a camera on a
> tripod?
Desktop recording and observation are most useful.
> Please let me know if you have ideas on what we could do or who we could
> contact.
Feel free to get in touch with me with any questions.  This is a very 
interesting research and I would be happy to give you any help you need 
or discuss the usability reports you will be using. I am also very keen 
to hear about your results and how your contribution to the conference went.
> Thanks,
> Søren


--

*CHARLINE POIRIER*
*User Research Programme Lead*
*Canonical*
*27th floor, 21-24 Millbank Tower*
*London SW1P 4QP UK*

*Tel: +44 (0) 20 7630 2491*
*Mob: +44 (0) 78 8695 4514*
*www.Ubuntu.com  <http://www.Ubuntu.com/>  <http://www.Ubuntu.com/>*
*www.Canonical.com<http://www.Canonical.com/>*