[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Ayatana] Idea: letting users of the testing distribution participate in testing and report feedback on different UI choices if they wish

Martín Soto ha scritto:
2009/6/16 Vincenzo Ciancia <ciancia@xxxxxxxxxxx <mailto:ciancia@xxxxxxxxxxx>> 

    Please, let's keep the "this is something that only power user
    like/dislike" old argument out of this discussion. I see this is not
    your intention, but as we are all power users this is an effective
    dialectic technique to lower the value of our observations.
I wouldn't want to lower the value of any observations you've made, as long as these are observations about other people's behavior and not about your own behavior, tastes or preferences.
Ok, then let's summarize: I can be defined a "power user" but I am seriously concerned with usability and not so much with customizability or feature-loadness. I think I gained enough experience in the past to know why the latter two are not as much important as the former. I started 12 years ago as an FVWM advocate who liked to take one week to write his own .fvwmrc that did a looney-tunes like introduction at login, and some snow close to xmas - when kde and gnome did not exist. Old times :)
This is not offtopic: I want to tell you how I was convinced to change mentality. I read the famous gnome usability studies for gnome 2.0. Then I stopped laughing at gnome 2.0 and its feature-less-ness. This is why I would suggest that usability studies get published, it didn't happen this time and we are still not convinced.
I believe you that you and Scott are not the only guys who hate this feature. Still, the problem with saying "there are 20 people in Launchpad who hate it too" is that all of you conform a self-selected sample.
The point is not if I hate or not something. The point is "how many" hate this. Now there are two ways to see this. Either you want to use launchpad features as a metric or not. In the first case you compare the number of duplicates, subscribers (not only comments!) with other bug reports. Comments come from vocal persons, but reports do not necessarily. So duplicates are perhaps better than comments.
But you need to know about statistical properties (what is the typical distribution of the number of duplicates???).
If you think that launchpad is biased as you explain, and I certainly AGREE WITH YOU (surprised??), then you have to invent another way. In both cases, saying that 20 duplicates means 20 users is a very bad estimate. You'd need to know the average of users that suffer from a bug w.r.t. those who take the burden to try to report it. AFAIK we do not have ANY CLUE about that. Unless we start messaging a bit in the default distribution about bugs we will never have this metric, but I guess that going out and talking to your friends can give _you_ a good idea on the impression of people about the popup. I did that and reactions are laughters or just "I don't care about updates anyway". 



    Now let's get to the point of which evidence we have that people do not
    like popups in general. For update-notification, if you want evidence,
    again, create a poll and find a way to gather the opinion of users. I
    won't do that because I already have good experience.
The risk of such a poll is the same: Self selection. Obviously, people are much more likely to participate if the are bothered by the feature, which will immediately introduce a strong bias.
Then a good idea *may be* to put a general poll in the testing distribution that all users are strongly encouraged to take. There you can discuss the 4-5 big changes of every distribution (those that typically go in the release notes). Just an idea.
Even better: we can actually implement, or mockup in a clear way, the possible design choices, and then in the poll ask users to test each choice for some time, and in the end tell us what they preferred. After all testing is for testing and for reporting. Why must testers jailed into bug finding? Can't they also be involved in the decision process? I will propose this as a separate thread but is ayatana the good place? Perhaps ubuntu-devel-discuss would be better. 

Although I'm a
scientist, I'm not an expert in this kind of research, so I guess I'll ask my poll-designing colleagues here at work what they would do in such a situation and see if they have a better answer. 

There must be one! Please report your findings.
You are speaking about pop-ups here, but the update notifier is rather a "pop-under". It remains discretely behind other windows until you select it. The only way it can be intrusive, as you already pointed out, is by getting in your way when you're trying to switch windows with Alt+Tab.
Not only, also by closing the window which is over. Then you find this and wonder "hey, since when this thing is here?". And you are certainly not guaranteed that closing e.g. evolution means not being already late to start serious work :) In any case, finding a window already opened the user will probably not want to act *right now*, and then will close it.
Achieving a less disturbing system is, of course, a valuable goal. The problem here is that if your system is, for example, running an insecure network stack or a file system module that may destroy all of your data, you'd rather be disturbed about it.
Yes this was a strongly advocated point in favour of the new system. I just still think that the red-border white triangle with an exclamation mark in the notification area would be as good. 



     > . I would expect most people to just confirm the updates and keep
     > going with their lives

    Are you saying that you really NEVER experienced an upgrade that creates
    a problem? I use my computer to work. Sometimes I just can't afford the
    risk that the thing breaks, even in minor aspects. E.g. when I am
    preparing a presentation or I am under a deadline. For this reasons the
    popup cannot be that frequent, it'd be annoying to people like me.
I'm starting to wonder if we are speaking about the same piece of software. In my case, the pop-under remains quietly in the background. I can ignore it for as long as I need. I was never forced by my system to apply updates when I didn't want to. Am I missing something here?
No come on. If I really am working I don't want other windows opened at all, I must be concentrated, so I close the window. For me it is important to work concentrated in a clean environment. I keep as less windows opened as possible because I must be able to find the right one shortly. Keeping a window that I don't need opened is annoying for me. Therefore I close it and then I forget to apply updates e.g. when I stop my work and can afford a 30 minutes ubuntu-fix-break :) And indeed usually things go well! Otherwise I wouldn't be using ubuntu at all. 


[...]

    Clear evidence will be obtained only when studies will be published. All
    of this **** is based on a study which has not been published. I do not
    work in usability and do not have the resources to do a test, but if you
    find any, and you need some cooperation, I'll be glad to help designing
    some experiments.
This sounds great. We seem to have enough scientists here to run a small University, and still don't manage to get our act together and try some actual usability studies. I'm not an expert either, but we surely can learn.
We could use the testing distribution as a testing environment only for those users who, prompted by a clear question (a popup window maybe :P) when they install karmic (or karmic+1, I do not think we are in time for k), decide to participate in usability tests, involving changing the system in various ways for a week or so and then report feedback in a number-crunchable way.
But experts on statistics and usability will be of great help for this to be actually useful. Clear to everybody I suppose. 

Vincenzo