launchpad-dev team mailing list archive

[Dan Harrop-Griffiths <dan@xxxxxxxxxxxxx>]

Bug tracker display options survey results
--------------------------------------------------------------

Last week I put together a short survey for the new custom bug
tracker, asking users which information items were most important to
see at the same time.

I’m not suggesting we make these changes now, at the end of the
project, but I think tackling them as an escalated bug in the
New Year would be valuable.

The survey size was relatively small (48 participants), however the
quantitative data here backs up many of the comments people have
relayed in previous rounds of one-to-one interview-based user testing,
most notably:

- Status is most important
- Status is more important than Importance (although Importance still
ranks highly)
- Bug heat isn’t useful/important


First question
-------------------

Which of the following are most important for you to see at the same time?
-----------------------------------------------------------------------------------------------------------

Answers ranged from ‘Essential’, down to ‘Can do without’. These were
the highest scorers, classified as ‘Essential’:

- Status: 77.1%
- Bug number: 62.5%
- Importance: 56.3%

Also showing up fairly highly under ‘Essential’:

-Package/Project/Series name: 33.3%.

Respondents also thought it was ‘quite important’ (one rank down from
‘Essential’ to see:

-Asignee: 39.6%
-Date bug last updated: 41.7%
-Bug Tags: 33.3%

At the other end of the scale, 52.1% of respondents said they ‘can do
without’ bug heat.

Recommendations
---------------------------

Given these findings, I’d recommend that we include ‘Status, Bug
number, Importance and Package/Project/Series display in the first
results line in the bug tracker.

I definitely think we can push ‘bug heat’ down to the second line,
given that over half our respondents could do without it. This has
also been echoed a number of times in one-on-one interviews.

If there is physically enough room to accommodate any additional items
on the first line after the items mentioned above, I’d recommend we
add one or more of these: Asignee, Date bug last updated, Bug Tags.
These items will no doubt take up a different amount of room to
display due to differences in text length/icons, etc, and as the
figures are so close (between 41.7% - 33.3%), and on a relatively
small sample size of participants, I’ll leave it to you guys to decide
what can fit onto the first line most comfortably.


Most popular items on first line
----------------------------------------------

My reasoning here is that if the most important information to users
appears on the first line, they’re less likely to need to see a second
line, unless they select more of the less popular options (such as
‘Bug tags’ – 2.1% said was ‘Essential’). This will help us provide a
more streamlined view for the majority of users.

Question 2
----------------

Which 'order by' option would you like to see as the default? Arrow up
(ascending) or arrow down (descending)?

We also asked participants what they thought the default order of the
list should be. The results closely mirror what we have already, with
a descending arrow (high to low) being the most popular default option
(Importance – 62.2% and Bug number – 51.1%). At first glance, it might
sound strange to have bug number’s effectively in reverse order –
however this means that the most recent bugs would be at the top of
the list, as they have the highest figure.

Results for Status
--------------------------

These results were a bit more complicated, with 55.3% wanting the
arrow up (ascending) option, and 31.6% wanting the arrow down
(descending) option.

I think this survey was a bit too basic to gain much value from these
figures, as it didn’t specify what any ascending/descending order
would specifying. This is something I’d need to look into with a more
general study of how users look at status. Eg: Would ‘ascending order’
make more sense with ‘New/Incomplete’ at the top, or ‘Fix released’ at
the top?’

That’s about it from me. If anyone has any questions or comments about
the data or recommendations, please drop me an email back.

My Best,

Dan HG