ubuntu-phone team mailing list archive

[Martin Albisetti <argentina@xxxxxxxxx>]

On Thu, Apr 9, 2015 at 5:50 AM, Niklas Wenzel
<nikwen.developer@xxxxxxxxx> wrote:

Thanks for bringing this up!
I completely agree that the current sorting is naive at best and it is
not intended to be the way we sort search results at all, just a
convenient way to get us bootstrapped.


> To me it seems that we agree we should improve the sorting algorithm. I
> acknowledge the problem of having no way to deal with unfair reviews but I
> believe this should overall be separated from the sorting problem, even
> though a better sorting mechanism might relativise some bad reviews.
>
> Until now, we've had two suggestions: the Bayesian average and the solution
> mentioned by Selene, where each app would get a 3 star rating by default
> which would be adjusted later when people start reviewing the application.
> In my opinion these are pretty good solutions, with each of them having
> their individual advantages or disadvantages.
> The Bayesian average tries to predict what would happen if plenty of people
> were asked to rate the app but apps without bad reviews only still rank
> better than those without any reviews.
> Selene's method solves the latter, but depending on the implementation it
> might be disadvantageous for apps with few reviews. It would be nice to see
> a code example here. Selene, is your music player open source?

There are a lot of good points about people gaming the system and
problems with different approaches. Luckily, this is not a new problem
at all. In fact, it's not even new to Ubuntu!  Matthew Paul Thomas
spent some time thinking about this years ago and put together a great
wiki page[1].
Which points to a great post on how to avoid the most common problems:
http://www.evanmiller.org/how-not-to-sort-by-average-rating.html
We also plan to add in other metrics over time that are harder to
game: # of downloads normalised by download date, retention (how many
people keep it installed?), usage (opt-in feature to let the store
know how often the app is used), quality (number of crashes) and
others. To try and avoid loops where apps get more downloads because
they started off higher on the list, climbing up higher and making it
harder for new or less well-known apps to surface, we'll likely
introduce a small amount of serendipity into the search results,
occasionally returning something high on the list that might not
normally make it.
So yes, we will improve this soon and improve it iteratively over
time, I expect us to be working on this on and off as long as the
store exists.


> My next question: I'd love to help implement the new ranking mechanism. Is
> the code for the app store backend available somewhere so that I can
> retrieve and modify it? If not, who is the person to talk to?

Unfortunately this piece isn't open source. The obscurity of how
search results are (will be) sorted in this case will help make it
harder to game the system by developers, making it a bit of a better
experience for users.
A bit down the line, though, we might be able to open up some amount
of data and have a contest to improve sorting, similar to what Netflix
has done in the past[2].

In the mean time we'll soon improve the sorting a bit to get us
through this next phase of growth, which is a fantastic problem to
have  :)


[1] https://wiki.ubuntu.com/SoftwareCenter/RatingsAndReviews
[2] http://en.wikipedia.org/wiki/Netflix_Prize

-- 
Martin