ubuntu-phone team mailing list archive

[Niklas Wenzel <nikwen.developer@xxxxxxxxx>]

Hi Martin,

Thank you very much for your reply. :)

I checked out the article you referred to in your post and it looks great.
I'd love to see that implemented in the store.
Given the fact that I can't help out myself, how long do you think will it
take you to get that set up?
I know you're quite busy at the moment but as an app developer whose app
"currently suffers from its popularity" I find that quite important.

Thanks again,
Niklas

2015-04-09 14:38 GMT+02:00 Martin Albisetti <argentina@xxxxxxxxx>:

> On Thu, Apr 9, 2015 at 5:50 AM, Niklas Wenzel
> <nikwen.developer@xxxxxxxxx> wrote:
>
> Thanks for bringing this up!
> I completely agree that the current sorting is naive at best and it is
> not intended to be the way we sort search results at all, just a
> convenient way to get us bootstrapped.
>
>
> > To me it seems that we agree we should improve the sorting algorithm. I
> > acknowledge the problem of having no way to deal with unfair reviews but
> I
> > believe this should overall be separated from the sorting problem, even
> > though a better sorting mechanism might relativise some bad reviews.
> >
> > Until now, we've had two suggestions: the Bayesian average and the
> solution
> > mentioned by Selene, where each app would get a 3 star rating by default
> > which would be adjusted later when people start reviewing the
> application.
> > In my opinion these are pretty good solutions, with each of them having
> > their individual advantages or disadvantages.
> > The Bayesian average tries to predict what would happen if plenty of
> people
> > were asked to rate the app but apps without bad reviews only still rank
> > better than those without any reviews.
> > Selene's method solves the latter, but depending on the implementation it
> > might be disadvantageous for apps with few reviews. It would be nice to
> see
> > a code example here. Selene, is your music player open source?
>
> There are a lot of good points about people gaming the system and
> problems with different approaches. Luckily, this is not a new problem
> at all. In fact, it's not even new to Ubuntu!  Matthew Paul Thomas
> spent some time thinking about this years ago and put together a great
> wiki page[1].
> Which points to a great post on how to avoid the most common problems:
> http://www.evanmiller.org/how-not-to-sort-by-average-rating.html
> We also plan to add in other metrics over time that are harder to
> game: # of downloads normalised by download date, retention (how many
> people keep it installed?), usage (opt-in feature to let the store
> know how often the app is used), quality (number of crashes) and
> others. To try and avoid loops where apps get more downloads because
> they started off higher on the list, climbing up higher and making it
> harder for new or less well-known apps to surface, we'll likely
> introduce a small amount of serendipity into the search results,
> occasionally returning something high on the list that might not
> normally make it.
> So yes, we will improve this soon and improve it iteratively over
> time, I expect us to be working on this on and off as long as the
> store exists.
>
>
> > My next question: I'd love to help implement the new ranking mechanism.
> Is
> > the code for the app store backend available somewhere so that I can
> > retrieve and modify it? If not, who is the person to talk to?
>
> Unfortunately this piece isn't open source. The obscurity of how
> search results are (will be) sorted in this case will help make it
> harder to game the system by developers, making it a bit of a better
> experience for users.
> A bit down the line, though, we might be able to open up some amount
> of data and have a contest to improve sorting, similar to what Netflix
> has done in the past[2].
>
> In the mean time we'll soon improve the sorting a bit to get us
> through this next phase of growth, which is a fantastic problem to
> have  :)
>
>
> [1] https://wiki.ubuntu.com/SoftwareCenter/RatingsAndReviews
> [2] http://en.wikipedia.org/wiki/Netflix_Prize
>
> --
> Martin
>