ubuntu-phone team mailing list archive

[Leo Arias <leo.arias@xxxxxxxxxxxxx>]

On Fri, Dec 13, 2013 at 4:25 PM, Barry Warsaw <barry@xxxxxxxxxx> wrote:

> AFAIK, system-image and system-settings are not autopilot tested, but again
> it's not clear that such testing would have caught *this* issue.  Not that
> autopiloting the upgrade sequence isn't still a good idea!  We need to talk
> about that!  But if the error message disappears so quickly to a human
> with no
> functional effect, it's unlikely that an autopilot test would have caught
> the
> problem.
>

For me, there are two problems.

First, we are only doing user acceptance testing for our UIs. We also need
to test our APIs from the black box point of view of their consumers,
writing a test client that will call all the published methods and assert
that they behave as documented. We need this for libraries, dbus services
and servers.

Second, we don't know who are the clients of our projects and what they are
doing with them. Of course it is impossible to get a list of all the people
using a piece of public code; but for teams inside Canonical, I always try
to drink a beer with the people who have written software I have consumed
and that has been useful for me (I'll pay for the beer if using the project
was a pleasant experience ;).

If we have to change the test client for the tests to pass, we will need to
communicate it to all our known clients. Ideally, getting a +1 on the merge
proposal of the change to make sure they all understand and agree with the
change. From these reviews we might decide to add a new method instead of
changing an existent one, or deprecate a part of the API for some time
before removing it, or maybe somebody comes with a clever solution to keep
the same signature. Then this MP will remain as the documentation of all
the details involved in the change, in case we forget. And the changes on
the sample client will serve as a guideline for the real clients to comply
with the new API.

I loved your long mail explaining the things that went wrong. Can we keep
doing it if we fail again? The tone that Barry used is great for learning
from our mistakes, and get ready for when we get more community and
external teams using the code we write.

Just a suggestion, I think it will be easier to read if we write the title
of the bugs involved. It won't make the mail much longer.

pura vida.
-- 
¡paz y baile!
http://one.ubuntu.com