ubuntu-phone team mailing list archive

[Oliver Grawert <ogra@xxxxxxxxxx>]

On Fr, 2013-11-29 at 13:38 +0100, Alexander Sack wrote:
> On Fri, Nov 29, 2013 at 1:25 PM, Oliver Grawert <ogra@xxxxxxxxxx> wrote:
> > hi,
> > On Fr, 2013-11-29 at 13:01 +0100, Alexander Sack wrote:
> >> On Fri, Nov 29, 2013 at 12:41 PM, Oliver Grawert <ogra@xxxxxxxxxx> wrote:
> >> > hi,
> >> > On Fr, 2013-11-29 at 11:32 +0100, Alexander Sack wrote:
> >> >> Hi,
> >> >>
> >> >> it seems you put a few changes up for discussion in one shot.
> >> >>
> >> >> Let's keep those separate and look at them one by one:
> >> >>
> >> >> >From what I see you basically propose three main things:
> >> >>
> >> >>  1. lets increase velocity of image production so we get 2-3 images
> >> >> produced in devel-proposed per day
> >> >>  2. make cron the technology we use to schedule and kick those images
> >> >> 2-3 times a day
> >> >>  3. increase manual testing done before "releasing" images create a
> >> >> broader touch-release team that will include avengers and manual
> >> >> testers and community etc.
> >> >>
> >> >> Let me look at them one by one and then give a bullet summary of what
> >> >> I believe we should indeed tweak for now...
> >> >>
> >> >> On 1.
> >> >> ======
> >> >>
> >> >> I think 1. is and was the goal. So I think noone disagrees with the
> >> >> benefits of having 2-3 checkpoints a day and we should just do it.
> >> >> Note: it actually always was that way when I ran the landing team and
> >> >> during release time. I believe we still do it, but if we don't we
> >> >> should certainly ensure that we get back to do this.
> >> > on the majority of days in the past we only had one image build per day
> >> > simply because there were to many landings to wait for and in the end we
> >> > had huge change sets that burned a lot of manpower when searching where
> >> > a regression comes from.
> >> >
> >>
> >> Let's fix that process problem first.
> >>
> >> All we need to do is to be strict about following the time windows for
> >> cutting images regardless of whether the image has a chance to get
> >> promoted or not. We haven't spelled things out like this before, so I
> >> am pretty confident that this discussion helped getting us there.
> > why would it matter at all if an image gets promoted ... in my ideal
> > world we would have builds triggered every time a change set enters from
> > proposed or at least every 2h ... only a minimal amount of these images
> > would be promoted at all, but we had a lot to pick the best one from ;)
> 
> The reason for that is in a CI world we feed that image back so new
> merge proposals get tested on top of the most recent known good state.
> And this need to happen frequently, so folks don't test on outdated
> stuff etc.
I don't see how it makes any difference if that latest image was built
by cron or a human ... or do we have a special "human touch" in manually
built images that I don't know about ?
:)

for the last two weeks I mostly behaved like cron wrt building images
(exactly in preparation for this request since so many people asked me
why we don't do cronned images, I'm pretty sad nobody of them speaks up
in this discussion) it doesn't seem to have interfered with anything.

> 
> And yes, we currently don't have enough automation to be sure it's
> dogfood quality. But fixing that should be done through investing in
> more automation rather than changing the approach to put high latency
> manual and avengers activities into the middle of our rapid CI loop.

it didn't change within the last 6 months, we still have two people
investing a significant amount of time into re-running the tests and
often enough end up with a dogfooder running the same test manually just
to prove the automation wrong (which in turn steals time from the
dogfooder to find regressions).

compared to direct uploads our "rapid" CI loop adds one day from
"commited in trunk" to "in image", what I'm trying with my proposal is
to free up resources and de-couple things so we can move faster, don't
exclude the community as much as we do now and actually have more
testers before we promote anything. simply giving us the chance to
detect more regressions ...

... how many of the avengers run their images in a foreign locale, did
any of them notice that there is no keyboard coming up anymore on non
english phones since we promoted the first image after r10 ?

having 6-10 people testing the image *after* promotion really doesn't
help us to catch regressions *before* promoting and our automated
testing is far from being where we need it for this.

ciao
	oli

Attachment: signature.asc
Description: This is a digitally signed message part