ubuntu-phone team mailing list archive

[Oliver Grawert <ogra@xxxxxxxxxx>]

hi,
On Do, 2013-11-28 at 17:01 +0000, Dave Morley wrote:
> On 28/11/13 12:13, Oliver Grawert wrote:
> > Hi,
> > 
> > As some might have noticed we recently had a few bad image releases into
> > the Trusty channel that contained regressions. 
> > We also tested out some changes to the build policies of the proposed
> > images that I would like to establish further.
> > 
> > Within the last weeks we raised the amount of built images per day to a
> > higher frequency (3 at most currently) ... the reason for this:
> > currently finding a regression in an image when only building a single
> > image per day results in a pretty big change set. Finding the offending
> > package that introduced a regression can take hours of weeding through
> > package changelogs and is often based on guesswork ... usually we are
> > guessing pretty well, but this still burns a lot of developer time we
> > should rather spend on fixing bugs and writing tests or features.
> > 
> > with building at least 3 images per day the change sets got a lot
> > smaller, finding issues becomes a charm and offending packages are easy
> > to identify ... 
> > 
> > My proposal here is to re-enable cron driven builds again. 
> > For a start this has to be every 8h since our testing infrastructure can
> > not cope with more frequent builds and over time (as our testing
> > infrastructure improves) to get to a frequency of doing a build every
> > 4h, this should give us a small enough set of changes to be a lot
> > quicker in finding regressions. I would like to switch cron back on by
> > end of this week on the above mentioned 8h schedule, it seems all
> > involved teams agreed that this is a good plan, please speak up if this
> > plan does not suit the way your team works (or speak up in support :) )
> 
> Yay!!!!  What times?  The likelihood is that each time zone will only
> ever see 2 releases a day I'm assuming.  But it would also be good to
> know if you should wait another 5 minutes for the next release before
> downloading, especially if I'm on 3g.
teh times will indeed be publically announced here, on a wikipage and on
G+ ;) you wont miss them ...
> 
> > 
> > The second part of my proposal goes a bit further than just adding a
> > cron entry ...
> > 
> > Today the release of an image is handled by the Landing Team, which was
> > traditionally only responsible for landing new upstream code inside the
> > image ... this task alone already hogs most resourcesof the team.
> > Releasing an image is usually pretty quickly decided as one topic of a
> > daily meeting based on a quick glance over the automated tests and by
> > asking for feedback from people that have done a short dogfooding
> > smoketest ...
> > 
> > I personally think the release process deserves a lot more attention,
> > way more input from other teams and a wider dogfooder audience. The
> > Landing team should concentrate on the landings of new code, the time
> > they need to invest in additional image testing is time they can not
> > spend on testing new landings which slows all of us down by creating an
> > artificial bottleneck now that all landings need to go through that
> > team. They should be able to fully concentrate on this process instead.
> > 
> > What I like to throw in as a discussion point is to form a new
> > "touch-release" team that consists of people from QA that bring in the
> > buglist of collected bugs and triages and also checks errors.ubuntu.com
> > regulary (we might need a "touch" filter there), one representative of
> > the avengers team (who dogfood the stable images), someone from the
> > ubuntu-cdimage team and a representative of the Landing team to give
> > input and get feedback on the recent landings from actual endusers.
> > 
> > This team should be a public focused team holding daily IRC meetings the
> > whole community can participate in (as opposed to privately held
> > team-only hangouts), I know there are plenty of users of the
> > trusty-proposed/devel-proposed images and I think we should really try
> > to have them participate in the release process and enable them to give
> > us feedback to:
> > 
> > a) regressions they run into that went unnoticed by the developers
> > b) new bugs the QA/triage teams have not yet seen
> > c) critical blockers we didn't even consider or see yet because they
> > only happen after a day or two of constant usage.
> > 
> > With the higher frequency of proposed builds it should then be possible
> > to pick the image with the best automated results from a former day (or
> > even two days so testers had more time for long term testing) that is
> > considered good by all participating parties and their individual
> > requirements.
> > 
> > Lets put more focus onto the Image releases and take load of the Landing
> > team to speed us all up and have the community more involved to actually
> > increase the quality of our images and be truly regression free.
> > 
> 
> I think that something based on these lines would make a certain amount
> of sense.  I would hold off on a meeting everyday though if the
> community are involved it is hard for everyone to turn up everyday
> particularly with timezone constraints and instead schedule them for
> Tuesdays and Thursday.  This would mean you roll out a stable-ish image
> based on the meeting and leaves you with a nice stable release over the
> weekend for people to really dogfood.
> 
> It might also be good, as we start the process of reducing the test
> failure rate, that we set a level that we won't release under.  For
> example if the overall image percentage drops below 90% it is simply
> ignored till the next release. This cuts down drastically on the image
> pool to release from.
> 
> Just some addition thoughts.
> 
> 
this makes a lot of sense ... I would still like to do meetings more
often than once a week, not every community tester needs to participate
in each meeting as long as we can aggregate the feedback somehow ...
probably daily is a bit to much though

ciao
	oli

Attachment: signature.asc
Description: This is a digitally signed message part