lubuntu-qa team mailing list archive
-
lubuntu-qa team
-
Mailing list archive
-
Message #01050
Re: ISO Testing Cadence Week
Only part of this has to do with Lubuntu specifically so apologies to Phill and the rest of the Lubuntu QA team, but the lions share - mostly ubiquity stuff - effects us too. I started iso/upgrade testing with Ubuntu so I feel I must carry through with some ongoing Ubuntu testing, and there will be times that it may be necessary to compare how casper and/or ubiquity perform with the various flavors of *buntu in general ;^)
This morning I saw something in the Ubuntu changelogs that peeked my interest so I spun a new disc (Ubuntu QQ i386-live 20120809.1) and gave it a sweet test drive with an existing Windows and PP Edubuntu install. When I chose install I was offered the option to "upgrade" 12.04 to 12.10 and I chose that option. It worked (and has in my experience always worked) quite well :^)
That's one of the new options offered by ubiquity since it was majorly upgraded in Maverick. Certainly there were bugs (one quite serious), and only Colin Watson would know what's going to change in the near future, but there is no new documentation regarding that new upgrade option :^(
More specific to iso-testing we've never adjusted the iso-testing test-cases to address the change. We still offer only manual partitioning, entire disc, and auto-resize. But ubiquity now offers much more than that. To be fair even prior to the change we never addressed "use largest continuous free space" in our test cases but with Lucid approaching EOL that's not an issue.
I do however think we should at some point update the iso-testing cases to reflect the new options. Evan Dandrea and Colin Watson should be invaluable in that regard, but I must say that Evan screwed up big time eliminating the "use largest continuous free space" option. Currently you can only tell what's going to happen in that regard by whether the lower right hand button says "continue" or "install".
But ultimately there are installation/upgrade options available that are NOT even being offered as test cases. That can NOT be portrayed as good in any way, shape, or form from a QA perspective. I helped write ISO-9000 specs for a multi-national company once so I'm not clueless - just dain bramaged, and slow now :^(
Now that my rant about ubiquity test-cases is out of the way you can take a breath and prepare for my super dumb question ;^)
I just don't understand the purpose of cadence testing. I missed Alpha 3 testing prior to July 26th due to health issues but according to my calendar the next Lubuntu is QQ Beta1 due on Sept 6th. Is that correct?
If so what is the purpose of "cadence testing" at this point in the cycle?
As previously mentioned I feel an absolute duty to still perform some Ubuntu testing on top of my Lubuntu testing and I know that Ubuntu hits 12.04.1 on Aug 23rd so I've tried to set aside the week of the 20th ;^)
Even though Lubuntu will not release a 12.04.1 iso some changes in the Ubuntu 12.04.1 repos could effect Lubuntu, specifically Abiword ;^)
How many brains have I fried besides my own?
Lance
PS: I feel much better with more oxygen getting to my brain :^)
PPS: I just hope I'm not being too much of a pain in the nether regions.
--- On Wed, 8/8/12, Nicholas Skaggs <nicholas.skaggs@xxxxxxxxxxxxx> wrote:
From: Nicholas Skaggs <nicholas.skaggs@xxxxxxxxxxxxx>
Subject: Re: [Lubuntu-qa] ISO Testing Cadence Week
To: "Phill Whiteside" <PhillW@xxxxxxxxxx>
Cc: "Lance" <lbsolost@xxxxxxxxx>, lubuntu-qa@xxxxxxxxxxxxxxxxxxx
Date: Wednesday, August 8, 2012, 4:01 PM
Lance, these bugs are perfect to bring
up. I will ask the release team directly both about there plans
for abiword, and that installer bug. It was slated to be fixed as
you can see, but clearly was not fixed as part of alpha 3.
I would echo phillw here; don't ever sit on a bug that isn't
getting good attention! I have raised attention to several bugs
this cycle brought up by folks like yourself and gotten
resolution. We can't always fix everything, but I don't like bugs
sitting without comment. If we can't fix it, let's talk about why
and workarounds. If we can, let's communicate the timing.Things
are very easy to slip through the cracks, and part of our job as a
community QA team is to point out things like this that slip
through. I should have an answer on Friday -- I'll push to have
follow-up comments on both bugs, so you'll see the response.
On the cadence testing, this is something the greater community
asked us to do. I think it's time we all talked about how it's
working out, and in general, how ISO testing, communication,
respins, etc are working out. We all know the importance of iso
testing, but we also know that it has issues because of so many
teams and people involved. You guys should be proud of your work
in testing, and it's my goal to see your work valued not only by
me, but by the rest of the community. After alpha 3 we said we
would meet and talk. It's been a week and a half, plenty of time
to reflect. Time now to talk and see what we can do to make beta 1
better. Expect to see an invite from me soon :-) Thanks,
Nicholas
On 08/08/2012 12:13 PM, Phill Whiteside wrote:
Hi Lance,
Bug #987418 is still active. On the defence of the release
team, I include a note from their meeting today:
(16:51:05) slangasek: I know 12.04.1
has pulled a lot of effort away from quantal development for
this first part of the cycle, but please make it a focus for the
next couple of weeks to get these back on track
The Abiword bug is also still very much alive and kicking.
@Nicholas could you check as to if any progress has been made
to put the 'stable' version of Abiword into precise &
quantal, the note from the abiword people about frustration at
ubuntu shipping an unstable is quite evident.
Lance, with regard to cadence testing, it is accepted that
the current set up does not make the reporting of such tests
easily available. It is being worked upon.
(15:48:25) phillw: balloons:
has any progress been made so that the daily builds do not wipe
out what has been tested on http://iso.qa.ubuntu.com/qatracker/milestones/219/builds
(15:52:06) balloons: phillw,
you mean in the same way as during a milestone?
(15:52:24) balloons: this week,
although it's harder for you to see the previous results, they
are not lost or wiped
(15:52:41) balloons: I am counting
and attempting to track them.
(15:52:42) phillw: balloons:
yes. otherwise no one knows what has been tested :)
(15:53:08) balloons: phillw,
yes, I know.. that is definitely an issue.. But we don't have a
good tech solution atm to the problem
(15:53:16) phillw: the
only way I can track is by turning on the history which makes
for a very long list!
(15:53:23) balloons: if we did a
traditional milestone, we'd lose results on the new iso releases
Honest, my good friend, these things are not being ignored!
Please do not feel down hearted.
If anyone has points that they wish raising at the weekly QA
meeting, please feel free to email me and I'll add them to the
agenda at https://wiki.ubuntu.com/QATeam/Meetings It
is important that we know of frustrations. I do try to keep
myself updated by the tracker and this mailing list as to any
issues that I perceive as needing a 'kick up the bum', but stuff
can sneak in under my radar and I rely on you fantastic people
to poke me to ensure that we do not have people thinking there
are serious bugs that are felt to be unloved. Having had
personally an un-loved bug, I do fully appreciate just how
frustrating it is!
Regards,
Phill.
On 8 August 2012 14:36, Lance <lbsolost@xxxxxxxxx>
wrote:
Before I even
get started please remember that I suffer from a
brain injury and my analysis may be extremely flawed
;^)
I rather think "cadence" should be changed to
"early"! Are we ready for iso-testing? This should
be a point where we see if we're even close to being
ready. If we're not ready how do we report that?
Regarding re-spins I don't know how we can possibly
improve that. If a serious bug is recognized by one
of the devs there will be a respin. But this is a
serious bug:
https://bugs.launchpad.net/ubuntu/+source/ubiquity/+bug/987418
It effects both Precise and Quantal but it seems to
get no love :^(
Another is making a decision on Abiword:
https://bugs.launchpad.net/ubuntu/+source/abiword/+bug/1019621
There comes a point where testers wonder if their
testing even matters. I've been testing since Hardy
and I've never had less confidence in the effect of
my effort :^(
When testers feel they're no longer being listened
to they will stop testing.
Lance
--- On Tue, 8/7/12, Phill Whiteside <PhillW@xxxxxxxxxx>
wrote:
From: Phill Whiteside <PhillW@xxxxxxxxxx>
Subject: Re: [Lubuntu-qa] ISO Testing Cadence Week
To: "Nicholas Skaggs" <nicholas.skaggs@xxxxxxxxxxxxx>
Cc: lubuntu-qa@xxxxxxxxxxxxxxxxxxx
Date: Tuesday, August 7, 2012, 8:16 PM
Hi Nicholas,
in the past, the question has been
raised about testers dong tests, only for
them to be deleted the next day as a
respin comes up. Are tests / bugs now
being carried over?
what is the difference between cadence
and the qa-testing milestones?
I will ensure our wiki area is updated
with your reply.
Regards,
Phill.
On 7 August 2012 21:36, Nicholas
Skaggs <nicholas.skaggs@xxxxxxxxxxxxx>
wrote:
Cheers
everyone. This week is a cadence
week for iso testing. What that
means for us is to test all of our
ubuntu isos mandatory testcases at
least once over the course of the
week. A big thank you to those who
are already leading the way on
this with testing:
gregfaith,
michael-kroll, fabiomarconi, and
doak-jackson
These 4 have already reported
results to the isotracker. Thanks!
You may have noticed that the i386
ubuntu images had issues building,
but by the time you read this
those should be resolved. If you
need help, check out the
walkthrough page on the wiki
and/or email the list. We're happy
to help anyone through there first
tests case.
https://wiki.ubuntu.com/Testing/ISO/Walkthrough
Happy Testing everyone!
Nicholas
--
Ubuntu-qa mailing list
Ubuntu-qa@xxxxxxxxxxxxxxxx
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-qa
--
https://wiki.ubuntu.com/phillw
-----Inline Attachment Follows-----
--
Mailing list: https://launchpad.net/~lubuntu-qa
Post to : lubuntu-qa@xxxxxxxxxxxxxxxxxxx
Unsubscribe : https://launchpad.net/~lubuntu-qa
More help : https://help.launchpad.net/ListHelp
--
https://wiki.ubuntu.com/phillw
Follow ups
References
