software-store-developers team mailing list archive

[Aaron Peachey <alpeachey@xxxxxxxxx>]

On Nov 9, 2011 8:27 PM, "Michael Nelson" <michael.nelson@xxxxxxxxxxxxx>
wrote:
>
> On Wed, Nov 9, 2011 at 5:45 AM, Aaron Peachey <alpeachey@xxxxxxxxx> wrote:
> > Hi all,
> >
> > Sorry for the long period of absence and silence - my work has been
> > taking up all of my time and for the foreseeable future I can't see
> > much changing so it's unlikely I'll be writing much code for at least
> > a month or so. Keen to get back into asap though!
>
> Hi Aaron! I hope you're able to do some interesting stuff for your work :)
Hi Michael,

> >
> > Congrats to everyone on achieving some great progress with the release
> > of USC in Oneiric.
> >
> > Re Gary's policy, it all makes sense and will be a good control
> > mechanism for us to ensure quality.
> > Having written a few changes for rnr-server and experienced a more
> > stringent merge proposal process I envisage USC moving more towards
> > that type of thing, where relevant unit tests and written and passing
> > in order for a peer review to be approved. Something we don't
> > necessarily do when proposing merges for USC is write test cases and
> > run all tests to ensure we haven't regressed anything.
>
> Given that you're one of the few client-people who have contributed to
> rnr-server, it'd be great to capture more of your thoughts there, so
> that we can together improve and evaluate the way we do this (it may
> be that our server-side review process has been too stringent?) In
> particular:
I don't find it overly stringent and am happy with the process. It's also
reflective of the fact that a server is much harder to have people test
during a dev cycle because it's essentially live as soon as it's released.

>  * how could the process of getting your branch landed been more fun
> (or less frustrating)?
As an amateur coder I'm more frustrated writing tests because I have a
natural tendency to just want to do cool stuff! However I didn't find much
about the process frustrating. For me,  only having an hour or two in the
evenings to make changes and resubmit mp is the frustrating part because a
reasonably simple change can take a week or two to finalise. However, this
is less to do with the process and more to do with my own schedules,
priority and timezone!

>  * was it clear that you weren't required to write the tests yourself
> - or how could that have been clearer, or was it even a helpful
> suggestion?
Yep it was clear that if I didn't have time to do something (or even
inclination to try) then someone else would pick it up. Being new to python
and very new to django (and very stubborn and ambitious) I was keen to do
it all properly, including the tests. I found the tests were actually good
to write when I got my head around them and very useful, especially when
some things are difficult to test in any other way.

>  * given that you did go and write the tests (and wrote great tests
> :-)), after your branches landed, did you feel that it was a
> worthwhile experience, or something that you'd find easier next time,
> or something that you'd avoid next time?
As above, the tests were definitely useful and I wouldn't propose a branch
now without having written appropriate tests and running the test suite.

> And this last question comes more from the fact that I (personally - I
> know everyone is different) find writing tests first much more fun
> (and often in my limited experience, leads to better implementations):
>  * Does your experience leave you feeling interested in trying a
> branch (not now of course, just when you've time) where you go through
> the pattern of: (1) write test that fails, (2) implement the fix and
> confirm test passes, (3) refactor if necessory, or leave you with no
> motivation (or somewhere between)?
It seems like a better idea to have the target set before writing the
solution. My main concern with this it's that alot of the tests I have
written needed some implementation detail to run (eg setting up the
factory) so personally I think I would need to know how to write tests that
are more generic and less reliant on how the code was written.

> I guess where I'm going is that we all want it to be fun to contribute
> - now and into the future. And that means it needs to be both easy and
> safe to contribute (safe in the sense that the pain that my branch
> landing today causes you in 2 months is minimised). Up until now, on
> the server side, my approach has been:
>  1) Don't require people to write tests themselves, but if they're
> keen to try, help wherever you can. Then as a reviewer, implement any
> missing tests for the branch.
>  2) Related to the above, encourage small branches (~200-600 lines) as
> it can be not only painful, but very difficult to write tests well
> *after* the implementation for large changes)
>
> How could that approach be improved, relaxed or changed (or abandoned)
> - thoughts?
I agree. I would suggest people should be encouraged to write tests for
their branches to ensure they understand their changes and catch any issues
that other ad hoc testing didn't catch (this happened to me a few times!)
I wouldn't recommend changing much else.

> > When I'm trying
> > to get changes written I hate nothing more than fixing regressed tests
> > and essentially it means a longer lifecycle time and potentially more
> > re-work until a change gets merged
>
> Yeah, I hate that also. But I see that as either (1) an issue with the
> state of the test suite, (2) an issue with the state of the code base,
> or (3) an issue with the process I'm using to make my change. From
> memory, you experienced (1) with one of your branches (a test that was
> failing when run against Postgres that wasn't related to your branch),
> and that was something that we could improve (we weren't at the time
> running our tests against Postgres), and did improve - hopefully
> reducing the pain for us all in the future.
Definitely. it was definitely frustrating at times but helped with my own
understanding and pointed out some changes that could be made.

> > but the quality of trunk as a
> > result of the process should be higher result (and the number of bug
> > reports for anyone using development releases should decrease)
>
> Yeah, definitely, and also the ease with which someone can safely
> contribute should continually increase (ie. I know my change isn't
> breaking anything else).
My only other comment is around the setup process for a local version of
the server.  it's reasonably straightforward but not overly well documented
and the chance to run into issues with the build is high. I'd suggest that
process needs to be made as simple as possible and very documented to
ensure the time it takes from deciding to write a change and actually
starting to write code is minimised.

> Thanks Aaron, and totally understand if you don't have time to reply :)
No problems, hope this is helpful.
Aaron

> -Michael
>