elementary-dev-community team mailing list archive

[Craig <weberc2@xxxxxxxxx>]

Hmm, that's a good point. While the barrier to entry is very high right
now, we could move some of that difficulty from things like simple
installation (I can't remember the last time following the installation
instructions yielded a successful build/install) to things that help
improve our quality. If it has to be hard it may as well be hard for a
reason.

Anyway, even if we do reject a bug fix for lack of testing, at least the
hard work of finding the bug is done for us. From there, we can use the
opportunity to teach the submitter how to test or, in the worst case, write
the test ourselves and implement the fix (since the rejected fix points is
to the cause of the bug).

I'm sure there will be some complications in practice, but it will move us
a step closer to quality and it will allow us to focus on implementing new
features rather than bug hunting. That might be enough in its own right to
attract developers (and likely talented ones at that!).
Agree with Julian. TDD is best suited when the software is developed using
agile development methodology. For something like elementary os, which is
community driven, TDD might not be practical. But writing test cases is a
must I believe. That way with regression testing it might make debugging
new bugs, which occurs when new features are added will get a lot easier.


On Wed, Aug 21, 2013 at 9:45 PM, Julien <spautz.julien@xxxxxxxxx> wrote:

> So if I'm not mistaken, TDD requires you to write tests *before*implementing something.
>
> This might work well for teams with enough developers that don't have to
> rely on volunteers and drive-by contributors, but I don't see this working
> out for elementary, at least not at this point.
>
> Many contributors fix a few small bugs here and there and don't have much
> experience with development, which is why we try to keep the entry barrier
> really low for new devs. When I started here, I had no experience with
> Vala, Gtk and Launchpad at all, and I just fixed some easy bugs,
> implemented some small features in various apps and so on.
>
> If we used TDD, we would have to reject most code by new devs, because
> they didn't implement tests, and then we'll have to explain to them how to
> use tests, and then they'll have to write tests, but that's bad because you
> have to write them before you write the actual code ...
>
> It's going to be a mess.
>
> What might work are regular unit tests, implemented by people who kind of
> know what they're doing (we need some documentation for that, maybe add it
> to the dev guide). This will give us some of the TDD benefits w/o deterring
> potential new devs.
>
> On Wed, Aug 21, 2013 at 3:29 PM, Craig <weberc2@xxxxxxxxx> wrote:
>
>
> I have never heard a project die because they decided to move to TDD. And
>> I heard about lots of hits on the matter.
>
>
> This. I've heard (and witnessed) a lot of projects drop their defect
> percentage from 20% to 3%.  A lot of new-to-TDD developers don't like it at
> first because it feels slower, but I don't think those people remember how
> much time is spent bug hunting at the end of a release cycle.
>
> I think the next step is to find a pilot project and get the lead
> developer(s) to agree to work toward TDD. This will probably look like:
>
> 1) Modifying the project structure to include _test directories
> 2) Creating tests with GLib.Test or some such
> 3) Coaching/mentoring the developers at TDD
> 4) Performing code reviews in which insufficiently tested code is sent
> back for revision
>
> After testing it out for a few months, the community can see how they like
> it.
>
> Does this sound like a reasonable approach? If so, would any project leads
> like to volunteer their project? And who in the community would like to
> participate in such an experiment?
>
>
> On Wed, Aug 21, 2013 at 4:34 AM, Alex Lourie <djay.il@xxxxxxxxx> wrote:
>
>> Albert
>>
>> We don't need to exaggerate. Though TDD is, indeed, a development
>> methodology, it is not supposed to completely change the way everyone works.
>>
>> Just consider that writing a good test takes about 10 minutes, and that
>> each developer writes one or two tests for new stuff they add (or the old
>> one they fix) from time to time. Then, in time, you'll have part of your
>> code tested, which is exactly what we're aiming for. Beginning is hard,
>> continue something is easier.
>>
>> I have never heard a project die because they decided to move to TDD. And
>> I heard about lots of hits on the matter.
>>
>>
>> On Tue, Aug 20, 2013 at 3:37 AM, Albert Palacios <optimisme@xxxxxxxxx>wrote:
>>
>>> Hi Craig,
>>>
>>> Thanks for your explanation and the linked video. I am still agnostic,
>>> but I don't want to be the only one who complained about the proposal.
>>>
>>> At this point I have more questions than answers, and those will
>>> probably be solved with a working example.
>>>
>>> But remember, *TDD is a development methodology* not a testing
>>> methodology. It will change our development work flow, and will probably
>>> move potential volunteers away from the project.
>>>
>>> TDD like every other development methodology does not fit every project
>>> or team. It can be a hit, and it can even the death of the project.
>>>
>>> Albert
>>>
>>>
>>>
>>>
>>> On Tue, Aug 20, 2013 at 12:50 AM, Craig <weberc2@xxxxxxxxx> wrote:
>>>
>>>> Hi Albert,
>>>>
>>>> Thanks for your response, you asked a lot of great questions. In
>>>> addition to Gufran's earlier response.
>>>>
>>>>  Can you prove that there will be huge benefits in time/resources?
>>>>
>>>>
>>>> Well, that depends on what you consider "proof". In the spring, my
>>>> company paid several thousand dollars to send me to a conference in San
>>>> Jose in which many software development authorities recited, "test driven
>>>> development pays itself off in iteration 0". That means the very first time
>>>> you write the code it has already paid for itself (because even before you
>>>> get the automatic-regression testing benefits, you've already got the
>>>> benefits of a better architecture and documentation--because the tests
>>>> _are_ the documentation!). I'm sure I can dig up lots of other resources,
>>>> but I think it should suffice to say that I've never heard an expert
>>>> comment on TDD except to say it's fastest way to develop quality software.
>>>> For more information addressing your specific concerns, see the Wikipedia
>>>> article's "benefits" section (and read on to the "shortcomings" as it is
>>>> also relevant:
>>>> http://en.wikipedia.org/wiki/Test-driven_development#Benefits. Even
>>>> more importantly, this short video:
>>>> http://www.youtube.com/watch?v=DodJQyHsmHI
>>>>
>>>>
>>>> Can you prove that there will be less bugs? (looks like that if tests
>>>>> are not right, bugs will populate equally).
>>>>
>>>> It's pretty hard to write bad tests if you're practicing TDD, because
>>>> you write the test first, watch it fail, insert the code you need to make
>>>> it pass, and then hopefully watch it pass. If you wrote a bad test, it very
>>>> probably will pass before you've written the code to make it pass (which
>>>> serves to alert the programmer that his test is bad or his software is
>>>> doing something unexpected) or the test will fail after he has correctly
>>>> written the next line of code (which serves to alert the programmer to
>>>> review both the code and his test and identify the source of the problem).
>>>> For this reason alone, many, many bugs are eliminated.
>>>>
>>>> From what's been said, looks like there will be an extra effort on
>>>>> development, adding complexity and more tools to know (not to say maintain).
>>>>
>>>> Besides the initial learning curve, development actually goes _faster_
>>>> with TDD (see the aforementioned Wikipedia article--I can provide more
>>>> resources on demand) because debugging time becomes exponentially more
>>>> expensive as time passes after the bug has been introduced. This is because
>>>> the bug can live anywhere in any code that has been added *since the last
>>>> time the tests were run* and because the programmer will have an
>>>> increasingly difficult time remembering the code he wrote at the time of
>>>> the bug as time progresses. With TDD, you are running the tests after every
>>>> change (generally you test every time you build), so as soon as you've
>>>> broken something you find out about it. This means that the bug is
>>>> guaranteed to live in the last change you made, which is a smaller sample
>>>> and fresher-in-your-mind than changes you made weeks ago. Regarding your
>>>> complexity concern, generally the process isn't complex (it's actually very
>>>> simple) and it _simplifies_ development once you learn how to do it. The
>>>> most complex part is figuring out how to integrate testing into the CMake
>>>> project, and that's only complicated because CMake is complicated.
>>>> Regarding tools, there are already testing tools available for Vala,
>>>> including GLib (so we don't have to maintain anything). Anyway, testing
>>>> tools don't take a terribly long time to learn.
>>>>
>>>>  Can we focus on the half done things before adding new projects?
>>>>> Granite is not ready, documentation is missing, not to talk about the bugs
>>>>> that survived Luna release ...
>>>>
>>>>
>>>> TDD is more valuable the sooner you start implementing it. Even if you
>>>> didn't write tests for old code and only started TDD with new code (and
>>>> existing defects), you would be doing yourself a huge favor. I'm not
>>>> suggesting that everyone stop what they're doing and go back and test every
>>>> line of code (although it would be a good thing to chip away at over time),
>>>> but practicing TDD on _new_ code can't hurt that much, can it?
>>>>
>>>> With that in mind, are there any arguments against TDD that outweigh
>>>> its merits?
>>>>
>>>> Thanks again for your questions! :)
>>>> Craig
>>>>
>>>>
>>>> On Mon, Aug 19, 2013 at 12:32 PM, Albert Palacios <optimisme@xxxxxxxxx>wrote:
>>>>
>>>>> Hi Craig and Gufran,
>>>>>
>>>>> I don't agree with TDD, and making a committee. Can you prove that
>>>>> there will be huge benefits in time/resources? Can you prove that there
>>>>> will be less bugs? (looks like that if tests are not right, bugs will
>>>>> populate equally). Can you prove that creating, modifying and fixing code
>>>>> is going to be easier?
>>>>>
>>>>> From what's been said, looks like there will be an extra effort on
>>>>> development, adding complexity and more tools to know (not to say
>>>>> maintain).  Can we focus on the half done things before adding new
>>>>> projects? Granite is not ready, documentation is missing, not to talk about
>>>>> the bugs that survived Luna release ...
>>>>>
>>>>>
>>>>> On Mon, Aug 19, 2013 at 7:27 PM, Gufran <dogabhai@xxxxxxxxx> wrote:
>>>>>
>>>>>> Count me in.
>>>>>> I cant really put forward any point on how should we proceed but I'd
>>>>>> definitely love to be with the team.
>>>>>> Yes, a *testing committee* is a good idea, maybe something
>>>>>> independent of dev community. We can write scripts to automate tests, and
>>>>>> we can do that in any language (Python for example), so it would be like a
>>>>>> Python coupling between software and tests.
>>>>>> Just my two cents :)
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, Aug 19, 2013 at 10:43 PM, Craig <weberc2@xxxxxxxxx> wrote:
>>>>>>
>>>>>>> This is cool and important, but I don't think it should stop the
>>>>>>> discussion on test driven development. Perhaps this could be a separate
>>>>>>> thread? It doesn't sound as though anyone is opposed to TDD, so can we
>>>>>>> confirm that? And if no one is opposed, how can we proceed? Can we start
>>>>>>> some kind of a "testing committee" to help determine what testing steps are
>>>>>>> needed, what framework to use, and how to integrate testing into the
>>>>>>> existing project structure (i.e., using CMake)?
>>>>>>>
>>>>>>> Does this sound like a good plan? Thoughts?
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Aug 19, 2013 at 12:05 PM, David Gomes <
>>>>>>> david@xxxxxxxxxxxxxxxx> wrote:
>>>>>>>
>>>>>>>> I'll work on it, so far we only have this I made:
>>>>>>>> https://dl.dropboxusercontent.com/u/19899464/reviewstutorial.html
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Aug 19, 2013 at 6:01 PM, Albert Palacios <
>>>>>>>> optimisme@xxxxxxxxx> wrote:
>>>>>>>>
>>>>>>>>> Hi Munchor,
>>>>>>>>>
>>>>>>>>> A contribution / bug fixing step by step guide is needed at the
>>>>>>>>> developers site. There was a .pdf before the new site change, but now it is
>>>>>>>>> impossible to find.
>>>>>>>>>
>>>>>>>>>  The problem with the old guide is that it encouraged to create
>>>>>>>>> your own branch instead of using the ~elementary-dev-community one (this is
>>>>>>>>> totally new for me). Obviously, bazaar guides doesn't teach you on using
>>>>>>>>> the "elementary-dev-community".
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Aug 19, 2013 at 6:57 PM, David Gomes <
>>>>>>>>> david@xxxxxxxxxxxxxxxx> wrote:
>>>>>>>>>
>>>>>>>>>> I always tell people if they make their branches owned by
>>>>>>>>>> ~elementary-dev-community I will volunteer to fix the code style myself. I
>>>>>>>>>> have all the free time and the will to do it, just people always make their
>>>>>>>>>> branches owned by themselves.
>>>>>>>>>>
>>>>>>>>>> ~David "Munchor" Gomes
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mon, Aug 19, 2013 at 4:29 PM, Albert Palacios Jimenez <
>>>>>>>>>> optimisme@xxxxxxxxx> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> Before talking about testing, and advanced development
>>>>>>>>>>> techniques for teams with resources, there is one easy and simple thing we
>>>>>>>>>>> can do to accelerate development.
>>>>>>>>>>>
>>>>>>>>>>> Sometimes (very often), bugs are stopped due spaces not
>>>>>>>>>>> following the "code style" guidelines. Adding a "code style" validator
>>>>>>>>>>> script before compiling, we can prevent uploads with spaces at the end of
>>>>>>>>>>> the lines ...  and save a lot of time.
>>>>>>>>>>>
>>>>>>>>>>> For example, just executing the next line before compiling:
>>>>>>>>>>>
>>>>>>>>>>> find . -type f -print0 | xargs -0 sed -i 's/[ \t]*$//'
>>>>>>>>>>>
>>>>>>>>>>> We will remove every "white space" at the end of any line,
>>>>>>>>>>> including new lines with tab spaces.
>>>>>>>>>>>
>>>>>>>>>>> This can sound stupid, but it is absurd to block bug fixes
>>>>>>>>>>> during several days due white spaces at the end of lines.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Mailing list: https://launchpad.net/~elementary-dev-community
>>>>>>>>>>> Post to     : elementary-dev-community@xxxxxxxxxxxxxxxxxxx
>>>>>>>>>>> Unsubscribe : https://launchpad.net/~elementary-dev-community
>>>>>>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Mailing list: https://launchpad.net/~elementary-dev-community
>>>>>>>> Post to     : elementary-dev-community@xxxxxxxxxxxxxxxxxxx
>>>>>>>> Unsubscribe : https://launchpad.net/~elementary-dev-community
>>>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Mailing list: https://launchpad.net/~elementary-dev-community
>>>>>>> Post to     : elementary-dev-community@xxxxxxxxxxxxxxxxxxx
>>>>>>> Unsubscribe : https://launchpad.net/~elementary-dev-community
>>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> Mailing list: https://launchpad.net/~elementary-dev-community
>>>>>> Post to     : elementary-dev-community@xxxxxxxxxxxxxxxxxxx
>>>>>> Unsubscribe : https://launchpad.net/~elementary-dev-community
>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>> --
>>> Mailing list: https://launchpad.net/~elementary-dev-community
>>> Post to     : elementary-dev-community@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~elementary-dev-community
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>>
>>
>>
>> --
>> Alex Lourie
>>
>
>
> --
> Mailing list: https://launchpad.net/~elementary-dev-community
> Post to     : elementary-dev-community@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~elementary-dev-community
> More help   : https://help.launchpad.net/ListHelp
>
>