elementary-dev-community team mailing list archive

["Daniel Foré" <daniel@xxxxxxxxxxxxxxxx>]

This all sounds great and I think everybody is pro-testing, however I've yet to see a reproduce-able example or a guide regarding any kind of tests being implemented (especially by those extremely vocal about their importance). Not books or articles about why testing is good, but something that actually shows a person how to write tests for their apps.

    
      


    So, as Linus would say, "Talk is cheap. Show me the code."

    —
Sent from Mailbox for iPhone

On Sun, Aug 18, 2013 at 3:47 PM, Craig <weberc2@xxxxxxxxx> wrote:

> Hi Alex,
> tl;dr: Unit tests are pretty much necessary to have an architecture on
> which you can run automatic system-level tests, and if you aren't
> automating then testing becomes too impractical.
> When you describe "system tests" you are actually describing what are
> called "acceptance tests" or "behavioral" tests (
> http://www.extremeprogramming.org/rules/functionaltests.html). Unit tests
> test small units of code such as classes or functions. Traditional TDD
> relies primarily upon unit tests, and those are primarily what I'm
> referring to.
> One of the primary purposes of unit testing is to ensure good code
> architecture. If you don't unit test, you probably won't have good access
> points for your acceptance tests (how do you verify that that Gtk.Label has
> the correct text when you can only access the top level window?), so
> automation will be out of the question. And if you aren't automating then
> you can't continuously integrate (running all tests every time a change is
> made to the repository in order to find bugs as soon as they are made).
> Honestly, if you aren't automating then testing becomes too impractical.
> On Aug 18, 2013 5:10 PM, "Alex Lourie" <djay.il@xxxxxxxxx> wrote:
>> Hi Craig
>>
>> For the clarification purposes, I'd like to separate 'automatic tests
>> (system testst)' and 'unittests'. I consider them different things.
>> Unittests are pieces of code that test some other pieces of the code.
>> System tests are scripts/code/steps that test that your program (or part of
>> it) works. Unittests are usually run automatically (by, say, unittesting
>> framework). System tests could be run automatically or manually. There are,
>> sometimes, frameworks for that, but in most cases it's either manual or
>> custom developed.
>>
>> Unittests are (usually) developed by the same developer who developed the
>> original code, just as in your TDD example. System tests are best developed
>> by external party (such as users).
>>
>> From here on, I can agree with you on point 1, and the naming.
>>
>> Basically, we all agree that having *testing *is a good practice and a
>> feasible way to manage the complexity of software. But unittesting cannot
>> test the logical connections between the blocks of code in the program.
>> That's the job for system testing.
>>
>> I don't care how we call it. The more *systematic *testing we do for
>> Elementary the better it's going to be, and the more chances we have to
>> sustain growth.
>>
>> So I would just like to see testing implemented. Any kind of it.
>>
>>
>> On Sun, Aug 18, 2013 at 10:56 PM, Craig <weberc2@xxxxxxxxx> wrote:
>>
>>> Hi Alex,
>>>
>>> To correct you on a couple of things:
>>>
>>> 1. TDD **does not** require you to have all or even several of the tests
>>> written before hand. It simply requires you to have the test written for
>>> the next change you are about to make. The idea is to write a test, run the
>>> test to watch it fail (this helps verify you wrote your test correctly),
>>> add the simplest code to make the test pass, run the test to watch it pass
>>> (and verify your code additions worked). Then you rinse and repeat.
>>>
>>> 2. TDD is actually a simplified form of what developers do already. That
>>> is, you usually write some code, run your code, then visually verify that
>>> it worked. TDD just crystalizes this process in code which can be executed
>>> later. TDD isn't hard, so it's well within the capacity of all of our devs;
>>> however, it does taking some getting used to. TDD is the best and fastest
>>> way to develop quality code and it's the ONLY practical way to raise the
>>> ceiling on the amount of code complexity a team of a given size can handle
>>> (there is a lot of research and professional heuristics about this). If a
>>> developer doesn't have the will to do this, they aren't taking their
>>> discipline seriously and, frankly, are a danger to any project that values
>>> quality. Besides, I've never met a developer who has hit the complexity
>>> ceiling *and* who has practiced TDD who doesn't advocate this kind of
>>> testing.
>>>
>>> With those corrections in mind, I can't see any difference between your
>>> first point and "real TDD". I agree that your 2nd point is a good idea.
>>> Automated testing can't capture everything, and it's definitely important
>>> to have some hands-on testing that we could run through a few times every
>>> release. But we should always be working toward automated testing so
>>> developers can, you know, develop.
>>>
>>>
>>> On Sun, Aug 18, 2013 at 2:05 PM, Alex Lourie <djay.il@xxxxxxxxx> wrote:
>>>
>>>> Ok, I have not added a single line of code to elementary yet, and gave
>>>> some decent amount of headache to real devs around here. I have though
>>>> participated in development of many products, small and large, and so here
>>>> it goes.
>>>>
>>>> TDD in its pure form requires having tests written and ready before the
>>>> code is written. Then failing tests are being implemented one by one,
>>>> having the code both tested and working in the end of this process. I
>>>> highly doubt that any of Elementary devs have the capacity or the will to
>>>> follow this. Especially, when many of our projects are built using 'try and
>>>> error' methodology, that is they follow the 'Hey! Let's try this!' rule.
>>>> Which is awesome, as it allows fast development and really quick release
>>>> cycles.
>>>>
>>>> Now, what I think would fit Elementary much better is simply testing
>>>> (yes, that QA thing everyone loves and adores) - the T from TDD, which is
>>>> the most important part. The best way to do that is, of course, to create a
>>>> bunch of automatic tests, but that's not really feasible either. So as I
>>>> see it, we can do 2 things:
>>>>
>>>> 1. Start looking into unittesting as much and as early as possible, and
>>>> have devs starting to create tests. This has the benefit of devs learning
>>>> to create unittests and having some of the code tested, and also,
>>>> potentially, at some point, it could help moving to real TDD.
>>>> 2. Start creating manual test procedures for basic staff. Yes, it
>>>> requires a lot of human power and time, but also do translations. This is
>>>> the most boring part of software engineering, but it is the one that can
>>>> bring balance to the force. Also, some of these can sometimes be automated
>>>> too, so it's not all that aweful.
>>>>
>>>> I personally think we need both 1 and 2. I am a strong believer in
>>>> testing as means to improve the product. But TDD is probably an overshoot
>>>> in our case. We need to start with *something*, and right now we don't
>>>> really have it.
>>>>
>>>>
>>>> On Sun, Aug 18, 2013 at 8:41 PM, Craig <weberc2@xxxxxxxxx> wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> I posted the following message on Google Plus yesterday, but it
>>>>> occurred to me that the weekend may not be prime time for checking that
>>>>> social network. I think this message is pretty important, so I want to post
>>>>> it again here: (I apologize in advance for its length)
>>>>>
>>>>> Congratulations to all the developers who made Luna such a success!
>>>>>>  You've done a great job and delivered an incredible Linux experience!
>>>>>>
>>>>>> I know I bring this up periodically, but I'm concerned that Luna + 1
>>>>>> and future releases will take more and more time to release, and/or that
>>>>>> you will quickly reach a ceiling with respect to the amount of code we'll
>>>>>> be able to maintain before quality degrades.
>>>>>>
>>>>>> The cause for my concern is the nature of complexity: as software
>>>>>> grows (that is, as code is added), bugs grow exponentially (complexity
>>>>>> increases exponentially with logic, and bugs grow linearly with
>>>>>> complexity). If we don't start working toward solutions that will scale
>>>>>> with this problem, we **will** hit a ceiling with respect to the amount of
>>>>>> complexity we will be able to support (this means fewer features or
>>>>>> less-powerful features). I promise.
>>>>>>
>>>>>> I know some in the community are working toward this goal, but I think
>>>>>> it's going to take a concerted effort on the part of the developers to take
>>>>>> this problem seriously. I urge you all to take this problem as seriously as
>>>>>> you take the rest of the user experience (because bugs are, at the end of
>>>>>> the day, a sharp degradation of the user experience).
>>>>>>
>>>>>> In my experience, the silver bullet for combating this problem is test
>>>>>> driven development. If you look around the software development industry,
>>>>>> code is improving, and it's largely because TDD is catching on. And Google
>>>>>> is a good role model in this regard (not just for us, but for
>>>>>> everyone--they are pioneers of code quality). If you're a developer and
>>>>>> you're unfamiliar with TDD, take some time and research it. It will pay
>>>>>> dividends immediately. If you have any questions about development, I'm
>>>>>> happy to provide my advice as a professional developer. Also, read up on
>>>>>> Google's testing strategies (I recommend
>>>>>> http://www.amazon.com/Google-Tests-Software-James-Whittaker/dp/0321803027_How Google Tests Software_).
>>>>>>
>>>>>> You guys are a _great_ UX shop, now let's become a great code shop. I
>>>>>> hope this analogy doesn't offend anyone who is passionate about their tech
>>>>>> brands, but my advice is this:
>>>>>>
>>>>>> Design like Apple, develop like Google.
>>>>>>
>>>>>> I really push you developers to continue to strive to hone your craft
>>>>>> the way Daniel and Cassidy (and any other UX designers) are learning to
>>>>>> hone theirs.
>>>>>>
>>>>>> P.S., Sorry for the book, and I hope you all take this as respectful,
>>>>>> constructive criticism. _Please_ ask me anything about development,
>>>>>> especially with respect to how we can keep quality high using processes
>>>>>> rather than sheer developer effort (so as to free you developers to work on
>>>>>> interesting problems rather than bug hunting).
>>>>>>
>>>>>> Thanks for reading,
>>>>>> Craig
>>>>>>
>>>>>
>>>>> --
>>>>> Mailing list: https://launchpad.net/~elementary-dev-community
>>>>> Post to     : elementary-dev-community@xxxxxxxxxxxxxxxxxxx
>>>>> Unsubscribe : https://launchpad.net/~elementary-dev-community
>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Alex Lourie
>>>>
>>>
>>>
>>
>>
>> --
>> Alex Lourie
>>