elementary-dev-community team mailing list archive
Mailing list archive
Re: Congratulations Luna developers!
tl;dr: Unit tests are pretty much necessary to have an architecture on
which you can run automatic system-level tests, and if you aren't
automating then testing becomes too impractical.
When you describe "system tests" you are actually describing what are
called "acceptance tests" or "behavioral" tests (
http://www.extremeprogramming.org/rules/functionaltests.html). Unit tests
test small units of code such as classes or functions. Traditional TDD
relies primarily upon unit tests, and those are primarily what I'm
One of the primary purposes of unit testing is to ensure good code
architecture. If you don't unit test, you probably won't have good access
points for your acceptance tests (how do you verify that that Gtk.Label has
the correct text when you can only access the top level window?), so
automation will be out of the question. And if you aren't automating then
you can't continuously integrate (running all tests every time a change is
made to the repository in order to find bugs as soon as they are made).
Honestly, if you aren't automating then testing becomes too impractical.
On Aug 18, 2013 5:10 PM, "Alex Lourie" <djay.il@xxxxxxxxx> wrote:
> Hi Craig
> For the clarification purposes, I'd like to separate 'automatic tests
> (system testst)' and 'unittests'. I consider them different things.
> Unittests are pieces of code that test some other pieces of the code.
> System tests are scripts/code/steps that test that your program (or part of
> it) works. Unittests are usually run automatically (by, say, unittesting
> framework). System tests could be run automatically or manually. There are,
> sometimes, frameworks for that, but in most cases it's either manual or
> custom developed.
> Unittests are (usually) developed by the same developer who developed the
> original code, just as in your TDD example. System tests are best developed
> by external party (such as users).
> From here on, I can agree with you on point 1, and the naming.
> Basically, we all agree that having *testing *is a good practice and a
> feasible way to manage the complexity of software. But unittesting cannot
> test the logical connections between the blocks of code in the program.
> That's the job for system testing.
> I don't care how we call it. The more *systematic *testing we do for
> Elementary the better it's going to be, and the more chances we have to
> sustain growth.
> So I would just like to see testing implemented. Any kind of it.
> On Sun, Aug 18, 2013 at 10:56 PM, Craig <weberc2@xxxxxxxxx> wrote:
>> Hi Alex,
>> To correct you on a couple of things:
>> 1. TDD **does not** require you to have all or even several of the tests
>> written before hand. It simply requires you to have the test written for
>> the next change you are about to make. The idea is to write a test, run the
>> test to watch it fail (this helps verify you wrote your test correctly),
>> add the simplest code to make the test pass, run the test to watch it pass
>> (and verify your code additions worked). Then you rinse and repeat.
>> 2. TDD is actually a simplified form of what developers do already. That
>> is, you usually write some code, run your code, then visually verify that
>> it worked. TDD just crystalizes this process in code which can be executed
>> later. TDD isn't hard, so it's well within the capacity of all of our devs;
>> however, it does taking some getting used to. TDD is the best and fastest
>> way to develop quality code and it's the ONLY practical way to raise the
>> ceiling on the amount of code complexity a team of a given size can handle
>> (there is a lot of research and professional heuristics about this). If a
>> developer doesn't have the will to do this, they aren't taking their
>> discipline seriously and, frankly, are a danger to any project that values
>> quality. Besides, I've never met a developer who has hit the complexity
>> ceiling *and* who has practiced TDD who doesn't advocate this kind of
>> With those corrections in mind, I can't see any difference between your
>> first point and "real TDD". I agree that your 2nd point is a good idea.
>> Automated testing can't capture everything, and it's definitely important
>> to have some hands-on testing that we could run through a few times every
>> release. But we should always be working toward automated testing so
>> developers can, you know, develop.
>> On Sun, Aug 18, 2013 at 2:05 PM, Alex Lourie <djay.il@xxxxxxxxx> wrote:
>>> Ok, I have not added a single line of code to elementary yet, and gave
>>> some decent amount of headache to real devs around here. I have though
>>> participated in development of many products, small and large, and so here
>>> it goes.
>>> TDD in its pure form requires having tests written and ready before the
>>> code is written. Then failing tests are being implemented one by one,
>>> having the code both tested and working in the end of this process. I
>>> highly doubt that any of Elementary devs have the capacity or the will to
>>> follow this. Especially, when many of our projects are built using 'try and
>>> error' methodology, that is they follow the 'Hey! Let's try this!' rule.
>>> Which is awesome, as it allows fast development and really quick release
>>> Now, what I think would fit Elementary much better is simply testing
>>> (yes, that QA thing everyone loves and adores) - the T from TDD, which is
>>> the most important part. The best way to do that is, of course, to create a
>>> bunch of automatic tests, but that's not really feasible either. So as I
>>> see it, we can do 2 things:
>>> 1. Start looking into unittesting as much and as early as possible, and
>>> have devs starting to create tests. This has the benefit of devs learning
>>> to create unittests and having some of the code tested, and also,
>>> potentially, at some point, it could help moving to real TDD.
>>> 2. Start creating manual test procedures for basic staff. Yes, it
>>> requires a lot of human power and time, but also do translations. This is
>>> the most boring part of software engineering, but it is the one that can
>>> bring balance to the force. Also, some of these can sometimes be automated
>>> too, so it's not all that aweful.
>>> I personally think we need both 1 and 2. I am a strong believer in
>>> testing as means to improve the product. But TDD is probably an overshoot
>>> in our case. We need to start with *something*, and right now we don't
>>> really have it.
>>> On Sun, Aug 18, 2013 at 8:41 PM, Craig <weberc2@xxxxxxxxx> wrote:
>>>> I posted the following message on Google Plus yesterday, but it
>>>> occurred to me that the weekend may not be prime time for checking that
>>>> social network. I think this message is pretty important, so I want to post
>>>> it again here: (I apologize in advance for its length)
>>>> Congratulations to all the developers who made Luna such a success!
>>>>> You've done a great job and delivered an incredible Linux experience!
>>>>> I know I bring this up periodically, but I'm concerned that Luna + 1
>>>>> and future releases will take more and more time to release, and/or that
>>>>> you will quickly reach a ceiling with respect to the amount of code we'll
>>>>> be able to maintain before quality degrades.
>>>>> The cause for my concern is the nature of complexity: as software
>>>>> grows (that is, as code is added), bugs grow exponentially (complexity
>>>>> increases exponentially with logic, and bugs grow linearly with
>>>>> complexity). If we don't start working toward solutions that will scale
>>>>> with this problem, we **will** hit a ceiling with respect to the amount of
>>>>> complexity we will be able to support (this means fewer features or
>>>>> less-powerful features). I promise.
>>>>> I know some in the community are working toward this goal, but I think
>>>>> it's going to take a concerted effort on the part of the developers to take
>>>>> this problem seriously. I urge you all to take this problem as seriously as
>>>>> you take the rest of the user experience (because bugs are, at the end of
>>>>> the day, a sharp degradation of the user experience).
>>>>> In my experience, the silver bullet for combating this problem is test
>>>>> driven development. If you look around the software development industry,
>>>>> code is improving, and it's largely because TDD is catching on. And Google
>>>>> is a good role model in this regard (not just for us, but for
>>>>> everyone--they are pioneers of code quality). If you're a developer and
>>>>> you're unfamiliar with TDD, take some time and research it. It will pay
>>>>> dividends immediately. If you have any questions about development, I'm
>>>>> happy to provide my advice as a professional developer. Also, read up on
>>>>> Google's testing strategies (I recommend
>>>>> http://www.amazon.com/Google-Tests-Software-James-Whittaker/dp/0321803027_How Google Tests Software_).
>>>>> You guys are a _great_ UX shop, now let's become a great code shop. I
>>>>> hope this analogy doesn't offend anyone who is passionate about their tech
>>>>> brands, but my advice is this:
>>>>> Design like Apple, develop like Google.
>>>>> I really push you developers to continue to strive to hone your craft
>>>>> the way Daniel and Cassidy (and any other UX designers) are learning to
>>>>> hone theirs.
>>>>> P.S., Sorry for the book, and I hope you all take this as respectful,
>>>>> constructive criticism. _Please_ ask me anything about development,
>>>>> especially with respect to how we can keep quality high using processes
>>>>> rather than sheer developer effort (so as to free you developers to work on
>>>>> interesting problems rather than bug hunting).
>>>>> Thanks for reading,
>>>> Mailing list: https://launchpad.net/~elementary-dev-community
>>>> Post to : elementary-dev-community@xxxxxxxxxxxxxxxxxxx
>>>> Unsubscribe : https://launchpad.net/~elementary-dev-community
>>>> More help : https://help.launchpad.net/ListHelp
>>> Alex Lourie
> Alex Lourie