← Back to team overview

elementary-dev-community team mailing list archive

Re: Test Driven Development

 

To everyone interested in implementing automated testing and QA procedures
I highly recommend reading logs Martin Pitt's Ubuntu Dev Week sessions on
the topic. They can be found at
http://irclogs.ubuntu.com/2013/01/31/%23ubuntu-classroom.html, his IRC nick
is "pitti".

I've added some things he mentioned to our document (
https://docs.google.com/document/d/1cTsWpeT0h4FD81T-xs4bEWlALsO__j3rL1A7iB-SWz0/edit),
especially on behavioral testing, but I still highly recommend reading the
original session.


2013/4/29 Jaap Broekhuizen <jaapz.b@xxxxxxxxx>

> Pal, that looks very interesting, please do upload it to launchpad so we
> can have a closer look :)
>
> In te mean time, I have created a google document to have a central point
> of investigation for elementary automated testing. Feel free to add
> information to the doc whenever you can, but please keep it clean! :) You
> can find it here:
> https://docs.google.com/document/d/1cTsWpeT0h4FD81T-xs4bEWlALsO__j3rL1A7iB-SWz0/edit
>
> I haven't found any BDD frameworks yet, but I have found some interesting
> testing frameworks.
>
> I think I'll set up a testing branch for granite some day later this week,
> maybe test out the different frameworks so we can see what suits us best.
> If anyone else wants to start setting up a branch like that, you are of
> course free to do so ;)
>
> --
> Jaap
>
> On Mon, Apr 29, 2013 at 3:08 AM, Pál Dorogi <pal.dorogi@xxxxxxxxx> wrote:
>
>> Hi,
>>
>> You can use cmake for unit test as it supports GLib's test. I use
>> MonoDevelop/Xamarin Studio for developing for huge projects coexists
>> /w cmake (as MD/XS does not support cmake). MD is for rapid
>> development but there is no internal Unit to support vala but C#
>> (Nunit) and some other languages. So, I run some cmake command before
>> and after MD build which runs cmake for cmake build and run test. For
>> example:
>>
>> before build: cmake .. in /build/ dir
>> after build in MD: run build/test/unit_test
>>
>> I added CMakeLists.txt into my MD project and I just need to sync
>> betwwen MD and that file when I add or remove a Vala source file
>> into/from the MD.
>>
>> I do not know how would it works /w launchpad as I do not know how its
>> packaging works /w cmake's unit test, but I think it should work.
>> You just need add some stanza in the project's root CMakeList.txt like
>> this, but it's not simpe as it's using some other features like
>> external projects and so on.
>> set (PROJECT_TEST tests)
>>
>> ...
>> enable_testing (true)
>> add_subdirectory (${PROJECT_TEST})
>>
>> and add create some CMakeList.txt in the ./test dir
>>
>> ###############################################################################
>> # Sources
>>
>> ###############################################################################
>> set (UNIT_TESTS unit_tests)
>>
>> set (VALA_SOURCES
>> Model/Address.vala
>> Model/Person.vala
>> Model/Gender.vala
>> ValidatorTest.vala
>>     TestMain.vala
>> )
>>
>> set (PKG_DEPS gtk+-3.0 glib-2.0 gee-1.0)
>>
>>
>> ################################################################################
>> # set (CMAKE_VERBOSE_MAKEFILE ON)
>> set (CMAKE_FIND_LIBRARY_SUFFIXES .so)
>>
>> # External Packages definitions.
>> set (EXTERN_PROJ dafunit)
>> set (EXTERN_SOURCE_DIR src)
>>
>> set (INTERN_PROJ dafvalidation)
>> set (INTERN_SOURCE_DIR ${PROJECT_SOURCE})
>>
>> include (ExternalProject)
>>
>> ExternalProject_Add (${EXTERN_PROJ}
>>     #PREFIX ../../${EXTERN_PROJ}
>>     SOURCE_DIR ../../../${EXTERN_PROJ}
>>     BINARY_DIR ${CMAKE_BINARY_DIR}/${EXTERN_PROJ}/build
>>     INSTALL_DIR ""
>>     UPDATE_COMMAND ""
>>     PATCH_COMMAND ""
>>     INSTALL_COMMAND ""
>> )
>>
>> ExternalProject_Get_Property(${EXTERN_PROJ} BINARY_DIR)
>> include_directories (${BINARY_DIR}/${EXTERN_SOURCE_DIR})
>> include_directories (${CMAKE_BINARY_DIR}/${INTERN_SOURCE_DIR})
>>
>> # PkgConfig
>> find_package (PkgConfig)
>> find_package (GObjectIntrospection 0.9.12)
>> include (GObjectIntrospectionMacros)
>>
>> pkg_check_modules(DEPS REQUIRED ${PKG_DEPS})
>>
>> set (CFLAGS ${DEPS_CFLAGS} ${DEPS_CFLAGS_OTHER})
>> add_definitions (${CFLAGS})
>> set (LIBS ${DEPS_LIBRARIES})
>> set(LIB_PATHS ${DEPS_LIBRARY_DIRS})
>> link_directories(${LIB_PATHS})
>>
>> # Does not work set (ENV{PKG_CONFIG_PATH} ${EXTERNAL_BINARY_DIR}/src)
>> vala_precompile (VALA_C
>> ${VALA_SOURCES}
>> PACKAGES
>> ${PKG_DEPS}
>>     posix
>> CUSTOM_VAPIS
>>     ${CMAKE_BINARY_DIR}/${INTERN_SOURCE_DIR}/${INTERN_PROJ}.vapi
>> ${BINARY_DIR}/${EXTERN_SOURCE_DIR}/${EXTERN_PROJ}.vapi
>> OPTIONS
>> )
>>
>> add_executable (${UNIT_TESTS} ${VALA_C})
>>
>> # Does not work add_dependencies (unit_tests dafvalidation)
>>
>> target_link_libraries(${UNIT_TESTS} ${LIBS})
>> target_link_libraries(${UNIT_TESTS}
>>
>> ${BINARY_DIR}/${EXTERN_SOURCE_DIR}/${CMAKE_FIND_LIBRARY_PREFIXES}${EXTERN_PROJ}${CMAKE_FIND_LIBRARY_SUFFIXES})
>> target_link_libraries(${UNIT_TESTS}
>>
>> ${CMAKE_BINARY_DIR}/${INTERN_SOURCE_DIR}/${CMAKE_FIND_LIBRARY_PREFIXES}${INTERN_PROJ}${CMAKE_FIND_LIBRARY_SUFFIXES})
>> add_test(${UNIT_TESTS} ${CMAKE_CURRENT_BINARY_DIR}/${UNIT_TESTS})
>> ###################################################
>>
>>
>> I am going to upload it to lp so, if you would like to have a look at
>> it just let me know and that case I will uploadid it on some day in
>> this week
>>
>> On 29 April 2013 07:19, Lochlan Bunn <loklaan@xxxxxxxxx> wrote:
>> > I have read alot about TTD, both in school and in persistent articles.
>> I've
>> > used it to develop a small gui based game, and I can say that I liked
>> the
>> > flow once I was used to it. I used JUnit & Eclipse, and that was all
>> that
>> > was needed the whole time.
>> >
>> > So when it comes to elementary dev, and vala/gtk/linux dev in general,
>> I'd
>> > be interested in reading/learning how to write unit test (suites) for
>> vala
>> > in respects to both CI, a la Launchpad, packaging, and moreso in an IDE.
>> >
>> >
>> > On 27 April 2013 07:48, Craig <weberc2@xxxxxxxxx> wrote:
>> >>
>> >> I agree wholeheartedly. And as Cassidy mentioned, we can use scratch as
>> >> the incubation project.  Would any devs be interested in volunteering
>> to
>> >> learn? Jaap, would you be interested in helping instruct?
>> >>
>> >> On Apr 26, 2013 3:25 PM, "Jaap Broekhuizen" <jaapz.b@xxxxxxxxx> wrote:
>> >>>
>> >>> I also think implementing Behavorial testing (applying BDD) is very
>> >>> relevant for us, as we are focussing a lot on user interface and
>> >>> interaction.
>> >>>
>> >>> So imo we should start on a project which we can use as a playground
>> for
>> >>> both unit an behavorial testing.
>> >>>
>> >>> Does anyone know of good vala bdd frameworks?
>> >>>
>> >>> Jaap
>> >>>
>> >>> Op 26 apr. 2013 22:21 schreef "Cassidy James" <
>> cassidy@xxxxxxxxxxxxxxxx>
>> >>> het volgende:
>> >>>>
>> >>>> I don't think we need any convincing; everything I've heard from the
>> >>>> devs is that we need to do this. It's just a matter of figuring out
>> a common
>> >>>> way of doing it.
>> >>>>
>> >>>> Craig, a relatively small/new project that could use testing is the
>> new
>> >>>> Scratch or even the new work going on with Contractor. Both are
>> (from what I
>> >>>> understand) fresh codebases and now might be the time to work  on
>> tests. I
>> >>>> recommend you hop into #elementary-dev and work with the devs on
>> getting
>> >>>> some tests worked out.
>> >>>>
>> >>>> Regards,
>> >>>> Cassidy
>> >>>>
>> >>>> On Apr 26, 2013 11:04 AM, "Pál Dorogi" <pal.dorogi@xxxxxxxxx> wrote:
>> >>>>>
>> >>>>> I dunno, I am a newbie here.
>> >>>>>
>> >>>>> On 26 April 2013 22:24, Craig <weberc2@xxxxxxxxx> wrote:
>> >>>>> > That's exactly what I'd like to know: how can I help. I can try
>> and
>> >>>>> > post
>> >>>>> > some tutorials, but I'd like to know who is interested and what
>> the
>> >>>>> > development community already knows.
>> >>>>> >
>> >>>>> > On Apr 26, 2013 6:39 AM, "Pál Dorogi" <pal.dorogi@xxxxxxxxx>
>> wrote:
>> >>>>> >>
>> >>>>> >> Hi Craig,
>> >>>>> >>
>> >>>>> >> I agree 100% /w you, but I think you should write some tutorials
>> and
>> >>>>> >> post them in your blog, if you have any. But in my opinion that
>> the
>> >>>>> >> human beings do not like "re-learn" things and the real OOP,
>> Design
>> >>>>> >> Patterns, SOLID, TDD etc. etc. are very steep and time for a
>> >>>>> >> non-real
>> >>>>> >> OOP/DP experienced Programmer/Developer.
>> >>>>> >> Also, the learning curve is very steep for these advanced stuffs
>> and
>> >>>>> >> needs long time to get there. But, nobody would not know how good
>> >>>>> >> are
>> >>>>> >> they until haven't learnt and used those stuffs, would they?.:)
>> >>>>> >>
>> >>>>> >> I did sine similar things, getting some new fresh things (TDD,
>> >>>>> >> MvvM/Presentation Model Design Pattern) to programming in Vala
>> >>>>> >>
>> >>>>> >>
>> >>>>> >> ((
>> http://ilapstech.blogspot.com/2013/04/advanced-programming-in-vala-dafs.html
>> )
>> >>>>> >> but you should keep in mind that this kind of new things (TDD,
>> DP,
>> >>>>> >> SOLDI, MVVM etc. etc.) are like evolution (evolution in
>> Programming)
>> >>>>> >> which needs some time to get it succeeded (or failed).:)
>> >>>>> >>
>> >>>>> >> On 26 April 2013 20:36, Craig <weberc2@xxxxxxxxx> wrote:
>> >>>>> >> > Hello everyone,
>> >>>>> >> >
>> >>>>> >> > I'm just leaving San Jose after having spent a week listening
>> to a
>> >>>>> >> > lot
>> >>>>> >> > of
>> >>>>> >> > smart people talk about, among other things, Test Driven
>> >>>>> >> > Development
>> >>>>> >> > (TDD).
>> >>>>> >> > I know I keep harping on this, but among the people who write
>> the
>> >>>>> >> > coolest,
>> >>>>> >> > best software (and other average software folks) TDD is seen as
>> >>>>> >> > absolutely
>> >>>>> >> > critical. I can't point to anything other discipline in the
>> >>>>> >> > software
>> >>>>> >> > world
>> >>>>> >> > that is of comparable importance. And here's why:
>> >>>>> >> >
>> >>>>> >> > When we start writing software, we can manage it with a couple
>> of
>> >>>>> >> > developers, perhaps all the way up through the first release;
>> >>>>> >> > however,
>> >>>>> >> > as we
>> >>>>> >> > add features, our software becomes more complex. It's hard for
>> us
>> >>>>> >> > to
>> >>>>> >> > remember what parts of our programs worked well before and what
>> >>>>> >> > parts
>> >>>>> >> > are
>> >>>>> >> > broken. We often make changes to the underlying architecture to
>> >>>>> >> > facilitate a
>> >>>>> >> > new feature, but we're not exactly sure if in doing so, we
>> broke
>> >>>>> >> > an
>> >>>>> >> > existing
>> >>>>> >> > feature. And we'll of course do a little ad hoc manual testing
>> to
>> >>>>> >> > verify
>> >>>>> >> > that things still work, but we're only going to really check
>> 5-10%
>> >>>>> >> > of
>> >>>>> >> > the
>> >>>>> >> > code that we most suspect would break. And even if we do power
>> >>>>> >> > through,
>> >>>>> >> > we're only going to ever check 60-70% of the code, and it's
>> all a
>> >>>>> >> > very
>> >>>>> >> > slow,
>> >>>>> >> > unreliable process. Soon we spend all of our time fighting bugs
>> >>>>> >> > and we
>> >>>>> >> > can
>> >>>>> >> > never get around to any interesting work. Does this pattern
>> sound
>> >>>>> >> > familiar?
>> >>>>> >> >
>> >>>>> >> > With TDD, you write a simple, small test for every piece of
>> >>>>> >> > interesting
>> >>>>> >> > code
>> >>>>> >> > you write, and every time you rebuild the project, all of your
>> old
>> >>>>> >> > tests
>> >>>>> >> > run. If you're writing good tests, you can be assured that all
>> of
>> >>>>> >> > your
>> >>>>> >> > code
>> >>>>> >> > works as you intend it to every single time you build, and if
>> >>>>> >> > someone
>> >>>>> >> > merges
>> >>>>> >> > in a bug, it will be caught immediately (and the test that
>> fails
>> >>>>> >> > will
>> >>>>> >> > give
>> >>>>> >> > you some good information about what broke/where the bug is
>> >>>>> >> > hiding).
>> >>>>> >> >
>> >>>>> >> > Of course, it takes time to write tests; however, it's still
>> much
>> >>>>> >> > less
>> >>>>> >> > time
>> >>>>> >> > than you would spend debugging your code. Furthermore, when you
>> >>>>> >> > write
>> >>>>> >> > tests
>> >>>>> >> > before you write your production code, you are forced to design
>> >>>>> >> > your
>> >>>>> >> > code
>> >>>>> >> > modularly just to make it testable. Among software
>> professionals,
>> >>>>> >> > TDD is
>> >>>>> >> > seen as the fastest way to write software. I mean, Luna has
>> been
>> >>>>> >> > 90%
>> >>>>> >> > complete for 90% of its development cycle, and this is a common
>> >>>>> >> > pattern
>> >>>>> >> > in
>> >>>>> >> > the software world.
>> >>>>> >> >
>> >>>>> >> > With all of this in mind, I'd like to know how I can help you
>> guys
>> >>>>> >> > start
>> >>>>> >> > practicing TDD? If this hasn't persuaded you, I'd appreciate
>> it if
>> >>>>> >> > you
>> >>>>> >> > would
>> >>>>> >> > respond and give your perspective so we can talk about it. I'm
>> >>>>> >> > very
>> >>>>> >> > interested in seeing you guys continue to put out great
>> software,
>> >>>>> >> > but
>> >>>>> >> > I'm
>> >>>>> >> > concerned that as you write more code, you're going to be
>> creating
>> >>>>> >> > more
>> >>>>> >> > for
>> >>>>> >> > yourselves to maintain and the amount of time you spend writing
>> >>>>> >> > new
>> >>>>> >> > software
>> >>>>> >> > is going to drop off exponentially as the complexity (as
>> >>>>> >> > complexity
>> >>>>> >> > produces
>> >>>>> >> > bugs) increases.
>> >>>>> >> >
>> >>>>> >> > Please let me know if/how I can help you.
>> >>>>> >> >
>> >>>>> >> > Craig
>> >>>>> >> >
>> >>>>> >> > --
>> >>>>> >> > Mailing list: https://launchpad.net/~elementary-dev-community
>> >>>>> >> > Post to     : elementary-dev-community@xxxxxxxxxxxxxxxxxxx
>> >>>>> >> > Unsubscribe : https://launchpad.net/~elementary-dev-community
>> >>>>> >> > More help   : https://help.launchpad.net/ListHelp
>> >>>>> >> >
>> >>>>>
>> >>>>> --
>> >>>>> Mailing list: https://launchpad.net/~elementary-dev-community
>> >>>>> Post to     : elementary-dev-community@xxxxxxxxxxxxxxxxxxx
>> >>>>> Unsubscribe : https://launchpad.net/~elementary-dev-community
>> >>>>> More help   : https://help.launchpad.net/ListHelp
>> >>>>
>> >>>>
>> >>>> --
>> >>>> Mailing list: https://launchpad.net/~elementary-dev-community
>> >>>> Post to     : elementary-dev-community@xxxxxxxxxxxxxxxxxxx
>> >>>> Unsubscribe : https://launchpad.net/~elementary-dev-community
>> >>>> More help   : https://help.launchpad.net/ListHelp
>> >>>>
>> >>>
>> >>> --
>> >>> Mailing list: https://launchpad.net/~elementary-dev-community
>> >>> Post to     : elementary-dev-community@xxxxxxxxxxxxxxxxxxx
>> >>> Unsubscribe : https://launchpad.net/~elementary-dev-community
>> >>> More help   : https://help.launchpad.net/ListHelp
>> >>>
>> >>
>> >> --
>> >> Mailing list: https://launchpad.net/~elementary-dev-community
>> >> Post to     : elementary-dev-community@xxxxxxxxxxxxxxxxxxx
>> >> Unsubscribe : https://launchpad.net/~elementary-dev-community
>> >> More help   : https://help.launchpad.net/ListHelp
>> >>
>> >
>> >
>> > --
>> > Mailing list: https://launchpad.net/~elementary-dev-community
>> > Post to     : elementary-dev-community@xxxxxxxxxxxxxxxxxxx
>> > Unsubscribe : https://launchpad.net/~elementary-dev-community
>> > More help   : https://help.launchpad.net/ListHelp
>> >
>>
>> --
>> Mailing list: https://launchpad.net/~elementary-dev-community
>> Post to     : elementary-dev-community@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~elementary-dev-community
>> More help   : https://help.launchpad.net/ListHelp
>>
>
>
> --
> Mailing list: https://launchpad.net/~elementary-dev-community
> Post to     : elementary-dev-community@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~elementary-dev-community
> More help   : https://help.launchpad.net/ListHelp
>
>


-- 
Sergey "Shnatsel" Davidoff
OS architect @ elementary

Follow ups

References