kicad-developers team mailing list archive

[Miguel Angel Ajo <miguelangel@xxxxxxx>]

KiCad project is full of awesome developers, you go elsewhere proposing unit testing, and you
have a 90% chance of hitting people's change rejection…. :)


Miguel Angel Ajo
http://www.nbee.es (http://www.nbee.es/)
+34911407752
skype: ajoajoajo


On Monday, 29 de April de 2013 at 19:10, Wayne Stambaugh wrote:

> On 4/29/2013 12:38 PM, Adam Wolf wrote:
> > If we evaluate cppunit, and run into issues when we try to make it
> > cross platform, I suggest looking at Google Test Framework, which is
> > like cppunit, but works better with regards to weird compilers.
> >  
> > Adam Wolf
> > Wayne and Layne, LLC
> >  
>  
>  
> If it helps, here is a nice test framework feature comparison chart
> (assuming it's correct and reasonably current) for just about every
> language.
>  
> http://en.wikipedia.org/wiki/List_of_unit_testing_frameworks
>  
> We could always create branch to try different ideas to see what works
> and what doesn't before making any decisions. Honestly, I don't have
> much experience in this area so I'm going to have to depend on someone
> else but I'm willing to do my part as best I can.
>  
> Wayne
>  
> >  
> > On Mon, Apr 29, 2013 at 11:33 AM, Dick Hollenbeck <dick@xxxxxxxxxxx (mailto:dick@xxxxxxxxxxx)> wrote:
> > > Miguel,
> > >  
> > > Sorry, I also was evaluating. So I apologize for being hypocritical on that.
> > >  
> > > But you'll notice that I was very careful to remain neutral still at this point, and I was
> > > analyzing mostly my own ideas.
> > >  
> > >  
> > > Still, my bad. You see, brainstorming does take some practice :)
> > >  
> > > I think we'll probably end up with a good solution anyways.
> > >  
> > >  
> > > Dick
> > >  
> > >  
> > >  
> > > for sounding On 04/29/2013 11:27 AM, Dick Hollenbeck wrote:
> > > > Technically, during the brainstorming process, it is best to gather ideas, and postpone
> > > > passing judgement on those you have already heard, until the evaluation process, which
> > > > comes later. There is a theoretical reason why this is not to be done, and it supposedly
> > > > thwarts creativity.
> > > >  
> > > > Creativity is the genesis of good ideas.
> > > >  
> > > > After the list of ideas is made, evaluation eventually begins. When it is agreed that
> > > > there are enough ideas to warrant the transition.
> > > >  
> > > > So your idea is CPPUnit.
> > > >  
> > > > Brainstorming is a fantastic tool, but admittedly it takes some understanding and
> > > > agreement to follow it.
> > > >  
> > > > The notion of thwarting creativity is specifically one of the things brainstorming tries
> > > > to avoid, since sometimes the best ideas end up being out of the norm. Improvements come
> > > > by challenging old ways. If they fall into the waste bucket during the evaluation
> > > > process, there is no harm at that point.
> > > >  
> > > >  
> > > > Dick
> > > >  
> > > >  
> > > >  
> > > >  
> > > > On 04/29/2013 10:40 AM, Miguel Angel Ajo wrote:
> > > > > I have the feeling that setting entry points from Python to C, when not needed will be an
> > > > > extra work and harder to maintain. In that
> > > > > case why not just using C++ standard testing frameworks?, like CPPUnit, or something that
> > > > > plays well with wx for UI testing?
> > > > >  
> > > > > http://wiki.wxwidgets.org/Tools <-- Testing
> > > > >  
> > > > >  
> > > > > We have two things to test:
> > > > >  
> > > > > 1) UI/behaviour , this is usually done launching dialogs manually, and calling click
> > > > > methods, reading form entries, checking output, etc…,
> > > > > Ideally it must be something like (warning, pseudocode)
> > > > >  
> > > > > dialog = MyDialog()
> > > > >  
> > > > > n = dialog->getListElements( 'list1' )
> > > > > dialog->fillField( 'Field1','Content1' )
> > > > > dialog->clickButton( 'Add' )
> > > > > assert dialog->getListElements('list1')==(n+1)
> > > > >  
> > > > > 2) Model testing: We call the back models with loads of tests to check their correct behavior
> > > > >  
> > > > > a) Normal ones, just model behaviour, correct DRC checks,
> > > > >  
> > > > > b) Models that generate outputs: In webkit, they just have pngs of the outputs
> > > > > renderings, and they xor/compare. In our case, we can make a diff of
> > > > > the output contents with expected ones, or try to render them to PNG & test theme.
> > > > > (warning: needs extra maintenance when outputs are enhanced or change, but it will catch
> > > > > unexpected changes)
> > > > >  
> > > > > c) Model testing from python, everything that's swigged out can be tested from there,
> > > > > and then we get a contract on what the available methods
> > > > > are to stay as fixed as possible. Even we get loads of examples out of this.
> > > > >  
> > > > >  
> > > > > The automated tests I've seen so far (in the last two years) generate an xml standardized
> > > > > files (junit xml format) that's used for reports (I use them in bamboo at nbee, and also
> > > > > with other client for web ui testing, I've even used it to generate VHDL testing outputs
> > > > > from ghdl) http://projects.nbee.es:8085/browse/S12-DEF the bad thing of bamboo is that it
> > > > > doesn't play well with bzr (AFAIK)
> > > > >  
> > > > >  
> > > > >  
> > > > >  
> > > > >  
> > > > >  
> > > > >  
> > > > > Miguel Angel Ajo
> > > > > http://www.nbee.es <http://www.nbee.es/>
> > > > > +34911407752
> > > > > skype: ajoajoajo
> > > > >  
> > > > > On Monday, 29 de April de 2013 at 17:06, Dick Hollenbeck wrote:
> > > > >  
> > > > > > >  
> > > > > > > We could address a) and b) by having a single DLL/DSO specific entrypoint for all tests,
> > > > > > > with a big switch in there to route to the actual test. This scaffolding has some
> > > > > > > maintenance cost, and begins to wash out the ease of using python on top, but perhaps not
> > > > > > > fully.
> > > > > > >  
> > > > > >  
> > > > > >  
> > > > > >  
> > > > > > In the python C implementation code you find instances of the paradigm of va_args (tuple)
> > > > > > with a C string telling which argument types are in the tuple. The C string is like a
> > > > > > printf format string.
> > > > > >  
> > > > > > So it would be possible to make this callable from python and pass all required arguments
> > > > > > for any worker function. You would have to instantiate the C++ instance before calling
> > > > > > any member function which is your worker function. But that could be done in the switch
> > > > > > within the test entrypoint.
> > > > > >  
> > > > > >  
> > > > > > test_result TestEntryPoint( aTestNum, formatStringTellingArgumentTypesInTuple, Tuple )
> > > > > > {
> > > > > > switch( aTestNum )
> > > > > > {
> > > > > > case XYZ:
> > > > > > {
> > > > > > // instantiate what is needed, say obj
> > > > > >  
> > > > > > // gleen into C++ arguments, those from Tuple by parsing
> > > > > > // formatStringTellingArgumentTypesInTuple
> > > > > > va_args...
> > > > > >  
> > > > > > // call object member func with C++ form of arguments in tuple.
> > > > > > result = obj->WorkerFunction( arg1, arg2, arg3 );
> > > > > >  
> > > > > > // destroy what we instantiated.
> > > > > > }
> > > > > > }
> > > > > >  
> > > > > > return test_result( result ) ; // telling what just happened.
> > > > > > }
> > > > > >  
> > > > > >  
> > > > > >  
> > > > > > The above could be a seed for better ideas. It is a means of testing any "worker
> > > > > > function" from python.
> > > > > >  
> > > > > >  
> > > > > >  
> > > > > >  
> > > > > >  
> > > > > >  
> > > > > >  
> > > > > >  
> > > > > > _______________________________________________
> > > > > > Mailing list: https://launchpad.net/~kicad-developers
> > > > > > Post to : kicad-developers@xxxxxxxxxxxxxxxxxxx <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>
> > > > > > Unsubscribe : https://launchpad.net/~kicad-developers
> > > > > > More help : https://help.launchpad.net/ListHelp
> > > > > >  
> > > > >  
> > > > >  
> > > >  
> > > >  
> > >  
> > >  
> > >  
> > > _______________________________________________
> > > Mailing list: https://launchpad.net/~kicad-developers
> > > Post to : kicad-developers@xxxxxxxxxxxxxxxxxxx (mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx)
> > > Unsubscribe : https://launchpad.net/~kicad-developers
> > > More help : https://help.launchpad.net/ListHelp
> > >  
> >  
> >  
> > _______________________________________________
> > Mailing list: https://launchpad.net/~kicad-developers
> > Post to : kicad-developers@xxxxxxxxxxxxxxxxxxx (mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx)
> > Unsubscribe : https://launchpad.net/~kicad-developers
> > More help : https://help.launchpad.net/ListHelp
> >  
>  
>  
>  
>  
> _______________________________________________
> Mailing list: https://launchpad.net/~kicad-developers
> Post to : kicad-developers@xxxxxxxxxxxxxxxxxxx (mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx)
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help : https://help.launchpad.net/ListHelp
>  
>