yade-dev team mailing list archive

[Anton Gladky <gladky.anton@xxxxxxxxx>]

Hi all,

please do not forget, that Yade has already a rich opportunity
to create semi-unit-tests (yade --test) and nice semi-integration-tests
(yade --check). Sure, one can create unit tests for pure C++-functions
using boost::unit_test, cppunit or googletest etc., but I would propose
first to extend existing tests, not to spread an energy for the connecting
the new framework into the Yade (+new dependencies, +new cmake-change,
+integration overhead).

And if the existing "yade --test" or "yade --check" are really not enough,
one need to analyze the problem deeper and then make a decision for the
connecting of newer frameworks.

Regards

Anton

Am Mo., 7. Jan. 2019 um 18:26 Uhr schrieb Janek Kozicki <janek_listy@xxxxx>:
>
> Bruno Chareyre said:     (by the date of Sun, 6 Jan 2019 17:08:12 +0100)
>
> > Thanks for raising this major issue. It would be great to populate unit
> > tests indeed.
> >
> > I recently
> > https://github.com/yade/trunk/commit/b5fbefc6463294f580296cb5727dbbfd733fa8a0
> > introduced a regression test for the utils module and I would like
> > to advertise it here. It is testing only one function from utils
> > currently.
>
> Very interesting, thank you!
>
> I have in mind writing testing code in C++, so that's a little different.
>
> > It needs volunteers to expand it (which can be done simply by reproducing
> > the logic of testUserCreatedInteraction() in more functions). If nobody is
> > going to add tests systematically - the ideal case - I would suggest at
> > least that:
> > *when a bug is fixed a unit test is added simultaneously*.
> > Fixing a bug usually gives a fresh vision of the behavior, which makes
> > writing a unit test easier.
> > Ultimately we could even collectively agree that a bug is not fixed if
> > there is no test proving it.
>
> I would go a bit further: assume that current yade version is the
> reference version. Then I would add test to every C++ function and class
> method. Store the results of tests as a reference result. When a bug
> is found, then the reference result would change. Or worse: it would
> turn out that although function is tested, the test did not catch the
> bug. And that would be great incentive to add a test case where this
> bug happened.
>
> And also this would be a lot easier for others: the code testing
> this function is already written, only another set of input
> parameters must be added to test this case where bug appeared.
>
> Yes, I know that it is a crazy amount of work.
>
> > About 1: would be great provided that it doesn't end-up in simply removing
> > examples which do not work.
>
> Definitely not. The main goal is to really fix all examples. This is
> a perfect opportunity for me to see the latest additions to yade! :)
>
> > Classifying examples is also an important point and I would discourage the
> > previous approach of moving failing scripts to a special
> > "examples/not-working" folder since it breaks the classification in
> > subfolders. Better rename them (something like *.py.fail) while keeping
> > them in their original location.
> > It is less clear if/how you intend to implement the "all examples must
> > work" policy. It is difficult to automatize testing of examples since they
> > are very heterogeneous. For instance some examples don't have a O.run() as
> > user is supposed to click "play" instead.
> > If the error happens after playing the error will not be detected. I
> > suspect many other special situations like this one.
>
> Maybe I would be able to implement this idea in following manner:
> run yade on each example with extra flag --test-example or such.
> This flag would mean that O.run() must be invoked anyways. If user
> has to click it, then yade instead does it. Some parts of examples
> would be untestable like interaction with GUI, in such cases a dummy
> function would be called instead (the point is that example.py need
> not be modified, the --test-example flag should take care of that). If
> examples produce some output file, than that is checked too. I am not
> sure how it will turn up. That's just a general idea.
>
>
> > About 2. I support the idea of investigating new techniques yet I don't
> > understand the suggestion very well. My impression is that all plugins are
> > already eligible for unit tests. For instance, testing a function from
> > utils in [1] did not need any change to the utils module itslef. All it
> > needs is to effectively design and write the unit tests for each other
> > function of each other class/module. That's indeed hundreds - if not
> > thousands - of tests.
>
> Well, time for me to learn what boost::unit_tests has to offer ;)
>
> The general idea within the framework is that it would be able to
> print a list of all publicly accessible C++ methods (not necessarily
> all of them being exported to python) which do not have an
> accompanying test.
>
> I don't know how to achieve this now. That's just an idea.
>
> Then using that list we would know the test coverage ;) If this list
> would someday become empty then we could say with confidence that we
> have 100% test coverage. If someone writes a new public function in
> some class, then even without exporting it to python, it would be
> caught and printed as a warning, that it has no accompanying test.
>
> Your _Tesselation<TT>::VertexHandle _Tesselation<TT>::move(…)
> should be caught automatically.
>
> I hope that it is possible. Maybe only a slight modification to
> YADE_PLUGIN or similar macro would be just enough? I don't know yet.
> Or maybe use some code for reading the library objects: it would go
> though all functions inside the binary library file, and try{}catch{}
> attempt to test them. I know that it is possible to read library
> symbols, I need to check how to do that.
> In that case each instance of _Tesselation<TT>::move(…) for all TT
> that ended up in the library file would be caught.
>
> --
> Janek Kozicki
>
> _______________________________________________
> Mailing list: https://launchpad.net/~yade-dev
> Post to     : yade-dev@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~yade-dev
> More help   : https://help.launchpad.net/ListHelp