openerp-expert-framework team mailing list archive

[Raphaël Valyi <rvalyi@xxxxxxxxx>]

Fabien,

Something important: yes as I said I think YAML makes sense and better if
that can be used for tests too.
Now, when you compare this:
http://piratepad.net/lPd4U1XdpS
and this:
http://piratepad.net/mATgKAXiac

I think it would be fair to recall that:

- I doubt non developer will really write your YAML files, no matter if they
look simple to us with tech skills (but I mean that's far better than XM,
glad I've been heard here)

- but at least non developer can perfectly read what OERPScenario actually
test (especially the result pages in plain English), I mean much better...
And they could easily also write English Scenario test that a developer
would translate to code implementing the few Ruby translations steps.

- you are not fair when you say you should learn Ruby: a test user should
only learn how to call methods on object. For a Python dev, that means like
4 hours at the very max, sometimes doing such effort is actually shorter
than learning an API (Okay in this case you don't throw a new API). In that
particular case, I'm pretty sure the OOOR API github.com/rvalyi/ooor is
still much easier to understand than the OpenERP one:
http://doc.openerp.com/developer/2_5_Objects_Fields_Methods/methods.html .
But okay your guys already suffered that pain...

- also you seems to consider learning bit of an other language is a pure
waste. I disagree and argue it makes you smarter and write better code
quicker in the future... If you consider you developer are not smart enough
to make such an effort, then I'm afraid are they good enough to work on an
ERP? OOOR is certainly not pointless, once you learn it, it becomes very
easy to perform some admin script in one liners, migrate or use the Kettle
ETL to interconnect OpenERP with anything, create Rails app connected to
OpenERP in a snap so a small step giving access to a whole new world of
possibilities rather than just a pure waste...

- some fundamental differences between your embedded modules and
OERPScenario ones:
   - embedded tests will only test with the current module and the modules
it depends on. BUT, a lot of OpenERP bugs occurs when you actually try
complex scenarios using an arbitrary modules combination... (especially
because not all methods are well designed enough to be properly inherited,
lots of overrider don't call super...).
   - your embedded tests test at the server side while OERPScenrio test at
the client side, closer to the user reality. Also, OERPScenario tests
important things such as on_changes, any buttons, wizards steps
(against across several modules), something you don't do with the current
test framework and where a lot of bug lies.


So overall, yes, those system are complementary. It would be stupid either
to forget about embedded tests and YAML.
I just wanted to make sure you understand the real benefit of something like
OERPScenario which is closer to real functional testing while the current
test suite is somewhat in between functional an unit testing. I also think
it's doesn't cost much to Tiny to run the two test suites after each commit
compared to the benefit. My fear would be you would have tried to rebuild a
full blown Cucumber clone in Python which is not possible as explained in
the links I passed here or that you just missed the advantages of
OERPScenario. I'm glad to see you started considering it.


Raphaël Valyi
http://www.akretion.com



2010/2/23 Fabien Pinckaers <fp@xxxxxxxxxxx>

> Joël,
>
> I hope you don't take this as a personal attack. We like and merge C2C
> proposals and we do really appreciate your work and it's clear that
> OpenERP scenario is great value add to OpenERP.
>
> I will explain the reason why we are evaluating a second option (which
> do not intend to replace OpenERP scenario but provide an alternative;
> people will be able to chose). I also think that both options can
> complement each other.
>
> Here are the reasons:
> 1. Like documentations, we think that the tests must be integrated in
> the OpenERP modules and not as separate scripts. It's important that
> every database installation or migration launches the whole test suite.
>
> 2. We don't want people to have to learn new languages and/or new
> libraries to be able to to test cases. We also need to keep clean
> dependencies against libraries used by OpenERP. (to keep things simple)
>
> 3. We think that test scenaris may be written/recorded by non
> developers. (without having to do some development) I know some may
> argue that this is the case with OEScenario but it is not. I tries some
> tests, and nearly all scenario I wanted to do required me to develop
> simple things in ruby.
>
> 4. For a Python software, we can difficultly rely on tests writen in
> ruby. It's a too big entry barrier for OpenERP new comers.
>
> I am not telling that we want to replace OEScenario. We like it, and it
> already helped a lot. But we think we also need an other alternative for
> the reason above. So, please, you should accept that we evaluate several
> options before choosing the right one.
>
>
> > I agree with you, I prefer OpenERP SA to spend more time improving the
> > framework and refactoring the code rather than coding a
> > concurrent to a ready-to-use community project.
>
> That's exactly what we want to do and that's probably the most important
> point ! And the reason why we wanted to write a yaml version.
>
> We are reviewing lots of parts of the framework. The XML files is
> clearly something that made us loss lots of time when developing new
> modules.
>
> XML files are quite heavy and the code that parses XML file have to be
> rewritten from scratch (check tools/convert.py). So, yes, if we chosed
> to analyse this option, that's because we think the OpenERP framework
> need to support yaml, and the XML parser have to be rewritten.
>
> So, we spend more time by improving the framework.
>
> Let just me remind you that OpenERP already has a scenario/test system
> integrated in the XML files (with assert, ...). So we are not
> developping a new engine, but just improving the way you can load data
> files for modules.
>
> Adding a yaml parser is only 50 lines of code more to the XML parser.
> So, we are clearly not reinventing the wheel, we are cleaning our code
> and add 50 lines of python to support yaml and XML formats.
>
> It's a great addition to OpenERP and I am sure everyone will be happy to
> write data files in yaml instead of XML. (with 50 lines more, our module
> recorder creates yaml files instead of XML ones, so we also have to
> automated creation of test cases). And, if these demo data can sustain
> the scenario system, it's very good too. Especially because we don't
> have to develop new things.
>
> OpenERP developers will not have to learn new things, because the yaml
> tool we propose is exactly the same than the .XML files in modules. No
> new things to learn.
>
> So, we are not creating a completly new test system, we just improve the
> existing one by support yaml. If you put that in perspective:
>
> 1. We developed a test engine in OpenERP
>  (clearly not easy to use due to XML)
> 2. C2C decided to not use it and write a completly other one
>  (for good reasons)
> 3. We think that, by cleaning the framework, we can do a as good one
> and that it will be better integrated in easier in OpenERP.
>  (for good reasons too)
>
> Having two options is better than nothing:
>  - C2C's solution is good to test from external scripts
>  - OpenERP is good to test within modules and package in OpenERP
> It does not seems to be a problem for myself: we will continue to
> advertise both and each one will be able to use the one it prefers.
> Open source is all about freedom: you can not argue against this, I think.
>
> > Unfortunately, this is not the first time... I hope this time, they will
> > admit and recognize our work. In a ideal world, I would even
> > dream about OpenERP SA supporting our project as the official Testing
> > system....
>
> Joël, we recognize your work ! We like it and communicate on it. But you
> should understand that we need:
> 1. A yaml parser for data files, instead of the current XML one
> 2. Something integrated in OpenERP modules, so that everyone can launch
> the tests and every installation/migration. Tests should be based on
> demo data. -> no need to write both.
> 3. OpenERP (which is in Python) can not rely on ruby and his libs
> 4. Tests must be writeable by non developers.
>
>
> >> So, I desperately call you to give OERPScenario (Cucumber) a shot
> >> because no matter the efforts you are willing to waste, you will not
> >> come close, as explained in this presentation. So, I rather suggest
> >> you picks up that incredible tool CampToCamp made and use it and
> >> rather use your remaining time refactoring the core of OpenERP for
> >> less bugs, rather than trying to make a clone that will fail to be as
> >> powerful. Note that in OERPScenario, you will only use Cucumber,
> >> meaning that even if you would consider valid the critics against
> >> Ruby, they just wouldn't apply as you don't have to write Ruby
> >> code yourself past using the Cucumber/OERPScenario.
>
> Of course we do it. Olivier is testing OERPScenario and will provide a
> feedback soon.
>
> The best way to evaluate is to compare a test case. Let me know if I am
> wrong, but as far as I understood, if I want to create this test:
>
>  - create a partner "Demour SA" with two contacts "Luc" and "Marc"
>  - check that the credit of the partner==0
>  - change the name of the partner to "Demour sa"
>  - test that the new name is "Demour sa"
>
> With OEScenario, you have to create these files:
>  - http://piratepad.net/lPd4U1XdpS
> (todo by Olivier)
>
> With Yaml OpenERP, I create this file in a module:
>  - http://piratepad.net/mATgKAXiac
>
>
> Note that OpenERP Yaml is only for the trunk, for the V5.0 version of
> OpenERP, I suggest to use OERPScenario (or the normal test system in XML
> modules)
>
> Thanks,
>
>
> --
> Fabien Pinckaers
> CEO Tiny - OpenERP Editor
> Chaussée de Namur 40
> B-1367 Grand-Rosière
> Belgium
> Phone: +32.81.81.37.00
> Fax: +32.81.73.35.01
> Web: http://openerp.com
>
> Great Achievements Start With Tiny Investments
>  -- Marty, 2005
>