openerp-expert-framework team mailing list archive

[Joël Grand-Guillaume <joel.grandguillaume@xxxxxxxxxxxxxx>]

Hi Fabien,


First of all, I must say I really appreciate you've finally decided to respond to 
the community interrogations !

I also appreciate that you join the expert* mailing lists and I hope that in the future
this will provide OpenERP SA a good way to discuss ALL problematics with the
community. It's also a good way for the community to expose their ideas, I'm
happy to ear you on this lists ! This is a key point for the project : Thanks you !

I was may be rude on my lasts posts, but honestly, I think I had to ! You don't really
provide us anything during the last months. I don't want to lists stuff here, I think
you're aware of all troubles... And you had your reasons...

As you provide me some reasons why you don't really want to support
OERPScenario, I'll give you my point of view / feed-back on that. This is the way to 
discuss a problematic.



> 1. Like documentations, we think that the tests must be integrated in
> the OpenERP modules and not as separate scripts. It's important that
> every database installation or migration launches the whole test suite.

I agree with you for Unit tests. But functional tests or scenario can also be tested
on each commit with a extern bot (through XML-RPC for example). I know LP provide an 
API for that.

What about open-object framework errors ? I think it's not that good to tests a tool reliability
with this same tool ! 

Finally, what about cross-module tests ? I witch module will you include them, ...

> 2. We don't want people to have to learn new languages and/or new
> libraries to be able to to test cases. We also need to keep clean
> dependencies against libraries used by OpenERP. (to keep things simple)

I agree with you on that one, at least on the technology to use. This will be better to keep 
only Python. But:

- OERPScenario don't has to be installed on the same server than OpenERP
- OERPScenario can be launched as an independent service among multi-openerp servers and
  you can even provide a community services to tests their OpenERP instances
- Ruby isn't that hard to learn for Python Devs !

> 3. We think that test scenaris may be written/recorded by non
> developers. (without having to do some development) I know some may
> argue that this is the case with OEScenario but it is not. I tries some
> tests, and nearly all scenario I wanted to do required me to develop
> simple things in ruby.

I do not agree with you here !!! This is may be the more important ! OERPScenario allow
business people to write the tests cases using "Gherkin" syntax, not in Ruby !!! Some technical 
people are required to "code" the tests case business people wrote !

I try with Luc Maurer and with 10minutes explanation, he write me a fully understandable test case
using the Gherkin syntax ! I'm pretty sure he won't succeed to write a YAML test !! As probably
any business people never will...

This is THE KEY POINT : The Gherkin syntax was design for that. You cannot bite them with YAML stuff. 
This allow you to clearly separate the business description and the "coding" of the tests cases.

Business people write the tests cases, as they know what they talk about
Technical people code them... 

If you can do both, well, it's good :) !

> 4. For a Python software, we can difficultly rely on tests writen in
> ruby. It's a too big entry barrier for OpenERP new comers.

Even if I agree with you on the ground, new comers will not write tests cases :) ! You need to know
the ERP system for years before you really can make good tests !

I don't think that saying you use Ruby and Cucumber Library to test OpenERP will ever cause you
some trouble, even for new comers...

> I am not telling that we want to replace OEScenario. We like it, and it
> already helped a lot. But we think we also need an other alternative for
> the reason above. So, please, you should accept that we evaluate several
> options before choosing the right one.

Well... What can I say...I accept you have another opinion, and I will respect your choice anyway... I
have no choice on that... 

>> I agree with you, I prefer OpenERP SA to spend more time improving the
>> framework and refactoring the code rather than coding a 
>> concurrent to a ready-to-use community project. 
> 
> That's exactly what we want to do and that's probably the most important
> point ! And the reason why we wanted to write a yaml version.
> 
> ..........
> 
> Having two options is better than nothing:
>  - C2C's solution is good to test from external scripts
>  - OpenERP is good to test within modules and package in OpenERP
> It does not seems to be a problem for myself: we will continue to
> advertise both and each one will be able to use the one it prefers.
> Open source is all about freedom: you can not argue against this, I think.
> 

As you said, Open Source is about freedom, let people choose the best one. Just a pain in the
ass for the "not choosen one" ;) !

>> Unfortunately, this is not the first time... I hope this time, they will
>> admit and recognize our work. In a ideal world, I would even
>> dream about OpenERP SA supporting our project as the official Testing
>> system....
> 
> Joël, we recognize your work ! We like it and communicate on it. But you
> should understand that we need:
> 1. A yaml parser for data files, instead of the current XML one
> 2. Something integrated in OpenERP modules, so that everyone can launch
> the tests and every installation/migration. Tests should be based on
> demo data. -> no need to write both.
> 3. OpenERP (which is in Python) can not rely on ruby and his libs
> 4. Tests must be writeable by non developers.

I never ever arg against YAML ! Let's go with YAML files into OpenERP, I'm happy
with that ! I also think that we should base part of the tests on demo data.

I just remember here that I implemented this in OERPScenario, so you can use the 
demo data and improve them when needed.

Non-devs people will NEVER EVER be able to write tests case in a technical way ! this is
why someone smart invent the Gherkin syntax... With this, business people only care on
the Scenario, tech people about coding it...


>>> So, I desperately call you to give OERPScenario (Cucumber) a shot
>>> .....
> 
> Of course we do it. Olivier is testing OERPScenario and will provide a
> feedback soon.
> 
> The best way to evaluate is to compare a test case. Let me know if I am
> wrong, but as far as I understood, if I want to create this test:
> 
>  - create a partner "Demour SA" with two contacts "Luc" and "Marc"
>  - check that the credit of the partner==0
>  - change the name of the partner to "Demour sa"
>  - test that the new name is "Demour sa"
> 
> With OEScenario, you have to create these files:
>  - http://piratepad.net/lPd4U1XdpS
> (todo by Olivier)
> 
> With Yaml OpenERP, I create this file in a module:
>  - http://piratepad.net/mATgKAXiac


This is a good point you tested both solutions, I appreciate ! OERPScenario has an issue with base_contact,
that's why it crashes on your test.

Please Fabien, take something realistic before taking the wrong decision !

Tests cases are not about creating a simple partner with 2 contacts ! On this "too" simple test
case, it's more easier on YAML, I agree.

But, just try with one other, more realistic one !! I give a "real" test case at the end of this mail... have a look.

I bet you won't succeed on this one with your YAML test system... And this is the kind of tests we need ! We
can provide good Helpers with OERPScenario. When they'll be complete, you'll be able to write tests as fast 
as a lighting ;) !

And we don't even discuss other problematics, like translations ! We can provide traduced tests cases as
the whole stuff around translation are included in the Cucumber Library.
 
Well, at the end, as you see, we really though about lots of things before coding one test tool in Ruby (I never 
coded in Ruby before). And we already though on all your point. I won't say we have the best solution ever, 
but I know it's a good one.

I'm not against your system, it have advantages also and I recognize them. I'm just saying that you should
concentrate your effort on refactoring the code, giving a good Unit Tests suite included into OpenERP 
and refactoring the code. Then, let us take the BDD part as we'll do it well ;) !

As you said, the community will take his choice... Tryton is coming fast. I want OpenERP to keep his position !!

Thanks for your consideration.


Regards,

Joël

 



EXAMPLE MORE COMPLEX TEST CASE :

  @invoicing @account @addons @reconciliation @bankstatement
  Scenario: validate_failure_rollback_on_statement
    Given I have recorded on the 1 jan 2009 a supplier invoice (in_invoice) of 1000,0 CHF without tax called MyFirstSupplierInvoiceBankStatement
    When I press the validate button
    Then I should see the invoice MyFirstSupplierInvoiceBankStatement open
    And the residual amount = 1000,0

    Given I have recorded on the 1 jan 2009 a supplier invoice (in_invoice) of 1000,0 CHF without tax called MySecondSupplierInvoiceBankStatement
    When I press the validate button
    Then I should see the invoice MySecondSupplierInvoiceBankStatement open
    And the residual amount = 1000,0
    
    Given I create a new bank statement called MyBankStatement with a CHF account journal
    And I import on the 1 jan 2009, the following invoice (order matters) : MySecondSupplierInvoiceBankStatement, MyFirstSupplierInvoiceBankStatement
    Then I should see an draft bank statement with 2 lines
    
    Given I take the created invoice MyFirstSupplierInvoiceBankStatement
    When I call the Pay invoice wizard
    And I completely pay the residual amount in CHF on the 13 sep 2009
    Then I should see a residual amount of 0.0 CHF.-
    And I should see the invoice MyFirstSupplierInvoiceBankStatement paid
	And the invoice should appear as paid invoice (checkbox tic)
    
    Given I take the bank statement called MyBankStatement
    When push the confirm button of the statement it should raise a warning because one invoice is already reconciled
    And no entries should be created by the bank statement
    

-- 

Joël Grand-Guillaume 
Division Manager
Business Solutions

Camptocamp SA
PSE A, CH-1015 Lausanne
 www.camptocamp.com 

Phone: +41 21 619 10 28
Office: +41 21 619 10 10
Fax: +41 21 619 10 00
Email: joel.grandguillaume@xxxxxxxxxxxxxx
http://www.camptocamp.com/fr/business-solutions/formations