openerp-expert-framework team mailing list archive

[Fabien Pinckaers <fp@xxxxxxxxxxx>]

I repost the evaluation made by Olivier, because I think he is not a
member of the lsit:

As explained, I have made an attempt at writing the above scenario using
oerpscenario (see http://piratepad.net/lPd4U1XdpS)

Take this as a first try by a java/python dev that must learn some ruby
on-the-fly to write the scenario. You can certainly provide advice to
improve the scenario and ruby stuff.

I probably haven't quite understood the philosophy of oerpscenario and
the way it is supposed to be used, so please correct me if I'm wrong in
the following early feedback:

1. It was definitely non-trivial to write the steps definition and get
the syntax right to make the test work, even with the basic
documentation provided with ooor and oerpscenario. The barrier is not
that small for an openobject python developer, I'm afraid.

2. Debugging errors is a challenge. Even if that's already the case with
OpenErp, at least I am already familiar with them.
E.g. here I have no idea why an ActiveResource suddenly calls a random
"allowed!" method, and how I could solve that.
(See bottom for some of the errors I encountered)
It feels a bit sluggish on my machine, but that's probably irrelevant.

3. I am not sure how this could scale for a large scenario base if we
have to rely on a small number of oerpscenario developers writing the
steps definition for everyone else.
(Compare this with recording the actual scenario in OpenERP directly
with an improved base_module_record that would output yaml?)

4. Finally, I really like cucumber's output for the test results.
Note to team: arrange something similar for regular OpenERP tests!


BTW, do any of you know other oerpscenario users/contributors that could
provide first impression feedback as well?



-------------------
For the record and for the oerpscenario experts, here's a shortlist of
the main errors I encountered. I can write bug reports about these on LP
if these are actual bugs (you can reply off-list):

- Creating a res.partner.address object using ResPartnerAddress.create()
or save() does not seem to work, because it calls the strange 'allowed!'
methods on the res.partner.address object via XMLRPC. I could not find a
workaround for this.

- Modifying a ResPartner instance and then saving it does not seem to
work either (see my example on piratepad). This seems to trigger a write
on all fields of the object, including the relationship fields, which
does not work (a.o.: passes an array of arrays instead of tuples).



PS: Raphael, I'm not in the openerp-expert-framework, thanks for adding
me  ;-)
And while at it, could you include ~openerp in all the expert teams so
that core team members can directly subscribe to the relevant
mailing-lists and follow the discussion there? (IIRC I already requested
this :p)





Joël Grand-Guillaume wrote:
> Hi Fabien,
> 
> 
> First of all, I must say I really appreciate you've finally decided to
> respond to 
> the community interrogations !
> 
> I also appreciate that you join the expert* mailing lists and I hope
> that in the future
> this will provide OpenERP SA a good way to discuss ALL problematics with the
> community. It's also a good way for the community to expose their ideas, I'm
> happy to ear you on this lists ! This is a key point for the project :
> Thanks you !
> 
> I was may be rude on my lasts posts, but honestly, I think I had to !
> You don't really
> provide us anything during the last months. I don't want to lists stuff
> here, I think
> you're aware of all troubles... And you had your reasons...
> 
> As you provide me some reasons why you don't really want to support
> OERPScenario, I'll give you my point of view / feed-back on that. This
> is the way to 
> discuss a problematic.
> 
> 
> 
>> 1. Like documentations, we think that the tests must be integrated in
>> the OpenERP modules and not as separate scripts. It's important that
>> every database installation or migration launches the whole test suite.
> 
> I agree with you for Unit tests. But functional tests or scenario can
> also be tested
> on each commit with a extern bot (through XML-RPC for example). I know
> LP provide an 
> API for that.
> 
> What about open-object framework errors ? I think it's not that good to
> tests a tool reliability
> with this same tool ! 
> 
> Finally, what about cross-module tests ? I witch module will you include
> them, ...
> 
>> 2. We don't want people to have to learn new languages and/or new
>> libraries to be able to to test cases. We also need to keep clean
>> dependencies against libraries used by OpenERP. (to keep things simple)
> 
> I agree with you on that one, at least on the technology to use. This
> will be better to keep 
> only Python. But:
> 
> - OERPScenario don't has to be installed on the same server than OpenERP
> - OERPScenario can be launched as an independent service among
> multi-openerp servers and
>   you can even provide a community services to tests their OpenERP instances
> - Ruby isn't that hard to learn for Python Devs !
> 
>> 3. We think that test scenaris may be written/recorded by non
>> developers. (without having to do some development) I know some may
>> argue that this is the case with OEScenario but it is not. I tries some
>> tests, and nearly all scenario I wanted to do required me to develop
>> simple things in ruby.
> 
> I do not agree with you here !!! This is may be the more important !
> OERPScenario allow
> business people to write the tests cases using "Gherkin" syntax, not in
> Ruby !!! Some technical 
> people are required to "code" the tests case business people wrote !
> 
> I try with Luc Maurer and with 10minutes explanation, he write me a
> fully understandable test case
> using the Gherkin syntax ! I'm pretty sure he won't succeed to write a
> YAML test !! As probably
> any business people never will...
> 
> This is THE KEY POINT : The Gherkin syntax was design for that. You
> cannot bite them with YAML stuff. 
> This allow you to clearly separate the business description and the
> "coding" of the tests cases.
> 
> Business people write the tests cases, as they know what they talk about
> Technical people code them... 
> 
> If you can do both, well, it's good :) !
> 
>> 4. For a Python software, we can difficultly rely on tests writen in
>> ruby. It's a too big entry barrier for OpenERP new comers.
> 
> Even if I agree with you on the ground, new comers will not write tests
> cases :) ! You need to know
> the ERP system for years before you really can make good tests !
> 
> I don't think that saying you use Ruby and Cucumber Library to test
> OpenERP will ever cause you
> some trouble, even for new comers...
> 
>> I am not telling that we want to replace OEScenario. We like it, and it
>> already helped a lot. But we think we also need an other alternative for
>> the reason above. So, please, you should accept that we evaluate several
>> options before choosing the right one.
> 
> Well... What can I say...I accept you have another opinion, and I will
> respect your choice anyway... I
> have no choice on that... 
> 
>>> I agree with you, I prefer OpenERP SA to spend more time improving the
>>> framework and refactoring the code rather than coding a
>>> concurrent to a ready-to-use community project.
>>
>> That's exactly what we want to do and that's probably the most important
>> point ! And the reason why we wanted to write a yaml version.
>>
>> ..........
>>
>> Having two options is better than nothing:
>>  - C2C's solution is good to test from external scripts
>>  - OpenERP is good to test within modules and package in OpenERP
>> It does not seems to be a problem for myself: we will continue to
>> advertise both and each one will be able to use the one it prefers.
>> Open source is all about freedom: you can not argue against this, I think.
>>
> 
> As you said, Open Source is about freedom, let people choose the best
> one. Just a pain in the
> ass for the "not choosen one" ;) !
> 
>>> Unfortunately, this is not the first time... I hope this time, they will
>>> admit and recognize our work. In a ideal world, I would even
>>> dream about OpenERP SA supporting our project as the official Testing
>>> system....
>>
>> Joël, we recognize your work ! We like it and communicate on it. But you
>> should understand that we need:
>> 1. A yaml parser for data files, instead of the current XML one
>> 2. Something integrated in OpenERP modules, so that everyone can launch
>> the tests and every installation/migration. Tests should be based on
>> demo data. -> no need to write both.
>> 3. OpenERP (which is in Python) can not rely on ruby and his libs
>> 4. Tests must be writeable by non developers.
> 
> I never ever arg against YAML ! Let's go with YAML files into OpenERP,
> I'm happy
> with that ! I also think that we should base part of the tests on demo data.
> 
> I just remember here that I implemented this in OERPScenario, so you can
> use the 
> demo data and improve them when needed.
> 
> Non-devs people will NEVER EVER be able to write tests case in a
> technical way ! this is
> why someone smart invent the Gherkin syntax... With this, business
> people only care on
> the Scenario, tech people about coding it...
> 
> 
>>>> So, I desperately call you to give OERPScenario (Cucumber) a shot
>>>> .....
>>
>> Of course we do it. Olivier is testing OERPScenario and will provide a
>> feedback soon.
>>
>> The best way to evaluate is to compare a test case. Let me know if I am
>> wrong, but as far as I understood, if I want to create this test:
>>
>>  - create a partner "Demour SA" with two contacts "Luc" and "Marc"
>>  - check that the credit of the partner==0
>>  - change the name of the partner to "Demour sa"
>>  - test that the new name is "Demour sa"
>>
>> With OEScenario, you have to create these files:
>>  - http://piratepad.net/lPd4U1XdpS
>> (todo by Olivier)
>>
>> With Yaml OpenERP, I create this file in a module:
>>  - http://piratepad.net/mATgKAXiac
> 
> 
> This is a good point you tested both solutions, I appreciate !
> OERPScenario has an issue with base_contact,
> that's why it crashes on your test.
> 
> Please Fabien, take something realistic before taking the wrong decision !
> 
> Tests cases are not about creating a simple partner with 2 contacts ! On
> this "too" simple test
> case, it's more easier on YAML, I agree.
> 
> But, just try with one other, more realistic one !! I give a "real" test
> case at the end of this mail... have a look.
> 
> I bet you won't succeed on this one with your YAML test system... And
> this is the kind of tests we need ! We
> can provide good Helpers with OERPScenario. When they'll be complete,
> you'll be able to write tests as fast 
> as a lighting ;) !
> 
> And we don't even discuss other problematics, like translations ! We can
> provide traduced tests cases as
> the whole stuff around translation are included in the Cucumber Library.
>  
> Well, at the end, as you see, we really though about lots of things
> before coding one test tool in Ruby (I never 
> coded in Ruby before). And we already though on all your point. I won't
> say we have the best solution ever, 
> but I know it's a good one.
> 
> I'm not against your system, it have advantages also and I recognize
> them. I'm just saying that you should
> concentrate your effort on refactoring the code, giving a good Unit
> Tests suite included into OpenERP 
> and refactoring the code. Then, let us take the BDD part as we'll do it
> well ;) !
> 
> As you said, the community will take his choice... Tryton is coming
> fast. I want OpenERP to keep his position !!
> 
> Thanks for your consideration.
> 
> 
> Regards,
> 
> Joël
> 
>  
> 
> 
> 
> EXAMPLE MORE COMPLEX TEST CASE :
> 
>   @invoicing @account @addons @reconciliation @bankstatement
>   Scenario: validate_failure_rollback_on_statement
>     Given I have recorded on the 1 jan 2009 a supplier invoice
> (in_invoice) of 1000,0 CHF without tax called
> MyFirstSupplierInvoiceBankStatement
>     When I press the validate button
>     Then I should see the invoice MyFirstSupplierInvoiceBankStatement open
>     And the residual amount = 1000,0
> 
>     Given I have recorded on the 1 jan 2009 a supplier invoice
> (in_invoice) of 1000,0 CHF without tax called
> MySecondSupplierInvoiceBankStatement
>     When I press the validate button
>     Then I should see the invoice MySecondSupplierInvoiceBankStatement open
>     And the residual amount = 1000,0
>     
>     Given I create a new bank statement called MyBankStatement with a
> CHF account journal
>     And I import on the 1 jan 2009, the following invoice (order
> matters) : MySecondSupplierInvoiceBankStatement,
> MyFirstSupplierInvoiceBankStatement
>     Then I should see an draft bank statement with 2 lines
>     
>     Given I take the created invoice MyFirstSupplierInvoiceBankStatement
>     When I call the Pay invoice wizard
>     And I completely pay the residual amount in CHF on the 13 sep 2009
>     Then I should see a residual amount of 0.0 CHF.-
>     And I should see the invoice MyFirstSupplierInvoiceBankStatement paid
> And the invoice should appear as paid invoice (checkbox tic)
>     
>     Given I take the bank statement called MyBankStatement
>     When push the confirm button of the statement it should raise a
> warning because one invoice is already reconciled
>     And no entries should be created by the bank statement
>     
> 
> -- 
> 
> 
> *Joël Grand-Guillaume** *
> 
> *Division Manager*
> *Business Solutions*
> *
> *
> *Camptocamp SA*
> PSE A, CH-1015 Lausanne
> 
>  www.camptocamp.com <http://www.camptocamp.com> 
> 
> 
> Phone: +41 21 619 10 28
> Office: +41 21 619 10 10
> Fax: +41 21 619 10 00
> Email: joel.grandguillaume@xxxxxxxxxxxxxx
> <mailto:joel.grandguillaume@xxxxxxxxxxxxxx>
> http://www.camptocamp.com/fr/business-solutions/formations
> 


-- 
Fabien Pinckaers
CEO Tiny - OpenERP Editor
Chaussée de Namur 40
B-1367 Grand-Rosière
Belgium
Phone: +32.81.81.37.00
Fax: +32.81.73.35.01
Web: http://openerp.com

Great Achievements Start With Tiny Investments
  -- Marty, 2005