dhis2-devs team mailing list archive

[Ngoc Thanh Nguyen <thanh.hispvietnam@xxxxxxxxx>]

Hi Bob,

Thanks for the feedbacks written in your busy time :-). But I have a
small comments on Selenium. Please read below.

On Thu, Jun 3, 2010 at 5:17 PM, Bob Jolliffe <bobjolliffe@xxxxxxxxx> wrote:
> Hi
>
> On 3 June 2010 10:50, Ngoc Thanh Nguyen <thanh.hispvietnam@xxxxxxxxx> wrote:
>> Hi Bob,
>>
>> Thanks for the comments. I have some replies below.
>>
>> On Thu, Jun 3, 2010 at 3:40 PM, Bob Jolliffe <bobjolliffe@xxxxxxxxx> wrote:
>>> Hi Thanh
>>>
>>> We did receive it and i had a quick read through.  Its great to see
>>> work being done on testing.  we need lots of it.
>>>
>>> Two comments/questions (to prove that I read it!):
>>> 1.  I am not entirely convinced by your replacement of TSL with Excel.
>>>  TSL is a formal grammar.  It has limitations which have been
>>> described elsewhere but I don't think that the typing of '[' has ever
>>> really featured highly :-)   I think you may be confusing the
>>> specification with the authoring tool when you say that your revised
>>> TSL is to "use excel".  So on the theory side I'd be a bit
>>> uncomfortable with this aspect of your paper.
>>> (BTW did you look at schematron as an alternative grammar for
>>> expressing tests?  I know one of the criticisms of TSL is that it does
>>> not allow for a very literate human-readable approach. schematron
>>> allows for a human readable assertions and - though xpath is usually
>>> used for schema validation - tests can be implemented in any language.
>>>  Perhaps an off-the wall thought ...)
>>
>> TSL in my paper is about the specification developed by Ostrand 1988.
>> They developed this specification within the method they called
>> Category Partition - a combination between Equivalence Partition and
>> Boundary Analysis. This specification as I feel is a bit complicated
>> because it required typing [ or ] character. My revision aims to save
>> time for typing this and also make it readable for a computer program.
>> I selected Excel because of its popularity and easy-to-use.
>>
>
> Yes I understand all that.  So you have selected excel as a tool for
> authoring your test cases.  Thats fine and I'm sure its a good tool
> for the job for the reasons you describe.  The point is that TSL is a
> language for expressing them.  What do you express? An "excel file".
> I really don't think thats the the same thing at all.  I guess what is
> missing is some way to describe those spreadsheets of yours so that
> they can be considered as a sort of language like TSL.
>
>> CPM is the most popular techniques in black box testing. Cause graph
>> effect is another method but it requires to draw graphical diagram so
>> it seems impossible in practice.
>>
>> XPath or Schematron seems to be out of the scope of CPM.
>
> Not at all.  See
> http://www.computer.org/portal/web/csdl/doi/10.1109/ICSEA.2006.9 for
> example.
>
> I agree XPath is well out of scope, but schematron is not bound to use
> XPath.  Just so happens that 99% of the time it does because it is
> primarily used for testing xml validity.  But I guess there is nothing
> in principle preventing the actual tests to be implemented as say
> javascript, or even selenium scripts.
>
>>
>>> 2.  I am not very clear - perhaps didn't read carefully enough - how
>>> you generate the actual tests from the source document.  It looks to
>>> me that it requires quite a significant understanding of the
>>> application , the classes etc.  I guess this is where understanding
>>> the grammar, if any, is important.  Could you generate selenium tests
>>> from it?  This strikes me as the most black-box kind of testing.
>>
>> No, CPM can not generate actual test cases and test data. It can
>> generate test frame (a combination of different input values). Tester
>> has to provide the test data, hence, to build test case. I find it
>> hard to have any automation tool to generate test cases (with test
>> data) automatically. And my tool just does a simple job: 1) read the
>> Excel file containing category and partition, 2) combination based on
>> their properties and constraints, 3) and build the test frames based
>> on the combination.
>
> OK.  Now I understand.
>>
>>> How easy would it be to use your tool to do wider test coverage of
>>> dhis?  Would be a good thing.
>>
>> Here we come with the test coverage. Black box is different from white
>> box. It can not produce any code coverage (statement, block, branch,
>> predicate etc). Not at all. It can not be used with mutation. It seems
>> there is no way to assess whether a test set is adequate - good
>> enough. Any one have better idea can correct me on this.
>>
>> But I am very confident that if we follow the CPM strictly, together
>> with the support from my tool, i.e. revise the category and partition
>> building back and forth, we can end up with a 100% test coverage.
>
> Sounds good.
>
>>
>> The challenge is, for web layer of DHIS2, we could not apply white box
>> testing. This is partial reason why there is no unit testing for
>> action class. It is so difficult to mock all the session and
>> interceptor at least in DHIS2. We have only one choice, that is black
>> box testing. And so far, we have not done any systematic testing on
>> web modules. Our current approach is more ad-hoc.
>
> This is where I was suggesting something like selenium might be
> useful.  Does anybody know of any formal testing language for web
> applications?  I am sure there is bound to be something.


I have tried Selenium, indeed, I have built several test cases on
Selenium and it run quite well. I also tried sahi, another web
automation test tool. These tool help tester to record their tests and
save in different format, hence they can be run later without manual
re-running. HOWEVER, what to be recoded in Selenium is really a
question. And CPM comes to fulfill this space.

So I recommend the following process: building test specification
(category, partition, properties, constrainst) --> use the tool to
generate the test frame (or test case if you want to call) --> record
these test cases using Selenium --> enjoy the automation forever.

Thanh

> Thanh, thanks for sharing this.  Now I must get back to work ... bloody spring.
>
> Regards
> Bob
>
>>
>> More input needed.
>> Thanh
>>
>>> Cheers
>>> Bob
>>>
>>> On 2 June 2010 12:03, Ngoc Thanh Nguyen <thanh.hispvietnam@xxxxxxxxx> wrote:
>>>> Hi all,
>>>>
>>>> this is a project I did for the testing course this semester. I hope
>>>> it could shed a light for testing aspect of dhis2.
>>>>
>>>> Notice: the case was built based on the assumption that the system
>>>> specification requires name of orgunit, group, and group set to be
>>>> have between 2-255 characters. Though dhis2 implemented it with text
>>>> field max char = 160.
>>>>
>>>> Thanh
>>>>
>>>> _______________________________________________
>>>> Mailing list: https://launchpad.net/~dhis2-devs
>>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>>> More help   : https://help.launchpad.net/ListHelp
>>>>
>>>>
>>>
>>
>