dhis2-devs team mailing list archive

[Bob Jolliffe <bobjolliffe@xxxxxxxxx>]

Hi

On 3 June 2010 10:50, Ngoc Thanh Nguyen <thanh.hispvietnam@xxxxxxxxx> wrote:
> Hi Bob,
>
> Thanks for the comments. I have some replies below.
>
> On Thu, Jun 3, 2010 at 3:40 PM, Bob Jolliffe <bobjolliffe@xxxxxxxxx> wrote:
>> Hi Thanh
>>
>> We did receive it and i had a quick read through.  Its great to see
>> work being done on testing.  we need lots of it.
>>
>> Two comments/questions (to prove that I read it!):
>> 1.  I am not entirely convinced by your replacement of TSL with Excel.
>>  TSL is a formal grammar.  It has limitations which have been
>> described elsewhere but I don't think that the typing of '[' has ever
>> really featured highly :-)   I think you may be confusing the
>> specification with the authoring tool when you say that your revised
>> TSL is to "use excel".  So on the theory side I'd be a bit
>> uncomfortable with this aspect of your paper.
>> (BTW did you look at schematron as an alternative grammar for
>> expressing tests?  I know one of the criticisms of TSL is that it does
>> not allow for a very literate human-readable approach. schematron
>> allows for a human readable assertions and - though xpath is usually
>> used for schema validation - tests can be implemented in any language.
>>  Perhaps an off-the wall thought ...)
>
> TSL in my paper is about the specification developed by Ostrand 1988.
> They developed this specification within the method they called
> Category Partition - a combination between Equivalence Partition and
> Boundary Analysis. This specification as I feel is a bit complicated
> because it required typing [ or ] character. My revision aims to save
> time for typing this and also make it readable for a computer program.
> I selected Excel because of its popularity and easy-to-use.
>

Yes I understand all that.  So you have selected excel as a tool for
authoring your test cases.  Thats fine and I'm sure its a good tool
for the job for the reasons you describe.  The point is that TSL is a
language for expressing them.  What do you express? An "excel file".
I really don't think thats the the same thing at all.  I guess what is
missing is some way to describe those spreadsheets of yours so that
they can be considered as a sort of language like TSL.

> CPM is the most popular techniques in black box testing. Cause graph
> effect is another method but it requires to draw graphical diagram so
> it seems impossible in practice.
>
> XPath or Schematron seems to be out of the scope of CPM.

Not at all.  See
http://www.computer.org/portal/web/csdl/doi/10.1109/ICSEA.2006.9 for
example.

I agree XPath is well out of scope, but schematron is not bound to use
XPath.  Just so happens that 99% of the time it does because it is
primarily used for testing xml validity.  But I guess there is nothing
in principle preventing the actual tests to be implemented as say
javascript, or even selenium scripts.

>
>> 2.  I am not very clear - perhaps didn't read carefully enough - how
>> you generate the actual tests from the source document.  It looks to
>> me that it requires quite a significant understanding of the
>> application , the classes etc.  I guess this is where understanding
>> the grammar, if any, is important.  Could you generate selenium tests
>> from it?  This strikes me as the most black-box kind of testing.
>
> No, CPM can not generate actual test cases and test data. It can
> generate test frame (a combination of different input values). Tester
> has to provide the test data, hence, to build test case. I find it
> hard to have any automation tool to generate test cases (with test
> data) automatically. And my tool just does a simple job: 1) read the
> Excel file containing category and partition, 2) combination based on
> their properties and constraints, 3) and build the test frames based
> on the combination.

OK.  Now I understand.
>
>> How easy would it be to use your tool to do wider test coverage of
>> dhis?  Would be a good thing.
>
> Here we come with the test coverage. Black box is different from white
> box. It can not produce any code coverage (statement, block, branch,
> predicate etc). Not at all. It can not be used with mutation. It seems
> there is no way to assess whether a test set is adequate - good
> enough. Any one have better idea can correct me on this.
>
> But I am very confident that if we follow the CPM strictly, together
> with the support from my tool, i.e. revise the category and partition
> building back and forth, we can end up with a 100% test coverage.

Sounds good.

>
> The challenge is, for web layer of DHIS2, we could not apply white box
> testing. This is partial reason why there is no unit testing for
> action class. It is so difficult to mock all the session and
> interceptor at least in DHIS2. We have only one choice, that is black
> box testing. And so far, we have not done any systematic testing on
> web modules. Our current approach is more ad-hoc.

This is where I was suggesting something like selenium might be
useful.  Does anybody know of any formal testing language for web
applications?  I am sure there is bound to be something.

Thanh, thanks for sharing this.  Now I must get back to work ... bloody spring.

Regards
Bob

>
> More input needed.
> Thanh
>
>> Cheers
>> Bob
>>
>> On 2 June 2010 12:03, Ngoc Thanh Nguyen <thanh.hispvietnam@xxxxxxxxx> wrote:
>>> Hi all,
>>>
>>> this is a project I did for the testing course this semester. I hope
>>> it could shed a light for testing aspect of dhis2.
>>>
>>> Notice: the case was built based on the assumption that the system
>>> specification requires name of orgunit, group, and group set to be
>>> have between 2-255 characters. Though dhis2 implemented it with text
>>> field max char = 160.
>>>
>>> Thanh
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~dhis2-devs
>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>>
>>
>