dhis2-devs-core team mailing list archive

[Jim Grace <jimgrace@xxxxxxxxx>]

Hi Jason and Mark,

API testing would be quite valuable - great! Thanks! And I agree that we
should aim towards developing a test suite for all API functions.

I like the simple test description syntax used by pyresttest. Have you
compare this tool with other REST testers?

Cheers,
Jim


On Fri, Aug 28, 2015 at 2:58 AM, Jason Pickering <
jason.p.pickering@xxxxxxxxx> wrote:

> Hi Morten,
> I think the idea would be to start with the schema and its various end
> points. Reason we were thinking this was
>
> 1) We could develop many test in a sort of automated way to see how this
> framework and approach works and if it is a good fit.
>
> 2) The schema endpoint in a way defines how we can "trust" the API, as the
> first point of contact potentially from external services. If the schema
> changes from one version to the next, that itself may be a test, but then
> this particular test failure was flagged because even within this version,
> the schema and its endpoint were not consistent. It would also help us
> potentially to develop generated payloads, if we can trust it. :)
>
> But, no. I think we would like to develop specific unit tests for certain
> operations...creating an orgunit, creating a user, deleting an orgunit
> group, etc and provide examples for app developers to create their own unit
> tests for their own apps/services to allow people to test against various
> versions of the application. I am working on one of these now, so once it
> is done, we can see what happens.
>
> Yes, I think the idea would to be integrate this into the CI server, which
> Mark has already done here (
> https://travis-ci.org/dhis2/dhis2-api-testing/builds/77504348).
>
> Regards,
> Jason
>
>
> On Fri, Aug 28, 2015 at 8:18 AM, Morten Olav Hansen <mortenoh@xxxxxxxxx>
> wrote:
>
>> Hi Jason
>>
>> That looks interesting, so this would target /api/schemas only? would be
>> interesting to see how other parts of the API could be tested also. I have
>> also been looking into doing testing using Spring MVC Testing framework.
>>
>> Are you thinking this could be integrated into our CI server also?
>>
>> --
>> Morten
>>
>> On Fri, Aug 28, 2015 at 12:41 PM, Jason Pickering <
>> jason.p.pickering@xxxxxxxxx> wrote:
>>
>>> Hi Morten,
>>> Mark and I have been doing some brainstorming and experimentation around
>>> the testing of the API. This came about after some discussions in the
>>> experts meeting around consistency of the API across different versions .
>>> Mark and I were thinking what we would like to have would be a suite of
>>> external tests of the API, mainly for regression purposes between
>>> versions.  The idea being to develop tests from one version based on the
>>> /api/schemas (i.e. stable)  end point, and then to test these against
>>> another version (i.e. dev) . Mark developed some code  to develop tests
>>> against the schemas <https://apps.dhis2.org/demo/api/schemas.json>
>>> endpoint  and then we plugged this into something called pyresttest
>>> <https://github.com/svanoort/pyresttest>. The repo of the API test for
>>> DHIS2 is here <https://github.com/dhis2/dhis2-api-testing>.
>>>
>>> Interestingly, it has picked up some inconsitencies between what is in
>>> the schema and what is in the schema end points.
>>>
>>> Note carefully here from this link
>>> <https://apps.dhis2.org/dev/api/schemas/programRuleAction.json>
>>>  (/api/programRuleActions)
>>>
>>>
>>> Note that "trackedEntityAttribute" is present here.
>>>
>>> However from https://apps.dhis2.org/demo/api/schemas.json, it is
>>> missing.
>>>
>>>
>>> This is one type of thing we would like to pick up.
>>>
>>> We are still thinking through the best way to approach the testing and
>>> your thoughts would be most welcome.
>>>
>>> Regards,
>>> Jason
>>>
>>>
>>
>
>
> --
> Jason P. Pickering
> email: jason.p.pickering@xxxxxxxxx
> tel:+46764147049
>
> --
> Mailing list: https://launchpad.net/~dhis2-devs-core
> Post to     : dhis2-devs-core@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~dhis2-devs-core
> More help   : https://help.launchpad.net/ListHelp
>
>