dhis2-devs-core team mailing list archive

[Jason Pickering <jason.p.pickering@xxxxxxxxx>]

Hi Jim.

No, not really. We just came across this yesterday afternoon, and it looked
like a good mix of simplicity, but yet quite functional. So, we just hacked
together some stuff to give it a spin.

We would be happy to consider other tools, but wanted to try this one out
and see what mileage we might get from it.

So far, I think Mark and I both file the results interesting. The ability
to integrate with CI is also interesting. Mark spun this up, which shows
which tests failed and passed https://travis-ci.org/dhis2/dhis2-api-testing

In addition to the test results between the /schema endpoints and the
various individual endpoints, I found another strange thing by accident.

 Trick question. Which of the following will "work" in the sense of
returning an HTTP 200 ?

curl  "
https://apps.dhis2.org/demo/api/sharing?type=dataElement&id=fbfJHSPpUQD"; -X
GET -H "Content-Type: application/json" -u admin:district

curl  "
https://apps.dhis2.org/demo/api/sharing?type=dataElement&id=fbfJHSPpUQD"; -X
GET -H "Content-Type: application/xml" -u admin:district

curl  "
https://apps.dhis2.org/demo/api/sharing?type=dataElement&id=fbfJHSPpUQD"; -X
GET -H "Content-Type: application/foobar" -u admin:district

Yeah, I know. Content-Type with a GET makes little sense, but give them a
try. It kind of stumped us.

Anyway, we would be happy to get more input on how to proceed, but feel
like we need to get somethign up and running pretty soon. Have done enough
talking about it at the experts meeting in Oslo!

Regards,
Jason



On Fri, Aug 28, 2015 at 2:22 PM, Jim Grace <jimgrace@xxxxxxxxx> wrote:

> Hi Jason and Mark,
>
> API testing would be quite valuable - great! Thanks! And I agree that we
> should aim towards developing a test suite for all API functions.
>
> I like the simple test description syntax used by pyresttest. Have you
> compare this tool with other REST testers?
>
> Cheers,
> Jim
>
>
> On Fri, Aug 28, 2015 at 2:58 AM, Jason Pickering <
> jason.p.pickering@xxxxxxxxx> wrote:
>
>> Hi Morten,
>> I think the idea would be to start with the schema and its various end
>> points. Reason we were thinking this was
>>
>> 1) We could develop many test in a sort of automated way to see how this
>> framework and approach works and if it is a good fit.
>>
>> 2) The schema endpoint in a way defines how we can "trust" the API, as
>> the first point of contact potentially from external services. If the
>> schema changes from one version to the next, that itself may be a test, but
>> then this particular test failure was flagged because even within this
>> version, the schema and its endpoint were not consistent. It would also
>> help us potentially to develop generated payloads, if we can trust it. :)
>>
>> But, no. I think we would like to develop specific unit tests for certain
>> operations...creating an orgunit, creating a user, deleting an orgunit
>> group, etc and provide examples for app developers to create their own unit
>> tests for their own apps/services to allow people to test against various
>> versions of the application. I am working on one of these now, so once it
>> is done, we can see what happens.
>>
>> Yes, I think the idea would to be integrate this into the CI server,
>> which Mark has already done here (
>> https://travis-ci.org/dhis2/dhis2-api-testing/builds/77504348).
>>
>> Regards,
>> Jason
>>
>>
>> On Fri, Aug 28, 2015 at 8:18 AM, Morten Olav Hansen <mortenoh@xxxxxxxxx>
>> wrote:
>>
>>> Hi Jason
>>>
>>> That looks interesting, so this would target /api/schemas only? would be
>>> interesting to see how other parts of the API could be tested also. I have
>>> also been looking into doing testing using Spring MVC Testing framework.
>>>
>>> Are you thinking this could be integrated into our CI server also?
>>>
>>> --
>>> Morten
>>>
>>> On Fri, Aug 28, 2015 at 12:41 PM, Jason Pickering <
>>> jason.p.pickering@xxxxxxxxx> wrote:
>>>
>>>> Hi Morten,
>>>> Mark and I have been doing some brainstorming and experimentation
>>>> around the testing of the API. This came about after some discussions in
>>>> the experts meeting around consistency of the API across different versions
>>>> .  Mark and I were thinking what we would like to have would be a suite of
>>>> external tests of the API, mainly for regression purposes between
>>>> versions.  The idea being to develop tests from one version based on the
>>>> /api/schemas (i.e. stable)  end point, and then to test these against
>>>> another version (i.e. dev) . Mark developed some code  to develop tests
>>>> against the schemas <https://apps.dhis2.org/demo/api/schemas.json>
>>>> endpoint  and then we plugged this into something called pyresttest
>>>> <https://github.com/svanoort/pyresttest>. The repo of the API test for
>>>> DHIS2 is here <https://github.com/dhis2/dhis2-api-testing>.
>>>>
>>>> Interestingly, it has picked up some inconsitencies between what is in
>>>> the schema and what is in the schema end points.
>>>>
>>>> Note carefully here from this link
>>>> <https://apps.dhis2.org/dev/api/schemas/programRuleAction.json>
>>>>  (/api/programRuleActions)
>>>>
>>>>
>>>> Note that "trackedEntityAttribute" is present here.
>>>>
>>>> However from https://apps.dhis2.org/demo/api/schemas.json, it is
>>>> missing.
>>>>
>>>>
>>>> This is one type of thing we would like to pick up.
>>>>
>>>> We are still thinking through the best way to approach the testing and
>>>> your thoughts would be most welcome.
>>>>
>>>> Regards,
>>>> Jason
>>>>
>>>>
>>>
>>
>>
>> --
>> Jason P. Pickering
>> email: jason.p.pickering@xxxxxxxxx
>> tel:+46764147049
>>
>> --
>> Mailing list: https://launchpad.net/~dhis2-devs-core
>> Post to     : dhis2-devs-core@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~dhis2-devs-core
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>


-- 
Jason P. Pickering
email: jason.p.pickering@xxxxxxxxx
tel:+46764147049