dhis2-devs team mailing list archive

[Pascal Brandt <pascal@xxxxxxxxx>]

Hi,

We have at least one use case where we'll be pushing data, so it sounds
like posting SDMX-HD or DXF to the dataValueSets resource is the right
option for us.

Thanks!

Pascal



On 6 August 2013 16:02, Jason Pickering <jason.p.pickering@xxxxxxxxx> wrote:

> Hi Pascal,
>
> I suppose this really depends on your scenario. The integration engine is
> useful for the case you describe, in that it can be scheduled to run on a
> periodic (but regular) basis from within DHIS2. If you need to run things
> on an ad-hoc basis, it may also be possible to manually trigger the engine,
> but I am not sure about this. Bob Joliffe would know best.
>
>  You could also push data to DHIS2 on a periodic basis as described in (2)
> of your mail. This is using the API of DHIS2.  DXF is the internal format
> of DHIS2, while SDMX-HD is a more neutral "standard". So, if your
> application can already "talk" SDMX-HD, then there is no need to develop an
> exporter to DXF.
>
> (1) describes a cross transformation, which will transform SDMX-HD XML to
> something which DHIS2 can understand. DXF can of course be imported
> natively by DHIS2. This is essentially what is described here<http://www.dhis2.org/doc/snapshot/en/user/html/ch26s04.html>,
> where a custom XSL is used to transform SDMX-HD.
>
> So, there are many ways, but it really depends on how you plan to do it.
>
> As for the documentation on DXF, there is not really anything at this
> point, other than the stuff in the manual as far as I know (but
> contributions are always welcome) :)
>
> So, it really comes down to whether you are going to pull or push. If you
> are pulling data from your legacy app, you will need to use the integration
> engine if you are doing it on a periodic basis. If you want to import your
> applications XML, you would need to develop an XSL to transform it, similar
> to the way it is done with SDMX-HD. Otherwise, you alter your legacy
> application to export to a format which DHIS2 understands (like DXF or CSV)
> and you push  this to DHIS2.
>
> Thats my understanding anyway, but others may have better info.
>
> Regards,
> Jason
>
>
> On Tue, Aug 6, 2013 at 3:53 PM, Pascal Brandt <pascal@xxxxxxxxx> wrote:
>
>> Hi,
>>
>> Thanks for the response Paulo. I also like the SDMX-HD and dataValueSets
>> (DXF) options, but am still interested to hear what the DHIS2 devs think.
>>
>> Ciao,
>> Pascal
>>
>>
>>
>> On 6 August 2013 15:45, Paulo Grácio <pgracio@xxxxxxxxxxxxxxxxxxxx>wrote:
>>
>>> Hi all,****
>>>
>>> ** **
>>>
>>> Please find my comments below. ****
>>>
>>> ** **
>>>
>>> Regards,****
>>>
>>> Paulo Grácio****
>>>
>>> ** **
>>>
>>> *From:* Pascal Brandt [mailto:pascal@xxxxxxxxx]
>>> *Sent:* terça-feira, 6 de Agosto de 2013 14:56
>>> *To:* Paulo Grácio
>>> *Cc:* Jason Pickering; DHIS 2 developers; Chris Seebregts; Carl Fourie;
>>> Ryan Crichton
>>>
>>> *Subject:* Re: [Dhis2-devs] Interoperability question****
>>>
>>> ** **
>>>
>>> Hi all,****
>>>
>>> ** **
>>>
>>> This thread is important for at least two separate projects that Jembi
>>> is working on.****
>>>
>>> ** **
>>>
>>> What we're looking for is a way to automatically import data into the
>>> DHIS. It's not a once-off import, but it's also not transactional (DHIS2
>>> forms aren't being completed on a regular basis). For a once-off, we could
>>> use CSV or DXF or one of the other options via the web interface and for
>>> transactional we could use the Web API.
>>>
>>> What would be the recommended way to do this periodic bulk data load? I
>>> see the following options:****
>>>
>>>    1. Sending data values using SDMX-HD<http://www.dhis2.org/doc/snapshot/en/user/html/ch25s08.html>
>>>    ****
>>>    2. Sending large bulks of data values<http://www.dhis2.org/doc/snapshot/en/user/html/ch25s09.html>
>>>    ****
>>>    3. Integration Engine<http://www.dhis2.org/doc/snapshot/en/user/html/ch26.html>
>>>    ****
>>>
>>> *[Paulo Grácio] IMHO option 1 is definitely the most appropriated. To
>>> have it automatically loaded into DHIS2 the best solution might be to
>>> implement an adapter that translates data to SDMX-HD before send it to
>>> DHIS2. Using this approach you also isolate your application specific data
>>> format from other systems, communication is always done using standard
>>> format. * *   *
>>>
>>> The main point is that needs to be automatic (no user interaction with
>>> the UI) and we need to be able to initiate the process on an ad-hoc bases. Is
>>> there a way to load CSV or DXF using the Web API? ****
>>>
>>> *[Paulo Grácio] Please have a look at section 25.6. Sending data values
>>> of DHIS2 User Manual.*
>>>
>>> http://apps.dhis2.org/demo/api/dataValueSets****
>>>
>>> * *
>>>
>>> Also, I'm having some trouble finding the documentation for the DXF
>>> format, is it available somewhere?****
>>>
>>> *[Paulo Grácio] By Morten Olav Hansen “This one is difficult, but we
>>> will try and provide something. Our api is very much a moving target
>>> though, ane because of some design choices we cannot just generate this.”
>>> *
>>>
>>> https://bugs.launchpad.net/dhis2/+bug/990783**
>>>
>>> ** **
>>>
>>> Thanks for your help.****
>>>
>>> ** **
>>>
>>> Regards,****
>>>
>>> Pascal****
>>>
>>> ** **
>>>
>>> ** **
>>>
>>> On 1 August 2013 18:50, Paulo Grácio <pgracio@xxxxxxxxxxxxxxxxxxxx>
>>> wrote:****
>>>
>>> Hi Jason,****
>>>
>>>  ****
>>>
>>> Basically what we are trying to achieved is a way to simple export data
>>> from a legacy system into dhis2.  Once this system only generates excel
>>> files as output we would like to import the generated file directly(without
>>> modifications) to DHIS2 and have an import summary. ****
>>>
>>>  ****
>>>
>>> We know that we can do it using several different approaches and we are
>>> considering them J.****
>>>
>>>  ****
>>>
>>> Regards,****
>>>
>>> Paulo Grácio****
>>>
>>>  ****
>>>
>>> *From:* Jason Pickering [mailto:jason.p.pickering@xxxxxxxxx]
>>> *Sent:* quinta-feira, 1 de Agosto de 2013 13:26
>>> *To:* Paulo Grácio****
>>>
>>>
>>> *Cc:* DHIS 2 developers
>>> *Subject:* Re: [Dhis2-devs] Interoperability question****
>>>
>>>  ****
>>>
>>> Hi Paulo,****
>>>
>>> I am not sure it would be worth it, but looks like it might work. Why
>>> can't you just simply export CSV instead? Seems that given the structure of
>>> your data, you could just simply use the CSV importer instead of setting up
>>> an integration route?****
>>>
>>>
>>> Regards,****
>>>
>>> Jason****
>>>
>>>  ****
>>>
>>>  ****
>>>
>>> On Wed, Jul 31, 2013 at 7:03 PM, Paulo Grácio <
>>> pgracio@xxxxxxxxxxxxxxxxxxxx> wrote:****
>>>
>>> Hi all,****
>>>
>>>  ****
>>>
>>> We are wondering about the best form to import in DHIS2 data from legacy
>>> applications.****
>>>
>>>  ****
>>>
>>> In chapter “26.4. Transforming data - a Java route” of the User Manual
>>> we see that there is a way to do this integration using Java routes, but we
>>> are having some difficulties in implementing a proof of concept.****
>>>
>>>  ****
>>>
>>> In our PoC scenario we have a XLS file sent by an external application
>>> (DataValues.xls in attach). Our idea is to create a Java route in order to
>>> import all data values that exist in that file.****
>>>
>>>  ****
>>>
>>> Following the tutorial “Customer 3: Excel via e-mail” presented in the
>>> Apache Camel page -
>>> http://camel.apache.org/tutorial-business-partners.html - we have been
>>> able to transform the XLS file in a list of DataValue objects. ****
>>>
>>>  ****
>>>
>>> Basically, we only had to create a XLSDataIn route and a
>>> ExcelConverterBean (files in attach). The XLSDataIn takes an InputStream of
>>> an XLS file and calls the ExcelConverterBean. The ExcelConverterBean
>>> iterates through the file, and for each data value row, creates a new
>>> DataValue object and adds that object to a list of DataValue objects.***
>>> *
>>>
>>>  ****
>>>
>>> Now we are having some issues because we don’t know the best way to
>>> insert these data values in DHIS2.****
>>>
>>>  ****
>>>
>>> The questions we are having are:****
>>>
>>> 1.       The approach that we have used to implement our PoC is the
>>> correct one? I mean, XLS -> List<DataValue> -> import is correct or we
>>> should use XLS -> CSV -> import?****
>>>
>>> 2.       How should we proceed, after the XLS file has been processed
>>> by the ExcelConverterBean?****
>>>
>>>  ****
>>>
>>>  ****
>>>
>>> Thanks for your support,****
>>>
>>> *Paulo Grácio <*%20your-email%20*>*
>>> *Technical Manager*
>>> Skype: paulojrgracio****
>>>
>>> *Critical Software Mozambique* <http://www.criticalsoftware.co.uk/>
>>> *Dependable Technologies for Critical Systems*****
>>>
>>> Critical Software Mozambique is a subsidiary of Critical Software, a
>>> CMMI® Level 5 rated Company <http://cmmiinstitute.com/>
>>> CMMI® is registered in the USPTO by CMU <http://www.cmu.edu/>****
>>>
>>> Rua Pereira Marinho, 179
>>> Bairro de Sommerchield
>>> Maputo
>>> Moçambique
>>> Phone: (+258) 214 951 45
>>> Fax: (+258) 214 951 46****
>>>
>>> DISCLAIMER: This message is confidential and may contain privileged
>>> information. It is for use only by the people or entities to whom it is
>>> addressed. If you are not an intended recipient, you should not disclose,
>>> distribute, copy, print, rely on or otherwise make use of this message. If
>>> an addressing or transmission error has misdirected it to you we would be
>>> grateful if you would please notify the sender by return, before deleting
>>> it from your system.****
>>>
>>>  ****
>>>
>>>  ****
>>>
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~dhis2-devs
>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>> More help   : https://help.launchpad.net/ListHelp****
>>>
>>>  ****
>>>
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~dhis2-devs
>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>> More help   : https://help.launchpad.net/ListHelp****
>>>
>>>
>>>
>>> ****
>>>
>>> ** **
>>>
>>> --
>>> *Pascal Brandt
>>> Senio**r Software Developer, Jembi Health Systems |  SOUTH AFRICA
>>> Mobile: +27 84 827 9342 | Office: +27 21 701 0939 | Skype: psbrandt
>>> E-mail: pascal@xxxxxxxxx* ****
>>>
>>
>>
>>
>> --
>> *Pascal Brandt
>> Senio**r Software Developer, Jembi Health Systems |  SOUTH AFRICA
>> Mobile: +27 84 827 9342 | Office: +27 21 701 0939 | Skype: psbrandt
>> E-mail: pascal@xxxxxxxxx*
>>
>
>


-- 
*Pascal Brandt
Senio**r Software Developer, Jembi Health Systems |  SOUTH AFRICA
Mobile: +27 84 827 9342 | Office: +27 21 701 0939 | Skype: psbrandt
E-mail: pascal@xxxxxxxxx*