dhis2-devs team mailing list archive

[Pascal Brandt <pascal@xxxxxxxxx>]

Thanks Lars.


On 8 August 2013 11:18, Lars Helge Øverland <larshelge@xxxxxxxxx> wrote:

> Agree with what is being said here - from the description it seems
> producing a DXF file and post it to the web api would be easiest.
>
> http://www.dhis2.org/doc/snapshot/en/user/html/ch25s06.html
>
> Lars
>
>
> On Tue, Aug 6, 2013 at 5:14 PM, Pascal Brandt <pascal@xxxxxxxxx> wrote:
>
>> Thanks Bob, we appreciate your input on this.
>>
>> Ciao,
>> Pascal
>>
>>
>> On 6 August 2013 17:11, Bob Jolliffe <bobjolliffe@xxxxxxxxx> wrote:
>>
>>> Hi Pascal
>>>
>>> I think posting dxf to the dataValueSets resource is the simplest option
>>> - as described at
>>> http://www.dhis2.org/doc/snapshot/en/user/html/ch25s06.html.
>>>
>>> While SDMX-HD is still lying unmaintained there is little point in
>>> wasting excessive effort on it, unless it is of course to revamp the
>>> process, but that is not today's proposition.
>>>
>>> In most cases I prefer to use codes rather than uids as the resulting
>>> datavaluesets are a bit easier to read and debug and it is reasonably
>>> possible that you might have the same codes (particularly for facilities)
>>> in both systems.  In which case you use the dataElementIdScheme="code"
>>> and orgUnitIdScheme="code" attributes as described at the bottom of
>>> http://www.dhis2.org/doc/snapshot/en/user/html/ch25s09.html.
>>>
>>> Regards
>>> Bob
>>>
>>>
>>>
>>> On 6 August 2013 15:12, Pascal Brandt <pascal@xxxxxxxxx> wrote:
>>>
>>>> Hi,
>>>>
>>>> We have at least one use case where we'll be pushing data, so it sounds
>>>> like posting SDMX-HD or DXF to the dataValueSets resource is the right
>>>> option for us.
>>>>
>>>> Thanks!
>>>>
>>>> Pascal
>>>>
>>>>
>>>>
>>>> On 6 August 2013 16:02, Jason Pickering <jason.p.pickering@xxxxxxxxx>wrote:
>>>>
>>>>> Hi Pascal,
>>>>>
>>>>> I suppose this really depends on your scenario. The integration engine
>>>>> is useful for the case you describe, in that it can be scheduled to run on
>>>>> a periodic (but regular) basis from within DHIS2. If you need to run things
>>>>> on an ad-hoc basis, it may also be possible to manually trigger the engine,
>>>>> but I am not sure about this. Bob Joliffe would know best.
>>>>>
>>>>>  You could also push data to DHIS2 on a periodic basis as described in
>>>>> (2) of your mail. This is using the API of DHIS2.  DXF is the internal
>>>>> format of DHIS2, while SDMX-HD is a more neutral "standard". So, if your
>>>>> application can already "talk" SDMX-HD, then there is no need to develop an
>>>>> exporter to DXF.
>>>>>
>>>>> (1) describes a cross transformation, which will transform SDMX-HD XML
>>>>> to something which DHIS2 can understand. DXF can of course be imported
>>>>> natively by DHIS2. This is essentially what is described here<http://www.dhis2.org/doc/snapshot/en/user/html/ch26s04.html>,
>>>>> where a custom XSL is used to transform SDMX-HD.
>>>>>
>>>>> So, there are many ways, but it really depends on how you plan to do
>>>>> it.
>>>>>
>>>>> As for the documentation on DXF, there is not really anything at this
>>>>> point, other than the stuff in the manual as far as I know (but
>>>>> contributions are always welcome) :)
>>>>>
>>>>> So, it really comes down to whether you are going to pull or push. If
>>>>> you are pulling data from your legacy app, you will need to use the
>>>>> integration engine if you are doing it on a periodic basis. If you want to
>>>>> import your applications XML, you would need to develop an XSL to transform
>>>>> it, similar to the way it is done with SDMX-HD. Otherwise, you alter your
>>>>> legacy application to export to a format which DHIS2 understands (like DXF
>>>>> or CSV) and you push  this to DHIS2.
>>>>>
>>>>> Thats my understanding anyway, but others may have better info.
>>>>>
>>>>> Regards,
>>>>> Jason
>>>>>
>>>>>
>>>>> On Tue, Aug 6, 2013 at 3:53 PM, Pascal Brandt <pascal@xxxxxxxxx>wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Thanks for the response Paulo. I also like the SDMX-HD and
>>>>>> dataValueSets (DXF) options, but am still interested to hear what the DHIS2
>>>>>> devs think.
>>>>>>
>>>>>> Ciao,
>>>>>> Pascal
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 6 August 2013 15:45, Paulo Grácio <pgracio@xxxxxxxxxxxxxxxxxxxx>wrote:
>>>>>>
>>>>>>> Hi all,****
>>>>>>>
>>>>>>> ** **
>>>>>>>
>>>>>>> Please find my comments below. ****
>>>>>>>
>>>>>>> ** **
>>>>>>>
>>>>>>> Regards,****
>>>>>>>
>>>>>>> Paulo Grácio****
>>>>>>>
>>>>>>> ** **
>>>>>>>
>>>>>>> *From:* Pascal Brandt [mailto:pascal@xxxxxxxxx]
>>>>>>> *Sent:* terça-feira, 6 de Agosto de 2013 14:56
>>>>>>> *To:* Paulo Grácio
>>>>>>> *Cc:* Jason Pickering; DHIS 2 developers; Chris Seebregts; Carl
>>>>>>> Fourie; Ryan Crichton
>>>>>>>
>>>>>>> *Subject:* Re: [Dhis2-devs] Interoperability question****
>>>>>>>
>>>>>>> ** **
>>>>>>>
>>>>>>> Hi all,****
>>>>>>>
>>>>>>> ** **
>>>>>>>
>>>>>>> This thread is important for at least two separate projects that
>>>>>>> Jembi is working on.****
>>>>>>>
>>>>>>> ** **
>>>>>>>
>>>>>>> What we're looking for is a way to automatically import data into
>>>>>>> the DHIS. It's not a once-off import, but it's also not transactional
>>>>>>> (DHIS2 forms aren't being completed on a regular basis). For a once-off, we
>>>>>>> could use CSV or DXF or one of the other options via the web interface and
>>>>>>> for transactional we could use the Web API.
>>>>>>>
>>>>>>> What would be the recommended way to do this periodic bulk data
>>>>>>> load? I see the following options:****
>>>>>>>
>>>>>>>    1. Sending data values using SDMX-HD<http://www.dhis2.org/doc/snapshot/en/user/html/ch25s08.html>
>>>>>>>    ****
>>>>>>>    2. Sending large bulks of data values<http://www.dhis2.org/doc/snapshot/en/user/html/ch25s09.html>
>>>>>>>    ****
>>>>>>>    3. Integration Engine<http://www.dhis2.org/doc/snapshot/en/user/html/ch26.html>
>>>>>>>    ****
>>>>>>>
>>>>>>> *[Paulo Grácio] IMHO option 1 is definitely the most appropriated.
>>>>>>> To have it automatically loaded into DHIS2 the best solution might be to
>>>>>>> implement an adapter that translates data to SDMX-HD before send it to
>>>>>>> DHIS2. Using this approach you also isolate your application specific data
>>>>>>> format from other systems, communication is always done using standard
>>>>>>> format. * *   *
>>>>>>>
>>>>>>> The main point is that needs to be automatic (no user interaction
>>>>>>> with the UI) and we need to be able to initiate the process on an ad-hoc
>>>>>>> bases. Is there a way to load CSV or DXF using the Web API? ****
>>>>>>>
>>>>>>> *[Paulo Grácio] Please have a look at section 25.6. Sending data
>>>>>>> values of DHIS2 User Manual.*
>>>>>>>
>>>>>>> http://apps.dhis2.org/demo/api/dataValueSets****
>>>>>>>
>>>>>>> * *
>>>>>>>
>>>>>>> Also, I'm having some trouble finding the documentation for the DXF
>>>>>>> format, is it available somewhere?****
>>>>>>>
>>>>>>> *[Paulo Grácio] By Morten Olav Hansen “This one is difficult, but
>>>>>>> we will try and provide something. Our api is very much a moving target
>>>>>>> though, ane because of some design choices we cannot just generate this.”
>>>>>>> *
>>>>>>>
>>>>>>> https://bugs.launchpad.net/dhis2/+bug/990783**
>>>>>>>
>>>>>>> ** **
>>>>>>>
>>>>>>> Thanks for your help.****
>>>>>>>
>>>>>>> ** **
>>>>>>>
>>>>>>> Regards,****
>>>>>>>
>>>>>>> Pascal****
>>>>>>>
>>>>>>> ** **
>>>>>>>
>>>>>>> ** **
>>>>>>>
>>>>>>> On 1 August 2013 18:50, Paulo Grácio <pgracio@xxxxxxxxxxxxxxxxxxxx>
>>>>>>> wrote:****
>>>>>>>
>>>>>>> Hi Jason,****
>>>>>>>
>>>>>>>  ****
>>>>>>>
>>>>>>> Basically what we are trying to achieved is a way to simple export
>>>>>>> data from a legacy system into dhis2.  Once this system only
>>>>>>> generates excel files as output we would like to import the generated file
>>>>>>> directly(without modifications) to DHIS2 and have an import summary.
>>>>>>> ****
>>>>>>>
>>>>>>>  ****
>>>>>>>
>>>>>>> We know that we can do it using several different approaches and we
>>>>>>> are considering them J.****
>>>>>>>
>>>>>>>  ****
>>>>>>>
>>>>>>> Regards,****
>>>>>>>
>>>>>>> Paulo Grácio****
>>>>>>>
>>>>>>>  ****
>>>>>>>
>>>>>>> *From:* Jason Pickering [mailto:jason.p.pickering@xxxxxxxxx]
>>>>>>> *Sent:* quinta-feira, 1 de Agosto de 2013 13:26
>>>>>>> *To:* Paulo Grácio****
>>>>>>>
>>>>>>>
>>>>>>> *Cc:* DHIS 2 developers
>>>>>>> *Subject:* Re: [Dhis2-devs] Interoperability question****
>>>>>>>
>>>>>>>  ****
>>>>>>>
>>>>>>> Hi Paulo,****
>>>>>>>
>>>>>>> I am not sure it would be worth it, but looks like it might work.
>>>>>>> Why can't you just simply export CSV instead? Seems that given the
>>>>>>> structure of your data, you could just simply use the CSV importer instead
>>>>>>> of setting up an integration route?****
>>>>>>>
>>>>>>>
>>>>>>> Regards,****
>>>>>>>
>>>>>>> Jason****
>>>>>>>
>>>>>>>  ****
>>>>>>>
>>>>>>>  ****
>>>>>>>
>>>>>>> On Wed, Jul 31, 2013 at 7:03 PM, Paulo Grácio <
>>>>>>> pgracio@xxxxxxxxxxxxxxxxxxxx> wrote:****
>>>>>>>
>>>>>>> Hi all,****
>>>>>>>
>>>>>>>  ****
>>>>>>>
>>>>>>> We are wondering about the best form to import in DHIS2 data from
>>>>>>> legacy applications.****
>>>>>>>
>>>>>>>  ****
>>>>>>>
>>>>>>> In chapter “26.4. Transforming data - a Java route” of the User
>>>>>>> Manual we see that there is a way to do this integration using Java routes,
>>>>>>> but we are having some difficulties in implementing a proof of concept.
>>>>>>> ****
>>>>>>>
>>>>>>>  ****
>>>>>>>
>>>>>>> In our PoC scenario we have a XLS file sent by an external
>>>>>>> application (DataValues.xls in attach). Our idea is to create a Java route
>>>>>>> in order to import all data values that exist in that file.****
>>>>>>>
>>>>>>>  ****
>>>>>>>
>>>>>>> Following the tutorial “Customer 3: Excel via e-mail” presented in
>>>>>>> the Apache Camel page -
>>>>>>> http://camel.apache.org/tutorial-business-partners.html - we have
>>>>>>> been able to transform the XLS file in a list of DataValue objects.
>>>>>>> ****
>>>>>>>
>>>>>>>  ****
>>>>>>>
>>>>>>> Basically, we only had to create a XLSDataIn route and a
>>>>>>> ExcelConverterBean (files in attach). The XLSDataIn takes an InputStream of
>>>>>>> an XLS file and calls the ExcelConverterBean. The ExcelConverterBean
>>>>>>> iterates through the file, and for each data value row, creates a new
>>>>>>> DataValue object and adds that object to a list of DataValue objects.
>>>>>>> ****
>>>>>>>
>>>>>>>  ****
>>>>>>>
>>>>>>> Now we are having some issues because we don’t know the best way to
>>>>>>> insert these data values in DHIS2.****
>>>>>>>
>>>>>>>  ****
>>>>>>>
>>>>>>> The questions we are having are:****
>>>>>>>
>>>>>>> 1.       The approach that we have used to implement our PoC is the
>>>>>>> correct one? I mean, XLS -> List<DataValue> -> import is correct or we
>>>>>>> should use XLS -> CSV -> import?****
>>>>>>>
>>>>>>> 2.       How should we proceed, after the XLS file has been
>>>>>>> processed by the ExcelConverterBean?****
>>>>>>>
>>>>>>>  ****
>>>>>>>
>>>>>>>  ****
>>>>>>>
>>>>>>> Thanks for your support,****
>>>>>>>
>>>>>>> *Paulo Grácio <*%20your-email%20*>*
>>>>>>> *Technical Manager*
>>>>>>> Skype: paulojrgracio****
>>>>>>>
>>>>>>> *Critical Software Mozambique* <http://www.criticalsoftware.co.uk/>
>>>>>>> *Dependable Technologies for Critical Systems*****
>>>>>>>
>>>>>>> Critical Software Mozambique is a subsidiary of Critical Software,
>>>>>>> a CMMI® Level 5 rated Company <http://cmmiinstitute.com/>
>>>>>>> CMMI® is registered in the USPTO by CMU <http://www.cmu.edu/>****
>>>>>>>
>>>>>>> Rua Pereira Marinho, 179
>>>>>>> Bairro de Sommerchield
>>>>>>> Maputo
>>>>>>> Moçambique
>>>>>>> Phone: (+258) 214 951 45
>>>>>>> Fax: (+258) 214 951 46****
>>>>>>>
>>>>>>> DISCLAIMER: This message is confidential and may contain privileged
>>>>>>> information. It is for use only by the people or entities to whom it is
>>>>>>> addressed. If you are not an intended recipient, you should not disclose,
>>>>>>> distribute, copy, print, rely on or otherwise make use of this message. If
>>>>>>> an addressing or transmission error has misdirected it to you we would be
>>>>>>> grateful if you would please notify the sender by return, before deleting
>>>>>>> it from your system.****
>>>>>>>
>>>>>>>  ****
>>>>>>>
>>>>>>>  ****
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Mailing list: https://launchpad.net/~dhis2-devs
>>>>>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>>>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>>>>>> More help   : https://help.launchpad.net/ListHelp****
>>>>>>>
>>>>>>>  ****
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Mailing list: https://launchpad.net/~dhis2-devs
>>>>>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>>>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>>>>>> More help   : https://help.launchpad.net/ListHelp****
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ****
>>>>>>>
>>>>>>> ** **
>>>>>>>
>>>>>>> --
>>>>>>> *Pascal Brandt
>>>>>>> Senio**r Software Developer, Jembi Health Systems |  SOUTH AFRICA
>>>>>>> Mobile: +27 84 827 9342 | Office: +27 21 701 0939 | Skype: psbrandt
>>>>>>> E-mail: pascal@xxxxxxxxx* ****
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Pascal Brandt
>>>>>> Senio**r Software Developer, Jembi Health Systems |  SOUTH AFRICA
>>>>>> Mobile: +27 84 827 9342 | Office: +27 21 701 0939 | Skype: psbrandt
>>>>>> E-mail: pascal@xxxxxxxxx*
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> *Pascal Brandt
>>>> Senio**r Software Developer, Jembi Health Systems |  SOUTH AFRICA
>>>> Mobile: +27 84 827 9342 | Office: +27 21 701 0939 | Skype: psbrandt
>>>> E-mail: pascal@xxxxxxxxx*
>>>>
>>>> _______________________________________________
>>>> Mailing list: https://launchpad.net/~dhis2-devs
>>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>>> More help   : https://help.launchpad.net/ListHelp
>>>>
>>>>
>>>
>>
>>
>> --
>> *Pascal Brandt
>> Senio**r Software Developer, Jembi Health Systems |  SOUTH AFRICA
>> Mobile: +27 84 827 9342 | Office: +27 21 701 0939 | Skype: psbrandt
>> E-mail: pascal@xxxxxxxxx*
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~dhis2-devs
>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~dhis2-devs
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>


-- 
*Pascal Brandt
Senio**r Software Developer, Jembi Health Systems |  SOUTH AFRICA
Mobile: +27 84 827 9342 | Office: +27 21 701 0939 | Skype: psbrandt
E-mail: pascal@xxxxxxxxx*