dhis2-users team mailing list archive

[Knut Staring <knutst@xxxxxxxxx>]

Hi Shinichi,

You are completely right that it is important to be able to easily
import the orgunit hierarchy and historical data into DHIS2. The
general term for such operations is Extract, Transform and Load (ETL),
for which there are a number of powerful general tools available. Here
are two:
http://kettle.pentaho.com/
http://www.talend.com/index.php
I have personally also used ODBC connections in Access to transform
and load Excel data into Postgres directly.

A disadvantage of going directly into the database is that one looses
the validations that the DHIS2 import mechanism can perform (in
addition to the database constraints), and of course it is burdensome
to learn to use the above tools. DHIS2 has the capability to import
XML files, and thus also modern Excel files (.xlsx), and this should
probably be the common way.However, given the many different types of
data and vast range of potential data sources, we will probably never
have a simple wizard that does it all (and there is no sense in trying
to replicate Kettle or Talend).

Still, we have been thinking about defining a suitable standard format
for such import, as well as providing the user with some assistance in
transforming the data into shape for loading. We have also done some
work on extraction of data automatically from a large number of Excel
files (e.g. one or more per district) using Python. Work remains
before this work will reach a stage where it is robust and generic
enough - but I think working with the Kenyan data could help move this
process forward. I will refresh my memory on the status of the Python
work and get back to you.

Knut


On Mon, Nov 8, 2010 at 8:42 AM, Shinichi Suzuki <shin461@xxxxxxxxx> wrote:
> Hello, Knut and dhis2_devs, users
>
> Do you have a data import tool written by Python and available to use?
> I need the tools for the following area.
> 1. Existing HIS EXCEL data import to DHIS2. We want to load about several
> years historical data which has been corrected into EXCEL files.
> 2. Import from Master Facility List (Web based Facility Master maintenance
> system) to DHIS2 for facility data synchronization. This MFL has a
> capability to export the data into EXCEL file. But MFL does not have a
> "Short name" and need to add the data manually. Then want to import it.
>
> If you do not have it or not available to use it, please advise me if
> possible.
> I believe this import capability is important for the new user of DHIS and
> also could use inter system connection more widely.
>
> Best regards,
>  Shinichi Suzuki
> -----------------------------------------------------
> Shinichi Suzuki
> MIS Division MOPHS: LG37 AFYA House, Nairobi, KENYA
> E-Mail: shin461@xxxxxxxxx
> Phone: 0712-754-963
> (JICA Senior Volunteer 21-4)
> -----------------------------------------------------
>
>
>
>



-- 
Cheers,
Knut Staring