dhis2-users team mailing list archive

[Isaiah Nyabuto <isaiahnyabuto@xxxxxxxxx>]

Hi Aron,

Thanks for sharing your scripts and its good to know we have the same Idea.

I often write several functions to access dhis2 APIs, I realized most of
this could be standardized into an R package - R dhis2 wrapper. This is
also something I foresee and I would like to write in the near future. If
possible, may I ask we work together on that?

Thanks.



On Mon, Jan 29, 2018 at 9:00 PM, Aaron C White <aaroncwhite@xxxxxxxxx>
wrote:

> Isaiah,
>
> I've found the best way is to convert rows of a CSV into JSON structure
> and then post to the appropriate endpoint using httr.
>
> I've made a slew of helper functions that make it easy for me to do along
> with some markdown files starting to explain things.  When I started
> working with dhis2, I was an analyst and most comfortable using dataframes,
> so this kind of fits that perspective.  I've learned a lot along the way
> and plan to refactor things at some point.  Eventually, I'd like to make it
> a real R package- just haven't had the time!
> https://github.com/aaroncwhite/dhis2R
>
> Feel free to PM me for more detail/explanation.
>
> Aaron
>
>
> --
> Aaron C. White
> 919.522.5585
>
> On Mon, Jan 29, 2018 at 12:24 PM, Jason Pickering <
> jason.p.pickering@xxxxxxxxx> wrote:
>
>> Hi Isaiah,
>>
>> Yes, R is definitely capable of creating a file which you can import into
>> DHIS2.
>>
>> If you want to use CSV, be sure to have your file with  the "row.names=FALSE
>> " parameter and be sure to convert all "NA" values to with " na = '' "
>>
>> You can also post the file directly to DHIS2. Have a look at
>> https://github.com/jason-p-pickering/dhis2-data-munging/b
>> lob/master/bootstrap/bootstrap.Rmd or more details of how to do this in
>> the section "Loading the data elements"
>>
>> Regards,
>> Jason
>>
>>
>> On Mon, Jan 29, 2018 at 6:04 PM, Timothy Harding <tharding@xxxxxxxxxxxxxx
>> > wrote:
>>
>>> Isaiah:
>>>
>>> org_kqmh_list_test.csv:
>>>
>>> Has an extra column
>>>
>>> [image: Inline image 1]
>>>
>>> 1, 2, 3, etc.. should not be in there unless the OUs are named 1, 2, 3.
>>>
>>> Probably should be LF instead of CRLF:
>>> [image: Inline image 2]
>>> but I'm not sure it makes a difference here.
>>>
>>> Also you've got a lot of NA's in there, not sure those are needed unless
>>> you are setting something to be NA, best just have ,, rather than ,NA,
>>>
>>> [image: Inline image 3]
>>>
>>>
>>> Good luck!
>>>
>>>
>>> *Timothy Harding*
>>> Sr. Systems Analyst, BAO Systems
>>> +1 202-536-1541 | tharding@xxxxxxxxxxxxxx | http://www.baosystems.com | Skype:
>>> hardingt@xxxxxxxxx | 2900 K Street, Suite 406, Washington D.C. 20007
>>> <https://maps.google.com/?q=2900+K+Street,+Suite+406,+Washington+D.C.+20007&entry=gmail&source=g>
>>>
>>> On Mon, Jan 29, 2018 at 11:30 AM, Isaiah Nyabuto <
>>> isaiahnyabuto@xxxxxxxxx> wrote:
>>>
>>>> Hi all,
>>>>
>>>> Thanks Timothy for sharing this. I am trying to import organisation
>>>> units metadata in csv file with UTF-8 encoding. When I import the file with
>>>> Metadata import in dhis2, the file is not imported at all and I do get the
>>>> summary report.
>>>>
>>>> I wrote the csv file with R function write.table.
>>>>
>>>> Its funny enough that when I write the same file using Excel and save
>>>> as csv file with UTF-8, it works well.
>>>>
>>>> I do not understand what could be the problem here, has anyone tried to
>>>> write a csv file with an R function that can successfully be imported in
>>>> the dhis2?
>>>>
>>>> Looking for your help. I have attached my csv files in this email.
>>>>
>>>> Thanks
>>>>
>>>>
>>>>
>>>>
>>>> ==========================
>>>> Isaiah Nyabuto
>>>> +254729572616 <+254%20729%20572616>
>>>> Bsc. Health Records Management and Informatics
>>>>
>>>> On Jan 17, 2018 17:51, "Thomas Lavreys" <thomaslavreys@xxxxxxxxx>
>>>> wrote:
>>>>
>>>>> Thanks for the in depth info! It will surely help along the way. I’ve
>>>>> only started with some testers, but I will probably run into some more
>>>>> issues when I load my real data.
>>>>>
>>>>>
>>>>>
>>>>> Thanks again!!
>>>>>
>>>>>
>>>>>
>>>>> Kind regards,
>>>>>
>>>>>
>>>>>
>>>>> Thomas
>>>>>
>>>>>
>>>>>
>>>>> *From:* Timothy Harding [mailto:tharding@xxxxxxxxxxxxxx]
>>>>> *Sent:* Wednesday, January 17, 2018 3:35 PM
>>>>> *To:* Thomas Lavreys <thomaslavreys@xxxxxxxxx>
>>>>> *Cc:* Adebusoye Anifalaje <busoye@xxxxxxxxxxxxxx>; dhis2-users <
>>>>> dhis2-users@xxxxxxxxxxxxxxxxxxx>
>>>>> *Subject:* Re: [Dhis2-users] Data element not found or not accessible
>>>>>
>>>>>
>>>>>
>>>>> Hello Thomas!
>>>>>
>>>>>
>>>>>
>>>>> I am glad you have solved your issue! I noticed a few gotchas so here
>>>>> are a couple of things to note when importing csv files in DHIS 2:
>>>>>
>>>>>
>>>>>
>>>>> 1. The first row
>>>>>
>>>>>
>>>>>
>>>>> dataelement Data element not found or not accessible
>>>>>
>>>>>
>>>>>
>>>>> [image: Inline image 1]
>>>>>
>>>>>
>>>>>
>>>>> is a* funny error*, because it is actually telling you something
>>>>> about a single row, found here in the documentation
>>>>> <https://docs.dhis2.org/master/en/developer/html/webapi_data_values.html#webapi_data_values_csv>
>>>>> :
>>>>>
>>>>> "dataelement","period","orgunit","catoptcombo","attroptcombo","value","storedby","lastupd","comment"
>>>>>
>>>>> DHIS 2 is literally complaining about the fact that it *cannot find a
>>>>> data element with the uid "dataelement"*, which is what is assumes
>>>>> you are trying to load in that top row. You can try yourself by loading a
>>>>> file with only this top line, the error will be the same, 1 ignored row,
>>>>> data element not found or not accessible. It is safe to ignore this error,
>>>>> I'd be worried if there was a UID called dataelement, which is actually a
>>>>> valid UID! Just extremely unlikely to pop up from the random UID generator.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> 2. Ignored
>>>>>
>>>>>
>>>>>
>>>>> Unlike the metadata importer, the data importer will report that it
>>>>> ignored *every row on a dry run*. It can make testing difficult
>>>>> because you don't know if it will actually import correctly or not as it
>>>>> reports back that every row was ignored. Two rules of thumb here:
>>>>>
>>>>>    1. The actual errored out row will be listed in the "Conflicts"
>>>>>    section, even in a dry run
>>>>>    2. Sadly, some errors you can only find on a real run, so always
>>>>>    upload your files with a comment, so you can find them later via the DB to
>>>>>    delete them if something goes wrong and all else fails
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> 3. File Type
>>>>>
>>>>>
>>>>>
>>>>> UTF-8 works well and I use it for all my CSV uploads. Be wary of
>>>>> EXCEL's CSV saving habits though, especially on a Mac. Your file should be *UTF-8
>>>>> without a BOM* (a byte order marker, DHIS 2 doesn't _really_ care, as
>>>>> it will error out on row one anyway (see above)) AND it should be standard *unix
>>>>> line feeds (LF)*. Excel for the Mac will default to saving it with
>>>>> circa 1990's CRs only AND a BOM (unless you are editing a CSV, in which
>>>>> case it will keep the settings it found when it opened it). Easiest way to
>>>>> check is to grab a copy of Notepad++ (windows) or BBedit (mac).
>>>>>
>>>>>
>>>>>
>>>>> BBedit:
>>>>>
>>>>> [image: Inline image 2]
>>>>>
>>>>>
>>>>>
>>>>> and
>>>>>
>>>>>
>>>>>
>>>>> [image: Inline image 3]
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Notepad++
>>>>>
>>>>> [image: Inline image 4]
>>>>>
>>>>>
>>>>>
>>>>> and
>>>>>
>>>>>
>>>>>
>>>>> [image: Inline image 5]
>>>>>
>>>>>
>>>>>
>>>>> Hope this helps!
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> *Timothy Harding*
>>>>> Sr. Systems Analyst, BAO Systems
>>>>>
>>>>> +1 202-536-1541 | tharding@xxxxxxxxxxxxxx | http://www.baosystems.com
>>>>> | Skype: hardingt@xxxxxxxxx | 2900 K Street, Suite 406, Washington
>>>>> D.C. 20007
>>>>> <https://maps.google.com/?q=2900+K+Street,+Suite+406,+Washington+D.C.+20007&entry=gmail&source=g>
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Jan 17, 2018 at 9:11 AM, Thomas Lavreys <
>>>>> thomaslavreys@xxxxxxxxx> wrote:
>>>>>
>>>>> I found the issue. The file was saved as a csv utf-8, which is
>>>>> different to a normal csv (aparrently).
>>>>>
>>>>>
>>>>>
>>>>> Thanks for the advice!
>>>>>
>>>>>
>>>>>
>>>>> Kind regards,
>>>>>
>>>>>
>>>>>
>>>>> Thomas
>>>>>
>>>>>
>>>>>
>>>>> *From:* Adebusoye Anifalaje [mailto:busoye@xxxxxxxxxxxxxx]
>>>>> *Sent:* Wednesday, January 17, 2018 12:31 PM
>>>>>
>>>>>
>>>>> *To:* Thomas Lavreys <thomaslavreys@xxxxxxxxx>
>>>>> *Cc:* dhis2-users@xxxxxxxxxxxxxxxxxxx
>>>>> *Subject:* Re: [Dhis2-users] Data element not found or not accessible
>>>>>
>>>>>
>>>>>
>>>>> Hi Thomas,
>>>>>
>>>>>
>>>>>
>>>>> I aslo see that you are using version 2.23, it is possible that the
>>>>> csv column structure is different for this version.  There might be someone
>>>>> on this forum who is currently working on this version and can provide
>>>>> input on that.
>>>>>
>>>>> - -
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> *Busoye Anifalaje (PhD)*
>>>>> Director of Services (Principal), BAO Systems
>>>>>
>>>>> UK: +44 7901-740-757 <+44%207901%20740757> | US: +1 682-307-0986
>>>>> <(682)%20307-0986>|
>>>>>
>>>>>  busoye@xxxxxxxxxxxxxx | http://www.baosystems.com |
>>>>>
>>>>> Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007
>>>>> <https://maps.google.com/?q=2900+K+Street,+Suite+406,+Washington+D.C.+20007&entry=gmail&source=g>
>>>>>
>>>>>
>>>>>
>>>>> On 17 Jan 2018, at 12:23, Thomas Lavreys <thomaslavreys@xxxxxxxxx>
>>>>> wrote:
>>>>>
>>>>>
>>>>>
>>>>> I tried it anyway, and it doesn’t recognize it:
>>>>>
>>>>>
>>>>>
>>>>> Import summary
>>>>>
>>>>> Import count
>>>>>
>>>>> *Type*
>>>>>
>>>>> *Count*
>>>>>
>>>>> Imported
>>>>>
>>>>> 0
>>>>>
>>>>> Updated
>>>>>
>>>>> 0
>>>>>
>>>>> Deleted
>>>>>
>>>>> 0
>>>>>
>>>>> Ignored
>>>>>
>>>>> 13
>>>>>
>>>>> Conflicts
>>>>>
>>>>> *Element*
>>>>>
>>>>> *Description*
>>>>>
>>>>> dataelement
>>>>>
>>>>> Data element not found or not accessible
>>>>>
>>>>> bjDvmb4bfuf
>>>>>
>>>>> Category option combo not found or not accessible
>>>>>
>>>>>
>>>>>
>>>>> So I’m guessing I need to find the catoptioncombo ID somewhere.
>>>>>
>>>>>
>>>>>
>>>>> Regards,
>>>>>
>>>>>
>>>>>
>>>>> Thomas
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> *From:* Adebusoye Anifalaje [mailto:busoye@xxxxxxxxxxxxxx
>>>>> <busoye@xxxxxxxxxxxxxx>]
>>>>> *Sent:* Wednesday, January 17, 2018 12:11 PM
>>>>> *To:* Thomas Lavreys <thomaslavreys@xxxxxxxxx>
>>>>> *Cc:* dhis2-users@xxxxxxxxxxxxxxxxxxx
>>>>> *Subject:* Re: [Dhis2-users] Data element not found or not accessible
>>>>>
>>>>>
>>>>>
>>>>> Hi Thomas,
>>>>>
>>>>>
>>>>>
>>>>> Can you try specifying the catoptioncombo even if it is default? This
>>>>> was an issue with one of the version builds a while ago.
>>>>>
>>>>>
>>>>>
>>>>> Let’s know if that helps.
>>>>>
>>>>>
>>>>>
>>>>> Cheers
>>>>>
>>>>>
>>>>>
>>>>> - -
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> *<image001.png>*
>>>>>
>>>>> *Busoye Anifalaje (PhD)*
>>>>> Director of Services (Principal), BAO Systems
>>>>>
>>>>> UK: +44 7901-740-757 <+44%207901%20740757> | US: +1 682-307-0986
>>>>> <(682)%20307-0986>|
>>>>>
>>>>>  busoye@xxxxxxxxxxxxxx | http://www.baosystems.com |
>>>>>
>>>>> Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007
>>>>> <https://maps.google.com/?q=2900+K+Street,+Suite+406,+Washington+D.C.+20007&entry=gmail&source=g>
>>>>>
>>>>>
>>>>>
>>>>> On 17 Jan 2018, at 12:02, Thomas Lavreys <thomaslavreys@xxxxxxxxx>
>>>>> wrote:
>>>>>
>>>>>
>>>>>
>>>>> Dear,
>>>>>
>>>>>
>>>>>
>>>>> I get the message “Data element not found or not accessible” when
>>>>> trying to import a csv file. I looked at some old threads, but failed to
>>>>> find an answer.
>>>>>
>>>>> The csv file is attached (with headers, but it gave the same answer
>>>>> without headers).
>>>>>
>>>>> Attached also the API page of the data element, the sharing of the
>>>>> data element (I have superuser access), and the orgunit page to confirm the
>>>>> ID. Also a screenshot of the import summary and my DHIS2 instance info.
>>>>>
>>>>>
>>>>>
>>>>> I guess I might be missing something but I can’t figure out what
>>>>> exactly.
>>>>>
>>>>>
>>>>>
>>>>> Thank you in advance for your help.
>>>>>
>>>>>
>>>>>
>>>>> Kind regards,
>>>>>
>>>>>
>>>>>
>>>>> Thomas Lavreys
>>>>>
>>>>>
>>>>>
>>>>> *Thomas Lavreys, MSc, PgC Trop Med*
>>>>>
>>>>> *Sustainability Department SOCFINCO*
>>>>>
>>>>> *Maisha Consulting bvba*
>>>>>
>>>>> *Bisschoppenhof 9*
>>>>>
>>>>> *3150 Tildonk*
>>>>>
>>>>> *Belgium*
>>>>>
>>>>>
>>>>>
>>>>> *Tel Socfin: +32-25-080050*
>>>>>
>>>>> *Tel Maisha: +32-16-850242 <+32%2016%2085%2002%2042>*
>>>>>
>>>>> *Cell phone: +32-474-717641 <+32%20474%2071%2076%2041>*
>>>>>
>>>>> *Fax: +32-16-850242 <+32%2016%2085%2002%2042>*
>>>>>
>>>>>
>>>>>
>>>>> <TEST.csv><data element id.JPG><sharing data element.JPG><orgunit
>>>>> id.JPG><import csv.JPG><instance info.JPG>_____________________
>>>>> __________________________
>>>>> Mailing list: https://launchpad.net/~dhis2-users
>>>>> Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
>>>>> Unsubscribe : https://launchpad.net/~dhis2-users
>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Mailing list: https://launchpad.net/~dhis2-users
>>>>> Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
>>>>> Unsubscribe : https://launchpad.net/~dhis2-users
>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Mailing list: https://launchpad.net/~dhis2-users
>>>>> Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
>>>>> Unsubscribe : https://launchpad.net/~dhis2-users
>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>
>>>>>
>>>
>>
>>
>> --
>> Jason P. Pickering
>> email: jason.p.pickering@xxxxxxxxx
>> tel:+46764147049 <+46%2076%20414%2070%2049>
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~dhis2-users
>> Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~dhis2-users
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~dhis2-users
> Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~dhis2-users
> More help   : https://help.launchpad.net/ListHelp
>
>


-- 
--------------------------------------------------------------------------------
Isaiah Nyabuto
+254 729 572 616
Bsc. Health Records Management & Informatics, JKUAT.