dhis2-users team mailing list archive

[Aaron C White <aaroncwhite@xxxxxxxxx>]

Isaiah,

I'd love to!

Aaron


--
Aaron C. White
919.522.5585

On Tue, Jan 30, 2018 at 12:25 AM, Isaiah Nyabuto <isaiahnyabuto@xxxxxxxxx>
wrote:

> Hi Aron,
>
> Thanks for sharing your scripts and its good to know we have the same Idea.
>
> I often write several functions to access dhis2 APIs, I realized most of
> this could be standardized into an R package - R dhis2 wrapper. This is
> also something I foresee and I would like to write in the near future. If
> possible, may I ask we work together on that?
>
> Thanks.
>
>
>
> On Mon, Jan 29, 2018 at 9:00 PM, Aaron C White <aaroncwhite@xxxxxxxxx>
> wrote:
>
>> Isaiah,
>>
>> I've found the best way is to convert rows of a CSV into JSON structure
>> and then post to the appropriate endpoint using httr.
>>
>> I've made a slew of helper functions that make it easy for me to do along
>> with some markdown files starting to explain things.  When I started
>> working with dhis2, I was an analyst and most comfortable using dataframes,
>> so this kind of fits that perspective.  I've learned a lot along the way
>> and plan to refactor things at some point.  Eventually, I'd like to make it
>> a real R package- just haven't had the time!
>> https://github.com/aaroncwhite/dhis2R
>>
>> Feel free to PM me for more detail/explanation.
>>
>> Aaron
>>
>>
>> --
>> Aaron C. White
>> 919.522.5585 <(919)%20522-5585>
>>
>> On Mon, Jan 29, 2018 at 12:24 PM, Jason Pickering <
>> jason.p.pickering@xxxxxxxxx> wrote:
>>
>>> Hi Isaiah,
>>>
>>> Yes, R is definitely capable of creating a file which you can import
>>> into DHIS2.
>>>
>>> If you want to use CSV, be sure to have your file with  the "row.names=FALSE
>>> " parameter and be sure to convert all "NA" values to with " na = '' "
>>>
>>> You can also post the file directly to DHIS2. Have a look at
>>> https://github.com/jason-p-pickering/dhis2-data-munging/b
>>> lob/master/bootstrap/bootstrap.Rmd or more details of how to do this in
>>> the section "Loading the data elements"
>>>
>>> Regards,
>>> Jason
>>>
>>>
>>> On Mon, Jan 29, 2018 at 6:04 PM, Timothy Harding <
>>> tharding@xxxxxxxxxxxxxx> wrote:
>>>
>>>> Isaiah:
>>>>
>>>> org_kqmh_list_test.csv:
>>>>
>>>> Has an extra column
>>>>
>>>> [image: Inline image 1]
>>>>
>>>> 1, 2, 3, etc.. should not be in there unless the OUs are named 1, 2, 3.
>>>>
>>>> Probably should be LF instead of CRLF:
>>>> [image: Inline image 2]
>>>> but I'm not sure it makes a difference here.
>>>>
>>>> Also you've got a lot of NA's in there, not sure those are needed
>>>> unless you are setting something to be NA, best just have ,, rather
>>>> than ,NA,
>>>>
>>>> [image: Inline image 3]
>>>>
>>>>
>>>> Good luck!
>>>>
>>>>
>>>> *Timothy Harding*
>>>> Sr. Systems Analyst, BAO Systems
>>>> +1 202-536-1541 | tharding@xxxxxxxxxxxxxx | http://www.baosystems.com |
>>>>  Skype: hardingt@xxxxxxxxx | 2900 K Street, Suite 406, Washington D.C.
>>>> 20007
>>>> <https://maps.google.com/?q=2900+K+Street,+Suite+406,+Washington+D.C.+20007&entry=gmail&source=g>
>>>>
>>>> On Mon, Jan 29, 2018 at 11:30 AM, Isaiah Nyabuto <
>>>> isaiahnyabuto@xxxxxxxxx> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> Thanks Timothy for sharing this. I am trying to import organisation
>>>>> units metadata in csv file with UTF-8 encoding. When I import the file with
>>>>> Metadata import in dhis2, the file is not imported at all and I do get the
>>>>> summary report.
>>>>>
>>>>> I wrote the csv file with R function write.table.
>>>>>
>>>>> Its funny enough that when I write the same file using Excel and save
>>>>> as csv file with UTF-8, it works well.
>>>>>
>>>>> I do not understand what could be the problem here, has anyone tried
>>>>> to write a csv file with an R function that can successfully be imported in
>>>>> the dhis2?
>>>>>
>>>>> Looking for your help. I have attached my csv files in this email.
>>>>>
>>>>> Thanks
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ==========================
>>>>> Isaiah Nyabuto
>>>>> +254729572616 <+254%20729%20572616>
>>>>> Bsc. Health Records Management and Informatics
>>>>>
>>>>> On Jan 17, 2018 17:51, "Thomas Lavreys" <thomaslavreys@xxxxxxxxx>
>>>>> wrote:
>>>>>
>>>>>> Thanks for the in depth info! It will surely help along the way. I’ve
>>>>>> only started with some testers, but I will probably run into some more
>>>>>> issues when I load my real data.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Thanks again!!
>>>>>>
>>>>>>
>>>>>>
>>>>>> Kind regards,
>>>>>>
>>>>>>
>>>>>>
>>>>>> Thomas
>>>>>>
>>>>>>
>>>>>>
>>>>>> *From:* Timothy Harding [mailto:tharding@xxxxxxxxxxxxxx]
>>>>>> *Sent:* Wednesday, January 17, 2018 3:35 PM
>>>>>> *To:* Thomas Lavreys <thomaslavreys@xxxxxxxxx>
>>>>>> *Cc:* Adebusoye Anifalaje <busoye@xxxxxxxxxxxxxx>; dhis2-users <
>>>>>> dhis2-users@xxxxxxxxxxxxxxxxxxx>
>>>>>> *Subject:* Re: [Dhis2-users] Data element not found or not accessible
>>>>>>
>>>>>>
>>>>>>
>>>>>> Hello Thomas!
>>>>>>
>>>>>>
>>>>>>
>>>>>> I am glad you have solved your issue! I noticed a few gotchas so here
>>>>>> are a couple of things to note when importing csv files in DHIS 2:
>>>>>>
>>>>>>
>>>>>>
>>>>>> 1. The first row
>>>>>>
>>>>>>
>>>>>>
>>>>>> dataelement Data element not found or not accessible
>>>>>>
>>>>>>
>>>>>>
>>>>>> [image: Inline image 1]
>>>>>>
>>>>>>
>>>>>>
>>>>>> is a* funny error*, because it is actually telling you something
>>>>>> about a single row, found here in the documentation
>>>>>> <https://docs.dhis2.org/master/en/developer/html/webapi_data_values.html#webapi_data_values_csv>
>>>>>> :
>>>>>>
>>>>>> "dataelement","period","orgunit","catoptcombo","attroptcombo","value","storedby","lastupd","comment"
>>>>>>
>>>>>> DHIS 2 is literally complaining about the fact that it *cannot find
>>>>>> a data element with the uid "dataelement"*, which is what is assumes
>>>>>> you are trying to load in that top row. You can try yourself by loading a
>>>>>> file with only this top line, the error will be the same, 1 ignored row,
>>>>>> data element not found or not accessible. It is safe to ignore this error,
>>>>>> I'd be worried if there was a UID called dataelement, which is actually a
>>>>>> valid UID! Just extremely unlikely to pop up from the random UID generator.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2. Ignored
>>>>>>
>>>>>>
>>>>>>
>>>>>> Unlike the metadata importer, the data importer will report that it
>>>>>> ignored *every row on a dry run*. It can make testing difficult
>>>>>> because you don't know if it will actually import correctly or not as it
>>>>>> reports back that every row was ignored. Two rules of thumb here:
>>>>>>
>>>>>>    1. The actual errored out row will be listed in the "Conflicts"
>>>>>>    section, even in a dry run
>>>>>>    2. Sadly, some errors you can only find on a real run, so always
>>>>>>    upload your files with a comment, so you can find them later via the DB to
>>>>>>    delete them if something goes wrong and all else fails
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> 3. File Type
>>>>>>
>>>>>>
>>>>>>
>>>>>> UTF-8 works well and I use it for all my CSV uploads. Be wary of
>>>>>> EXCEL's CSV saving habits though, especially on a Mac. Your file should be *UTF-8
>>>>>> without a BOM* (a byte order marker, DHIS 2 doesn't _really_ care,
>>>>>> as it will error out on row one anyway (see above)) AND it should be
>>>>>> standard *unix line feeds (LF)*. Excel for the Mac will default to
>>>>>> saving it with circa 1990's CRs only AND a BOM (unless you are editing a
>>>>>> CSV, in which case it will keep the settings it found when it opened it).
>>>>>> Easiest way to check is to grab a copy of Notepad++ (windows) or BBedit
>>>>>> (mac).
>>>>>>
>>>>>>
>>>>>>
>>>>>> BBedit:
>>>>>>
>>>>>> [image: Inline image 2]
>>>>>>
>>>>>>
>>>>>>
>>>>>> and
>>>>>>
>>>>>>
>>>>>>
>>>>>> [image: Inline image 3]
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Notepad++
>>>>>>
>>>>>> [image: Inline image 4]
>>>>>>
>>>>>>
>>>>>>
>>>>>> and
>>>>>>
>>>>>>
>>>>>>
>>>>>> [image: Inline image 5]
>>>>>>
>>>>>>
>>>>>>
>>>>>> Hope this helps!
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> *Timothy Harding*
>>>>>> Sr. Systems Analyst, BAO Systems
>>>>>>
>>>>>> +1 202-536-1541 | tharding@xxxxxxxxxxxxxx | http://www.baosystems.com
>>>>>>  | Skype: hardingt@xxxxxxxxx | 2900 K Street, Suite 406, Washington
>>>>>> D.C. 20007
>>>>>> <https://maps.google.com/?q=2900+K+Street,+Suite+406,+Washington+D.C.+20007&entry=gmail&source=g>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Jan 17, 2018 at 9:11 AM, Thomas Lavreys <
>>>>>> thomaslavreys@xxxxxxxxx> wrote:
>>>>>>
>>>>>> I found the issue. The file was saved as a csv utf-8, which is
>>>>>> different to a normal csv (aparrently).
>>>>>>
>>>>>>
>>>>>>
>>>>>> Thanks for the advice!
>>>>>>
>>>>>>
>>>>>>
>>>>>> Kind regards,
>>>>>>
>>>>>>
>>>>>>
>>>>>> Thomas
>>>>>>
>>>>>>
>>>>>>
>>>>>> *From:* Adebusoye Anifalaje [mailto:busoye@xxxxxxxxxxxxxx]
>>>>>> *Sent:* Wednesday, January 17, 2018 12:31 PM
>>>>>>
>>>>>>
>>>>>> *To:* Thomas Lavreys <thomaslavreys@xxxxxxxxx>
>>>>>> *Cc:* dhis2-users@xxxxxxxxxxxxxxxxxxx
>>>>>> *Subject:* Re: [Dhis2-users] Data element not found or not accessible
>>>>>>
>>>>>>
>>>>>>
>>>>>> Hi Thomas,
>>>>>>
>>>>>>
>>>>>>
>>>>>> I aslo see that you are using version 2.23, it is possible that the
>>>>>> csv column structure is different for this version.  There might be someone
>>>>>> on this forum who is currently working on this version and can provide
>>>>>> input on that.
>>>>>>
>>>>>> - -
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> *Busoye Anifalaje (PhD)*
>>>>>> Director of Services (Principal), BAO Systems
>>>>>>
>>>>>> UK: +44 7901-740-757 <+44%207901%20740757> | US: +1 682-307-0986
>>>>>> <(682)%20307-0986>|
>>>>>>
>>>>>>  busoye@xxxxxxxxxxxxxx | http://www.baosystems.com |
>>>>>>
>>>>>> Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007
>>>>>> <https://maps.google.com/?q=2900+K+Street,+Suite+406,+Washington+D.C.+20007&entry=gmail&source=g>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 17 Jan 2018, at 12:23, Thomas Lavreys <thomaslavreys@xxxxxxxxx>
>>>>>> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>> I tried it anyway, and it doesn’t recognize it:
>>>>>>
>>>>>>
>>>>>>
>>>>>> Import summary
>>>>>>
>>>>>> Import count
>>>>>>
>>>>>> *Type*
>>>>>>
>>>>>> *Count*
>>>>>>
>>>>>> Imported
>>>>>>
>>>>>> 0
>>>>>>
>>>>>> Updated
>>>>>>
>>>>>> 0
>>>>>>
>>>>>> Deleted
>>>>>>
>>>>>> 0
>>>>>>
>>>>>> Ignored
>>>>>>
>>>>>> 13
>>>>>>
>>>>>> Conflicts
>>>>>>
>>>>>> *Element*
>>>>>>
>>>>>> *Description*
>>>>>>
>>>>>> dataelement
>>>>>>
>>>>>> Data element not found or not accessible
>>>>>>
>>>>>> bjDvmb4bfuf
>>>>>>
>>>>>> Category option combo not found or not accessible
>>>>>>
>>>>>>
>>>>>>
>>>>>> So I’m guessing I need to find the catoptioncombo ID somewhere.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>>
>>>>>>
>>>>>> Thomas
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> *From:* Adebusoye Anifalaje [mailto:busoye@xxxxxxxxxxxxxx
>>>>>> <busoye@xxxxxxxxxxxxxx>]
>>>>>> *Sent:* Wednesday, January 17, 2018 12:11 PM
>>>>>> *To:* Thomas Lavreys <thomaslavreys@xxxxxxxxx>
>>>>>> *Cc:* dhis2-users@xxxxxxxxxxxxxxxxxxx
>>>>>> *Subject:* Re: [Dhis2-users] Data element not found or not accessible
>>>>>>
>>>>>>
>>>>>>
>>>>>> Hi Thomas,
>>>>>>
>>>>>>
>>>>>>
>>>>>> Can you try specifying the catoptioncombo even if it is default? This
>>>>>> was an issue with one of the version builds a while ago.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Let’s know if that helps.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Cheers
>>>>>>
>>>>>>
>>>>>>
>>>>>> - -
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> *<image001.png>*
>>>>>>
>>>>>> *Busoye Anifalaje (PhD)*
>>>>>> Director of Services (Principal), BAO Systems
>>>>>>
>>>>>> UK: +44 7901-740-757 <+44%207901%20740757> | US: +1 682-307-0986
>>>>>> <(682)%20307-0986>|
>>>>>>
>>>>>>  busoye@xxxxxxxxxxxxxx | http://www.baosystems.com |
>>>>>>
>>>>>> Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007
>>>>>> <https://maps.google.com/?q=2900+K+Street,+Suite+406,+Washington+D.C.+20007&entry=gmail&source=g>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 17 Jan 2018, at 12:02, Thomas Lavreys <thomaslavreys@xxxxxxxxx>
>>>>>> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>> Dear,
>>>>>>
>>>>>>
>>>>>>
>>>>>> I get the message “Data element not found or not accessible” when
>>>>>> trying to import a csv file. I looked at some old threads, but failed to
>>>>>> find an answer.
>>>>>>
>>>>>> The csv file is attached (with headers, but it gave the same answer
>>>>>> without headers).
>>>>>>
>>>>>> Attached also the API page of the data element, the sharing of the
>>>>>> data element (I have superuser access), and the orgunit page to confirm the
>>>>>> ID. Also a screenshot of the import summary and my DHIS2 instance info.
>>>>>>
>>>>>>
>>>>>>
>>>>>> I guess I might be missing something but I can’t figure out what
>>>>>> exactly.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Thank you in advance for your help.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Kind regards,
>>>>>>
>>>>>>
>>>>>>
>>>>>> Thomas Lavreys
>>>>>>
>>>>>>
>>>>>>
>>>>>> *Thomas Lavreys, MSc, PgC Trop Med*
>>>>>>
>>>>>> *Sustainability Department SOCFINCO*
>>>>>>
>>>>>> *Maisha Consulting bvba*
>>>>>>
>>>>>> *Bisschoppenhof 9*
>>>>>>
>>>>>> *3150 Tildonk*
>>>>>>
>>>>>> *Belgium*
>>>>>>
>>>>>>
>>>>>>
>>>>>> *Tel Socfin: +32-25-080050*
>>>>>>
>>>>>> *Tel Maisha: +32-16-850242 <+32%2016%2085%2002%2042>*
>>>>>>
>>>>>> *Cell phone: +32-474-717641 <+32%20474%2071%2076%2041>*
>>>>>>
>>>>>> *Fax: +32-16-850242 <+32%2016%2085%2002%2042>*
>>>>>>
>>>>>>
>>>>>>
>>>>>> <TEST.csv><data element id.JPG><sharing data element.JPG><orgunit
>>>>>> id.JPG><import csv.JPG><instance info.JPG>_____________________
>>>>>> __________________________
>>>>>> Mailing list: https://launchpad.net/~dhis2-users
>>>>>> Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
>>>>>> Unsubscribe : https://launchpad.net/~dhis2-users
>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Mailing list: https://launchpad.net/~dhis2-users
>>>>>> Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
>>>>>> Unsubscribe : https://launchpad.net/~dhis2-users
>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Mailing list: https://launchpad.net/~dhis2-users
>>>>>> Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
>>>>>> Unsubscribe : https://launchpad.net/~dhis2-users
>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>
>>>>>>
>>>>
>>>
>>>
>>> --
>>> Jason P. Pickering
>>> email: jason.p.pickering@xxxxxxxxx
>>> tel:+46764147049 <+46%2076%20414%2070%2049>
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~dhis2-users
>>> Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~dhis2-users
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>>
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~dhis2-users
>> Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~dhis2-users
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>
>
> --
> ------------------------------------------------------------
> --------------------
> Isaiah Nyabuto
> +254 729 572 616
> Bsc. Health Records Management & Informatics, JKUAT.
>