dhis2-users team mailing list archive

[Isaiah Nyabuto <isaiahnyabuto@xxxxxxxxx>]

Hi Aaron,

Thats perfect. Lets keep in touch for a skype session soon. my skype id is;
Isaiah Nyabuto

Thanks.



On Tue, Jan 30, 2018 at 11:41 PM, Aaron C White <aaroncwhite@xxxxxxxxx>
wrote:

> Isaiah,
>
> I'd love to!
>
> Aaron
>
>
> --
> Aaron C. White
> 919.522.5585
>
> On Tue, Jan 30, 2018 at 12:25 AM, Isaiah Nyabuto <isaiahnyabuto@xxxxxxxxx>
> wrote:
>
>> Hi Aron,
>>
>> Thanks for sharing your scripts and its good to know we have the same
>> Idea.
>>
>> I often write several functions to access dhis2 APIs, I realized most of
>> this could be standardized into an R package - R dhis2 wrapper. This is
>> also something I foresee and I would like to write in the near future. If
>> possible, may I ask we work together on that?
>>
>> Thanks.
>>
>>
>>
>> On Mon, Jan 29, 2018 at 9:00 PM, Aaron C White <aaroncwhite@xxxxxxxxx>
>> wrote:
>>
>>> Isaiah,
>>>
>>> I've found the best way is to convert rows of a CSV into JSON structure
>>> and then post to the appropriate endpoint using httr.
>>>
>>> I've made a slew of helper functions that make it easy for me to do
>>> along with some markdown files starting to explain things.  When I started
>>> working with dhis2, I was an analyst and most comfortable using dataframes,
>>> so this kind of fits that perspective.  I've learned a lot along the way
>>> and plan to refactor things at some point.  Eventually, I'd like to make it
>>> a real R package- just haven't had the time!
>>> https://github.com/aaroncwhite/dhis2R
>>>
>>> Feel free to PM me for more detail/explanation.
>>>
>>> Aaron
>>>
>>>
>>> --
>>> Aaron C. White
>>> 919.522.5585 <(919)%20522-5585>
>>>
>>> On Mon, Jan 29, 2018 at 12:24 PM, Jason Pickering <
>>> jason.p.pickering@xxxxxxxxx> wrote:
>>>
>>>> Hi Isaiah,
>>>>
>>>> Yes, R is definitely capable of creating a file which you can import
>>>> into DHIS2.
>>>>
>>>> If you want to use CSV, be sure to have your file with  the "row.names=FALSE
>>>> " parameter and be sure to convert all "NA" values to with " na = '' "
>>>>
>>>> You can also post the file directly to DHIS2. Have a look at
>>>> https://github.com/jason-p-pickering/dhis2-data-munging/b
>>>> lob/master/bootstrap/bootstrap.Rmd or more details of how to do this
>>>> in the section "Loading the data elements"
>>>>
>>>> Regards,
>>>> Jason
>>>>
>>>>
>>>> On Mon, Jan 29, 2018 at 6:04 PM, Timothy Harding <
>>>> tharding@xxxxxxxxxxxxxx> wrote:
>>>>
>>>>> Isaiah:
>>>>>
>>>>> org_kqmh_list_test.csv:
>>>>>
>>>>> Has an extra column
>>>>>
>>>>> [image: Inline image 1]
>>>>>
>>>>> 1, 2, 3, etc.. should not be in there unless the OUs are named 1, 2, 3.
>>>>>
>>>>> Probably should be LF instead of CRLF:
>>>>> [image: Inline image 2]
>>>>> but I'm not sure it makes a difference here.
>>>>>
>>>>> Also you've got a lot of NA's in there, not sure those are needed
>>>>> unless you are setting something to be NA, best just have ,, rather
>>>>> than ,NA,
>>>>>
>>>>> [image: Inline image 3]
>>>>>
>>>>>
>>>>> Good luck!
>>>>>
>>>>>
>>>>> *Timothy Harding*
>>>>> Sr. Systems Analyst, BAO Systems
>>>>> +1 202-536-1541 | tharding@xxxxxxxxxxxxxx | http://www.baosystems.com
>>>>> | Skype: hardingt@xxxxxxxxx | 2900 K Street, Suite 406, Washington
>>>>> D.C. 20007
>>>>> <https://maps.google.com/?q=2900+K+Street,+Suite+406,+Washington+D.C.+20007&entry=gmail&source=g>
>>>>>
>>>>> On Mon, Jan 29, 2018 at 11:30 AM, Isaiah Nyabuto <
>>>>> isaiahnyabuto@xxxxxxxxx> wrote:
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> Thanks Timothy for sharing this. I am trying to import organisation
>>>>>> units metadata in csv file with UTF-8 encoding. When I import the file with
>>>>>> Metadata import in dhis2, the file is not imported at all and I do get the
>>>>>> summary report.
>>>>>>
>>>>>> I wrote the csv file with R function write.table.
>>>>>>
>>>>>> Its funny enough that when I write the same file using Excel and save
>>>>>> as csv file with UTF-8, it works well.
>>>>>>
>>>>>> I do not understand what could be the problem here, has anyone tried
>>>>>> to write a csv file with an R function that can successfully be imported in
>>>>>> the dhis2?
>>>>>>
>>>>>> Looking for your help. I have attached my csv files in this email.
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> ==========================
>>>>>> Isaiah Nyabuto
>>>>>> +254729572616 <+254%20729%20572616>
>>>>>> Bsc. Health Records Management and Informatics
>>>>>>
>>>>>> On Jan 17, 2018 17:51, "Thomas Lavreys" <thomaslavreys@xxxxxxxxx>
>>>>>> wrote:
>>>>>>
>>>>>>> Thanks for the in depth info! It will surely help along the way.
>>>>>>> I’ve only started with some testers, but I will probably run into some more
>>>>>>> issues when I load my real data.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Thanks again!!
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Kind regards,
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Thomas
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *From:* Timothy Harding [mailto:tharding@xxxxxxxxxxxxxx]
>>>>>>> *Sent:* Wednesday, January 17, 2018 3:35 PM
>>>>>>> *To:* Thomas Lavreys <thomaslavreys@xxxxxxxxx>
>>>>>>> *Cc:* Adebusoye Anifalaje <busoye@xxxxxxxxxxxxxx>; dhis2-users <
>>>>>>> dhis2-users@xxxxxxxxxxxxxxxxxxx>
>>>>>>> *Subject:* Re: [Dhis2-users] Data element not found or not
>>>>>>> accessible
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Hello Thomas!
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> I am glad you have solved your issue! I noticed a few gotchas so
>>>>>>> here are a couple of things to note when importing csv files in DHIS 2:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 1. The first row
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> dataelement Data element not found or not accessible
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> [image: Inline image 1]
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> is a* funny error*, because it is actually telling you something
>>>>>>> about a single row, found here in the documentation
>>>>>>> <https://docs.dhis2.org/master/en/developer/html/webapi_data_values.html#webapi_data_values_csv>
>>>>>>> :
>>>>>>>
>>>>>>> "dataelement","period","orgunit","catoptcombo","attroptcombo","value","storedby","lastupd","comment"
>>>>>>>
>>>>>>> DHIS 2 is literally complaining about the fact that it *cannot find
>>>>>>> a data element with the uid "dataelement"*, which is what is
>>>>>>> assumes you are trying to load in that top row. You can try yourself by
>>>>>>> loading a file with only this top line, the error will be the same, 1
>>>>>>> ignored row, data element not found or not accessible. It is safe to ignore
>>>>>>> this error, I'd be worried if there was a UID called dataelement, which is
>>>>>>> actually a valid UID! Just extremely unlikely to pop up from the random UID
>>>>>>> generator.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2. Ignored
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Unlike the metadata importer, the data importer will report that it
>>>>>>> ignored *every row on a dry run*. It can make testing difficult
>>>>>>> because you don't know if it will actually import correctly or not as it
>>>>>>> reports back that every row was ignored. Two rules of thumb here:
>>>>>>>
>>>>>>>    1. The actual errored out row will be listed in the "Conflicts"
>>>>>>>    section, even in a dry run
>>>>>>>    2. Sadly, some errors you can only find on a real run, so always
>>>>>>>    upload your files with a comment, so you can find them later via the DB to
>>>>>>>    delete them if something goes wrong and all else fails
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 3. File Type
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> UTF-8 works well and I use it for all my CSV uploads. Be wary of
>>>>>>> EXCEL's CSV saving habits though, especially on a Mac. Your file should be *UTF-8
>>>>>>> without a BOM* (a byte order marker, DHIS 2 doesn't _really_ care,
>>>>>>> as it will error out on row one anyway (see above)) AND it should be
>>>>>>> standard *unix line feeds (LF)*. Excel for the Mac will default to
>>>>>>> saving it with circa 1990's CRs only AND a BOM (unless you are editing a
>>>>>>> CSV, in which case it will keep the settings it found when it opened it).
>>>>>>> Easiest way to check is to grab a copy of Notepad++ (windows) or BBedit
>>>>>>> (mac).
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> BBedit:
>>>>>>>
>>>>>>> [image: Inline image 2]
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> and
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> [image: Inline image 3]
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Notepad++
>>>>>>>
>>>>>>> [image: Inline image 4]
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> and
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> [image: Inline image 5]
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Hope this helps!
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *Timothy Harding*
>>>>>>> Sr. Systems Analyst, BAO Systems
>>>>>>>
>>>>>>> +1 202-536-1541 | tharding@xxxxxxxxxxxxxx |
>>>>>>> http://www.baosystems.com | Skype: hardingt@xxxxxxxxx | 2900 K
>>>>>>> Street, Suite 406, Washington D.C. 20007
>>>>>>> <https://maps.google.com/?q=2900+K+Street,+Suite+406,+Washington+D.C.+20007&entry=gmail&source=g>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Jan 17, 2018 at 9:11 AM, Thomas Lavreys <
>>>>>>> thomaslavreys@xxxxxxxxx> wrote:
>>>>>>>
>>>>>>> I found the issue. The file was saved as a csv utf-8, which is
>>>>>>> different to a normal csv (aparrently).
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Thanks for the advice!
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Kind regards,
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Thomas
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *From:* Adebusoye Anifalaje [mailto:busoye@xxxxxxxxxxxxxx]
>>>>>>> *Sent:* Wednesday, January 17, 2018 12:31 PM
>>>>>>>
>>>>>>>
>>>>>>> *To:* Thomas Lavreys <thomaslavreys@xxxxxxxxx>
>>>>>>> *Cc:* dhis2-users@xxxxxxxxxxxxxxxxxxx
>>>>>>> *Subject:* Re: [Dhis2-users] Data element not found or not
>>>>>>> accessible
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Hi Thomas,
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> I aslo see that you are using version 2.23, it is possible that the
>>>>>>> csv column structure is different for this version.  There might be someone
>>>>>>> on this forum who is currently working on this version and can provide
>>>>>>> input on that.
>>>>>>>
>>>>>>> - -
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *Busoye Anifalaje (PhD)*
>>>>>>> Director of Services (Principal), BAO Systems
>>>>>>>
>>>>>>> UK: +44 7901-740-757 <+44%207901%20740757> | US: +1 682-307-0986
>>>>>>> <(682)%20307-0986>|
>>>>>>>
>>>>>>>  busoye@xxxxxxxxxxxxxx | http://www.baosystems.com |
>>>>>>>
>>>>>>> Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007
>>>>>>> <https://maps.google.com/?q=2900+K+Street,+Suite+406,+Washington+D.C.+20007&entry=gmail&source=g>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 17 Jan 2018, at 12:23, Thomas Lavreys <thomaslavreys@xxxxxxxxx>
>>>>>>> wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> I tried it anyway, and it doesn’t recognize it:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Import summary
>>>>>>>
>>>>>>> Import count
>>>>>>>
>>>>>>> *Type*
>>>>>>>
>>>>>>> *Count*
>>>>>>>
>>>>>>> Imported
>>>>>>>
>>>>>>> 0
>>>>>>>
>>>>>>> Updated
>>>>>>>
>>>>>>> 0
>>>>>>>
>>>>>>> Deleted
>>>>>>>
>>>>>>> 0
>>>>>>>
>>>>>>> Ignored
>>>>>>>
>>>>>>> 13
>>>>>>>
>>>>>>> Conflicts
>>>>>>>
>>>>>>> *Element*
>>>>>>>
>>>>>>> *Description*
>>>>>>>
>>>>>>> dataelement
>>>>>>>
>>>>>>> Data element not found or not accessible
>>>>>>>
>>>>>>> bjDvmb4bfuf
>>>>>>>
>>>>>>> Category option combo not found or not accessible
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> So I’m guessing I need to find the catoptioncombo ID somewhere.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Regards,
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Thomas
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *From:* Adebusoye Anifalaje [mailto:busoye@xxxxxxxxxxxxxx
>>>>>>> <busoye@xxxxxxxxxxxxxx>]
>>>>>>> *Sent:* Wednesday, January 17, 2018 12:11 PM
>>>>>>> *To:* Thomas Lavreys <thomaslavreys@xxxxxxxxx>
>>>>>>> *Cc:* dhis2-users@xxxxxxxxxxxxxxxxxxx
>>>>>>> *Subject:* Re: [Dhis2-users] Data element not found or not
>>>>>>> accessible
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Hi Thomas,
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Can you try specifying the catoptioncombo even if it is default?
>>>>>>> This was an issue with one of the version builds a while ago.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Let’s know if that helps.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Cheers
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> - -
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *<image001.png>*
>>>>>>>
>>>>>>> *Busoye Anifalaje (PhD)*
>>>>>>> Director of Services (Principal), BAO Systems
>>>>>>>
>>>>>>> UK: +44 7901-740-757 <+44%207901%20740757> | US: +1 682-307-0986
>>>>>>> <(682)%20307-0986>|
>>>>>>>
>>>>>>>  busoye@xxxxxxxxxxxxxx | http://www.baosystems.com |
>>>>>>>
>>>>>>> Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007
>>>>>>> <https://maps.google.com/?q=2900+K+Street,+Suite+406,+Washington+D.C.+20007&entry=gmail&source=g>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 17 Jan 2018, at 12:02, Thomas Lavreys <thomaslavreys@xxxxxxxxx>
>>>>>>> wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Dear,
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> I get the message “Data element not found or not accessible” when
>>>>>>> trying to import a csv file. I looked at some old threads, but failed to
>>>>>>> find an answer.
>>>>>>>
>>>>>>> The csv file is attached (with headers, but it gave the same answer
>>>>>>> without headers).
>>>>>>>
>>>>>>> Attached also the API page of the data element, the sharing of the
>>>>>>> data element (I have superuser access), and the orgunit page to confirm the
>>>>>>> ID. Also a screenshot of the import summary and my DHIS2 instance info.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> I guess I might be missing something but I can’t figure out what
>>>>>>> exactly.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Thank you in advance for your help.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Kind regards,
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Thomas Lavreys
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *Thomas Lavreys, MSc, PgC Trop Med*
>>>>>>>
>>>>>>> *Sustainability Department SOCFINCO*
>>>>>>>
>>>>>>> *Maisha Consulting bvba*
>>>>>>>
>>>>>>> *Bisschoppenhof 9*
>>>>>>>
>>>>>>> *3150 Tildonk*
>>>>>>>
>>>>>>> *Belgium*
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *Tel Socfin: +32-25-080050*
>>>>>>>
>>>>>>> *Tel Maisha: +32-16-850242 <+32%2016%2085%2002%2042>*
>>>>>>>
>>>>>>> *Cell phone: +32-474-717641 <+32%20474%2071%2076%2041>*
>>>>>>>
>>>>>>> *Fax: +32-16-850242 <+32%2016%2085%2002%2042>*
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> <TEST.csv><data element id.JPG><sharing data element.JPG><orgunit
>>>>>>> id.JPG><import csv.JPG><instance info.JPG>_____________________
>>>>>>> __________________________
>>>>>>> Mailing list: https://launchpad.net/~dhis2-users
>>>>>>> Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
>>>>>>> Unsubscribe : https://launchpad.net/~dhis2-users
>>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Mailing list: https://launchpad.net/~dhis2-users
>>>>>>> Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
>>>>>>> Unsubscribe : https://launchpad.net/~dhis2-users
>>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Mailing list: https://launchpad.net/~dhis2-users
>>>>>>> Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
>>>>>>> Unsubscribe : https://launchpad.net/~dhis2-users
>>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>>
>>>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Jason P. Pickering
>>>> email: jason.p.pickering@xxxxxxxxx
>>>> tel:+46764147049 <+46%2076%20414%2070%2049>
>>>>
>>>> _______________________________________________
>>>> Mailing list: https://launchpad.net/~dhis2-users
>>>> Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
>>>> Unsubscribe : https://launchpad.net/~dhis2-users
>>>> More help   : https://help.launchpad.net/ListHelp
>>>>
>>>>
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~dhis2-users
>>> Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~dhis2-users
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>>
>>
>>
>> --
>> ------------------------------------------------------------
>> --------------------
>> Isaiah Nyabuto
>> +254 729 572 616
>> Bsc. Health Records Management & Informatics, JKUAT.
>>
>
>


-- 
--------------------------------------------------------------------------------
Isaiah Nyabuto
+254 729 572 616
Bsc. Health Records Management & Informatics, JKUAT.