dhis2-users team mailing list archive

[Jason Pickering <jason.p.pickering@xxxxxxxxx>]

Hi Isaiah,

Yes, R is definitely capable of creating a file which you can import into
DHIS2.

If you want to use CSV, be sure to have your file with  the "row.names=FALSE
" parameter and be sure to convert all "NA" values to with " na = '' "

You can also post the file directly to DHIS2. Have a look at
https://github.com/jason-p-pickering/dhis2-data-munging/blob/master/bootstrap/bootstrap.Rmd
or more details of how to do this in the section "Loading the data
elements"

Regards,
Jason


On Mon, Jan 29, 2018 at 6:04 PM, Timothy Harding <tharding@xxxxxxxxxxxxxx>
wrote:

> Isaiah:
>
> org_kqmh_list_test.csv:
>
> Has an extra column
>
> [image: Inline image 1]
>
> 1, 2, 3, etc.. should not be in there unless the OUs are named 1, 2, 3.
>
> Probably should be LF instead of CRLF:
> [image: Inline image 2]
> but I'm not sure it makes a difference here.
>
> Also you've got a lot of NA's in there, not sure those are needed unless
> you are setting something to be NA, best just have ,, rather than ,NA,
>
> [image: Inline image 3]
>
>
> Good luck!
>
>
> *Timothy Harding*
> Sr. Systems Analyst, BAO Systems
> +1 202-536-1541 | tharding@xxxxxxxxxxxxxx | http://www.baosystems.com | Skype:
> hardingt@xxxxxxxxx | 2900 K Street, Suite 406, Washington D.C. 20007
> <https://maps.google.com/?q=2900+K+Street,+Suite+406,+Washington+D.C.+20007&entry=gmail&source=g>
>
> On Mon, Jan 29, 2018 at 11:30 AM, Isaiah Nyabuto <isaiahnyabuto@xxxxxxxxx>
> wrote:
>
>> Hi all,
>>
>> Thanks Timothy for sharing this. I am trying to import organisation units
>> metadata in csv file with UTF-8 encoding. When I import the file with
>> Metadata import in dhis2, the file is not imported at all and I do get the
>> summary report.
>>
>> I wrote the csv file with R function write.table.
>>
>> Its funny enough that when I write the same file using Excel and save as
>> csv file with UTF-8, it works well.
>>
>> I do not understand what could be the problem here, has anyone tried to
>> write a csv file with an R function that can successfully be imported in
>> the dhis2?
>>
>> Looking for your help. I have attached my csv files in this email.
>>
>> Thanks
>>
>>
>>
>>
>> ==========================
>> Isaiah Nyabuto
>> +254729572616 <+254%20729%20572616>
>> Bsc. Health Records Management and Informatics
>>
>> On Jan 17, 2018 17:51, "Thomas Lavreys" <thomaslavreys@xxxxxxxxx> wrote:
>>
>>> Thanks for the in depth info! It will surely help along the way. I’ve
>>> only started with some testers, but I will probably run into some more
>>> issues when I load my real data.
>>>
>>>
>>>
>>> Thanks again!!
>>>
>>>
>>>
>>> Kind regards,
>>>
>>>
>>>
>>> Thomas
>>>
>>>
>>>
>>> *From:* Timothy Harding [mailto:tharding@xxxxxxxxxxxxxx]
>>> *Sent:* Wednesday, January 17, 2018 3:35 PM
>>> *To:* Thomas Lavreys <thomaslavreys@xxxxxxxxx>
>>> *Cc:* Adebusoye Anifalaje <busoye@xxxxxxxxxxxxxx>; dhis2-users <
>>> dhis2-users@xxxxxxxxxxxxxxxxxxx>
>>> *Subject:* Re: [Dhis2-users] Data element not found or not accessible
>>>
>>>
>>>
>>> Hello Thomas!
>>>
>>>
>>>
>>> I am glad you have solved your issue! I noticed a few gotchas so here
>>> are a couple of things to note when importing csv files in DHIS 2:
>>>
>>>
>>>
>>> 1. The first row
>>>
>>>
>>>
>>> dataelement Data element not found or not accessible
>>>
>>>
>>>
>>> [image: Inline image 1]
>>>
>>>
>>>
>>> is a* funny error*, because it is actually telling you something about
>>> a single row, found here in the documentation
>>> <https://docs.dhis2.org/master/en/developer/html/webapi_data_values.html#webapi_data_values_csv>
>>> :
>>>
>>> "dataelement","period","orgunit","catoptcombo","attroptcombo","value","storedby","lastupd","comment"
>>>
>>> DHIS 2 is literally complaining about the fact that it *cannot find a
>>> data element with the uid "dataelement"*, which is what is assumes you
>>> are trying to load in that top row. You can try yourself by loading a file
>>> with only this top line, the error will be the same, 1 ignored row, data
>>> element not found or not accessible. It is safe to ignore this error, I'd
>>> be worried if there was a UID called dataelement, which is actually a valid
>>> UID! Just extremely unlikely to pop up from the random UID generator.
>>>
>>>
>>>
>>>
>>>
>>> 2. Ignored
>>>
>>>
>>>
>>> Unlike the metadata importer, the data importer will report that it
>>> ignored *every row on a dry run*. It can make testing difficult because
>>> you don't know if it will actually import correctly or not as it reports
>>> back that every row was ignored. Two rules of thumb here:
>>>
>>>    1. The actual errored out row will be listed in the "Conflicts"
>>>    section, even in a dry run
>>>    2. Sadly, some errors you can only find on a real run, so always
>>>    upload your files with a comment, so you can find them later via the DB to
>>>    delete them if something goes wrong and all else fails
>>>
>>>
>>>
>>>
>>>
>>> 3. File Type
>>>
>>>
>>>
>>> UTF-8 works well and I use it for all my CSV uploads. Be wary of EXCEL's
>>> CSV saving habits though, especially on a Mac. Your file should be *UTF-8
>>> without a BOM* (a byte order marker, DHIS 2 doesn't _really_ care, as
>>> it will error out on row one anyway (see above)) AND it should be standard *unix
>>> line feeds (LF)*. Excel for the Mac will default to saving it with
>>> circa 1990's CRs only AND a BOM (unless you are editing a CSV, in which
>>> case it will keep the settings it found when it opened it). Easiest way to
>>> check is to grab a copy of Notepad++ (windows) or BBedit (mac).
>>>
>>>
>>>
>>> BBedit:
>>>
>>> [image: Inline image 2]
>>>
>>>
>>>
>>> and
>>>
>>>
>>>
>>> [image: Inline image 3]
>>>
>>>
>>>
>>>
>>>
>>> Notepad++
>>>
>>> [image: Inline image 4]
>>>
>>>
>>>
>>> and
>>>
>>>
>>>
>>> [image: Inline image 5]
>>>
>>>
>>>
>>> Hope this helps!
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> *Timothy Harding*
>>> Sr. Systems Analyst, BAO Systems
>>>
>>> +1 202-536-1541 | tharding@xxxxxxxxxxxxxx | http://www.baosystems.com | Skype:
>>> hardingt@xxxxxxxxx | 2900 K Street, Suite 406, Washington D.C. 20007
>>> <https://maps.google.com/?q=2900+K+Street,+Suite+406,+Washington+D.C.+20007&entry=gmail&source=g>
>>>
>>>
>>>
>>> On Wed, Jan 17, 2018 at 9:11 AM, Thomas Lavreys <thomaslavreys@xxxxxxxxx>
>>> wrote:
>>>
>>> I found the issue. The file was saved as a csv utf-8, which is different
>>> to a normal csv (aparrently).
>>>
>>>
>>>
>>> Thanks for the advice!
>>>
>>>
>>>
>>> Kind regards,
>>>
>>>
>>>
>>> Thomas
>>>
>>>
>>>
>>> *From:* Adebusoye Anifalaje [mailto:busoye@xxxxxxxxxxxxxx]
>>> *Sent:* Wednesday, January 17, 2018 12:31 PM
>>>
>>>
>>> *To:* Thomas Lavreys <thomaslavreys@xxxxxxxxx>
>>> *Cc:* dhis2-users@xxxxxxxxxxxxxxxxxxx
>>> *Subject:* Re: [Dhis2-users] Data element not found or not accessible
>>>
>>>
>>>
>>> Hi Thomas,
>>>
>>>
>>>
>>> I aslo see that you are using version 2.23, it is possible that the csv
>>> column structure is different for this version.  There might be someone on
>>> this forum who is currently working on this version and can provide input
>>> on that.
>>>
>>> - -
>>>
>>>
>>>
>>>
>>> *Busoye Anifalaje (PhD)*
>>> Director of Services (Principal), BAO Systems
>>>
>>> UK: +44 7901-740-757 <+44%207901%20740757> | US: +1 682-307-0986
>>> <(682)%20307-0986>|
>>>
>>>  busoye@xxxxxxxxxxxxxx | http://www.baosystems.com |
>>>
>>> Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007
>>> <https://maps.google.com/?q=2900+K+Street,+Suite+406,+Washington+D.C.+20007&entry=gmail&source=g>
>>>
>>>
>>>
>>> On 17 Jan 2018, at 12:23, Thomas Lavreys <thomaslavreys@xxxxxxxxx>
>>> wrote:
>>>
>>>
>>>
>>> I tried it anyway, and it doesn’t recognize it:
>>>
>>>
>>>
>>> Import summary
>>>
>>> Import count
>>>
>>> *Type*
>>>
>>> *Count*
>>>
>>> Imported
>>>
>>> 0
>>>
>>> Updated
>>>
>>> 0
>>>
>>> Deleted
>>>
>>> 0
>>>
>>> Ignored
>>>
>>> 13
>>>
>>> Conflicts
>>>
>>> *Element*
>>>
>>> *Description*
>>>
>>> dataelement
>>>
>>> Data element not found or not accessible
>>>
>>> bjDvmb4bfuf
>>>
>>> Category option combo not found or not accessible
>>>
>>>
>>>
>>> So I’m guessing I need to find the catoptioncombo ID somewhere.
>>>
>>>
>>>
>>> Regards,
>>>
>>>
>>>
>>> Thomas
>>>
>>>
>>>
>>>
>>>
>>> *From:* Adebusoye Anifalaje [mailto:busoye@xxxxxxxxxxxxxx
>>> <busoye@xxxxxxxxxxxxxx>]
>>> *Sent:* Wednesday, January 17, 2018 12:11 PM
>>> *To:* Thomas Lavreys <thomaslavreys@xxxxxxxxx>
>>> *Cc:* dhis2-users@xxxxxxxxxxxxxxxxxxx
>>> *Subject:* Re: [Dhis2-users] Data element not found or not accessible
>>>
>>>
>>>
>>> Hi Thomas,
>>>
>>>
>>>
>>> Can you try specifying the catoptioncombo even if it is default? This
>>> was an issue with one of the version builds a while ago.
>>>
>>>
>>>
>>> Let’s know if that helps.
>>>
>>>
>>>
>>> Cheers
>>>
>>>
>>>
>>> - -
>>>
>>>
>>>
>>>
>>>
>>>
>>> *<image001.png>*
>>>
>>> *Busoye Anifalaje (PhD)*
>>> Director of Services (Principal), BAO Systems
>>>
>>> UK: +44 7901-740-757 <+44%207901%20740757> | US: +1 682-307-0986
>>> <(682)%20307-0986>|
>>>
>>>  busoye@xxxxxxxxxxxxxx | http://www.baosystems.com |
>>>
>>> Skype: busoye | 2900 K Street, Suite 406, Washington D.C. 20007
>>> <https://maps.google.com/?q=2900+K+Street,+Suite+406,+Washington+D.C.+20007&entry=gmail&source=g>
>>>
>>>
>>>
>>> On 17 Jan 2018, at 12:02, Thomas Lavreys <thomaslavreys@xxxxxxxxx>
>>> wrote:
>>>
>>>
>>>
>>> Dear,
>>>
>>>
>>>
>>> I get the message “Data element not found or not accessible” when trying
>>> to import a csv file. I looked at some old threads, but failed to find an
>>> answer.
>>>
>>> The csv file is attached (with headers, but it gave the same answer
>>> without headers).
>>>
>>> Attached also the API page of the data element, the sharing of the data
>>> element (I have superuser access), and the orgunit page to confirm the ID.
>>> Also a screenshot of the import summary and my DHIS2 instance info.
>>>
>>>
>>>
>>> I guess I might be missing something but I can’t figure out what exactly.
>>>
>>>
>>>
>>> Thank you in advance for your help.
>>>
>>>
>>>
>>> Kind regards,
>>>
>>>
>>>
>>> Thomas Lavreys
>>>
>>>
>>>
>>> *Thomas Lavreys, MSc, PgC Trop Med*
>>>
>>> *Sustainability Department SOCFINCO*
>>>
>>> *Maisha Consulting bvba*
>>>
>>> *Bisschoppenhof 9*
>>>
>>> *3150 Tildonk*
>>>
>>> *Belgium*
>>>
>>>
>>>
>>> *Tel Socfin: +32-25-080050*
>>>
>>> *Tel Maisha: +32-16-850242 <+32%2016%2085%2002%2042>*
>>>
>>> *Cell phone: +32-474-717641 <+32%20474%2071%2076%2041>*
>>>
>>> *Fax: +32-16-850242 <+32%2016%2085%2002%2042>*
>>>
>>>
>>>
>>> <TEST.csv><data element id.JPG><sharing data element.JPG><orgunit
>>> id.JPG><import csv.JPG><instance info.JPG>_____________________
>>> __________________________
>>> Mailing list: https://launchpad.net/~dhis2-users
>>> Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~dhis2-users
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~dhis2-users
>>> Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~dhis2-users
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>>
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~dhis2-users
>>> Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~dhis2-users
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>>
>


-- 
Jason P. Pickering
email: jason.p.pickering@xxxxxxxxx
tel:+46764147049