dhis2-devs team mailing list archive

[Lars Helge Øverland <larshelge@xxxxxxxxx>]

Hi Tim,

yes I agree we must make it simpler to export a full data set including
category option combos. It is a common use-case. I have started on a
blueprint for it for 2.23 here:

https://blueprints.launchpad.net/dhis2/+spec/data-set-exchange

regards,

Lars


On Tue, Dec 15, 2015 at 4:10 PM, Timothy Harding <tharding@xxxxxxxxxxxxxx>
wrote:

> Hello DHIS2 Devs and Community!
>
> I've been doing some digging and it appears that this is about the easiest
> way to do this. I've got a question for the devs though: Is it a bug or
> intended behavior that when I export a dataset with dependencies it sends
> me a file sans CategoryOptionCombos?
>
>
>
>
> *Timothy Harding*
> Sr. Systems Analyst, BAO Systems
> +1 202-536-1541 | tharding@xxxxxxxxxxxxxx | http://www.baosystems.com | Skype:
> hardingt@xxxxxxxxx | 2900 K Street, Suite 404, Washington D.C. 20007
>
> On Wed, Nov 18, 2015 at 11:36 AM, Timothy Harding <tharding@xxxxxxxxxxxxxx
> > wrote:
>
>> Hello Hello!, Hot on the heels of my previous query, I have another one
>> associated with CatOptionCombos
>>
>> *What is the best way to migrate a single dataset and its constituents
>> from one DHIS2 system to another such that I can then export datavalues
>> from the source to the target DHIS2 instance?* I can get most of the way
>> there with the following:
>>
>> 1. Select Dataset as a detailed metadata export:
>> [image: Inline image 1]
>>
>> 2. Export with all dependencies
>> [image: Inline image 2]
>>
>> 3. Import the resulting file into a different DHIS2 instance
>>
>> This process will fail initially, but with manual edits, you can work
>> around the gotchas listed below:
>>
>>    - The users are considered a dependency and exported, if the roles
>>    aren't the same between the two systems it creates problems, so I typically
>>    delete the <users></users> section. They only cause problems when updating
>>    certain bits of metadata, the system will complain that it can't find the
>>    user, but otherwise will import just fine. Perhaps something like the
>>    following? Just throwing it out there 😁
>>
>> [image: Inline image 3]
>>
>>    - Categories and Category Options are out of order in the XML output,
>>    but the system seems to reorder them on import, so no issues here
>>    - *CategoryOptionCombos are not exported*, with all the other
>>    dependencies. Further more, there is no "CategoryOptionCombos", (or
>>    "CategoryOptions)" listed to choose from in the metadata detailed exporter
>>    😞. Sure these can all be generated with the maintenance operation,
>>    but the result will be CatOptionCombos with *completely different
>>    UIDs* than the source system, which would break any ability to upload
>>    data from the source server. Furthermore, there are *some *references
>>    to categoryOptionCombo in the dataset XML with dependencies file if you
>>    have any <compulsoryDataElementOperands>, but this won't import them, just
>>    cause the system not to find the categoryOptionCombo it is referencing.
>>    - I work around this by doing something like this:
>>       https://apps.dhis2.org/demo/api/categoryOptionCombos.xml?fields=name,id,displayName,categoryCombo[name,id],categoryOptions[name,id]&paging=false&filter=categoryCombo.id:eq:m2jTvAj5kkm
>>       and repeat for each category combo covered by this dataset, and cut and
>>       paste the results into the XML file generated in step 2. This is hugely
>>       time consuming if there are a lot of catcombos used by the dataset in
>>       question. Sadly the API wont let me filter on just a dataset, because there
>>       is no reverse link between category combos and data elements (one way only)
>>    - Defaults for Category Options, Categories, Category Combinations
>>    need to be updated (Find&Replace) with the UIDs in use by the target
>>    system. Not sure exactly how to fix this, other than making an exception in
>>    the import process for any of those three with the name "default".
>>    Otherwise it is manual work on the export file each time.
>>
>> * ERROR 2015-11-18 09:53:57,138 ERROR: duplicate key value violates
>> unique constraint "uk_pbj3u1nk9vnuof8f47utvowmv"
>>   Detail: Key (name)=(default) already exists. (SqlExceptionHelper.java
>> [taskScheduler-1])
>>
>>    - dissaggregation vs DISSAGGREGATION, in v2.20, the *exporter *keeps
>>    them lowercase, and the *importer *now expects uppercase. Why the
>>    distinction though?
>>
>> * ERROR 2015-11-18 09:45:36,496
>> com.fasterxml.jackson.databind.exc.InvalidFormatException: Can not
>> construct instance of org.hisp.dhis.common.DataDimensionType from String
>> value 'disaggregation': value not one of declared Enum instance names:
>> [DISAGGREGATION, ATTRIBUTE]
>>
>> Once I contend with all the gotchas I listed, *I am able to get an
>> import from a source DHIS2 installation to the Target DHIS2 installation
>> with the proper UIDs for the categoryOptionCombos*, but it makes me
>> think that there has to be an easier way. Am I missing a button or a step
>> somewhere? The "default" UIDs and the catoptioncombos are the hairiest part
>> of the whole process and probably wouldn't be easy for most users.
>>
>> The only other thing I can think of is to use the vanilla metadata
>> exporter, but this is not feasible when there are 15 datasets in the source
>> system and I only need to export 1.
>>
>> [image: Inline image 4]
>>
>>
>> *Timothy Harding*
>> Sr. Systems Analyst, BAO Systems
>> +1 202-536-1541 | tharding@xxxxxxxxxxxxxx | http://www.baosystems.com | Skype:
>> hardingt@xxxxxxxxx | 2900 K Street, Suite 404, Washington D.C. 20007
>>
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~dhis2-devs
> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~dhis2-devs
> More help   : https://help.launchpad.net/ListHelp
>
>


-- 
Lars Helge Øverland
Lead developer, DHIS 2
University of Oslo
Skype: larshelgeoverland
http://www.dhis2.org <https://www.dhis2.org>