dhis2-devs team mailing list archive

[Morten Olav Hansen <morten@xxxxxxxxx>]

Ok, thanks Vanya, I'm looking into it now

-- 
Morten Olav Hansen
Senior Engineer, DHIS 2
University of Oslo
http://www.dhis2.org

On Thu, Sep 1, 2016 at 7:28 PM, Vanya Seth <vanyas@xxxxxxxxxxxxxxxx> wrote:

> Hi Morten
>
> Thanks for the response.
>
> We ran into another issue while trying to do a full export and import.
>
> As per the previous conversations we took care of all data migrations (as
> per the new constraints on the DB- pretty much caught by the Importer
> during the validation phase itself).
>
> Having done that we run into this issue:
> The 'categoryoptiongroup' import is failing with constraint violation for
> the UID. We checked the data and there is no repeating UID in the  database.
>
> The error text is attached for your reference, as well the payload used
> for the import.
>
> Thanks for all the help in advance.
>
> Regards
> Vanya
>
>
>
> On Thu, Sep 1, 2016 at 11:10 AM, Morten Olav Hansen <morten@xxxxxxxxx>
> wrote:
>
>> Thanks Aamer, I'm looking at a similar bug right now.. seems some objects
>> don't get their deps properly attached..
>>
>> --
>> Morten Olav Hansen
>> Senior Engineer, DHIS 2
>> University of Oslo
>> http://www.dhis2.org
>>
>> On Thu, Sep 1, 2016 at 12:36 PM, Aamer Mohammed <aamerm@xxxxxxxxxxxxxxxx>
>> wrote:
>>
>>> Hi Morten,
>>>
>>> Please find the payload for below request
>>> *curl -H "Content-Type: application/json" -X GET -u username:password
>>> "http://localhost:8888/api/24/metadata?filter=created:gt:2016-07
>>> <http://localhost:8888/api/24/metadata?filter=created:gt:2016-07>" >
>>> metadata.224.newapi.created.json*
>>>
>>> Import the payload in a fresh instance for import using
>>>
>>> *curl -H "Content-Type: application/json" -X POST --data
>>> @metadata.224.newapi.created.json -u admin:district
>>> “http://localhost:8080/api/24/metadata?atomicMode=NONE
>>> <http://localhost:8080/api/24/metadata?atomicMode=NONE>” >
>>> output_created_besteffort.txt*
>>>
>>> If the same payload is run with atomicMode=ALL, I am getting 'Invalid
>>> references' errors which is acceptable. But if it is run with
>>> atomicMode=NONE, it is throwing the error for which the complete stack
>>> trace is attached earlier. filename: '
>>> *output_created_besteffort_trace.txt'*
>>>
>>> Let me know if the complete payload or any other details are required
>>> from my end.
>>>
>>>
>>> Thanks
>>> Aamer.
>>>
>>>
>>>
>>> On Thu, Sep 1, 2016 at 8:52 AM, Morten Olav Hansen <morten@xxxxxxxxx>
>>> wrote:
>>>
>>>> Hi Aamer
>>>>
>>>> Could you please share the payload of the object where this happen?
>>>>
>>>> --
>>>> Morten Olav Hansen
>>>> Senior Engineer, DHIS 2
>>>> University of Oslo
>>>> http://www.dhis2.org
>>>>
>>>> On Fri, Aug 26, 2016 at 9:13 PM, Aamer Mohammed <
>>>> aamerm@xxxxxxxxxxxxxxxx> wrote:
>>>>
>>>>> Hi Morten,
>>>>>
>>>>> The Importer was run with atomicMode=ALL. Though the payload is huge,
>>>>> only few errors were shown in ImportSummary. As per errors in previous
>>>>> ImportSummary, I have done below.
>>>>> 1) ensured the 'short name' field is unique for each of the 'data
>>>>> elements' and 'data element group' as well
>>>>> 2) UID of admin, Super User, tracked entity was changed in payload to
>>>>> be same to where it is getting imported.
>>>>> I have manually resolved them and ran the importer again with
>>>>> atomicMode=ALL.
>>>>>
>>>>> I got exception as attached in trace file 'output_besteffort_trace.txt'
>>>>> Would these kind of issues be known only after the import has failed?
>>>>> and we need to resolve them one-by-one and as-and-when the importer throws
>>>>> them? Is there any note which has the details about the constraints being
>>>>> introduced in a new DHIS version?
>>>>>
>>>>> Alternatively, If we export a filtered set of metadata using '
>>>>> /api/24/metadata?filter=lastUpdated:gt:2016-05' and run the importer
>>>>> with atomicMode=NONE
>>>>>
>>>>> *curl -H "Content-Type: application/json" -X POST --data
>>>>> @metadata.224.newapi.json -u admin:district
>>>>> “http://localhost:8080/api/24/metadata?atomicMode=NONE
>>>>> <http://localhost:8080/api/24/metadata?atomicMode=NONE>” >
>>>>> output_created_besteffort_trace.txt*
>>>>>
>>>>> Getting below exception in trace. Complete trace also attached.
>>>>> -----
>>>>> * INFO 2016-08-26 15:58:38,217 (admin) Creating 57 object(s) of type
>>>>> ReportTable (DefaultObjectBundleService.java [qtp289378424-12])
>>>>> org.hibernate.TransientObjectException: object references an unsaved
>>>>> transient instance - save the transient instance before flushing:
>>>>> org.hisp.dhis.indicator.Indicator
>>>>> ------
>>>>> Any suggestions please.
>>>>>
>>>>>
>>>>> Thanks
>>>>> Aamer.
>>>>>
>>>>>
>>>>> On Tue, Aug 23, 2016 at 11:19 AM, Morten Olav Hansen <morten@xxxxxxxxx
>>>>> > wrote:
>>>>>
>>>>>> Hi Aamer
>>>>>>
>>>>>> Probably what is happening is that the new database has some
>>>>>> constraints that the old one doesn't. We have had some issues with
>>>>>> hibernate in the past, and for certain databases that means that many of
>>>>>> the constraints was not applied..
>>>>>>
>>>>>> If you look at the messages, you will see that e.g `shortName` must
>>>>>> be unique for data elements.. the list of errors is not big though,
>>>>>> probably you can go through the payload and manually update the objects
>>>>>> which has issues?
>>>>>>
>>>>>> Another issue I see is that you are importing users, and those UIDs
>>>>>> was only stabilized in 2.23.. so you might want to also manually update
>>>>>> those (again, should not be a big job), this is not done as a startup
>>>>>> routine.. just done when creating the admin user etc in a fresh database
>>>>>>
>>>>>> --
>>>>>> Morten Olav Hansen
>>>>>> Senior Engineer, DHIS 2
>>>>>> University of Oslo
>>>>>> http://www.dhis2.org
>>>>>>
>>>>>> On Tue, Aug 23, 2016 at 12:00 PM, Aamer Mohammed <
>>>>>> aamerm@xxxxxxxxxxxxxxxx> wrote:
>>>>>>
>>>>>>> Any thoughts please on the below approach. Let us know if any
>>>>>>> details are required from our end around this.
>>>>>>>
>>>>>>> On Wed, Aug 17, 2016 at 1:18 PM, Aamer Mohammed <
>>>>>>> aamerm@xxxxxxxxxxxxxxxx> wrote:
>>>>>>>
>>>>>>>> Hi Morten,
>>>>>>>>
>>>>>>>> We have our metadata configured on a DHIS 2.21 instance. We have
>>>>>>>> sequentially upgraded this instance to 2.22, 2.23 and 2.24 by following the
>>>>>>>> upgrade notes for each of them.
>>>>>>>>
>>>>>>>> We want to export the metadata from this upgraded 2.24 instance and
>>>>>>>> import it to a fresh instance which is already on 2.24. We followed the
>>>>>>>> below steps.
>>>>>>>> 1) Export the metadata from the upgraded 2.24 instance using the
>>>>>>>> new exporter
>>>>>>>> *curl -H "Content-Type: application/json" -X GET -u
>>>>>>>> username:password http://xxxx:8080/api/24/metadata.json
>>>>>>>> <http://xxxx:8080/api/24/metadata.json> > metadata.224.newapi.json *
>>>>>>>>
>>>>>>>> 2) Import the metadata using the new importer to a fresh instance
>>>>>>>> which is already on 2.24
>>>>>>>> *curl -H "Content-Type: application/json" -X POST --data
>>>>>>>> @metadata.224.newapi.json -u username:password
>>>>>>>> http://yyyy:8080/api/24/metadata <http://yyyy:8080/api/24/metadata> >
>>>>>>>> output.txt*
>>>>>>>>
>>>>>>>> We see that none of the objects are getting imported. The complete
>>>>>>>> response of the importer is attached below. Any thoughts please. Also let
>>>>>>>> me know if any further details are required from our end.
>>>>>>>>
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> Aamer.
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
>
> --
> With Regards
> ThoughtWorks Technologies
> Hyderabad
>
> --Stay Hungry Stay Foolish!!
>