← Back to team overview

dhis2-devs team mailing list archive

Re: Bugs in DHIS new importer.

 

Hi Morten,

I've just tried with eventCharts and eventReports (25x and 23x of each type
at once) and it works fine.
Thanks for your support.

Regards!


*Sergio Valenzuela*
Fundación EHAS
Telemedicina para países en desarrollo -* www.ehas.org
<http://www.ehas.org/>*
E.T.S.I. de Telecomunicación A.101-9L. Ciudad Universitaria 30, 28040
Madrid - España
Tlfno. (+34) 91 488 87 41: (+34) 687 67 17 62



2016-07-29 11:06 GMT+02:00 Morten Olav Hansen <morten@xxxxxxxxx>:

> Hi Sergio
>
> Can you again try with latest 2.24?
>
> --
> Morten Olav Hansen
> Senior Engineer, DHIS 2
> University of Oslo
> http://www.dhis2.org
>
> On Thu, Jul 28, 2016 at 4:20 PM, Sergio Valenzuela <
> sergio.valenzuela@xxxxxxxx> wrote:
>
>> Ok. Thanks a lot.
>>
>>
>> *Sergio Valenzuela*
>> Fundación EHAS
>> Telemedicina para países en desarrollo -* www.ehas.org
>> <http://www.ehas.org/>*
>> E.T.S.I. de Telecomunicación A.101-9L. Ciudad Universitaria 30, 28040
>> Madrid - España
>> Tlfno. (+34) 91 488 87 41: (+34) 687 67 17 62
>>
>>
>>
>> 2016-07-28 16:19 GMT+02:00 Morten Olav Hansen <morten@xxxxxxxxx>:
>>
>>> Hm ok, seems to be a different error (ProgramIndicator this time), I
>>> will have a look tomorrow and fix it
>>>
>>> --
>>> Morten Olav Hansen
>>> Senior Engineer, DHIS 2
>>> University of Oslo
>>> http://www.dhis2.org
>>>
>>> On Thu, Jul 28, 2016 at 4:07 PM, Sergio Valenzuela <
>>> sergio.valenzuela@xxxxxxxx> wrote:
>>>
>>>> Hello again,
>>>>
>>>> I've testing the fix and I encountered some troubles. The previous
>>>> error appears again.
>>>> When I tried to import 1 event chart the importer seemed to work fine.
>>>> I tried 2 and It worked too. Finally I tried 238, and this is what I got:
>>>>
>>>> {
>>>>   "httpStatus": "Internal Server Error",
>>>>   "httpStatusCode": 500,
>>>>   "status": "ERROR",
>>>>   "message": "object references an unsaved transient instance - save
>>>> the transient instance before flushing:
>>>> org.hisp.dhis.program.ProgramIndicator"
>>>> }
>>>>
>>>> I tried 9 and It worked.
>>>> I tried 20 and It crashed again. After this try I was not able to
>>>> import a single one any more.
>>>>
>>>> I attach the stacktrace of every try I made. The payload I used is the
>>>> same as before but with a list of event charts instead of only one.
>>>>
>>>> Regards!
>>>>
>>>>
>>>> *Sergio Valenzuela*
>>>> Fundación EHAS
>>>> Telemedicina para países en desarrollo -* www.ehas.org
>>>> <http://www.ehas.org/>*
>>>> E.T.S.I. de Telecomunicación A.101-9L. Ciudad Universitaria 30, 28040
>>>> Madrid - España
>>>> Tlfno. (+34) 91 488 87 41: (+34) 687 67 17 62
>>>>
>>>>
>>>>
>>>> 2016-07-28 13:51 GMT+02:00 Sergio Valenzuela <
>>>> sergio.valenzuela@xxxxxxxx>:
>>>>
>>>>> Hi,
>>>>>
>>>>> I have just tested it again with Build revision: 23527 and now it
>>>>> works perfectly.
>>>>>
>>>>> Thanks Morten.
>>>>>
>>>>> Regards!
>>>>>
>>>>>
>>>>> *Sergio Valenzuela*
>>>>> Fundación EHAS
>>>>> Telemedicina para países en desarrollo -* www.ehas.org
>>>>> <http://www.ehas.org/>*
>>>>> E.T.S.I. de Telecomunicación A.101-9L. Ciudad Universitaria 30, 28040
>>>>> Madrid - España
>>>>> Tlfno. (+34) 91 488 87 41: (+34) 687 67 17 62
>>>>>
>>>>>
>>>>>
>>>>> 2016-07-28 13:28 GMT+02:00 Morten Olav Hansen <morten@xxxxxxxxx>:
>>>>>
>>>>>> Hi Sergio
>>>>>>
>>>>>> Please try again with latest 2.24
>>>>>>
>>>>>> --
>>>>>> Morten Olav Hansen
>>>>>> Senior Engineer, DHIS 2
>>>>>> University of Oslo
>>>>>> http://www.dhis2.org
>>>>>>
>>>>>> On Thu, Jul 28, 2016 at 10:56 AM, Sergio Valenzuela <
>>>>>> sergio.valenzuela@xxxxxxxx> wrote:
>>>>>>
>>>>>>> Great! Thanks Morten.
>>>>>>>
>>>>>>>
>>>>>>> *Sergio Valenzuela*
>>>>>>> Fundación EHAS
>>>>>>> Telemedicina para países en desarrollo -* www.ehas.org
>>>>>>> <http://www.ehas.org/>*
>>>>>>> E.T.S.I. de Telecomunicación A.101-9L. Ciudad Universitaria 30,
>>>>>>> 28040 Madrid - España
>>>>>>> Tlfno. (+34) 91 488 87 41: (+34) 687 67 17 62
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2016-07-28 10:44 GMT+02:00 Morten Olav Hansen <morten@xxxxxxxxx>:
>>>>>>>
>>>>>>>> Hi Sergio
>>>>>>>>
>>>>>>>> I could reproduce your error, will try and have a fix ready today
>>>>>>>> and backport to 2.24.
>>>>>>>>
>>>>>>>> --
>>>>>>>> Morten Olav Hansen
>>>>>>>> Senior Engineer, DHIS 2
>>>>>>>> University of Oslo
>>>>>>>> http://www.dhis2.org
>>>>>>>>
>>>>>>>> On Wed, Jul 27, 2016 at 4:30 PM, Sergio Valenzuela <
>>>>>>>> sergio.valenzuela@xxxxxxxx> wrote:
>>>>>>>>
>>>>>>>>> Thanks Morten. Keep me up please.
>>>>>>>>>
>>>>>>>>> Best regards!
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> *Sergio Valenzuela*
>>>>>>>>> Fundación EHAS
>>>>>>>>> Telemedicina para países en desarrollo -* www.ehas.org
>>>>>>>>> <http://www.ehas.org/>*
>>>>>>>>> E.T.S.I. de Telecomunicación A.101-9L. Ciudad Universitaria 30,
>>>>>>>>> 28040 Madrid - España
>>>>>>>>> Tlfno. (+34) 91 488 87 41: (+34) 687 67 17 62
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 2016-07-27 16:28 GMT+02:00 Morten Olav Hansen <morten@xxxxxxxxx>:
>>>>>>>>>
>>>>>>>>>> Hi Sergio
>>>>>>>>>>
>>>>>>>>>> Not sure exactly what is wrong there, but will try and reproduce
>>>>>>>>>> and fix tomorrow.
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Morten Olav Hansen
>>>>>>>>>> Senior Engineer, DHIS 2
>>>>>>>>>> University of Oslo
>>>>>>>>>> http://www.dhis2.org
>>>>>>>>>>
>>>>>>>>>> On Wed, Jul 27, 2016 at 4:11 PM, Sergio Valenzuela <
>>>>>>>>>> sergio.valenzuela@xxxxxxxx> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hello devs,
>>>>>>>>>>>
>>>>>>>>>>> I'm getting the "Exception occurred while trying to import the
>>>>>>>>>>> metadata. object references an unsaved transient instance - save the
>>>>>>>>>>> transient instance before flushing: org.hisp.dhis.dataelement.DataElement
>>>>>>>>>>> (MetadataSyncTask.java [taskScheduler-9])" error when using the
>>>>>>>>>>> new exporter/importer (/api/24/metadata). I'm exporting event charts and
>>>>>>>>>>> event reports from one DHIS2 instance to another (same version same build:
>>>>>>>>>>> 2.24 and 23522). Metadata is the same in both instances.
>>>>>>>>>>>
>>>>>>>>>>> Full stack trace attached.
>>>>>>>>>>> URL: http://localhost:8080/api/24/metadata
>>>>>>>>>>> Example eventCharts Payload:
>>>>>>>>>>> {
>>>>>>>>>>>   "date": "2016-07-27T12:54:31.851+0000",
>>>>>>>>>>>   "eventCharts": [
>>>>>>>>>>>     {
>>>>>>>>>>>       "lastUpdated": "2016-07-26T15:57:08.779+0000",
>>>>>>>>>>>       "id": "uM4KpocRONz",
>>>>>>>>>>>       "created": "2016-07-26T15:57:08.779+0000",
>>>>>>>>>>>       "name": "Acurenam: Mortalidad total: Nº de muertes",
>>>>>>>>>>>       "showData": true,
>>>>>>>>>>>       "publicAccess": "rw------",
>>>>>>>>>>>       "userOrganisationUnitChildren": false,
>>>>>>>>>>>       "type": "LINE",
>>>>>>>>>>>       "hideEmptyRows": false,
>>>>>>>>>>>       "userOrganisationUnit": false,
>>>>>>>>>>>       "hideNaData": false,
>>>>>>>>>>>       "completedOnly": false,
>>>>>>>>>>>       "sortOrder": 0,
>>>>>>>>>>>       "collapseDataDimensions": false,
>>>>>>>>>>>       "userOrganisationUnitGrandChildren": false,
>>>>>>>>>>>       "hideSubtitle": false,
>>>>>>>>>>>       "outputType": "EVENT",
>>>>>>>>>>>       "title": "Número de muertes",
>>>>>>>>>>>       "hideLegend": false,
>>>>>>>>>>>       "externalAccess": false,
>>>>>>>>>>>       "hideTitle": false,
>>>>>>>>>>>       "regression": false,
>>>>>>>>>>>       "program": {
>>>>>>>>>>>         "id": "yBICFtJAYFR"
>>>>>>>>>>>       },
>>>>>>>>>>>       "relativePeriods": {
>>>>>>>>>>>         "thisYear": false,
>>>>>>>>>>>         "quartersLastYear": false,
>>>>>>>>>>>         "last52Weeks": false,
>>>>>>>>>>>         "thisWeek": false,
>>>>>>>>>>>         "lastMonth": false,
>>>>>>>>>>>         "monthsThisYear": false,
>>>>>>>>>>>         "last2SixMonths": false,
>>>>>>>>>>>         "thisQuarter": false,
>>>>>>>>>>>         "last12Months": false,
>>>>>>>>>>>         "last5FinancialYears": false,
>>>>>>>>>>>         "thisSixMonth": false,
>>>>>>>>>>>         "lastQuarter": false,
>>>>>>>>>>>         "thisFinancialYear": false,
>>>>>>>>>>>         "last4Weeks": false,
>>>>>>>>>>>         "last3Months": false,
>>>>>>>>>>>         "thisMonth": false,
>>>>>>>>>>>         "last5Years": false,
>>>>>>>>>>>         "last6BiMonths": false,
>>>>>>>>>>>         "lastFinancialYear": false,
>>>>>>>>>>>         "last6Months": false,
>>>>>>>>>>>         "quartersThisYear": false,
>>>>>>>>>>>         "monthsLastYear": false,
>>>>>>>>>>>         "lastWeek": false,
>>>>>>>>>>>         "thisBimonth": false,
>>>>>>>>>>>         "lastBimonth": false,
>>>>>>>>>>>         "lastSixMonth": false,
>>>>>>>>>>>         "lastYear": false,
>>>>>>>>>>>         "last12Weeks": true,
>>>>>>>>>>>         "last4Quarters": false
>>>>>>>>>>>       },
>>>>>>>>>>>       "programStage": {
>>>>>>>>>>>         "id": "PgGkk7ELhw7"
>>>>>>>>>>>       },
>>>>>>>>>>>       "user": {
>>>>>>>>>>>         "id": "mGMEUNgjTYA"
>>>>>>>>>>>       },
>>>>>>>>>>>       "attributeDimensions": [],
>>>>>>>>>>>       "translations": [],
>>>>>>>>>>>       "filterDimensions": [
>>>>>>>>>>>         "ou"
>>>>>>>>>>>       ],
>>>>>>>>>>>       "organisationUnitGroups": [],
>>>>>>>>>>>       "itemOrganisationUnitGroups": [],
>>>>>>>>>>>       "userGroupAccesses": [],
>>>>>>>>>>>       "programIndicatorDimensions": [],
>>>>>>>>>>>       "columnDimensions": [
>>>>>>>>>>>         "RFGJRT5bFaC"
>>>>>>>>>>>       ],
>>>>>>>>>>>       "organisationUnitLevels": [],
>>>>>>>>>>>       "dataElementDimensions": [
>>>>>>>>>>>         {
>>>>>>>>>>>           "filter": "IN:O28",
>>>>>>>>>>>           "dataElement": {
>>>>>>>>>>>             "id": "RFGJRT5bFaC"
>>>>>>>>>>>           }
>>>>>>>>>>>         }
>>>>>>>>>>>       ],
>>>>>>>>>>>       "periods": [],
>>>>>>>>>>>       "organisationUnits": [
>>>>>>>>>>>         {
>>>>>>>>>>>           "id": "JnEJ2mR2WlB"
>>>>>>>>>>>         }
>>>>>>>>>>>       ],
>>>>>>>>>>>       "rowDimensions": [
>>>>>>>>>>>         "pe"
>>>>>>>>>>>       ]
>>>>>>>>>>>     }
>>>>>>>>>>>   ]
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> Hope you can help me out with this.
>>>>>>>>>>>
>>>>>>>>>>> Best regards!
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> *Sergio Valenzuela*
>>>>>>>>>>> Fundación EHAS
>>>>>>>>>>> Telemedicina para países en desarrollo -* www.ehas.org
>>>>>>>>>>> <http://www.ehas.org/>*
>>>>>>>>>>> E.T.S.I. de Telecomunicación A.101-9L. Ciudad Universitaria 30,
>>>>>>>>>>> 28040 Madrid - España
>>>>>>>>>>> Tlfno. (+34) 91 488 87 41: (+34) 687 67 17 62
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> 2016-06-29 5:18 GMT+02:00 Morten Olav Hansen <morten@xxxxxxxxx>:
>>>>>>>>>>>
>>>>>>>>>>>> @vanyas: Yes this was caught during the validation phase
>>>>>>>>>>>>> itself. The point here would be to catch such issues during configuring
>>>>>>>>>>>>> metadata itself. The use case was: We created some data set at HQ, synced
>>>>>>>>>>>>> to local. Then we created a custom attribute for data set and made it
>>>>>>>>>>>>> mandatory. Then when this version (with custom attribute info) syncs to
>>>>>>>>>>>>> local, we get this validation issues with the importer. Any general
>>>>>>>>>>>>> guideline for such cases.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> I don't disagree on that, but it's not really related to the
>>>>>>>>>>>> importer. Not sure we have any guidelines for that, in general you would
>>>>>>>>>>>> never actually change an attribute like that I think (I wouldn't mind it
>>>>>>>>>>>> being blocked), of course.. people make mistakes.. and we don't really want
>>>>>>>>>>>> to block edits either..
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> We do not support any data approval types at this point (at
>>>>>>>>>>>>>> least nothing has been done to try and support them, it's untested)
>>>>>>>>>>>>>>
>>>>>>>>>>>>> @vanyas: Whats the plan for this in that case? If its part of
>>>>>>>>>>>>> metadata this association should get synced.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> We have a blueprint for adding support for this:
>>>>>>>>>>>>
>>>>>>>>>>>> https://blueprints.launchpad.net/dhis2/+spec/exchange-data-approval
>>>>>>>>>>>>
>>>>>>>>>>>> It's currently set to 2.25, but nobody is assigned..
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Morten
>>>>>>>>>>>>
>>>>>>>>>>>> - when we updated the existing program with adding combination
>>>>>>>>>>>>>>> of categories and synched.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> This should be fixed now
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - while updating attributes in the program with registration.
>>>>>>>>>>>>>>> * ERROR 2016-06-21 21:23:24,325 ERROR: duplicate key value
>>>>>>>>>>>>>>> violates unique constraint ""programtrackedentityattribute_unique_key""
>>>>>>>>>>>>>>>   Detail: Key (programid, trackedentityattributeid)=(255,
>>>>>>>>>>>>>>> 240) already exists. (SqlExceptionHelper.java [taskScheduler-6])
>>>>>>>>>>>>>>> * ERROR 2016-06-21 21:23:24,385 Exception occurred while
>>>>>>>>>>>>>>> trying to import the metadata. could not execute statement
>>>>>>>>>>>>>>> (MetadataImportHandler.java [taskScheduler-6])
>>>>>>>>>>>>>>> org.hibernate.exception.ConstraintViolationException: could
>>>>>>>>>>>>>>> not execute statement at
>>>>>>>>>>>>>>> org.hibernate.exception.internal.SQLStateConversionDelegate.convert(SQLStateConversionDelegate.java:112)"Error
>>>>>>>>>>>>>>> on Edit Program Stage page. Press cancel and exception is thrown. Analytics
>>>>>>>>>>>>>>> Export Error. Process failed. null.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Please try again, I have done several fixes related to issues
>>>>>>>>>>>>>> like this (at least similar issues)
>>>>>>>>>>>>>>
>>>>>>>>>>>>> @vanyas : Thanks. Sure will check.
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - Charts, pivot tables, Report Table/ Pivot Table, Event
>>>>>>>>>>>>>>> Report, Event Chart, Validation Rule
>>>>>>>>>>>>>>> Metadata Sync failed for the above metadata With the below
>>>>>>>>>>>>>>> runtime exception.
>>>>>>>>>>>>>>> --- Exception occurred while trying to import the metadata.
>>>>>>>>>>>>>>> object references an unsaved transient instance - save the transient
>>>>>>>>>>>>>>> instance before flushing: org.hisp.dhis.dataelement.DataElement
>>>>>>>>>>>>>>> (MetadataSyncTask.java [taskScheduler-9])
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> This should be fixed
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - Admin user when added to dashboard item is not syncing with
>>>>>>>>>>>>>>> metadat sync.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Will look into it
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - When deleted a legend at HQ and synced to local. The legend
>>>>>>>>>>>>>>> at Local also is deleted on UI (exists in DB). Also, legend is getting
>>>>>>>>>>>>>>> duplicated with a different UID in DB.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Will look into it
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - Metadata Filter (Import-Export --> Metadata Detailed
>>>>>>>>>>>>>>> Export)
>>>>>>>>>>>>>>> modification to properties like name, description is
>>>>>>>>>>>>>>> successfully synced but the selection is not synced, the payload also have
>>>>>>>>>>>>>>> the details of the updated data when synced.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Not sure how much time we should spend on this, the metadata
>>>>>>>>>>>>>> filter functionality should be replaced in future releases (it's not very
>>>>>>>>>>>>>> stable, and has not been updated to support new types etc)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - custom forms are synced properly when created, but while
>>>>>>>>>>>>>>> Updation of custom form not reflecting on UI. However, payload has the full
>>>>>>>>>>>>>>> information of updates.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Will look into it, is the issue present for both data set and
>>>>>>>>>>>>>> program stages?
>>>>>>>>>>>>>>
>>>>>>>>>>>>> @vanya: This was tested with data sets only.
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Morten
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Thanks & Regards,
>>>>>>>>>>>>>>> Nalinikanth M
>>>>>>>>>>>>>>> Quality Analyst
>>>>>>>>>>>>>>> Email nalinim@xxxxxxxxxxxxxxxx
>>>>>>>>>>>>>>> Telephone +91 9052234588 <+91+9052234588>
>>>>>>>>>>>>>>> [image: ThoughtWorks]
>>>>>>>>>>>>>>> <http://www.thoughtworks.com/?utm_campaign=archana-chillala-signature&utm_medium=email&utm_source=thoughtworks-email-signature-generator>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> With Regards
>>>>>>>>>>>>> ThoughtWorks Technologies
>>>>>>>>>>>>> Hyderabad
>>>>>>>>>>>>>
>>>>>>>>>>>>> --Stay Hungry Stay Foolish!!
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> Mailing list: https://launchpad.net/~dhis2-devs
>>>>>>>>>>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>>>>>>>>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>>>>>>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Follow ups

References