dhis2-users team mailing list archive

[Morten Olav Hansen <morten@xxxxxxxxx>]

On Thu, Apr 12, 2018 at 1:54 PM, Olav Poppe <olavpo@xxxxxxxxxx> wrote:

> Hi,  I managed to solve the problem in the end. It was indeed caused by
> one single object being shared with a group I had not included in export…
>

Good

I’m still left with some questions though:
> - even with skipValidation=true the import failed until I identified the
> extra user group - is that a bug, or are validation of user group sharing
> never skipped?
>

No, it simply bypasses the expensive validation phase... it does not ignore
any errors (so they end up throwing hibernate issues), as it says in the
docs, it should really only be used for cases where your metadata is pre
approved to work... if not, it makes it very hard to find issues


> - is "skipValidation=true" the same as "Reference mode" in the GUI? And
> why does neither actually skip/allow invalid references as it sound like
> they do?
>

No, its very different. In 2.28 and later we now have
skipSharing=true/false for this exact use-case, in 2.30 we will also have
skipSharing for exports.


> - could we have a proper error message when this happens?
>

It does not give you a message for normal cases? (i.e. no skip validation)

--
Morten


>
> Olav
>
>
>
>
>
> 12. apr. 2018 kl. 07:40 skrev Morten Olav Hansen <morten@xxxxxxxxx>:
>
> Hi Olav
>
> I see that you include sharing in these payloads, are you sure all user
> groups and users are included? did you try with skipSharing=true ?
> skipValidation is not really recommended, as it will not validate
> anything.. but still requires things to be correct (if its not, it will
> result in lots of hibernate exceptions..)
>
>
> --
> Morten Olav Hansen
> Senior Engineer, DHIS 2
> University of Oslo
> http://www.dhis2.org
>
> On Wed, Apr 11, 2018 at 7:52 PM, Olav Poppe <olavpo@xxxxxxxxxx> wrote:
>
>> Thanks, JM. This is an event progrmame, so it’s not the TE.
>>
>> BUT I just managed to identify the culprit: One optionset was shared with
>> a group not included in the export…
>>
>> Morten, is valudation of sharing not skipped even with
>> skipValidation=true? Would be nice with some feedback on *what* fails..
>>
>> Olav
>>
>>
>> 11. apr. 2018 kl. 13:42 skrev JM Alcantara <jmalcantara1@xxxxxxxxx>:
>>
>> Hi Olav
>> I didn’t check the dates on the thread, I thought Knut was still waiting
>> for an answer.
>> Have you been able to determine which dependencies are affecting the
>> import? Also, if this is a tracker program did you check if the UID of the
>> TE is the same you have in the system?
>> Best regards,
>> JM
>>
>>
>> El 07/04/2018, a las 11:00, Olav Poppe <olavpo@xxxxxxxxxx> escribió:
>>
>> Hi, I might be creating some confusion by reviving Knut’s old thread.
>>
>> I heared from Knut that he resolved it by importing via the API, using
>> the skipValidation=true option to get around dependency issues.
>>
>> I’m getting the same error, but in this case importing a tracker
>> programme. And in my case, skipValidation=true does not help. I initially
>> tried with a metadata file which I had manually put together based on a
>> couple of different exports, but even with a "vanilla" programme dependency
>> export file I get this error, even with skipValidation=true.
>>
>> Isaiah, thanks for the suggestion of doing this object by object - that
>> might work, but I need to share this metadata file with different people
>> and asking each one to import 10-15 files in order is not a good solutiom.
>>
>> Any other suggestions?
>>
>> Regards
>>
>>
>>
>>
>>
>>
>> 7. apr. 2018 kl. 10:47 skrev JM Alcantara <jmalcantara1@xxxxxxxxx>:
>>
>> Hi Knut
>> Try splitting your import by org unit levels and do the import in order,
>> starting with the lowest level (1, 2, 3...)
>> The problem happens because the import creates the objects out of order
>> in regards to dependencies, in your case an org unit before its parent.
>> Best regards,
>> JM
>>
>> On Fri, 6 Apr 2018, 15:57 Olav Poppe, <olavpo@xxxxxxxxxx> wrote:
>>
>>> I’m having the same issue and error importing into an empty 2.27
>>> database. Though in this case a slightly modified dependency export of a
>>> tracker programme, not orgunits.
>>>
>>> Knut, did you find a solution?
>>>
>>> Sending you the payload, Morten.
>>>
>>> Olav
>>>
>>>
>>> 12. jan. 2018 kl. 08:39 skrev Morten Olav Hansen <morten@xxxxxxxxx>:
>>>
>>> That sounds weird Knut, would you mind sharing the org unit payload?
>>>
>>> --
>>> Morten Olav Hansen
>>> Senior Engineer, DHIS 2
>>> University of Oslo
>>> http://www.dhis2.org
>>>
>>> On Fri, Jan 12, 2018 at 2:19 AM, Knut Staring <knutst@xxxxxxxxx> wrote:
>>>
>>>> I have tried doing metadata export without sharing from 2.27 and
>>>> importing into 2.27, just the OU hierarchy and levels.
>>>>
>>>> Importing into a clean instance gives this:
>>>>
>>>>  INFO  2018-01-11 20:16:06,750 [Level: INFO, category: METADATA_IMPORT,
>>>> time: Thu Jan 11 20:16:06 CET 2018, message: (admin) Creating 8 object(s)
>>>> of type OrganisationUnitLevel] (InMemoryNotifier.java [taskScheduler-4])
>>>> * ERROR 2018-01-11 20:16:07,764 Unexpected error occurred in scheduled
>>>> task. (TaskUtils.java [taskScheduler-4])
>>>> org.hibernate.TransientObjectException: object references an unsaved
>>>> transient instance - save the transient instance before flushing:
>>>> org.hisp.dhis.organisationunit.OrganisationUnit
>>>>         at org.hibernate.engine.internal.ForeignKeys.getEntityIdentifie
>>>> rIfNotUnsaved(ForeignKeys.java:279)
>>>>         at org.hibernate.type.EntityType.getIdentifier(EntityType.java:
>>>> 462)
>>>>         at org.hibernate.type.ManyToOneType.isDirty(ManyToOneType.java:
>>>> 281)
>>>>         at org.hibernate.type.ManyToOneType.isDirty(ManyToOneType.java:
>>>> 291)
>>>>         at org.hibernate.type.TypeHelper.findDirty(TypeHelper.java:322)
>>>>         at org.hibernate.persister.entity.AbstractEntityPersister.findD
>>>> irty(AbstractEntityPersister.java:4204)
>>>>         at org.hibernate.event.internal.DefaultFlushEntityEventListener
>>>> .dirtyCheck(DefaultFlushEntityEventListener.java:528)
>>>>         at org.hibernate.event.internal.DefaultFlushEntityEventListener
>>>> .isUpdateNecessary(DefaultFlushEntityEventListener.java:215)
>>>>         at org.hibernate.event.internal.DefaultFlushEntityEventListener
>>>> .onFlushEntity(DefaultFlushEntityEventListener.java:142)
>>>>         at org.hibernate.event.internal.AbstractFlushingEventListener.f
>>>> lushEntities(AbstractFlushingEventListener.java:216)
>>>>         at org.hibernate.event.internal.AbstractFlushingEventListener.f
>>>> lushEverythingToExecutions(AbstractFlushingEventListener.java:85)
>>>>         at org.hibernate.event.internal.DefaultFlushEventListener.onFlu
>>>> sh(DefaultFlushEventListener.java:38)
>>>>         at org.hibernate.internal.SessionImpl.flush(SessionImpl.java:
>>>> 1295)
>>>>         at org.hisp.dhis.dbms.HibernateDbmsManager.clearSession(Hiberna
>>>> teDbmsManager.java:339)
>>>>         at org.hisp.dhis.dxf2.metadata.objectbundle.DefaultObjectBundle
>>>> Service.commit(DefaultObjectBundleService.java:180)
>>>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>         at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
>>>>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
>>>> Source)
>>>>         at java.lang.reflect.Method.invoke(Unknown Source)
>>>>         at org.springframework.aop.support.AopUtils.invokeJoinpointUsin
>>>> gReflection(AopUtils.java:333)
>>>>         at org.springframework.aop.framework.ReflectiveMethodInvocation
>>>> .invokeJoinpoint(ReflectiveMethodInvocation.java:190)
>>>>         at org.springframework.aop.framework.ReflectiveMethodInvocation
>>>> .proceed(ReflectiveMethodInvocation.java:157)
>>>>         at org.springframework.transaction.interceptor.TransactionInter
>>>> ceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
>>>>         at org.springframework.transaction.interceptor.TransactionAspec
>>>> tSupport.invokeWithinTransaction(TransactionAspectSupport.java:282)
>>>>         at org.springframework.transaction.interceptor.TransactionInter
>>>> ceptor.invoke(TransactionInterceptor.java:96)
>>>>         at org.springframework.aop.framework.ReflectiveMethodInvocation
>>>> .proceed(ReflectiveMethodInvocation.java:179)
>>>>         at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(
>>>> JdkDynamicAopProxy.java:213)
>>>>         at com.sun.proxy.$Proxy189.commit(Unknown Source)
>>>>         at org.hisp.dhis.dxf2.metadata.DefaultMetadataImportService.imp
>>>> ortMetadata(DefaultMetadataImportService.java:127)
>>>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>         at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
>>>>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
>>>> Source)
>>>>         at java.lang.reflect.Method.invoke(Unknown Source)
>>>>         at org.springframework.aop.support.AopUtils.invokeJoinpointUsin
>>>> gReflection(AopUtils.java:333)
>>>>         at org.springframework.aop.framework.ReflectiveMethodInvocation
>>>> .invokeJoinpoint(ReflectiveMethodInvocation.java:190)
>>>>         at org.springframework.aop.framework.ReflectiveMethodInvocation
>>>> .proceed(ReflectiveMethodInvocation.java:157)
>>>>         at org.springframework.transaction.interceptor.TransactionInter
>>>> ceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
>>>>         at org.springframework.transaction.interceptor.TransactionAspec
>>>> tSupport.invokeWithinTransaction(TransactionAspectSupport.java:282)
>>>>         at org.springframework.transaction.interceptor.TransactionInter
>>>> ceptor.invoke(TransactionInterceptor.java:96)
>>>>         at org.springframework.aop.framework.ReflectiveMethodInvocation
>>>> .proceed(ReflectiveMethodInvocation.java:179)
>>>>         at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(
>>>> JdkDynamicAopProxy.java:213)
>>>>         at com.sun.proxy.$Proxy191.importMetadata(Unknown Source)
>>>>         at org.hisp.dhis.importexport.action.util.ImportMetaDataTask.
>>>> call(ImportMetaDataTask.java:99)
>>>>         at org.hisp.dhis.security.SecurityContextRunnable.run(SecurityC
>>>> ontextRunnable.java:57)
>>>>         at org.springframework.scheduling.support.DelegatingErrorHandli
>>>> ngRunnable.run(DelegatingErrorHandlingRunnable.java:54)
>>>>         at java.util.concurrent.Executors$RunnableAdapter.call(Unknown
>>>> Source)
>>>>         at java.util.concurrent.FutureTask.run(Unknown Source)
>>>>         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Unknown
>>>> Source)
>>>>         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
>>>> Source)
>>>>         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown
>>>> Source)
>>>>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
>>>> Source)
>>>>         at java.lang.Thread.run(Unknown Source)
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Mailing list: https://launchpad.net/~dhis2-users
>>>> Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
>>>> Unsubscribe : https://launchpad.net/~dhis2-users
>>>> More help   : https://help.launchpad.net/ListHelp
>>>>
>>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~dhis2-users
>>> Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~dhis2-users
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~dhis2-users
>>> Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~dhis2-users
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>
>>
>>
>>
>
>