dhis2-devs team mailing list archive
-
dhis2-devs team
-
Mailing list archive
-
Message #38263
Re: [Branch ~dhis2-devs-core/dhis2/trunk] Rev 19296: WIP proper error reporting on failed GML parsing during import
So. The stack size thing was just a qualified guess derived from the stack
trace. It's pretty obviously not the cause. I'll wait on feedback from
Morten before going any further on this.
On Jun 27, 2015 6:50 PM, "Calle Hedberg" <calle.hedberg@xxxxxxxxx> wrote:
> Halvdan,
>
> I understand from your last email that the root cause of the problem is a
> 2.19 software issue - the meta data importer in 2.19 is not able to handle
> OrgUnit attribute values when present. That makes sense, and it is likely
> that the root cause of this bug / design flaw is a 2.19 change (since the
> same import worked fine in 2.18). Morten and/or Lars will hopefully know
> what change(s) caused this.
>
> With regard to JVM running out of heap space, and in relation to the
> (outdated - very!) chapter 8.2.8 you referenced:
>
> Firstly, with regard to the 150+ instances we are hosting on our data
> centre servers, we do have some constraints at the moment: Those servers
> have 32GB RAM only, and most instances are in reality running on 1-3GB.
> That will only change from next week, when we start moving to new servers
> with (initially) 128GB RAM. That should on average provide 6-8GB per
> instance (varying between 2 and 32GB).
>
> Secondly, finding the correct Java 7 and 8 (64 bits) and Tomcat 8.23 (64
> bits) memory allocations on my Windows notebook (32GB RAM) has been a
> nightmare - the people writing up java and tomcat documentation are
> useless, they seem unable to give any quantified advice at all where any
> normal person can find it. (In comparison, chapter 8.2.8 although outdated
> is super-clear). I am currently using the following setting for Java:
>
> JAVA_OPTS: -Xms4096m -Xmx8192m -XX:PermSize=512m -XX:MaxPermSize=1024m
>
> For some reason I/we do not understand, it seems Tomcat 8.23 does not use
> CATALINA_OPTS settings, so I ended up using the "Configure Tomcat" and then
> specify the following under the "Java" tab:
> Java options:
> -Dcatalina.home=C:\Program Files\Apache Software Foundation\Tomcat 8.0
> -Dcatalina.base=C:\Program Files\Apache Software Foundation\Tomcat 8.0
> -Djava.endorsed.dirs=C:\Program Files\Apache Software Foundation\Tomcat
> 8.0\endorsed
> -Djava.io.tmpdir=C:\Program Files\Apache Software Foundation\Tomcat
> 8.0\temp
> -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
> -Djava.util.logging.config.file=C:\Program Files\Apache Software
> Foundation\Tomcat 8.0\conf\logging.properties
> -XX:MaxPermSize=4096m
>
> (NOTE: the -XX value here is different from what I currently have in the
> JAVA_OPTS environmental variable - I cannot find any info on which of them
> takes precedence
>
> Also under that same "Java" tab on "Configure Tomcat":
> Initial memory pool: 4096MB
> Maximum memory pool: 8192MB
> Thread stack size: 4096KB (4MB)
>
> Based on your initial advice to increase the stack size, I tried to
> increase it to 8MB (GML import still crashing), 64MB (GML import still
> crashing), and 132MB (GML import went completely haywire). Which is
> understandable, given your subsequent finding that the root cause of the
> problem was not related to heap/stack settings.
>
> So I've set the thread stack size back to 4096KB (4MB) - even if I don't
> know whether that's correct or not (seems high to my layman eyes, but maybe
> DHIS2 require it?)
>
> If anybody actually KNOWS what memory/stack sizes that I optimal for a
> 32GB Windows machine, I would appreciate the advice...
>
> Regards
> Calle
>
> On 26 June 2015 at 20:26, Halvdan Grelland <halvdanhg@xxxxxxxxx> wrote:
>
>> Ok, I could have a look, but am preoccupied at the moment. Will get back
>> to you.
>>
>> In the meantime, it seems that you're facing a stack overflow error,
>> which could be caused by the JVM running out of heap space. Could you
>> please check the allocated heap for tomcat? If it is not being set (in the
>> CATALINA_OPTS env variable), please set it to something quite large and try
>> again.
>>
>> Under chapter 8.2.8 at the following link is a brief guide
>> https://www.dhis2.org/doc/snapshot/en/implementer/html/ch08s02.html
>>
>> Please let me know how this works out for you. :)
>>
>> 2015-06-26 11:44 GMT+02:00 Calle Hedberg <calle.hedberg@xxxxxxxxx>:
>>
>>> Halvdan,
>>>
>>> Here's a stripped down version of the Free State instance (no records in
>>> the data value table). Restore it as za_fs_s (psql -U dhis
>>> -f za_fs_s_26062015ext.sql -d za_fs_s, password dhis)
>>>
>>> You have got that single-record country GML file already.
>>>
>>> I would appreciate any help in figuring out what's happening. Just note
>>> that I'm using Postgresql 9.4, Java 7.79 64bits, and tomcat 8.23.
>>>
>>> Best regards
>>> Calle
>>>
>>> On 26 June 2015 at 11:17, Calle Hedberg <calle.hedberg@xxxxxxxxx> wrote:
>>>
>>>> Hi,
>>>>
>>>> I ran it again (revision 19247) and pulled out the first part of the
>>>> log error:
>>>>
>>>> * INFO 2015-06-26 11:04:18,338 User 'Calle_Hedberg' started import at
>>>> Fri Jun 26 11:04:18 CAT 2015 (DefaultImportService.java [taskScheduler-1])
>>>> * INFO 2015-06-26 11:04:18,348 [Level: INFO, category:
>>>> METADATA_IMPORT, time: Fri Jun 26 11:04:18 CAT 2015, message: Importing
>>>> meta-data] (InMemoryNotifier.java [taskScheduler-1])
>>>> * INFO 2015-06-26 11:04:18,348 Building object-bridge maps
>>>> (preheatCache: true). (DefaultObjectBridge.java [taskScheduler-1])
>>>> * INFO 2015-06-26 11:04:21,141 Building object-bridge maps took 2.79
>>>> seconds. (DefaultObjectBridge.java [taskScheduler-1])
>>>> * INFO 2015-06-26 11:04:21,141 [Level: INFO, category:
>>>> METADATA_IMPORT, time: Fri Jun 26 11:04:21 CAT 2015, message: Importing 1
>>>> OrganisationUnits] (InMemoryNotifier.java [taskScheduler-1])
>>>> * INFO 2015-06-26 11:04:21,151 Deleted objects associated with object
>>>> of type AttributeValue (DefaultDeletionManager.java [taskScheduler-1])
>>>> * INFO 2015-06-26 11:04:21,151 Deleted objects associated with object
>>>> of type AttributeValue (DefaultDeletionManager.java [taskScheduler-1])
>>>> * INFO 2015-06-26 11:04:21,151 Deleted objects associated with object
>>>> of type AttributeValue (DefaultDeletionManager.java [taskScheduler-1])
>>>> * ERROR 2015-06-26 11:04:21,432 Unexpected error occurred in scheduled
>>>> task. (TaskUtils.java [taskScheduler-1])
>>>> java.lang.StackOverflowError
>>>> at java.util.regex.Pattern$BmpCharProperty.<init>(Unknown Source)
>>>> at java.util.regex.Pattern$BmpCharProperty.<init>(Unknown Source)
>>>> at java.util.regex.Pattern$BitClass.<init>(Unknown Source)
>>>> at java.util.regex.Pattern.clazz(Unknown Source)
>>>> at java.util.regex.Pattern.sequence(Unknown Source)
>>>> at java.util.regex.Pattern.expr(Unknown Source)
>>>> at java.util.regex.Pattern.compile(Unknown Source)
>>>> at java.util.regex.Pattern.<init>(Unknown Source)
>>>> at java.util.regex.Pattern.compile(Unknown Source)
>>>> at java.lang.String.replaceAll(Unknown Source)
>>>> at com.google.common.base.MoreObjects.simpleName(MoreObjects.java:134)
>>>> at
>>>> com.google.common.base.MoreObjects.toStringHelper(MoreObjects.java:95)
>>>> at org.hisp.dhis.attribute.Attribute.toString(Attribute.java:396)
>>>> at sun.reflect.GeneratedMethodAccessor626.invoke(Unknown Source)
>>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
>>>> at java.lang.reflect.Method.invoke(Unknown Source)
>>>> at
>>>> org.hibernate.proxy.pojo.javassist.JavassistLazyInitializer.invoke(JavassistLazyInitializer.java:196)
>>>> at
>>>> org.hisp.dhis.attribute.Attribute_$$_jvst804_10.toString(Attribute_$$_jvst804_10.java)
>>>> at java.lang.String.valueOf(Unknown Source)
>>>> at java.lang.StringBuilder.append(Unknown Source)
>>>> at
>>>> org.hisp.dhis.attribute.AttributeValue.toString(AttributeValue.java:122)
>>>> at java.lang.String.valueOf(Unknown Source)
>>>> at java.lang.StringBuilder.append(Unknown Source)
>>>> at java.util.AbstractCollection.toString(Unknown Source)
>>>> at
>>>> org.hibernate.collection.internal.PersistentSet.toString(PersistentSet.java:327)
>>>> at java.lang.String.valueOf(Unknown Source)
>>>> at java.lang.StringBuilder.append(Unknown Source)
>>>> at
>>>> com.google.common.base.MoreObjects$ToStringHelper.toString(MoreObjects.java:359)
>>>>
>>>> Any ideas?
>>>>
>>>> Regards
>>>> Calle
>>>>
>>>> On 26 June 2015 at 10:12, Calle Hedberg <calle.hedberg@xxxxxxxxx>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Firstly, yes we are using a custom revision - although it is SUPPOSED
>>>>> to be based on a 2.19 version from a few days ago. I will check that.
>>>>>
>>>>> BUT I have tried the import using revision 19247 too, and the
>>>>> problem/behaviour is exactly the same - it hangs. Whereas if I import into
>>>>> an older version of the same provincial instance that still runs on 2.18,
>>>>> it imports fine.
>>>>>
>>>>> The fact that the same GML file imports in the Sierra Leone demo - I
>>>>> cannot explain that, and the tomcat log only shows that
>>>>> Attribute.toString() looping...
>>>>>
>>>>> Can I create a stripped down version of the instance and upload it for
>>>>> you to test?
>>>>>
>>>>> Regards
>>>>> Calle
>>>>>
>>>>>
>>>>>
>>>>> On 26 June 2015 at 04:09, Morten Olav Hansen <mortenoh@xxxxxxxxx>
>>>>> wrote:
>>>>>
>>>>>> It seems to be looping in Attribute.toString(), I remember this was
>>>>>> an issue at some point.. but that was a long long time ago.. so maybe you
>>>>>> need a revision bump
>>>>>>
>>>>>> --
>>>>>> Morten
>>>>>>
>>>>>> On Fri, Jun 26, 2015 at 6:52 AM, Halvdan Grelland <
>>>>>> halvdanhg@xxxxxxxxx> wrote:
>>>>>>
>>>>>>> I tried your example GML on both our demo servers (dev on 2.20 and
>>>>>>> demo on 2.19, same rev as your server) and it works just fine.
>>>>>>>
>>>>>>> If you're experiencing it hanging and not finishing you might be
>>>>>>> experiencing some kind of performance and/or memory issue. The coordinate
>>>>>>> set is fairly large (though it works well on our demos, but does take some
>>>>>>> time to finish). You can verify by trying yourself to create the orgunit
>>>>>>> (with the exact name) and import the GML on the Sierra Leone demo.
>>>>>>>
>>>>>>> The fact that the dry run seems to "work" for you (as well as the
>>>>>>> stack trace you provided) also points at the issue not residing in the GML
>>>>>>> processing but rather in the actual import stage (the GML importer uses the
>>>>>>> metadata importer as it's backend).
>>>>>>>
>>>>>>> Another thing: The instance you linked to is using a revision of 16783
>>>>>>> and ver. 2.19, which is sept. 2014 (current is 2.19 is at 19247).
>>>>>>> Are you using a custom build? Does it contain customizations to the core in
>>>>>>> any way? That would also be a possible source of issues, obviously.
>>>>>>>
>>>>>>> You could try deploying this war
>>>>>>> https://apps.dhis2.org/ci/job/dhis-2.19/lastBuild/artifact/dhis-2/dhis-web/dhis-web-portal/target/dhis.war
>>>>>>> and replicate the issue using the za South Africa orgunit.
>>>>>>>
>>>>>>> 2015-06-26 0:35 GMT+02:00 Calle Hedberg <calle.hedberg@xxxxxxxxx>:
>>>>>>>
>>>>>>>> Halvdan,
>>>>>>>>
>>>>>>>> The GML files I'm trying to import using 2.19 is the same GML files
>>>>>>>> I've been using successfully for 2.17, 2.18 (so it's using NAME). Something
>>>>>>>> in the import has changed:
>>>>>>>>
>>>>>>>> I have attached the small GML file I'm trying to import - it has
>>>>>>>> the international boundary of SA, so there is only ONE GIS record (top node
>>>>>>>> in all our OrgHierarchies). It imports with no problems in 2.18.
>>>>>>>>
>>>>>>>> In 2.19, if I choose YES for dry run, it will complete with usual
>>>>>>>> import summary etc.
>>>>>>>>
>>>>>>>> If I choose NO for dry run, I get
>>>>>>>> 2015-06-25 22:18:15Importing 1 OrganisationUnits 2015-06-25
>>>>>>>> 22:18:11Importing meta-data
>>>>>>>> and there it hangs. Nothing gets imported.
>>>>>>>>
>>>>>>>> The online instance you can try to import into is
>>>>>>>> http://fs.dhis.dhmis.org/staging/
>>>>>>>>
>>>>>>>> When I try to import into a local instance, it seems to enter some
>>>>>>>> kind of endless loop based on the tomcat log:
>>>>>>>> at org.hisp.dhis.attribute.Attribute.toString(Attribute.java:397)
>>>>>>>> at sun.reflect.GeneratedMethodAccessor723.invoke(Unknown Source)
>>>>>>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
>>>>>>>> at java.lang.reflect.Method.invoke(Unknown Source)
>>>>>>>> at
>>>>>>>> org.hibernate.proxy.pojo.javassist.JavassistLazyInitializer.invoke(JavassistLazyInitializer.java:192)
>>>>>>>> at
>>>>>>>> org.hisp.dhis.attribute.Attribute_$$_javassist_16.toString(Attribute_$$_javassist_16.java)
>>>>>>>> at java.lang.String.valueOf(Unknown Source)
>>>>>>>> at java.lang.StringBuilder.append(Unknown Source)
>>>>>>>> at
>>>>>>>> org.hisp.dhis.attribute.AttributeValue.toString(AttributeValue.java:136)
>>>>>>>> at java.lang.String.valueOf(Unknown Source)
>>>>>>>> at java.lang.StringBuilder.append(Unknown Source)
>>>>>>>> at java.util.AbstractCollection.toString(Unknown Source)
>>>>>>>> at
>>>>>>>> org.hibernate.collection.internal.PersistentSet.toString(PersistentSet.java:327)
>>>>>>>> at java.lang.String.valueOf(Unknown Source)
>>>>>>>> at java.lang.StringBuilder.append(Unknown Source)
>>>>>>>> at
>>>>>>>> com.google.common.base.MoreObjects$ToStringHelper.toString(MoreObjects.java:359)
>>>>>>>> at org.hisp.dhis.attribute.Attribute.toString(Attribute.java:397)
>>>>>>>> at sun.reflect.GeneratedMethodAccessor723.invoke(Unknown Source)
>>>>>>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
>>>>>>>> at java.lang.reflect.Method.invoke(Unknown Source)
>>>>>>>> at
>>>>>>>> org.hibernate.proxy.pojo.javassist.JavassistLazyInitializer.invoke(JavassistLazyInitializer.java:192)
>>>>>>>> at
>>>>>>>> org.hisp.dhis.attribute.Attribute_$$_javassist_16.toString(Attribute_$$_javassist_16.java)
>>>>>>>> at java.lang.String.valueOf(Unknown Source)
>>>>>>>> at java.lang.StringBuilder.append(Unknown Source)
>>>>>>>> at
>>>>>>>> org.hisp.dhis.attribute.AttributeValue.toString(AttributeValue.java:136)
>>>>>>>> at java.lang.String.valueOf(Unknown Source)
>>>>>>>> at java.lang.StringBuilder.append(Unknown Source)
>>>>>>>> at java.util.AbstractCollection.toString(Unknown Source)
>>>>>>>> at
>>>>>>>> org.hibernate.collection.internal.PersistentSet.toString(PersistentSet.java:327)
>>>>>>>> at java.lang.String.valueOf(Unknown Source)
>>>>>>>> at java.lang.StringBuilder.append(Unknown Source)
>>>>>>>> at
>>>>>>>> com.google.common.base.MoreObjects$ToStringHelper.toString(MoreObjects.java:359)
>>>>>>>> at org.hisp.dhis.attribute.Attribute.toString(Attribute.java:397)
>>>>>>>> at sun.reflect.GeneratedMethodAccessor723.invoke(Unknown Source)
>>>>>>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
>>>>>>>> at java.lang.reflect.Method.invoke(Unknown Source)
>>>>>>>>
>>>>>>>> On 25 June 2015 at 18:16, Halvdan Grelland <halvdanhg@xxxxxxxxx>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Calle,
>>>>>>>>>
>>>>>>>>> The commit you've replied to implements (but not fully, therefore
>>>>>>>>> 'WIP', there's another commit following it pretty soon after) better error
>>>>>>>>> reporting feedback to the user through the interface. I.e. not having to
>>>>>>>>> see the stacktrace on the server to determine if and why something went
>>>>>>>>> wrong. I believe you have been among the people requesting this too :)
>>>>>>>>>
>>>>>>>>> Also, as you're referencing, we've made some other changes to GML
>>>>>>>>> import. In particular the importer will now match on uid, code or name, as
>>>>>>>>> oppsed to the former mode of matching only on name. This is not reflected
>>>>>>>>> in the UI as it is a backend enhancement only, and does not change the
>>>>>>>>> import options at all. It IS, however, documented and ready for you to read
>>>>>>>>> about. Short version: it matches on a prioritized list of uid -> code ->
>>>>>>>>> name. Long version:
>>>>>>>>> https://www.dhis2.org/doc/snapshot/en/user/html/ch18s02.html
>>>>>>>>>
>>>>>>>>> Be adviced that the commit you've referenced is towards
>>>>>>>>> development trunk, and is not backported to 2.19, the match on multiple
>>>>>>>>> identifiers, however, was introduced in 2.19 and should work in current
>>>>>>>>> 2.19 stable. If you're having problems with it you can send me a full stack
>>>>>>>>> trace and preferably the GML input data.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 2015-06-25 17:54 GMT+02:00 Calle Hedberg <calle.hedberg@xxxxxxxxx>
>>>>>>>>> :
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> I'm unable to import GML files using 2.19 (the same files import
>>>>>>>>>> fine in 2.18) - the import simply hangs (endless rolling progress bar icon).
>>>>>>>>>>
>>>>>>>>>> Will the above bug-fix sort out that problem - I do not
>>>>>>>>>> understand what the above actually means, or if the GML format has changed
>>>>>>>>>> in some way.
>>>>>>>>>>
>>>>>>>>>> Note also that the final 2.19 release was supposed to include a
>>>>>>>>>> more diversified GML import - but I cannot see WHERE to do that (I still
>>>>>>>>>> see the same two buttons: choose file and import, either dry run or not)
>>>>>>>>>>
>>>>>>>>>> Regards
>>>>>>>>>> Calle
>>>>>>>>>>
>>>>>>>>>> On 5 June 2015 at 14:57, <noreply@xxxxxxxxxxxxx> wrote:
>>>>>>>>>>
>>>>>>>>>>> ------------------------------------------------------------
>>>>>>>>>>> revno: 19296
>>>>>>>>>>> committer: Halvdan Hoem Grelland <halvdanhg@xxxxxxxxx>
>>>>>>>>>>> branch nick: dhis2
>>>>>>>>>>> timestamp: Fri 2015-06-05 14:56:34 +0200
>>>>>>>>>>> message:
>>>>>>>>>>> WIP proper error reporting on failed GML parsing during import
>>>>>>>>>>> added:
>>>>>>>>>>>
>>>>>>>>>>> dhis-2/dhis-services/dhis-service-dxf2/src/main/java/org/hisp/dhis/dxf2/gml/GmlPreProcessingResult.java
>>>>>>>>>>> modified:
>>>>>>>>>>>
>>>>>>>>>>> dhis-2/dhis-services/dhis-service-dxf2/src/main/java/org/hisp/dhis/dxf2/gml/DefaultGmlImportService.java
>>>>>>>>>>>
>>>>>>>>>>> dhis-2/dhis-services/dhis-service-dxf2/src/main/java/org/hisp/dhis/dxf2/gml/GmlImportService.java
>>>>>>>>>>>
>>>>>>>>>>> dhis-2/dhis-services/dhis-service-dxf2/src/test/java/org/hisp/dhis/dxf2/gml/GmlImportServiceTest.java
>>>>>>>>>>>
>>>>>>>>>>> dhis-2/dhis-support/dhis-support-system/src/main/java/org/hisp/dhis/system/scheduling/SpringScheduler.java
>>>>>>>>>>>
>>>>>>>>>>> dhis-2/dhis-web/dhis-web-importexport/src/main/java/org/hisp/dhis/importexport/action/dxf2/MetaDataImportAction.java
>>>>>>>>>>>
>>>>>>>>>>> dhis-2/dhis-web/dhis-web-importexport/src/main/java/org/hisp/dhis/importexport/action/util/ImportMetaDataGmlTask.java
>>>>>>>>>>>
>>>>>>>>>>> dhis-2/dhis-web/dhis-web-importexport/src/main/resources/org/hisp/dhis/importexport/i18n_module.properties
>>>>>>>>>>>
>>>>>>>>>>> dhis-2/dhis-web/dhis-web-importexport/src/main/webapp/dhis-web-importexport/importMetaDataSummary.vm
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> lp:dhis2
>>>>>>>>>>> https://code.launchpad.net/~dhis2-devs-core/dhis2/trunk
>>>>>>>>>>>
>>>>>>>>>>> Your team DHIS 2 developers is subscribed to branch lp:dhis2.
>>>>>>>>>>> To unsubscribe from this branch go to
>>>>>>>>>>> https://code.launchpad.net/~dhis2-devs-core/dhis2/trunk/+edit-subscription
>>>>>>>>>>>
>>>>>>>>>>> === modified file
>>>>>>>>>>> 'dhis-2/dhis-services/dhis-service-dxf2/src/main/java/org/hisp/dhis/dxf2/gml/DefaultGmlImportService.java'
>>>>>>>>>>> ---
>>>>>>>>>>> dhis-2/dhis-services/dhis-service-dxf2/src/main/java/org/hisp/dhis/dxf2/gml/DefaultGmlImportService.java
>>>>>>>>>>> 2015-06-03 15:00:19 +0000
>>>>>>>>>>> +++
>>>>>>>>>>> dhis-2/dhis-services/dhis-service-dxf2/src/main/java/org/hisp/dhis/dxf2/gml/DefaultGmlImportService.java
>>>>>>>>>>> 2015-06-05 12:56:34 +0000
>>>>>>>>>>> @@ -32,6 +32,8 @@
>>>>>>>>>>> import com.google.common.base.Strings;
>>>>>>>>>>> import com.google.common.collect.Iterators;
>>>>>>>>>>> import com.google.common.collect.Maps;
>>>>>>>>>>> +import org.apache.commons.io.IOUtils;
>>>>>>>>>>> +import org.hibernate.Hibernate;
>>>>>>>>>>> import org.hisp.dhis.common.IdentifiableObjectManager;
>>>>>>>>>>> import org.hisp.dhis.common.IdentifiableProperty;
>>>>>>>>>>> import org.hisp.dhis.common.MergeStrategy;
>>>>>>>>>>> @@ -88,13 +90,91 @@
>>>>>>>>>>> // GmlImportService implementation
>>>>>>>>>>> //
>>>>>>>>>>> -------------------------------------------------------------------------
>>>>>>>>>>>
>>>>>>>>>>> + @Transactional
>>>>>>>>>>> + @Override
>>>>>>>>>>> + public GmlPreProcessingResult preProcessGml( InputStream
>>>>>>>>>>> inputStream )
>>>>>>>>>>> + {
>>>>>>>>>>> + InputStream dxfStream = null;
>>>>>>>>>>> + MetaData metaData = null;
>>>>>>>>>>> +
>>>>>>>>>>> + try
>>>>>>>>>>> + {
>>>>>>>>>>> + dxfStream = transformGml( inputStream );
>>>>>>>>>>> + metaData = renderService.fromXml( dxfStream,
>>>>>>>>>>> MetaData.class );
>>>>>>>>>>> + }
>>>>>>>>>>> + catch ( IOException | TransformerException e )
>>>>>>>>>>> + {
>>>>>>>>>>> + return GmlPreProcessingResult.failure( e );
>>>>>>>>>>> + }
>>>>>>>>>>> + finally
>>>>>>>>>>> + {
>>>>>>>>>>> + IOUtils.closeQuietly( dxfStream );
>>>>>>>>>>> + }
>>>>>>>>>>> +
>>>>>>>>>>> + Map<String, OrganisationUnit> uidMap =
>>>>>>>>>>> Maps.newHashMap(), codeMap = Maps.newHashMap(), nameMap = Maps.newHashMap();
>>>>>>>>>>> +
>>>>>>>>>>> + matchAndFilterOnIdentifiers(
>>>>>>>>>>> metaData.getOrganisationUnits(), uidMap, codeMap, nameMap );
>>>>>>>>>>> +
>>>>>>>>>>> + Map<String, OrganisationUnit> persistedUidMap =
>>>>>>>>>>> getMatchingPersistedOrgUnits( uidMap.keySet(), IdentifiableProperty.UID );
>>>>>>>>>>> + Map<String, OrganisationUnit> persistedCodeMap =
>>>>>>>>>>> getMatchingPersistedOrgUnits( codeMap.keySet(), IdentifiableProperty.CODE );
>>>>>>>>>>> + Map<String, OrganisationUnit> persistedNameMap =
>>>>>>>>>>> getMatchingPersistedOrgUnits( nameMap.keySet(), IdentifiableProperty.NAME );
>>>>>>>>>>> +
>>>>>>>>>>> + Iterator<OrganisationUnit> persistedIterator =
>>>>>>>>>>> Iterators.concat( persistedUidMap.values().iterator(),
>>>>>>>>>>> + persistedCodeMap.values().iterator(),
>>>>>>>>>>> persistedNameMap.values().iterator() );
>>>>>>>>>>> +
>>>>>>>>>>> + while ( persistedIterator.hasNext() )
>>>>>>>>>>> + {
>>>>>>>>>>> + OrganisationUnit persisted =
>>>>>>>>>>> persistedIterator.next(), imported = null;
>>>>>>>>>>> +
>>>>>>>>>>> + if ( !Strings.isNullOrEmpty( persisted.getUid() )
>>>>>>>>>>> && uidMap.containsKey( persisted.getUid() ) )
>>>>>>>>>>> + {
>>>>>>>>>>> + imported = uidMap.get( persisted.getUid() );
>>>>>>>>>>> + }
>>>>>>>>>>> + else if ( !Strings.isNullOrEmpty(
>>>>>>>>>>> persisted.getCode() ) && codeMap.containsKey( persisted.getCode() ) )
>>>>>>>>>>> + {
>>>>>>>>>>> + imported = codeMap.get( persisted.getCode() );
>>>>>>>>>>> + }
>>>>>>>>>>> + else if ( !Strings.isNullOrEmpty(
>>>>>>>>>>> persisted.getName() ) && nameMap.containsKey( persisted.getName() ) )
>>>>>>>>>>> + {
>>>>>>>>>>> + imported = nameMap.get( persisted.getName() );
>>>>>>>>>>> + }
>>>>>>>>>>> +
>>>>>>>>>>> + if ( imported == null || imported.getCoordinates()
>>>>>>>>>>> == null || imported.getFeatureType() == null )
>>>>>>>>>>> + {
>>>>>>>>>>> + continue; // Failed to dereference a persisted
>>>>>>>>>>> entity for this org unit or geo data incomplete/missing, therefore ignore
>>>>>>>>>>> + }
>>>>>>>>>>> +
>>>>>>>>>>> + mergeNonGeoData( persisted, imported );
>>>>>>>>>>> + }
>>>>>>>>>>> +
>>>>>>>>>>> + return GmlPreProcessingResult.success( metaData );
>>>>>>>>>>> + }
>>>>>>>>>>> +
>>>>>>>>>>> @Override
>>>>>>>>>>> public MetaData fromGml( InputStream inputStream )
>>>>>>>>>>> throws IOException, TransformerException
>>>>>>>>>>> {
>>>>>>>>>>> - InputStream dxfStream = transformGml( inputStream );
>>>>>>>>>>> - MetaData metaData = renderService.fromXml( dxfStream,
>>>>>>>>>>> MetaData.class );
>>>>>>>>>>> - dxfStream.close();
>>>>>>>>>>> + InputStream dxfStream;
>>>>>>>>>>> + MetaData metaData;
>>>>>>>>>>> +
>>>>>>>>>>> + try
>>>>>>>>>>> + {
>>>>>>>>>>> + dxfStream = transformGml( inputStream );
>>>>>>>>>>> + }
>>>>>>>>>>> + catch (Exception e)
>>>>>>>>>>> + {
>>>>>>>>>>> + dxfStream = null;
>>>>>>>>>>> + }
>>>>>>>>>>> +
>>>>>>>>>>> + if(dxfStream != null)
>>>>>>>>>>> + {
>>>>>>>>>>> + metaData = renderService.fromXml( dxfStream,
>>>>>>>>>>> MetaData.class );
>>>>>>>>>>> + dxfStream.close();
>>>>>>>>>>> + }
>>>>>>>>>>> + else
>>>>>>>>>>> + {
>>>>>>>>>>> + return null;
>>>>>>>>>>> + }
>>>>>>>>>>>
>>>>>>>>>>> Map<String, OrganisationUnit> uidMap =
>>>>>>>>>>> Maps.newHashMap(), codeMap = Maps.newHashMap(), nameMap = Maps.newHashMap();
>>>>>>>>>>>
>>>>>>>>>>> @@ -143,6 +223,7 @@
>>>>>>>>>>> importService.importMetaData( userUid, fromGml(
>>>>>>>>>>> inputStream ), importOptions, taskId );
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> + @Transactional
>>>>>>>>>>> @Override
>>>>>>>>>>> public void importGml( MetaData metaData, String userUid,
>>>>>>>>>>> ImportOptions importOptions, TaskId taskId )
>>>>>>>>>>> {
>>>>>>>>>>>
>>>>>>>>>>> === modified file
>>>>>>>>>>> 'dhis-2/dhis-services/dhis-service-dxf2/src/main/java/org/hisp/dhis/dxf2/gml/GmlImportService.java'
>>>>>>>>>>> ---
>>>>>>>>>>> dhis-2/dhis-services/dhis-service-dxf2/src/main/java/org/hisp/dhis/dxf2/gml/GmlImportService.java
>>>>>>>>>>> 2015-06-03 15:00:19 +0000
>>>>>>>>>>> +++
>>>>>>>>>>> dhis-2/dhis-services/dhis-service-dxf2/src/main/java/org/hisp/dhis/dxf2/gml/GmlImportService.java
>>>>>>>>>>> 2015-06-05 12:56:34 +0000
>>>>>>>>>>> @@ -62,6 +62,24 @@
>>>>>>>>>>> throws IOException, TransformerException;
>>>>>>>>>>>
>>>>>>>>>>> /**
>>>>>>>>>>> + * Pre-process a GML document. The process, in short,
>>>>>>>>>>> entails the following:
>>>>>>>>>>> + * <ol>
>>>>>>>>>>> + * <li>Parse the GML payload and transform it into DXF2
>>>>>>>>>>> format</li>
>>>>>>>>>>> + * <li>Get the given identifiers (uid, code or name)
>>>>>>>>>>> from the parsed payload and fetch
>>>>>>>>>>> + * the corresponding entities from the DB</li>
>>>>>>>>>>> + * <li>Merge the geospatial data given in the input GML
>>>>>>>>>>> into DB entities and return</li>
>>>>>>>>>>> + * </ol>
>>>>>>>>>>> + *
>>>>>>>>>>> + * The result of this process in returned in a {@link
>>>>>>>>>>> GmlPreProcessingResult} which
>>>>>>>>>>> + * encapsulates the returned {@link MetaData} object or the
>>>>>>>>>>> exception in cause of parse
>>>>>>>>>>> + * failure due to IO errors or malformed input.
>>>>>>>>>>> + *
>>>>>>>>>>> + * @param gmlInputStream the InputStream providing the GML
>>>>>>>>>>> input.
>>>>>>>>>>> + * @return a GmlPreProcessingResult representing the end
>>>>>>>>>>> result of the process.
>>>>>>>>>>> + */
>>>>>>>>>>> + GmlPreProcessingResult preProcessGml( InputStream
>>>>>>>>>>> gmlInputStream );
>>>>>>>>>>> +
>>>>>>>>>>> + /**
>>>>>>>>>>> * Imports GML data and merges the geospatial data updates
>>>>>>>>>>> into the database.
>>>>>>>>>>> * See {@link #fromGml(InputStream)} for details on the
>>>>>>>>>>> underlying process.
>>>>>>>>>>> *
>>>>>>>>>>>
>>>>>>>>>>> === added file
>>>>>>>>>>> 'dhis-2/dhis-services/dhis-service-dxf2/src/main/java/org/hisp/dhis/dxf2/gml/GmlPreProcessingResult.java'
>>>>>>>>>>> ---
>>>>>>>>>>> dhis-2/dhis-services/dhis-service-dxf2/src/main/java/org/hisp/dhis/dxf2/gml/GmlPreProcessingResult.java
>>>>>>>>>>> 1970-01-01 00:00:00 +0000
>>>>>>>>>>> +++
>>>>>>>>>>> dhis-2/dhis-services/dhis-service-dxf2/src/main/java/org/hisp/dhis/dxf2/gml/GmlPreProcessingResult.java
>>>>>>>>>>> 2015-06-05 12:56:34 +0000
>>>>>>>>>>> @@ -0,0 +1,70 @@
>>>>>>>>>>> +package org.hisp.dhis.dxf2.gml;
>>>>>>>>>>> +
>>>>>>>>>>> +import org.hisp.dhis.dxf2.metadata.MetaData;
>>>>>>>>>>> +
>>>>>>>>>>> +import java.io.InputStream;
>>>>>>>>>>> +
>>>>>>>>>>> +/**
>>>>>>>>>>> + * Wraps the result of {@link
>>>>>>>>>>> GmlImportService#preProcessGml(InputStream)}.
>>>>>>>>>>> + * This is necessary when performing GML import on a context
>>>>>>>>>>> where exceptions
>>>>>>>>>>> + * due to malformed input cannot be caught, thus leaving the
>>>>>>>>>>> user uninformed of
>>>>>>>>>>> + * the error. This class will wrap the failure and provide the
>>>>>>>>>>> Throwable to the
>>>>>>>>>>> + * consuming class on error or the resulting MetaData object on
>>>>>>>>>>> success.
>>>>>>>>>>> + *
>>>>>>>>>>> + * @author Halvdan Hoem grelland <halvdanhg@xxxxxxxxx>
>>>>>>>>>>> + */
>>>>>>>>>>> +public final class GmlPreProcessingResult
>>>>>>>>>>> +{
>>>>>>>>>>> + private boolean isSuccess;
>>>>>>>>>>> + private MetaData resultMetaData;
>>>>>>>>>>> + private Throwable throwable;
>>>>>>>>>>> +
>>>>>>>>>>> + private GmlPreProcessingResult(){}
>>>>>>>>>>> +
>>>>>>>>>>> + public static GmlPreProcessingResult success( MetaData
>>>>>>>>>>> resultMetaData ) {
>>>>>>>>>>> + GmlPreProcessingResult result = new
>>>>>>>>>>> GmlPreProcessingResult();
>>>>>>>>>>> + result.setResultMetaData( resultMetaData );
>>>>>>>>>>> + result.setSuccess( true );
>>>>>>>>>>> +
>>>>>>>>>>> + return result;
>>>>>>>>>>> + }
>>>>>>>>>>> +
>>>>>>>>>>> + public static GmlPreProcessingResult failure( Throwable
>>>>>>>>>>> throwable )
>>>>>>>>>>> + {
>>>>>>>>>>> + GmlPreProcessingResult result = new
>>>>>>>>>>> GmlPreProcessingResult();
>>>>>>>>>>> + result.setThrowable( throwable );
>>>>>>>>>>> + result.setSuccess( false );
>>>>>>>>>>> +
>>>>>>>>>>> + return result;
>>>>>>>>>>> + }
>>>>>>>>>>> +
>>>>>>>>>>> + private void setSuccess( boolean isSuccess )
>>>>>>>>>>> + {
>>>>>>>>>>> + this.isSuccess = isSuccess;
>>>>>>>>>>> + }
>>>>>>>>>>> +
>>>>>>>>>>> + public boolean isSuccess()
>>>>>>>>>>> + {
>>>>>>>>>>> + return isSuccess;
>>>>>>>>>>> + }
>>>>>>>>>>> +
>>>>>>>>>>> + private void setResultMetaData( MetaData resultMetaData )
>>>>>>>>>>> + {
>>>>>>>>>>> + this.resultMetaData = resultMetaData;
>>>>>>>>>>> + }
>>>>>>>>>>> +
>>>>>>>>>>> + public MetaData getResultMetaData()
>>>>>>>>>>> + {
>>>>>>>>>>> + return resultMetaData;
>>>>>>>>>>> + }
>>>>>>>>>>> +
>>>>>>>>>>> + private void setThrowable( Throwable throwable )
>>>>>>>>>>> + {
>>>>>>>>>>> + this.throwable = throwable;
>>>>>>>>>>> + }
>>>>>>>>>>> +
>>>>>>>>>>> + public Throwable getThrowable()
>>>>>>>>>>> + {
>>>>>>>>>>> + return throwable;
>>>>>>>>>>> + }
>>>>>>>>>>> +}
>>>>>>>>>>>
>>>>>>>>>>> === modified file
>>>>>>>>>>> 'dhis-2/dhis-services/dhis-service-dxf2/src/test/java/org/hisp/dhis/dxf2/gml/GmlImportServiceTest.java'
>>>>>>>>>>> ---
>>>>>>>>>>> dhis-2/dhis-services/dhis-service-dxf2/src/test/java/org/hisp/dhis/dxf2/gml/GmlImportServiceTest.java
>>>>>>>>>>> 2015-01-17 07:41:26 +0000
>>>>>>>>>>> +++
>>>>>>>>>>> dhis-2/dhis-services/dhis-service-dxf2/src/test/java/org/hisp/dhis/dxf2/gml/GmlImportServiceTest.java
>>>>>>>>>>> 2015-06-05 12:56:34 +0000
>>>>>>>>>>> @@ -117,4 +117,6 @@
>>>>>>>>>>> assertEquals( 1, units.get(
>>>>>>>>>>> "Blindern").getCoordinatesAsList().get( 0 ).getNumberOfCoordinates() );
>>>>>>>>>>> assertEquals( 76, units.get( "Forskningsparken"
>>>>>>>>>>> ).getCoordinatesAsList().get(0).getNumberOfCoordinates() );
>>>>>>>>>>> }
>>>>>>>>>>> +
>>>>>>>>>>> + // TODO Add test for
>>>>>>>>>>> GmlImportService#preProcessGml(InputStream)
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> === modified file
>>>>>>>>>>> 'dhis-2/dhis-support/dhis-support-system/src/main/java/org/hisp/dhis/system/scheduling/SpringScheduler.java'
>>>>>>>>>>> ---
>>>>>>>>>>> dhis-2/dhis-support/dhis-support-system/src/main/java/org/hisp/dhis/system/scheduling/SpringScheduler.java
>>>>>>>>>>> 2015-05-30 13:36:07 +0000
>>>>>>>>>>> +++
>>>>>>>>>>> dhis-2/dhis-support/dhis-support-system/src/main/java/org/hisp/dhis/system/scheduling/SpringScheduler.java
>>>>>>>>>>> 2015-06-05 12:56:34 +0000
>>>>>>>>>>> @@ -77,7 +77,7 @@
>>>>>>>>>>> {
>>>>>>>>>>> taskExecutor.execute( task );
>>>>>>>>>>> }
>>>>>>>>>>> -
>>>>>>>>>>> +
>>>>>>>>>>> @Override
>>>>>>>>>>> public boolean scheduleTask( String key, Runnable task,
>>>>>>>>>>> String cronExpr )
>>>>>>>>>>> {
>>>>>>>>>>>
>>>>>>>>>>> === modified file
>>>>>>>>>>> 'dhis-2/dhis-web/dhis-web-importexport/src/main/java/org/hisp/dhis/importexport/action/dxf2/MetaDataImportAction.java'
>>>>>>>>>>> ---
>>>>>>>>>>> dhis-2/dhis-web/dhis-web-importexport/src/main/java/org/hisp/dhis/importexport/action/dxf2/MetaDataImportAction.java
>>>>>>>>>>> 2015-05-28 16:10:07 +0000
>>>>>>>>>>> +++
>>>>>>>>>>> dhis-2/dhis-web/dhis-web-importexport/src/main/java/org/hisp/dhis/importexport/action/dxf2/MetaDataImportAction.java
>>>>>>>>>>> 2015-06-05 12:56:34 +0000
>>>>>>>>>>> @@ -187,7 +187,7 @@
>>>>>>>>>>> }
>>>>>>>>>>> else if ( "gml".equals( importFormat ) )
>>>>>>>>>>> {
>>>>>>>>>>> - scheduler.executeTask( new ImportMetaDataGmlTask(
>>>>>>>>>>> userId, gmlImportService, importOptions, in, taskId ) );
>>>>>>>>>>> + scheduler.executeTask( new ImportMetaDataGmlTask(
>>>>>>>>>>> userId, gmlImportService, notifier, importOptions, in, taskId ) );
>>>>>>>>>>> }
>>>>>>>>>>> else if ( "json".equals( importFormat ) ||
>>>>>>>>>>> "xml".equals( importFormat ) )
>>>>>>>>>>> {
>>>>>>>>>>>
>>>>>>>>>>> === modified file
>>>>>>>>>>> 'dhis-2/dhis-web/dhis-web-importexport/src/main/java/org/hisp/dhis/importexport/action/util/ImportMetaDataGmlTask.java'
>>>>>>>>>>> ---
>>>>>>>>>>> dhis-2/dhis-web/dhis-web-importexport/src/main/java/org/hisp/dhis/importexport/action/util/ImportMetaDataGmlTask.java
>>>>>>>>>>> 2015-04-11 14:06:51 +0000
>>>>>>>>>>> +++
>>>>>>>>>>> dhis-2/dhis-web/dhis-web-importexport/src/main/java/org/hisp/dhis/importexport/action/util/ImportMetaDataGmlTask.java
>>>>>>>>>>> 2015-06-05 12:56:34 +0000
>>>>>>>>>>> @@ -28,14 +28,18 @@
>>>>>>>>>>> * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
>>>>>>>>>>> */
>>>>>>>>>>>
>>>>>>>>>>> +import org.apache.commons.lang.exception.ExceptionUtils;
>>>>>>>>>>> import org.apache.commons.logging.Log;
>>>>>>>>>>> import org.apache.commons.logging.LogFactory;
>>>>>>>>>>> import org.hisp.dhis.dxf2.common.ImportOptions;
>>>>>>>>>>> import org.hisp.dhis.dxf2.gml.GmlImportService;
>>>>>>>>>>> +import org.hisp.dhis.dxf2.gml.GmlPreProcessingResult;
>>>>>>>>>>> import org.hisp.dhis.scheduling.TaskId;
>>>>>>>>>>> +import org.hisp.dhis.system.notification.NotificationLevel;
>>>>>>>>>>> +import org.hisp.dhis.system.notification.Notifier;
>>>>>>>>>>> +import org.springframework.web.util.HtmlUtils;
>>>>>>>>>>> +import org.xml.sax.SAXParseException;
>>>>>>>>>>>
>>>>>>>>>>> -import javax.xml.transform.TransformerException;
>>>>>>>>>>> -import java.io.IOException;
>>>>>>>>>>> import java.io.InputStream;
>>>>>>>>>>>
>>>>>>>>>>> /**
>>>>>>>>>>> @@ -50,12 +54,15 @@
>>>>>>>>>>>
>>>>>>>>>>> private String userUid;
>>>>>>>>>>>
>>>>>>>>>>> +
>>>>>>>>>>> //
>>>>>>>>>>> -------------------------------------------------------------------------
>>>>>>>>>>> // Dependencies
>>>>>>>>>>> //
>>>>>>>>>>> -------------------------------------------------------------------------
>>>>>>>>>>>
>>>>>>>>>>> private GmlImportService gmlImportService;
>>>>>>>>>>>
>>>>>>>>>>> + private Notifier notifier;
>>>>>>>>>>> +
>>>>>>>>>>> private ImportOptions importOptions;
>>>>>>>>>>>
>>>>>>>>>>> private InputStream inputStream;
>>>>>>>>>>> @@ -64,11 +71,12 @@
>>>>>>>>>>> // Constructors
>>>>>>>>>>> //
>>>>>>>>>>> -------------------------------------------------------------------------
>>>>>>>>>>>
>>>>>>>>>>> - public ImportMetaDataGmlTask( String userUid,
>>>>>>>>>>> GmlImportService gmlImportService,
>>>>>>>>>>> + public ImportMetaDataGmlTask( String userUid,
>>>>>>>>>>> GmlImportService gmlImportService, Notifier notifier,
>>>>>>>>>>> ImportOptions importOptions, InputStream inputStream,
>>>>>>>>>>> TaskId taskId )
>>>>>>>>>>> {
>>>>>>>>>>> this.userUid = userUid;
>>>>>>>>>>> this.gmlImportService = gmlImportService;
>>>>>>>>>>> + this.notifier = notifier;
>>>>>>>>>>> this.importOptions = importOptions;
>>>>>>>>>>> this.inputStream = inputStream;
>>>>>>>>>>> this.taskId = taskId;
>>>>>>>>>>> @@ -83,15 +91,37 @@
>>>>>>>>>>> {
>>>>>>>>>>> importOptions.setImportStrategy( "update" ); // Force
>>>>>>>>>>> update only for GML import
>>>>>>>>>>>
>>>>>>>>>>> - try
>>>>>>>>>>> - {
>>>>>>>>>>> - gmlImportService.importGml( inputStream, userUid,
>>>>>>>>>>> importOptions, taskId );
>>>>>>>>>>> - }
>>>>>>>>>>> - catch ( IOException | TransformerException e )
>>>>>>>>>>> - {
>>>>>>>>>>> - log.error( "Unable to read GML data from input
>>>>>>>>>>> stream", e );
>>>>>>>>>>> -
>>>>>>>>>>> - throw new RuntimeException( "Failed to parse GML
>>>>>>>>>>> input stream", e );
>>>>>>>>>>> - }
>>>>>>>>>>> + GmlPreProcessingResult gmlPreProcessingResult =
>>>>>>>>>>> gmlImportService.preProcessGml( inputStream );
>>>>>>>>>>> +
>>>>>>>>>>> + if ( !gmlPreProcessingResult.isSuccess() )
>>>>>>>>>>> + {
>>>>>>>>>>> + Throwable throwable =
>>>>>>>>>>> gmlPreProcessingResult.getThrowable();
>>>>>>>>>>> + String message = createErrorMessage( throwable );
>>>>>>>>>>> +
>>>>>>>>>>> + notifier.notify( taskId, NotificationLevel.ERROR,
>>>>>>>>>>> message, false );
>>>>>>>>>>> + log.error( "GML import failed: " + message,
>>>>>>>>>>> throwable );
>>>>>>>>>>> +
>>>>>>>>>>> + return;
>>>>>>>>>>> + }
>>>>>>>>>>> +
>>>>>>>>>>> + gmlImportService.importGml(
>>>>>>>>>>> gmlPreProcessingResult.getResultMetaData(), userUid, importOptions, taskId
>>>>>>>>>>> );
>>>>>>>>>>> + }
>>>>>>>>>>> +
>>>>>>>>>>> + private String createErrorMessage( Throwable throwable )
>>>>>>>>>>> + {
>>>>>>>>>>> + String message = "";
>>>>>>>>>>> + Throwable rootThrowable = ExceptionUtils.getRootCause(
>>>>>>>>>>> throwable );
>>>>>>>>>>> +
>>>>>>>>>>> + if ( rootThrowable instanceof SAXParseException )
>>>>>>>>>>> + {
>>>>>>>>>>> + SAXParseException e = (SAXParseException)
>>>>>>>>>>> rootThrowable;
>>>>>>>>>>> + message += "Syntax error on line " +
>>>>>>>>>>> e.getLineNumber() + ". " + e.getMessage();
>>>>>>>>>>> + }
>>>>>>>>>>> + else
>>>>>>>>>>> + {
>>>>>>>>>>> + message += rootThrowable.getMessage();
>>>>>>>>>>> + }
>>>>>>>>>>> +
>>>>>>>>>>> + return HtmlUtils.htmlEscape( message );
>>>>>>>>>>> }
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> === modified file
>>>>>>>>>>> 'dhis-2/dhis-web/dhis-web-importexport/src/main/resources/org/hisp/dhis/importexport/i18n_module.properties'
>>>>>>>>>>> ---
>>>>>>>>>>> dhis-2/dhis-web/dhis-web-importexport/src/main/resources/org/hisp/dhis/importexport/i18n_module.properties
>>>>>>>>>>> 2015-02-20 11:17:31 +0000
>>>>>>>>>>> +++
>>>>>>>>>>> dhis-2/dhis-web/dhis-web-importexport/src/main/resources/org/hisp/dhis/importexport/i18n_module.properties
>>>>>>>>>>> 2015-06-05 12:56:34 +0000
>>>>>>>>>>> @@ -300,6 +300,7 @@
>>>>>>>>>>> ignored=Ignored
>>>>>>>>>>> conflicts=Conflicts
>>>>>>>>>>> no_conflicts_found=No conflicts found
>>>>>>>>>>> +no_import_summary_available=No summary available
>>>>>>>>>>> type=Type
>>>>>>>>>>> count=Count
>>>>>>>>>>> export_as_xml=Export as XML
>>>>>>>>>>>
>>>>>>>>>>> === modified file
>>>>>>>>>>> 'dhis-2/dhis-web/dhis-web-importexport/src/main/webapp/dhis-web-importexport/importMetaDataSummary.vm'
>>>>>>>>>>> ---
>>>>>>>>>>> dhis-2/dhis-web/dhis-web-importexport/src/main/webapp/dhis-web-importexport/importMetaDataSummary.vm
>>>>>>>>>>> 2013-03-21 09:15:24 +0000
>>>>>>>>>>> +++
>>>>>>>>>>> dhis-2/dhis-web/dhis-web-importexport/src/main/webapp/dhis-web-importexport/importMetaDataSummary.vm
>>>>>>>>>>> 2015-06-05 12:56:34 +0000
>>>>>>>>>>> @@ -1,4 +1,6 @@
>>>>>>>>>>> <h3>$i18n.getString( "import_summary" )</h3>
>>>>>>>>>>> +
>>>>>>>>>>> +#if( $summary ) ## ImportSummary can be null on failed GML
>>>>>>>>>>> import (pre-processing fails)
>>>>>>>>>>> <h4>$i18n.getString( "import_count" )</h4>
>>>>>>>>>>>
>>>>>>>>>>> $summary.importCount.imported Imported<br/>
>>>>>>>>>>> @@ -54,3 +56,6 @@
>>>>>>>>>>> #else
>>>>>>>>>>> <p>$i18n.getString( "no_conflicts_found" )</p>
>>>>>>>>>>> #end
>>>>>>>>>>> +#else
>>>>>>>>>>> + $i18n.getString( "no_summary_available" )
>>>>>>>>>>> +#end
>>>>>>>>>>> \ No newline at end of file
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Mailing list: https://launchpad.net/~dhis2-devs
>>>>>>>>>>> Post to : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>>>>>>>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>>>>>>>>>> More help : https://help.launchpad.net/ListHelp
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>>
>>>>>>>>>> *******************************************
>>>>>>>>>>
>>>>>>>>>> Calle Hedberg
>>>>>>>>>>
>>>>>>>>>> 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
>>>>>>>>>>
>>>>>>>>>> Tel/fax (home): +27-21-685-6472
>>>>>>>>>>
>>>>>>>>>> Cell: +27-82-853-5352
>>>>>>>>>>
>>>>>>>>>> Iridium SatPhone: +8816-315-19274
>>>>>>>>>>
>>>>>>>>>> Email: calle.hedberg@xxxxxxxxx
>>>>>>>>>>
>>>>>>>>>> Skype: calle_hedberg
>>>>>>>>>>
>>>>>>>>>> *******************************************
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Mailing list: https://launchpad.net/~dhis2-devs
>>>>>>>>>> Post to : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>>>>>>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>>>>>>>>> More help : https://help.launchpad.net/ListHelp
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>>
>>>>>>>> *******************************************
>>>>>>>>
>>>>>>>> Calle Hedberg
>>>>>>>>
>>>>>>>> 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
>>>>>>>>
>>>>>>>> Tel/fax (home): +27-21-685-6472
>>>>>>>>
>>>>>>>> Cell: +27-82-853-5352
>>>>>>>>
>>>>>>>> Iridium SatPhone: +8816-315-19274
>>>>>>>>
>>>>>>>> Email: calle.hedberg@xxxxxxxxx
>>>>>>>>
>>>>>>>> Skype: calle_hedberg
>>>>>>>>
>>>>>>>> *******************************************
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Mailing list: https://launchpad.net/~dhis2-devs
>>>>>>> Post to : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>>>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>>>>>> More help : https://help.launchpad.net/ListHelp
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> *******************************************
>>>>>
>>>>> Calle Hedberg
>>>>>
>>>>> 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
>>>>>
>>>>> Tel/fax (home): +27-21-685-6472
>>>>>
>>>>> Cell: +27-82-853-5352
>>>>>
>>>>> Iridium SatPhone: +8816-315-19274
>>>>>
>>>>> Email: calle.hedberg@xxxxxxxxx
>>>>>
>>>>> Skype: calle_hedberg
>>>>>
>>>>> *******************************************
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> *******************************************
>>>>
>>>> Calle Hedberg
>>>>
>>>> 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
>>>>
>>>> Tel/fax (home): +27-21-685-6472
>>>>
>>>> Cell: +27-82-853-5352
>>>>
>>>> Iridium SatPhone: +8816-315-19274
>>>>
>>>> Email: calle.hedberg@xxxxxxxxx
>>>>
>>>> Skype: calle_hedberg
>>>>
>>>> *******************************************
>>>>
>>>>
>>>
>>>
>>> --
>>>
>>> *******************************************
>>>
>>> Calle Hedberg
>>>
>>> 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
>>>
>>> Tel/fax (home): +27-21-685-6472
>>>
>>> Cell: +27-82-853-5352
>>>
>>> Iridium SatPhone: +8816-315-19274
>>>
>>> Email: calle.hedberg@xxxxxxxxx
>>>
>>> Skype: calle_hedberg
>>>
>>> *******************************************
>>>
>>>
>>
>
>
> --
>
> *******************************************
>
> Calle Hedberg
>
> 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
>
> Tel/fax (home): +27-21-685-6472
>
> Cell: +27-82-853-5352
>
> Iridium SatPhone: +8816-315-19274
>
> Email: calle.hedberg@xxxxxxxxx
>
> Skype: calle_hedberg
>
> *******************************************
>
>
Follow ups
References
