dhis2-devs team mailing list archive

[Halvdan Grelland <halvdanhg@xxxxxxxxx>]

I tried your example GML on both our demo servers (dev on 2.20 and demo on
2.19, same rev as your server) and it works just fine.

If you're experiencing it hanging and not finishing you might be
experiencing some kind of performance and/or memory issue. The coordinate
set is fairly large (though it works well on our demos, but does take some
time to finish). You can verify by trying yourself to create the orgunit
(with the exact name) and import the GML on the Sierra Leone demo.

The fact that the dry run seems to "work" for you (as well as the stack
trace you provided) also points at the issue not residing in the GML
processing but rather in the actual import stage (the GML importer uses the
metadata importer as it's backend).

Another thing: The instance you linked to is using a revision of 16783 and
ver. 2.19, which is sept. 2014 (current is 2.19 is at 19247). Are you using
a custom build? Does it contain customizations to the core in any way? That
would also be a possible source of issues, obviously.

You could try deploying this war
https://apps.dhis2.org/ci/job/dhis-2.19/lastBuild/artifact/dhis-2/dhis-web/dhis-web-portal/target/dhis.war
and replicate the issue using the za South Africa orgunit.

2015-06-26 0:35 GMT+02:00 Calle Hedberg <calle.hedberg@xxxxxxxxx>:

> Halvdan,
>
> The GML files I'm trying to import using 2.19 is the same GML files I've
> been using successfully for 2.17, 2.18 (so it's using NAME). Something in
> the import has changed:
>
> I have attached the small GML file I'm trying to import - it has the
> international boundary of SA, so there is only ONE GIS record (top node in
> all our OrgHierarchies). It imports with no problems in 2.18.
>
> In 2.19, if I choose YES for dry run, it will complete with usual import
> summary etc.
>
> If I choose NO for dry run, I get
> 2015-06-25 22:18:15Importing 1 OrganisationUnits  2015-06-25 22:18:11Importing
> meta-data
> and there it hangs. Nothing gets imported.
>
> The online instance you can try to import into is
> http://fs.dhis.dhmis.org/staging/
>
> When I try to import into a local instance, it seems to enter some kind of
> endless loop based on the tomcat log:
>   at org.hisp.dhis.attribute.Attribute.toString(Attribute.java:397)
>   at sun.reflect.GeneratedMethodAccessor723.invoke(Unknown Source)
>   at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
>   at java.lang.reflect.Method.invoke(Unknown Source)
>   at
> org.hibernate.proxy.pojo.javassist.JavassistLazyInitializer.invoke(JavassistLazyInitializer.java:192)
>   at
> org.hisp.dhis.attribute.Attribute_$$_javassist_16.toString(Attribute_$$_javassist_16.java)
>   at java.lang.String.valueOf(Unknown Source)
>   at java.lang.StringBuilder.append(Unknown Source)
>   at
> org.hisp.dhis.attribute.AttributeValue.toString(AttributeValue.java:136)
>   at java.lang.String.valueOf(Unknown Source)
>   at java.lang.StringBuilder.append(Unknown Source)
>   at java.util.AbstractCollection.toString(Unknown Source)
>   at
> org.hibernate.collection.internal.PersistentSet.toString(PersistentSet.java:327)
>   at java.lang.String.valueOf(Unknown Source)
>   at java.lang.StringBuilder.append(Unknown Source)
>   at
> com.google.common.base.MoreObjects$ToStringHelper.toString(MoreObjects.java:359)
>   at org.hisp.dhis.attribute.Attribute.toString(Attribute.java:397)
>   at sun.reflect.GeneratedMethodAccessor723.invoke(Unknown Source)
>   at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
>   at java.lang.reflect.Method.invoke(Unknown Source)
>   at
> org.hibernate.proxy.pojo.javassist.JavassistLazyInitializer.invoke(JavassistLazyInitializer.java:192)
>   at
> org.hisp.dhis.attribute.Attribute_$$_javassist_16.toString(Attribute_$$_javassist_16.java)
>   at java.lang.String.valueOf(Unknown Source)
>   at java.lang.StringBuilder.append(Unknown Source)
>   at
> org.hisp.dhis.attribute.AttributeValue.toString(AttributeValue.java:136)
>   at java.lang.String.valueOf(Unknown Source)
>   at java.lang.StringBuilder.append(Unknown Source)
>   at java.util.AbstractCollection.toString(Unknown Source)
>   at
> org.hibernate.collection.internal.PersistentSet.toString(PersistentSet.java:327)
>   at java.lang.String.valueOf(Unknown Source)
>   at java.lang.StringBuilder.append(Unknown Source)
>   at
> com.google.common.base.MoreObjects$ToStringHelper.toString(MoreObjects.java:359)
>   at org.hisp.dhis.attribute.Attribute.toString(Attribute.java:397)
>   at sun.reflect.GeneratedMethodAccessor723.invoke(Unknown Source)
>   at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
>   at java.lang.reflect.Method.invoke(Unknown Source)
>
> On 25 June 2015 at 18:16, Halvdan Grelland <halvdanhg@xxxxxxxxx> wrote:
>
>> Hi Calle,
>>
>> The commit you've replied to implements (but not fully, therefore 'WIP',
>> there's another commit following it pretty soon after) better error
>> reporting feedback to the user through the interface. I.e. not having to
>> see the stacktrace on the server to determine if and why something went
>> wrong. I believe you have been among the people requesting this too :)
>>
>> Also, as you're referencing, we've made some other changes to GML import.
>> In particular the importer will now match on uid, code or name, as oppsed
>> to the former mode of matching only on name. This is not reflected in the
>> UI as it is a backend enhancement only, and does not change the import
>> options at all. It IS, however, documented and ready for you to read about.
>> Short version: it matches on a prioritized list of uid -> code -> name.
>> Long version:
>> https://www.dhis2.org/doc/snapshot/en/user/html/ch18s02.html
>>
>> Be adviced that the commit you've referenced is towards development
>> trunk, and is not backported to 2.19, the match on multiple identifiers,
>> however, was introduced in 2.19 and should work in current 2.19 stable. If
>> you're having problems with it you can send me a full stack trace and
>> preferably the GML input data.
>>
>>
>>
>> 2015-06-25 17:54 GMT+02:00 Calle Hedberg <calle.hedberg@xxxxxxxxx>:
>>
>>> Hi,
>>>
>>> I'm unable to import GML files using 2.19 (the same files import fine in
>>> 2.18) - the import simply hangs (endless rolling progress bar icon).
>>>
>>> Will the above bug-fix sort out that problem - I do not understand what
>>> the above actually means, or if the GML format has changed in some way.
>>>
>>> Note also that the final 2.19 release was supposed to include a more
>>> diversified GML import - but I cannot see WHERE to do that (I still see the
>>> same two buttons: choose file and import, either dry run or not)
>>>
>>> Regards
>>> Calle
>>>
>>> On 5 June 2015 at 14:57, <noreply@xxxxxxxxxxxxx> wrote:
>>>
>>>> ------------------------------------------------------------
>>>> revno: 19296
>>>> committer: Halvdan Hoem Grelland <halvdanhg@xxxxxxxxx>
>>>> branch nick: dhis2
>>>> timestamp: Fri 2015-06-05 14:56:34 +0200
>>>> message:
>>>>   WIP proper error reporting on failed GML parsing during import
>>>> added:
>>>>
>>>> dhis-2/dhis-services/dhis-service-dxf2/src/main/java/org/hisp/dhis/dxf2/gml/GmlPreProcessingResult.java
>>>> modified:
>>>>
>>>> dhis-2/dhis-services/dhis-service-dxf2/src/main/java/org/hisp/dhis/dxf2/gml/DefaultGmlImportService.java
>>>>
>>>> dhis-2/dhis-services/dhis-service-dxf2/src/main/java/org/hisp/dhis/dxf2/gml/GmlImportService.java
>>>>
>>>> dhis-2/dhis-services/dhis-service-dxf2/src/test/java/org/hisp/dhis/dxf2/gml/GmlImportServiceTest.java
>>>>
>>>> dhis-2/dhis-support/dhis-support-system/src/main/java/org/hisp/dhis/system/scheduling/SpringScheduler.java
>>>>
>>>> dhis-2/dhis-web/dhis-web-importexport/src/main/java/org/hisp/dhis/importexport/action/dxf2/MetaDataImportAction.java
>>>>
>>>> dhis-2/dhis-web/dhis-web-importexport/src/main/java/org/hisp/dhis/importexport/action/util/ImportMetaDataGmlTask.java
>>>>
>>>> dhis-2/dhis-web/dhis-web-importexport/src/main/resources/org/hisp/dhis/importexport/i18n_module.properties
>>>>
>>>> dhis-2/dhis-web/dhis-web-importexport/src/main/webapp/dhis-web-importexport/importMetaDataSummary.vm
>>>>
>>>>
>>>> --
>>>> lp:dhis2
>>>> https://code.launchpad.net/~dhis2-devs-core/dhis2/trunk
>>>>
>>>> Your team DHIS 2 developers is subscribed to branch lp:dhis2.
>>>> To unsubscribe from this branch go to
>>>> https://code.launchpad.net/~dhis2-devs-core/dhis2/trunk/+edit-subscription
>>>>
>>>> === modified file
>>>> 'dhis-2/dhis-services/dhis-service-dxf2/src/main/java/org/hisp/dhis/dxf2/gml/DefaultGmlImportService.java'
>>>> ---
>>>> dhis-2/dhis-services/dhis-service-dxf2/src/main/java/org/hisp/dhis/dxf2/gml/DefaultGmlImportService.java
>>>>   2015-06-03 15:00:19 +0000
>>>> +++
>>>> dhis-2/dhis-services/dhis-service-dxf2/src/main/java/org/hisp/dhis/dxf2/gml/DefaultGmlImportService.java
>>>>   2015-06-05 12:56:34 +0000
>>>> @@ -32,6 +32,8 @@
>>>>  import com.google.common.base.Strings;
>>>>  import com.google.common.collect.Iterators;
>>>>  import com.google.common.collect.Maps;
>>>> +import org.apache.commons.io.IOUtils;
>>>> +import org.hibernate.Hibernate;
>>>>  import org.hisp.dhis.common.IdentifiableObjectManager;
>>>>  import org.hisp.dhis.common.IdentifiableProperty;
>>>>  import org.hisp.dhis.common.MergeStrategy;
>>>> @@ -88,13 +90,91 @@
>>>>      // GmlImportService implementation
>>>>      //
>>>> -------------------------------------------------------------------------
>>>>
>>>> +    @Transactional
>>>> +    @Override
>>>> +    public GmlPreProcessingResult preProcessGml( InputStream
>>>> inputStream )
>>>> +    {
>>>> +        InputStream dxfStream = null;
>>>> +        MetaData metaData = null;
>>>> +
>>>> +        try
>>>> +        {
>>>> +            dxfStream = transformGml( inputStream );
>>>> +            metaData = renderService.fromXml( dxfStream,
>>>> MetaData.class );
>>>> +        }
>>>> +        catch ( IOException | TransformerException e )
>>>> +        {
>>>> +            return GmlPreProcessingResult.failure( e );
>>>> +        }
>>>> +        finally
>>>> +        {
>>>> +            IOUtils.closeQuietly( dxfStream );
>>>> +        }
>>>> +
>>>> +        Map<String, OrganisationUnit> uidMap  = Maps.newHashMap(),
>>>> codeMap = Maps.newHashMap(), nameMap = Maps.newHashMap();
>>>> +
>>>> +        matchAndFilterOnIdentifiers( metaData.getOrganisationUnits(),
>>>> uidMap, codeMap, nameMap );
>>>> +
>>>> +        Map<String, OrganisationUnit> persistedUidMap  =
>>>> getMatchingPersistedOrgUnits( uidMap.keySet(),  IdentifiableProperty.UID );
>>>> +        Map<String, OrganisationUnit> persistedCodeMap =
>>>> getMatchingPersistedOrgUnits( codeMap.keySet(), IdentifiableProperty.CODE );
>>>> +        Map<String, OrganisationUnit> persistedNameMap =
>>>> getMatchingPersistedOrgUnits( nameMap.keySet(), IdentifiableProperty.NAME );
>>>> +
>>>> +        Iterator<OrganisationUnit> persistedIterator =
>>>> Iterators.concat( persistedUidMap.values().iterator(),
>>>> +            persistedCodeMap.values().iterator(),
>>>> persistedNameMap.values().iterator() );
>>>> +
>>>> +        while ( persistedIterator.hasNext() )
>>>> +        {
>>>> +            OrganisationUnit persisted = persistedIterator.next(),
>>>> imported = null;
>>>> +
>>>> +            if ( !Strings.isNullOrEmpty( persisted.getUid() ) &&
>>>> uidMap.containsKey( persisted.getUid() ) )
>>>> +            {
>>>> +                imported = uidMap.get( persisted.getUid() );
>>>> +            }
>>>> +            else if ( !Strings.isNullOrEmpty( persisted.getCode() ) &&
>>>> codeMap.containsKey( persisted.getCode() ) )
>>>> +            {
>>>> +                imported = codeMap.get( persisted.getCode() );
>>>> +            }
>>>> +            else if ( !Strings.isNullOrEmpty( persisted.getName() ) &&
>>>> nameMap.containsKey( persisted.getName() ) )
>>>> +            {
>>>> +                imported = nameMap.get( persisted.getName() );
>>>> +            }
>>>> +
>>>> +            if ( imported == null || imported.getCoordinates() == null
>>>> || imported.getFeatureType() == null )
>>>> +            {
>>>> +                continue; // Failed to dereference a persisted entity
>>>> for this org unit or geo data incomplete/missing, therefore ignore
>>>> +            }
>>>> +
>>>> +            mergeNonGeoData( persisted, imported );
>>>> +        }
>>>> +
>>>> +        return GmlPreProcessingResult.success( metaData );
>>>> +    }
>>>> +
>>>>      @Override
>>>>      public MetaData fromGml( InputStream inputStream )
>>>>          throws IOException, TransformerException
>>>>      {
>>>> -        InputStream dxfStream = transformGml( inputStream );
>>>> -        MetaData metaData = renderService.fromXml( dxfStream,
>>>> MetaData.class );
>>>> -        dxfStream.close();
>>>> +        InputStream dxfStream;
>>>> +        MetaData metaData;
>>>> +
>>>> +        try
>>>> +        {
>>>> +            dxfStream = transformGml( inputStream );
>>>> +        }
>>>> +        catch (Exception e)
>>>> +        {
>>>> +            dxfStream = null;
>>>> +        }
>>>> +
>>>> +        if(dxfStream != null)
>>>> +        {
>>>> +            metaData = renderService.fromXml( dxfStream,
>>>> MetaData.class );
>>>> +            dxfStream.close();
>>>> +        }
>>>> +        else
>>>> +        {
>>>> +            return null;
>>>> +        }
>>>>
>>>>          Map<String, OrganisationUnit> uidMap  = Maps.newHashMap(),
>>>> codeMap = Maps.newHashMap(), nameMap = Maps.newHashMap();
>>>>
>>>> @@ -143,6 +223,7 @@
>>>>          importService.importMetaData( userUid, fromGml( inputStream ),
>>>> importOptions, taskId );
>>>>      }
>>>>
>>>> +    @Transactional
>>>>      @Override
>>>>      public void importGml( MetaData metaData, String userUid,
>>>> ImportOptions importOptions, TaskId taskId )
>>>>      {
>>>>
>>>> === modified file
>>>> 'dhis-2/dhis-services/dhis-service-dxf2/src/main/java/org/hisp/dhis/dxf2/gml/GmlImportService.java'
>>>> ---
>>>> dhis-2/dhis-services/dhis-service-dxf2/src/main/java/org/hisp/dhis/dxf2/gml/GmlImportService.java
>>>>  2015-06-03 15:00:19 +0000
>>>> +++
>>>> dhis-2/dhis-services/dhis-service-dxf2/src/main/java/org/hisp/dhis/dxf2/gml/GmlImportService.java
>>>>  2015-06-05 12:56:34 +0000
>>>> @@ -62,6 +62,24 @@
>>>>          throws IOException, TransformerException;
>>>>
>>>>      /**
>>>> +     * Pre-process a GML document. The process, in short, entails the
>>>> following:
>>>> +     * <ol>
>>>> +     *     <li>Parse the GML payload and transform it into DXF2
>>>> format</li>
>>>> +     *     <li>Get the given identifiers (uid, code or name) from the
>>>> parsed payload and fetch
>>>> +     *     the corresponding entities from the DB</li>
>>>> +     *     <li>Merge the geospatial data given in the input GML into
>>>> DB entities and return</li>
>>>> +     * </ol>
>>>> +     *
>>>> +     * The result of this process in returned in a {@link
>>>> GmlPreProcessingResult} which
>>>> +     * encapsulates the returned {@link MetaData} object or the
>>>> exception in cause of parse
>>>> +     * failure due to IO errors or malformed input.
>>>> +     *
>>>> +     * @param gmlInputStream the InputStream providing the GML input.
>>>> +     * @return a GmlPreProcessingResult representing the end result of
>>>> the process.
>>>> +     */
>>>> +    GmlPreProcessingResult preProcessGml( InputStream gmlInputStream );
>>>> +
>>>> +    /**
>>>>       * Imports GML data and merges the geospatial data updates into
>>>> the database.
>>>>       * See {@link #fromGml(InputStream)} for details on the underlying
>>>> process.
>>>>       *
>>>>
>>>> === added file
>>>> 'dhis-2/dhis-services/dhis-service-dxf2/src/main/java/org/hisp/dhis/dxf2/gml/GmlPreProcessingResult.java'
>>>> ---
>>>> dhis-2/dhis-services/dhis-service-dxf2/src/main/java/org/hisp/dhis/dxf2/gml/GmlPreProcessingResult.java
>>>>    1970-01-01 00:00:00 +0000
>>>> +++
>>>> dhis-2/dhis-services/dhis-service-dxf2/src/main/java/org/hisp/dhis/dxf2/gml/GmlPreProcessingResult.java
>>>>    2015-06-05 12:56:34 +0000
>>>> @@ -0,0 +1,70 @@
>>>> +package org.hisp.dhis.dxf2.gml;
>>>> +
>>>> +import org.hisp.dhis.dxf2.metadata.MetaData;
>>>> +
>>>> +import java.io.InputStream;
>>>> +
>>>> +/**
>>>> + * Wraps the result of {@link
>>>> GmlImportService#preProcessGml(InputStream)}.
>>>> + * This is necessary when performing GML import on a context where
>>>> exceptions
>>>> + * due to malformed input cannot be caught, thus leaving the user
>>>> uninformed of
>>>> + * the error. This class will wrap the failure and provide the
>>>> Throwable to the
>>>> + * consuming class on error or the resulting MetaData object on
>>>> success.
>>>> + *
>>>> + * @author Halvdan Hoem grelland <halvdanhg@xxxxxxxxx>
>>>> + */
>>>> +public final class GmlPreProcessingResult
>>>> +{
>>>> +    private boolean isSuccess;
>>>> +    private MetaData resultMetaData;
>>>> +    private Throwable throwable;
>>>> +
>>>> +    private GmlPreProcessingResult(){}
>>>> +
>>>> +    public static GmlPreProcessingResult success( MetaData
>>>> resultMetaData ) {
>>>> +        GmlPreProcessingResult result = new GmlPreProcessingResult();
>>>> +        result.setResultMetaData( resultMetaData );
>>>> +        result.setSuccess( true );
>>>> +
>>>> +        return result;
>>>> +    }
>>>> +
>>>> +    public static GmlPreProcessingResult failure( Throwable throwable )
>>>> +    {
>>>> +        GmlPreProcessingResult result = new GmlPreProcessingResult();
>>>> +        result.setThrowable( throwable );
>>>> +        result.setSuccess( false );
>>>> +
>>>> +        return result;
>>>> +    }
>>>> +
>>>> +    private void setSuccess( boolean isSuccess )
>>>> +    {
>>>> +        this.isSuccess = isSuccess;
>>>> +    }
>>>> +
>>>> +    public boolean isSuccess()
>>>> +    {
>>>> +        return isSuccess;
>>>> +    }
>>>> +
>>>> +    private void setResultMetaData( MetaData resultMetaData )
>>>> +    {
>>>> +        this.resultMetaData = resultMetaData;
>>>> +    }
>>>> +
>>>> +    public MetaData getResultMetaData()
>>>> +    {
>>>> +        return resultMetaData;
>>>> +    }
>>>> +
>>>> +    private void setThrowable( Throwable throwable )
>>>> +    {
>>>> +        this.throwable = throwable;
>>>> +    }
>>>> +
>>>> +    public Throwable getThrowable()
>>>> +    {
>>>> +        return throwable;
>>>> +    }
>>>> +}
>>>>
>>>> === modified file
>>>> 'dhis-2/dhis-services/dhis-service-dxf2/src/test/java/org/hisp/dhis/dxf2/gml/GmlImportServiceTest.java'
>>>> ---
>>>> dhis-2/dhis-services/dhis-service-dxf2/src/test/java/org/hisp/dhis/dxf2/gml/GmlImportServiceTest.java
>>>>      2015-01-17 07:41:26 +0000
>>>> +++
>>>> dhis-2/dhis-services/dhis-service-dxf2/src/test/java/org/hisp/dhis/dxf2/gml/GmlImportServiceTest.java
>>>>      2015-06-05 12:56:34 +0000
>>>> @@ -117,4 +117,6 @@
>>>>          assertEquals( 1, units.get(
>>>> "Blindern").getCoordinatesAsList().get( 0 ).getNumberOfCoordinates() );
>>>>          assertEquals( 76, units.get( "Forskningsparken"
>>>> ).getCoordinatesAsList().get(0).getNumberOfCoordinates() );
>>>>      }
>>>> +
>>>> +    // TODO Add test for GmlImportService#preProcessGml(InputStream)
>>>>  }
>>>>
>>>> === modified file
>>>> 'dhis-2/dhis-support/dhis-support-system/src/main/java/org/hisp/dhis/system/scheduling/SpringScheduler.java'
>>>> ---
>>>> dhis-2/dhis-support/dhis-support-system/src/main/java/org/hisp/dhis/system/scheduling/SpringScheduler.java
>>>> 2015-05-30 13:36:07 +0000
>>>> +++
>>>> dhis-2/dhis-support/dhis-support-system/src/main/java/org/hisp/dhis/system/scheduling/SpringScheduler.java
>>>> 2015-06-05 12:56:34 +0000
>>>> @@ -77,7 +77,7 @@
>>>>      {
>>>>          taskExecutor.execute( task );
>>>>      }
>>>> -
>>>> +
>>>>      @Override
>>>>      public boolean scheduleTask( String key, Runnable task, String
>>>> cronExpr )
>>>>      {
>>>>
>>>> === modified file
>>>> 'dhis-2/dhis-web/dhis-web-importexport/src/main/java/org/hisp/dhis/importexport/action/dxf2/MetaDataImportAction.java'
>>>> ---
>>>> dhis-2/dhis-web/dhis-web-importexport/src/main/java/org/hisp/dhis/importexport/action/dxf2/MetaDataImportAction.java
>>>>       2015-05-28 16:10:07 +0000
>>>> +++
>>>> dhis-2/dhis-web/dhis-web-importexport/src/main/java/org/hisp/dhis/importexport/action/dxf2/MetaDataImportAction.java
>>>>       2015-06-05 12:56:34 +0000
>>>> @@ -187,7 +187,7 @@
>>>>          }
>>>>          else if ( "gml".equals( importFormat ) )
>>>>          {
>>>> -            scheduler.executeTask( new ImportMetaDataGmlTask( userId,
>>>> gmlImportService, importOptions, in, taskId ) );
>>>> +            scheduler.executeTask( new ImportMetaDataGmlTask( userId,
>>>> gmlImportService, notifier, importOptions, in, taskId ) );
>>>>          }
>>>>          else if ( "json".equals( importFormat ) || "xml".equals(
>>>> importFormat ) )
>>>>          {
>>>>
>>>> === modified file
>>>> 'dhis-2/dhis-web/dhis-web-importexport/src/main/java/org/hisp/dhis/importexport/action/util/ImportMetaDataGmlTask.java'
>>>> ---
>>>> dhis-2/dhis-web/dhis-web-importexport/src/main/java/org/hisp/dhis/importexport/action/util/ImportMetaDataGmlTask.java
>>>>      2015-04-11 14:06:51 +0000
>>>> +++
>>>> dhis-2/dhis-web/dhis-web-importexport/src/main/java/org/hisp/dhis/importexport/action/util/ImportMetaDataGmlTask.java
>>>>      2015-06-05 12:56:34 +0000
>>>> @@ -28,14 +28,18 @@
>>>>   * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
>>>>   */
>>>>
>>>> +import org.apache.commons.lang.exception.ExceptionUtils;
>>>>  import org.apache.commons.logging.Log;
>>>>  import org.apache.commons.logging.LogFactory;
>>>>  import org.hisp.dhis.dxf2.common.ImportOptions;
>>>>  import org.hisp.dhis.dxf2.gml.GmlImportService;
>>>> +import org.hisp.dhis.dxf2.gml.GmlPreProcessingResult;
>>>>  import org.hisp.dhis.scheduling.TaskId;
>>>> +import org.hisp.dhis.system.notification.NotificationLevel;
>>>> +import org.hisp.dhis.system.notification.Notifier;
>>>> +import org.springframework.web.util.HtmlUtils;
>>>> +import org.xml.sax.SAXParseException;
>>>>
>>>> -import javax.xml.transform.TransformerException;
>>>> -import java.io.IOException;
>>>>  import java.io.InputStream;
>>>>
>>>>  /**
>>>> @@ -50,12 +54,15 @@
>>>>
>>>>      private String userUid;
>>>>
>>>> +
>>>>      //
>>>> -------------------------------------------------------------------------
>>>>      // Dependencies
>>>>      //
>>>> -------------------------------------------------------------------------
>>>>
>>>>      private GmlImportService gmlImportService;
>>>>
>>>> +    private Notifier notifier;
>>>> +
>>>>      private ImportOptions importOptions;
>>>>
>>>>      private InputStream inputStream;
>>>> @@ -64,11 +71,12 @@
>>>>      // Constructors
>>>>      //
>>>> -------------------------------------------------------------------------
>>>>
>>>> -    public ImportMetaDataGmlTask( String userUid, GmlImportService
>>>> gmlImportService,
>>>> +    public ImportMetaDataGmlTask( String userUid, GmlImportService
>>>> gmlImportService, Notifier notifier,
>>>>          ImportOptions importOptions, InputStream inputStream, TaskId
>>>> taskId )
>>>>      {
>>>>          this.userUid = userUid;
>>>>          this.gmlImportService = gmlImportService;
>>>> +        this.notifier = notifier;
>>>>          this.importOptions = importOptions;
>>>>          this.inputStream = inputStream;
>>>>          this.taskId = taskId;
>>>> @@ -83,15 +91,37 @@
>>>>      {
>>>>          importOptions.setImportStrategy( "update" ); // Force update
>>>> only for GML import
>>>>
>>>> -        try
>>>> -        {
>>>> -            gmlImportService.importGml( inputStream, userUid,
>>>> importOptions, taskId );
>>>> -        }
>>>> -        catch ( IOException | TransformerException e )
>>>> -        {
>>>> -            log.error( "Unable to read GML data from input stream", e
>>>> );
>>>> -
>>>> -            throw new RuntimeException( "Failed to parse GML input
>>>> stream", e );
>>>> -        }
>>>> +        GmlPreProcessingResult gmlPreProcessingResult =
>>>> gmlImportService.preProcessGml( inputStream );
>>>> +
>>>> +        if ( !gmlPreProcessingResult.isSuccess() )
>>>> +        {
>>>> +            Throwable throwable =
>>>> gmlPreProcessingResult.getThrowable();
>>>> +            String message = createErrorMessage( throwable );
>>>> +
>>>> +            notifier.notify( taskId, NotificationLevel.ERROR, message,
>>>> false );
>>>> +            log.error( "GML import failed: " + message, throwable );
>>>> +
>>>> +            return;
>>>> +        }
>>>> +
>>>> +        gmlImportService.importGml(
>>>> gmlPreProcessingResult.getResultMetaData(), userUid, importOptions, taskId
>>>> );
>>>> +    }
>>>> +
>>>> +    private String createErrorMessage( Throwable throwable )
>>>> +    {
>>>> +        String message = "";
>>>> +        Throwable rootThrowable = ExceptionUtils.getRootCause(
>>>> throwable );
>>>> +
>>>> +        if ( rootThrowable instanceof SAXParseException )
>>>> +        {
>>>> +            SAXParseException e = (SAXParseException) rootThrowable;
>>>> +            message += "Syntax error on line " + e.getLineNumber() +
>>>> ". " + e.getMessage();
>>>> +        }
>>>> +        else
>>>> +        {
>>>> +            message += rootThrowable.getMessage();
>>>> +        }
>>>> +
>>>> +        return HtmlUtils.htmlEscape( message );
>>>>      }
>>>>  }
>>>>
>>>> === modified file
>>>> 'dhis-2/dhis-web/dhis-web-importexport/src/main/resources/org/hisp/dhis/importexport/i18n_module.properties'
>>>> ---
>>>> dhis-2/dhis-web/dhis-web-importexport/src/main/resources/org/hisp/dhis/importexport/i18n_module.properties
>>>> 2015-02-20 11:17:31 +0000
>>>> +++
>>>> dhis-2/dhis-web/dhis-web-importexport/src/main/resources/org/hisp/dhis/importexport/i18n_module.properties
>>>> 2015-06-05 12:56:34 +0000
>>>> @@ -300,6 +300,7 @@
>>>>  ignored=Ignored
>>>>  conflicts=Conflicts
>>>>  no_conflicts_found=No conflicts found
>>>> +no_import_summary_available=No summary available
>>>>  type=Type
>>>>  count=Count
>>>>  export_as_xml=Export as XML
>>>>
>>>> === modified file
>>>> 'dhis-2/dhis-web/dhis-web-importexport/src/main/webapp/dhis-web-importexport/importMetaDataSummary.vm'
>>>> ---
>>>> dhis-2/dhis-web/dhis-web-importexport/src/main/webapp/dhis-web-importexport/importMetaDataSummary.vm
>>>>       2013-03-21 09:15:24 +0000
>>>> +++
>>>> dhis-2/dhis-web/dhis-web-importexport/src/main/webapp/dhis-web-importexport/importMetaDataSummary.vm
>>>>       2015-06-05 12:56:34 +0000
>>>> @@ -1,4 +1,6 @@
>>>>  <h3>$i18n.getString( "import_summary" )</h3>
>>>> +
>>>> +#if( $summary ) ## ImportSummary can be null on failed GML import
>>>> (pre-processing fails)
>>>>  <h4>$i18n.getString( "import_count" )</h4>
>>>>
>>>>  $summary.importCount.imported Imported<br/>
>>>> @@ -54,3 +56,6 @@
>>>>  #else
>>>>  <p>$i18n.getString( "no_conflicts_found" )</p>
>>>>  #end
>>>> +#else
>>>> +    $i18n.getString( "no_summary_available" )
>>>> +#end
>>>> \ No newline at end of file
>>>>
>>>>
>>>> _______________________________________________
>>>> Mailing list: https://launchpad.net/~dhis2-devs
>>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>>> More help   : https://help.launchpad.net/ListHelp
>>>>
>>>>
>>>
>>>
>>> --
>>>
>>> *******************************************
>>>
>>> Calle Hedberg
>>>
>>> 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
>>>
>>> Tel/fax (home): +27-21-685-6472
>>>
>>> Cell: +27-82-853-5352
>>>
>>> Iridium SatPhone: +8816-315-19274
>>>
>>> Email: calle.hedberg@xxxxxxxxx
>>>
>>> Skype: calle_hedberg
>>>
>>> *******************************************
>>>
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~dhis2-devs
>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>>
>>
>
>
> --
>
> *******************************************
>
> Calle Hedberg
>
> 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
>
> Tel/fax (home): +27-21-685-6472
>
> Cell: +27-82-853-5352
>
> Iridium SatPhone: +8816-315-19274
>
> Email: calle.hedberg@xxxxxxxxx
>
> Skype: calle_hedberg
>
> *******************************************
>
>