dhis2-devs team mailing list archive

[Bob Jolliffe <bobjolliffe@xxxxxxxxx>]

On 15 November 2010 14:31, Knut Staring <knutst@xxxxxxxxx> wrote:
> Actually, I was mainly thinking about the logs. There was nothing of
> much of interest at all in the log file that Roger posted, which is why
> I created a bug and not a blueprint.

I think the problem is compounded by use of the batchhandler.  Because
we process many dxf lines and then pass them on to batchhandler to do
its bit, its not so easy to trace the problem of the particular insert
back to the originating dxf element.  And with multiple inserts, as we
use,  this would be even more difficult.  As I said, I don't think
it's impossible to do this, but it is probably not worth the effort.

I think we certainly could do a better job of validating the dxf
against a relaxng or w3c schema and/or test business logic against a
schematron ruleset.  But there would be a problem of doing with this
currently because the dxf file can be extremely large for a big data
dump and so we would be constantly exposed to risk of exhausting
memory if we tried to validate the whole file (data+metadata).  The
solution I think we all probably see but have yet to implement is to
separate the metadata and data import processes more completely.  The
metadata - which is after all the complicated beast where we might
likely find tricky errors - we can validate using a schema based
parser.

The data import has more of a bulk efficiency import requirement but
also a far simpler validation requirement.

When I get the chance I'd like to look more at this.  Currently I am
caught up with (i) doing my tax returns and (ii) remoting for excel
pivot tables :-)  Hopefully get head up soon.

Cheers
Bob

>
> After messing a bit on my own machine, I got a more informative message:
>
> "Caused by: org.h2.jdbc.JdbcSQLException: NULL not allowed for column
> "NAME"; SQL statement:"
>
> So the problem had nothing to do with IDs in this case, but seems to be
> caused by a blank name, which was interpreted as NULL.
>
> But would it not be feasible to pinpoint the place in the file that
> caused this? Would not the information be readily available? I am
> thinking primarily for the log, which is often too opaque and brief to
> be really useful, and it's a hassle to step through these big DXF files
> with thousands of orgunits.
>
> --
> missing-feedback-on-import-failure
> https://bugs.launchpad.net/bugs/675527
> You received this bug notification because you are a member of DHIS 2
> coordinators, which is the registrant for DHIS.
>
> Status in DHIS 2 - District Health Information Software: New
>
> Bug description:
> When a DXF import fails, the user should get feedback on which XML element causes the failure, ideally with the line in the import file.
>
>
>

-- 
missing-feedback-on-import-failure
https://bugs.launchpad.net/bugs/675527
You received this bug notification because you are a member of DHIS 2
developers, which is subscribed to DHIS.

Status in DHIS 2 - District Health Information Software: New

Bug description:
When a DXF import fails, the user should get feedback on which XML element causes the failure, ideally with the line in the import file.