dhis2-devs team mailing list archive

Thread
Date
[Bug 1337490] [NEW] CSV meta data import does not report correct result

To: dhis2-devs@xxxxxxxxxxxxxxxxxxx
From: Olav P <1337490@xxxxxxxxxxxxxxxxxx>
Date: Thu, 03 Jul 2014 18:35:39 -0000
Reply-to: Bug 1337490 <1337490@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

The CSV meta data import reports that X number of data elements have
been imported, even when they have not. For example, of not including
all columns/parameters (such as aggregation operator), it will report
that the import was successful even when it was not, and the tomcat log
it will show something like this:

* ERROR 2014-07-03 18:30:51,203 Unexpected error occurred in scheduled task. (TaskUtils.java [taskScheduler-1])
org.springframework.dao.DataIntegrityViolationException: could not execute statement; SQL [n/a]; constraint [aggregationtype]; nested exception is org.hibernate.exception.ConstraintViolationException: could not execute statement
	at org.springframework.orm.hibernate4.SessionFactoryUtils.convertHibernateAccessException(SessionFactoryUtils.java:138)
	at org.springframework.orm.hibernate4.HibernateTransactionManager.convertHibernateAccessException(HibernateTransactionManager.java:680)
	at org.springframework.orm.hibernate4.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:562)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:755)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:724)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:475)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:270)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:94)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
	at com.sun.proxy.$Proxy125.importMetaData(Unknown Source)
	at org.hisp.dhis.importexport.action.util.ImportMetaDataCsvTask.run(ImportMetaDataCsvTask.java:88)
	at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:53)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)
Caused by: org.hibernate.exception.ConstraintViolationException: could not execute statement
	at org.hibernate.exception.internal.SQLStateConversionDelegate.convert(SQLStateConversionDelegate.java:128)
	at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:49)
	at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:125)
	at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:110)
	at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:136)
	at org.hibernate.engine.jdbc.batch.internal.NonBatchingBatch.addToBatch(NonBatchingBatch.java:58)
	at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:3058)
	at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:3499)
	at org.hibernate.action.internal.EntityInsertAction.execute(EntityInsertAction.java:88)
	at org.hibernate.engine.spi.ActionQueue.execute(ActionQueue.java:362)
	at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:354)
	at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:275)
	at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:328)
	at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:52)
	at org.hibernate.internal.SessionImpl.flush(SessionImpl.java:1233)
	at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:403)
	at org.hibernate.engine.transaction.internal.jdbc.JdbcTransaction.beforeTransactionCommit(JdbcTransaction.java:101)
	at org.hibernate.engine.transaction.spi.AbstractTransactionImpl.commit(AbstractTransactionImpl.java:175)
	at org.springframework.orm.hibernate4.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:554)
	... 17 more
Caused by: org.postgresql.util.PSQLException: ERROR: null value in column "aggregationtype" violates not-null constraint
  Detail: Failing row contains (29983, v8yKRzdHqIh, null, 2014-07-03 18:30:51.107, 2014-07-03 18:30:51.107, Condom observations, Condom observations, null, null, t, string, null, longText, aggregate, null, null, null, null, f, null, null, null, null, null).
	at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2161)
	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1890)
	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255)
	at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:560)
	at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:417)
	at org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2Statement.java:363)
	at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeUpdate(NewProxyPreparedStatement.java:105)
	at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:133)
	... 31 more

** Affects: dhis2
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of DHIS 2
developers, which is subscribed to DHIS.
https://bugs.launchpad.net/bugs/1337490

Title:
  CSV meta data import does not report correct result

Status in DHIS 2:
  New

Bug description:
  The CSV meta data import reports that X number of data elements have
  been imported, even when they have not. For example, of not including
  all columns/parameters (such as aggregation operator), it will report
  that the import was successful even when it was not, and the tomcat
  log it will show something like this:

  * ERROR 2014-07-03 18:30:51,203 Unexpected error occurred in scheduled task. (TaskUtils.java [taskScheduler-1])
  org.springframework.dao.DataIntegrityViolationException: could not execute statement; SQL [n/a]; constraint [aggregationtype]; nested exception is org.hibernate.exception.ConstraintViolationException: could not execute statement
  	at org.springframework.orm.hibernate4.SessionFactoryUtils.convertHibernateAccessException(SessionFactoryUtils.java:138)
  	at org.springframework.orm.hibernate4.HibernateTransactionManager.convertHibernateAccessException(HibernateTransactionManager.java:680)
  	at org.springframework.orm.hibernate4.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:562)
  	at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:755)
  	at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:724)
  	at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:475)
  	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:270)
  	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:94)
  	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
  	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
  	at com.sun.proxy.$Proxy125.importMetaData(Unknown Source)
  	at org.hisp.dhis.importexport.action.util.ImportMetaDataCsvTask.run(ImportMetaDataCsvTask.java:88)
  	at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:53)
  	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
  	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
  	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
  	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  	at java.lang.Thread.run(Thread.java:744)
  Caused by: org.hibernate.exception.ConstraintViolationException: could not execute statement
  	at org.hibernate.exception.internal.SQLStateConversionDelegate.convert(SQLStateConversionDelegate.java:128)
  	at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:49)
  	at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:125)
  	at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:110)
  	at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:136)
  	at org.hibernate.engine.jdbc.batch.internal.NonBatchingBatch.addToBatch(NonBatchingBatch.java:58)
  	at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:3058)
  	at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:3499)
  	at org.hibernate.action.internal.EntityInsertAction.execute(EntityInsertAction.java:88)
  	at org.hibernate.engine.spi.ActionQueue.execute(ActionQueue.java:362)
  	at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:354)
  	at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:275)
  	at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:328)
  	at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:52)
  	at org.hibernate.internal.SessionImpl.flush(SessionImpl.java:1233)
  	at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:403)
  	at org.hibernate.engine.transaction.internal.jdbc.JdbcTransaction.beforeTransactionCommit(JdbcTransaction.java:101)
  	at org.hibernate.engine.transaction.spi.AbstractTransactionImpl.commit(AbstractTransactionImpl.java:175)
  	at org.springframework.orm.hibernate4.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:554)
  	... 17 more
  Caused by: org.postgresql.util.PSQLException: ERROR: null value in column "aggregationtype" violates not-null constraint
    Detail: Failing row contains (29983, v8yKRzdHqIh, null, 2014-07-03 18:30:51.107, 2014-07-03 18:30:51.107, Condom observations, Condom observations, null, null, t, string, null, longText, aggregate, null, null, null, null, f, null, null, null, null, null).
  	at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2161)
  	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1890)
  	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255)
  	at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:560)
  	at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:417)
  	at org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2Statement.java:363)
  	at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeUpdate(NewProxyPreparedStatement.java:105)
  	at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:133)
  	... 31 more

To manage notifications about this bug go to:
https://bugs.launchpad.net/dhis2/+bug/1337490/+subscriptions
Follow ups

[Bug 1337490] Re: CSV meta data import does not report correct result
From: Robin Martens, 2014-07-21
[Bug 1337490] [NEW] CSV meta data import does not report correct result
From: Olav P, 2014-07-03
References

[Bug 1337490] [NEW] CSV meta data import does not report correct result
From: Olav P, 2014-07-03