dhis2-devs team mailing list archive

Thread
Date
Re: Fwd: [Dhis-dev] Export process from 1.4 to 2.0

To: Lars Helge Øverland <larshelge@xxxxxxxxx>
From: Jason Pickering <jason.p.pickering@xxxxxxxxx>
Date: Tue, 17 Mar 2009 14:16:38 +0200
Cc: dhis-users@xxxxxxxxxxxxxxxx, dhis2-devs@xxxxxxxxxxxxxxxxxxx
In-reply-to: <4c28a4a80903110340k563a99e4pcb11572209dbdc0b@mail.gmail.com>
Hi Lars,
I have made the required changes to DHIS, and now it reports using the
Postgreql backend.


About DHIS 2

Current user:
    admin
Version:
    2.0.1-SNAPSHOT
Build date:
    2009-03-03 17:07
User agent:
    Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.6)
Gecko/2009011913 Firefox/3.0.6
External configuration directory:
    c:\dhis2
Environment variable:
    DHIS2_HOME
Database type:
    PostgreSQL
Database name:
    dhis2_zm
Database user:
    dhis


I have tried method (1) on a clean database and obtain the following
errors (from the Tomcat log)

* INFO  14:04:16,440 Environment variable DHIS2_HOME points to
c:\dhis2 (DefaultLocationManager.java [Thread-1])
* INFO  14:04:18,643 Hibernate properties file found:
jar:file:/C:/Program%20Files/Apache%20Software%20Foundation/Tomcat%206.0/webapps/dhis/WEB-INF/lib/dhis-support-hibernate-2.0.1-SNAPSHOT.jar!/hibernate-default.properties
(DefaultHibernateConfigurationProvider.java [Thread-1])
* INFO  14:04:18,643 Hibernate properties file found:
jar:file:/C:/Program%20Files/Apache%20Software%20Foundation/Tomcat%206.0/webapps/dhis/WEB-INF/lib/dhis-web-maintenance-dataadmin.jar!/hibernate.properties
(DefaultHibernateConfigurationProvider.java [Thread-1])
* INFO  14:04:18,658 Hibernate properties file found:
c:\dhis2\hibernate.properties
(DefaultHibernateConfigurationProvider.java [Thread-1])
* INFO  14:04:42,969 Executing startup routine [1 of 11, runlevel 0]:
DataElementDefaultDimensionPopulator
(DefaultStartupRoutineExecutor.java [Thread-1])
* INFO  14:04:59,093 Added default dataelement dimension
(DataElementDefaultDimensionPopulator.java [Thread-1])
* INFO  14:04:59,125 Executing startup routine [2 of 11, runlevel 0]:
DataSetShortNamePopulator (DefaultStartupRoutineExecutor.java
[Thread-1])
* INFO  14:04:59,140 Populated DataSet short name and code
(DataSetShortNamePopulator.java [Thread-1])
* INFO  14:04:59,140 Executing startup routine [3 of 11, runlevel 0]:
CategoryOptionShortNamePopulator (DefaultStartupRoutineExecutor.java
[Thread-1])
* INFO  14:04:59,218 Populated CategoryOption shortname
(CategoryOptionShortNamePopulator.java [Thread-1])
* INFO  14:04:59,218 Executing startup routine [4 of 11, runlevel 0]:
OrganisationUnitHierarchyVerifier (DefaultStartupRoutineExecutor.java
[Thread-1])
* WARN  14:04:59,546 firstResult/maxResults specified with collection
fetch; applying in memory! (QueryTranslatorImpl.java [Thread-1])
* INFO  14:04:59,578 Added organistion unit hierarchy
(OrganisationUnitHierarchyVerifier.java [Thread-1])
* INFO  14:04:59,578 Executing startup routine [5 of 11, runlevel 0]:
OpenHealthDataSourceWriter (DefaultStartupRoutineExecutor.java
[Thread-1])
* INFO  14:04:59,578 Environment variable OPENHEALTH_HOME not set
(OpenHealthDataSourceWriter.java [Thread-1])
* INFO  14:04:59,578 Executing startup routine [6 of 11, runlevel 0]:
ReportTableUpgrader (DefaultStartupRoutineExecutor.java [Thread-1])
* INFO  14:04:59,593 Upgraded report tables (ReportTableUpgrader.java
[Thread-1])
* INFO  14:04:59,593 Executing startup routine [7 of 11, runlevel 1]:
PeriodTypePopulator (DefaultStartupRoutineExecutor.java [Thread-1])
* INFO  14:04:59,625 Executing startup routine [8 of 11, runlevel 2]:
MultiDimensionExpressionUpgrader (DefaultStartupRoutineExecutor.java
[Thread-1])
* INFO  14:04:59,656 Executing startup routine [9 of 11, runlevel 3]:
AggregationTableCreator (DefaultStartupRoutineExecutor.java
[Thread-1])
* INFO  14:04:59,750 Created table aggregateddatavalue
(AggregationTableCreator.java [Thread-1])
* INFO  14:04:59,750 Created table aggregatedindicatorvalue
(AggregationTableCreator.java [Thread-1])
* INFO  14:04:59,828 Created index crosstab on table datavalue
(AggregationTableCreator.java [Thread-1])
* INFO  14:04:59,843 Created table aggregateddatasetcompleteness
(AggregationTableCreator.java [Thread-1])
* INFO  14:04:59,843 Executing startup routine [10 of 11, runlevel 3]:
DataElementCategoryOptionComboStartupRoutine
(DefaultStartupRoutineExecutor.java [Thread-1])
* INFO  14:04:59,906 Verified CategoryOptionCombos
(DataElementCategoryOptionComboStartupRoutine.java [Thread-1])
* INFO  14:04:59,906 Executing startup routine [11 of 11, runlevel
10]: I18nUpgrader (DefaultStartupRoutineExecutor.java [Thread-1])
* INFO  14:05:00,125 I18n Upgrader running using locale engelska
(Storbritannien) (I18nUpgrader.java [Thread-1])
* INFO  14:05:00,187 I81n Upgrader: DataElementCategory (1)
(I18nUpgrader.java [Thread-1])
* INFO  14:05:00,203 I81n Upgrader: DataElementCategoryOption (1)
(I18nUpgrader.java [Thread-1])
* INFO  14:05:00,218 I81n Upgrader: DataElementCategoryCombo (1)
(I18nUpgrader.java [Thread-1])
* INFO  14:05:00,218 All startup routines done
(DefaultStartupRoutineExecutor.java [Thread-1])
* INFO  14:05:02,765 Reading datasource config file from
datasources.xml,
file:/C:/Program%20Files/Apache%20Software%20Foundation/Tomcat%206.0/webapps/openhealth-fp/WEB-INF/classes/datasources.xml
(MondrianOlapQueryProcessor.java [Thread-1])
* INFO  14:05:02,999 xmla output: '<Discover
xmlns="urn:schemas-microsoft-com:xml-analysis">
<RequestType>MDSCHEMA_DIMENSIONS</RequestType>
<Restrictions>
  <RestrictionList>
    <CATALOG_NAME>OpenHealth</CATALOG_NAME>
    <CUBE_NAME>Diseases</CUBE_NAME>
      </RestrictionList>
</Restrictions>
<Properties>
  <PropertyList>
    <DataSourceInfo>OpenHealthData</DataSourceInfo>
    <Catalog>OpenHealth</Catalog>
    <Format>Tabular</Format>
    <Content>SchemaData</Content>
  </PropertyList>
</Properties>
</Discover>
' (MondrianOlapQueryProcessor.java [Thread-1])
17 mar 14:05:12 ERROR [geoserver.global] -
<omitting Geoserver startup messages>

* INFO  14:06:48,946 Imported DataElements
(DefaultDhis14FileImportService.java [Thread-16])
* WARN  14:06:52,837 Value is null for key: '908' (LoggingHashMap.java
[Thread-16])
* ERROR 14:06:52,837 The process threw exception (ProcessExecutor.java
[Thread-16])
java.lang.RuntimeException: Query with RowHandler failed
	at org.hisp.dhis.importexport.dhis14.file.query.IbatisQueryManager.queryWithRowhandler(IbatisQueryManager.java:141)
	at org.hisp.dhis.importexport.dhis14.file.query.IbatisQueryManager.queryWithRowhandler(IbatisQueryManager.java:124)
	at org.hisp.dhis.importexport.dhis14.file.importer.DefaultDhis14FileImportService.importCalculatedDataElements(DefaultDhis14FileImportService.java:346)
	at org.hisp.dhis.importexport.dhis14.file.importer.DefaultDhis14FileImportService.importData(DefaultDhis14FileImportService.java:257)
	at org.hisp.dhis.importexport.ImportInternalProcess.executeStatements(ImportInternalProcess.java:84)
	at org.hisp.dhis.system.process.AbstractStatementInternalProcess.execute(AbstractStatementInternalProcess.java:85)
	at org.hisp.dhis.system.process.AbstractStatementInternalProcess.execute(AbstractStatementInternalProcess.java:37)
	at org.amplecode.cave.process.ProcessExecutor.run(ProcessExecutor.java:126)
	at java.lang.Thread.run(Unknown Source)
Caused by: com.ibatis.common.jdbc.exception.NestedSQLException:
--- The error occurred in sqlmap/calculatedDataElement.sqlmap.xml.
--- The error occurred while applying a result map.
--- Check the calculatedDataElement.calculatedDataElementResultMap.
--- The error happened while setting a property on the result object.
--- Cause: java.lang.NullPointerException
Caused by: java.lang.NullPointerException
	at com.ibatis.sqlmap.engine.mapping.statement.GeneralStatement.executeQueryWithCallback(GeneralStatement.java:188)
	at com.ibatis.sqlmap.engine.mapping.statement.GeneralStatement.executeQueryWithRowHandler(GeneralStatement.java:133)
	at com.ibatis.sqlmap.engine.impl.SqlMapExecutorDelegate.queryWithRowHandler(SqlMapExecutorDelegate.java:644)
	at com.ibatis.sqlmap.engine.impl.SqlMapSessionImpl.queryWithRowHandler(SqlMapSessionImpl.java:121)
	at com.ibatis.sqlmap.engine.impl.SqlMapClientImpl.queryWithRowHandler(SqlMapClientImpl.java:98)
	at org.hisp.dhis.importexport.dhis14.file.query.IbatisQueryManager.queryWithRowhandler(IbatisQueryManager.java:135)
	... 8 more
Caused by: java.lang.NullPointerException
	at org.hisp.dhis.expression.DefaultExpressionService.convertExpression(DefaultExpressionService.java:187)
	at sun.reflect.GeneratedMethodAccessor199.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:198)
	at $Proxy35.convertExpression(Unknown Source)
	at org.hisp.dhis.importexport.dhis14.file.rowhandler.CalculatedDataElementRowHandler.handleRow(CalculatedDataElementRowHandler.java:115)
	at com.ibatis.sqlmap.engine.mapping.statement.RowHandlerCallback.handleResultObject(RowHandlerCallback.java:76)
	at com.ibatis.sqlmap.engine.execution.SqlExecutor.handleResults(SqlExecutor.java:395)
	at com.ibatis.sqlmap.engine.execution.SqlExecutor.executeQuery(SqlExecutor.java:185)
	at com.ibatis.sqlmap.engine.mapping.statement.GeneralStatement.sqlExecuteQuery(GeneralStatement.java:205)
	at com.ibatis.sqlmap.engine.mapping.statement.GeneralStatement.executeQueryWithCallback(GeneralStatement.java:173)
	... 13 more


I would assume this would be caused by the fact I am using a
completely new database.

So essentially, I would like to perform a completely clean
installation from 1.4 to 2.0. We will need to do this on a regular
basis, as 2.0 (at least for the time being in Zambia) will only be
used for reporting and analysis. No data will be entered or modified.
Each quarter, a new import from the master 1.4 database will need to
take place.

Thanks in advance for your help.

I will attempt the second procedure now,and see what happens .

Best regards,
Jason


On Wed, Mar 11, 2009 at 12:40 PM, Lars Helge Øverland
<larshelge@xxxxxxxxx> wrote:
>
>>
>>
>> Here are the details of the system..
>>
>>
>> Current user:
>>    admin
>> Version:
>>    2.0.1-SNAPSHOT
>> Build date:
>>    2009-03-03 17:07
>> User agent:
>>    Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.6)
>> Gecko/2009011913 Firefox/3.0.6
>> External configuration directory:
>>    Not set
>> Environment variable:
>>    DHIS2_HOME
>> Database type:
>>    H2
>> Database name:
>>    dhis2
>> Database user:
>>    sa
>>
>> I do not have a real database setup yet. I am thinking that this may
>> have to do with the lack of an external configuration directory.
>>
>> Let me try and set it, but I assume that there should be a more
>> graceful error returned. :)
>>
>> Regards,
>> Jason
>
>
> Hi Jason,
>
> you need to install DHIS 2 properly:
>
> - Set the DHIS2_HOME environment variable to any folder. In this folder the
> system will create a subfolder called "importexport" where the dhis14 import
> config file will be stored.
>
> - Also set up a database, your system is now running on the default H2
> in-memory database. Install postgresql/mysql, put the attached
> hibernate.properties in the DHIS2_HOME directory and modify it to your db
> setup.
>
> You could also set the "import type" to analysis to reveal problems in the
> DHIS 1.4 datafile related to unique field/foreign key contraints (access is
> more lenient that postgres/mysql).
>
> Then try again, let me know if it works.
>
>
>
> Lars
>
>
>
>
>
>
>
>
Follow ups

Re: Fwd: [Dhis-dev] Export process from 1.4 to 2.0
From: Lars Helge Øverland, 2009-03-18
References

Fwd: [Dhis-dev] Export process from 1.4 to 2.0
From: Jason Pickering, 2009-03-10
Re: Fwd: [Dhis-dev] Export process from 1.4 to 2.0
From: Lars Helge Øverland, 2009-03-11