← Back to team overview

dhis2-users team mailing list archive

  1. dhis2-users team
  2. Mailing list archive
  3. Message #07131

Re: inport meta data to option set

  Thread Previous • Date Previous • • Thread Next 
A small update. I noticed the analytics module is also not working anymore so it’s probably not linked specifically to the import module. Here’s the catalina.out log from the analytics:


* INFO  2015-05-07 15:07:18,046 [Level: INFO, category: DATAMART, time: Thu May 07 15:07:18 CAT 2015, message: Analytics table update process started] (InMemoryNotifier.java [taskScheduler-3])
* INFO  2015-05-07 15:07:18,047 [Level: INFO, category: DATAMART, time: Thu May 07 15:07:18 CAT 2015, message: Updating resource tables] (InMemoryNotifier.java [taskScheduler-3])
* WARN  2015-05-07 15:07:18,050 SQL Error: 0, SQLState: 42703 (SqlExceptionHelper.java [taskScheduler-3])
* ERROR 2015-05-07 15:07:18,050 ERROR: column this_.query does not exist
  Position: 275 (SqlExceptionHelper.java [taskScheduler-3])
* INFO  2015-05-07 15:07:18,051 [Level: ERROR, category: DATAMART, time: Thu May 07 15:07:18 CAT 2015, message: Process failed: could not extract ResultSet] (InMemoryNotifier.java [taskScheduler-3])

I read in a previous post (see attached) this could be linked to the incorrect cleaning of the webapps folder when upgrading to 2.18 (which we recently did). Can this indeed be the issue and how to ensure it is properly cleaned when upgrading?

Regards,

Robin

From: Dhis2-users [mailto:dhis2-users-bounces+martens=sher.be@xxxxxxxxxxxxxxxxxxx] On Behalf Of Robin Martens
Sent: 07 May 2015 17:50
To: Knut Staring
Cc: dhis2-users@xxxxxxxxxxxxxxxxxxx
Subject: Re: [Dhis2-users] inport meta data to option set

Thanks Knut. However, I tried both with and without quotation marks. No difference. I also tried both adding an option to an existing option set and creating a new option set, also no difference…

From: Knut Staring [mailto:knutst@xxxxxxxxx]
Sent: 07 May 2015 15:14
To: Robin Martens
Cc: Lars Helge Øverland; Simon Muyambo; dhis2-users@xxxxxxxxxxxxxxxxxxx
Subject: Re: [Dhis2-users] inport meta data to option set

I think all the headers have to be lower case, as in the manual?

And everything else has to have quotation marks, including the optionsetname (Sector)

On Thu, May 7, 2015 at 2:57 PM, Robin Martens <martens@xxxxxxx<mailto:martens@xxxxxxx>> wrote:
Hi Lars,

The changed format also doesn’t work, have the following error in Catalina. I wanted to see if I could recreate the error on the demo server but can’t access the import/export module. Is this temporarily (reset at midnight) or permanently?


* INFO  2015-05-07 14:34:41,316 User 'admin' started import at Thu May 07 14:34:41 CAT 2015 (DefaultImportService.java [taskScheduler-7])
* INFO  2015-05-07 14:34:41,316 [Level: INFO, category: METADATA_IMPORT, time: Thu May 07 14:34:41 CAT 2015, message: Importing meta-data] (InMemoryNotifier.java [taskScheduler-7])
* INFO  2015-05-07 14:34:41,316 Building object-bridge maps (preheatCache: true). (DefaultObjectBridge.java [taskScheduler-7])
* WARN  2015-05-07 14:34:41,381 SQL Error: 0, SQLState: 42703 (SqlExceptionHelper.java [taskScheduler-7])
* ERROR 2015-05-07 14:34:41,382 ERROR: column this_.query does not exist
  Position: 275 (SqlExceptionHelper.java [taskScheduler-7])

My import settings were:


•        CSV

•        Dry run YES

•        Option set

Attached the file I tried to import (just a single option for a single optionset).

Regards,

Robin



From: Lars Helge Øverland [mailto:larshelge@xxxxxxxxx<mailto:larshelge@xxxxxxxxx>]
Sent: 29 April 2015 22:36
To: Robin Martens
Cc: Simon Muyambo; Knut Staring; dhis2-users@xxxxxxxxxxxxxxxxxxx<mailto:dhis2-users@xxxxxxxxxxxxxxxxxxx>

Subject: Re: [Dhis2-users] inport meta data to option set

Hi there,

the CSV option set import format changed in 2.18. The option code is now required. Have a look at the docs:

https://www.dhis2.org/doc/snapshot/en/developer/html/ch01s09.html


best regards,

Lars



On Wed, Apr 29, 2015 at 7:22 PM, Robin Martens <martens@xxxxxxx<mailto:martens@xxxxxxx>> wrote:
Hi all,

Did anyone encounter the same issue as the one described below? I.e. import conflicts when importing option sets (via CSV)?

Thanks in advance,

Robin

From: Dhis2-users [mailto:dhis2-users-bounces+martens<mailto:dhis2-users-bounces%2Bmartens>=sher.be@xxxxxxxxxxxxxxxxxxx<mailto:sher.be@xxxxxxxxxxxxxxxxxxx>] On Behalf Of Robin Martens
Sent: 28 April 2015 16:35
To: Simon Muyambo; 'Knut Staring'

Cc: dhis2-users@xxxxxxxxxxxxxxxxxxx<mailto:dhis2-users@xxxxxxxxxxxxxxxxxxx>
Subject: Re: [Dhis2-users] inport meta data to option set

Hi Simon and Knut,

I’m currently facing the same issue. Did you manage to solve it at the time and if so, how?

Kind regards,

Robin

From: Dhis2-users [mailto:dhis2-users-bounces+martens<mailto:dhis2-users-bounces%2Bmartens>=sher.be@xxxxxxxxxxxxxxxxxxx<mailto:sher.be@xxxxxxxxxxxxxxxxxxx>] On Behalf Of Simon Muyambo
Sent: 21 December 2014 02:17
To: 'Knut Staring'; 'Simon Muyambo'
Cc: dhis2-users@xxxxxxxxxxxxxxxxxxx<mailto:dhis2-users@xxxxxxxxxxxxxxxxxxx>
Subject: Re: [Dhis2-users] inport meta data to option set

Dear Knut

I have created option set and exported it in JSON. I then deleted the optionset and tried to import it. Below are the same errors I am getting with csv file. Never seems to be able to import members of the option set

Current user:
admin
Version:
2.17
Build revision:
17561
Build date:
2014-11-21 12:21


{"created":"2014-12-20T23:44:06.855+0000","optionSets":[{"id":"etvtSKUhoSZ","name":"prog","created":"2014-12-

20T23:42:20.447+0000","lastUpdated":"2014-12-20T23:43:21.551+0000","options":

[{"id":"v0DNS4VKf40","code":"g1","name":"George","created":"2014-12-20T23:43:07.726+0000","lastUpdated":"2014-12-

20T23:43:07.726+0000"},{"id":"Ghi9DN5UYfu","code":"t1","name":"Tom","created":"2014-12-20T23:43:21.550+0000","lastUpdated":"2014-

12-20T23:43:21.550+0000"}],"version":1}]}

Import summary
Import count
1 Imported
0 Updated
0 Ignored
Type Summary
Type      Imported             Updated              Ignored
OptionSet           1              0              0
Conflicts
Type      Element               Description
OptionSet           prog       Unknown reference to IdentifiableObject{id=0, uid='v0DNS4VKf40', code='g1', name='George', created=Sun Dec 21 01:43:07 CAT 2014, lastUpdated=Sun Dec 21 01:43:07 CAT 2014} (Option) on object IdentifiableObject{id=3, uid='etvtSKUhoSZ', code='null', name='prog', created=Sun Dec 21 01:42:20 CAT 2014, lastUpdated=Sun Dec 21 02:09:50 CAT 2014} (OptionSet).
OptionSet           prog       Unknown reference to IdentifiableObject{id=0, uid='Ghi9DN5UYfu', code='t1', name='Tom', created=Sun Dec 21 01:43:21 CAT 2014, lastUpdated=Sun Dec 21 01:43:21 CAT 2014} (Option) on object IdentifiableObject{id=3, uid='etvtSKUhoSZ', code='null', name='prog', created=Sun Dec 21 01:42:20 CAT 2014, lastUpdated=Sun Dec 21 02:09:50 CAT 2014} (OptionSet).

Regards
Simon Muyambo


From: Knut Staring [mailto:knutst@xxxxxxxxx<mailto:knutst@xxxxxxxxx>]
Sent: 19 December 2014 21:49
To: Simon Muyambo
Cc: Jason Pickering; dhis2-users@xxxxxxxxxxxxxxxxxxx<mailto:dhis2-users@xxxxxxxxxxxxxxxxxxx>
Subject: Re: [Dhis2-users] inport meta data to option set

I think you need to have quotes aroudn any text strings, including the codes

On Fri, Dec 19, 2014 at 8:40 PM, Simon Muyambo <smmuyambo@xxxxxxxxx<mailto:smmuyambo@xxxxxxxxx>> wrote:
Dear Knut and Jason

Thanks for the advice. I have run some tests with unique codes but not sure what the error means

name,uid,code,option
"Colorsm",,1b,"Blue1"
"Colorsm",,2g,"Green1"
"Gendersm",,3f,"Female1"
"Gendersm",,4m,"Male1"
Import summary
Import count
2 Imported
2 Updated
0 Ignored
Type Summary
Type

Imported

Updated

Ignored

OptionSet

2

2

0

Conflicts
Type

Element

Description

OptionSet

Gendersm

Unknown reference to IdentifiableObject{id=0, uid='Ja0hb0d1XGK', code='Female1', name='Female1', created=Fri Dec 19 21:34:19 CAT 2014, lastUpdated=Fri Dec 19 21:34:19 CAT 2014} (Option) on object IdentifiableObject{id=0, uid='qvy5Iof8De3', code='3f', name='Gendersm', created=null, lastUpdated=null} (OptionSet).

OptionSet

Colorsm

Unknown reference to IdentifiableObject{id=0, uid='jPyLUhETxWH', code='Green1', name='Green1', created=Fri Dec 19 21:34:19 CAT 2014, lastUpdated=Fri Dec 19 21:34:19 CAT 2014} (Option) on object IdentifiableObject{id=0, uid='JGfUphTsLuk', code='2g', name='Colorsm', created=null, lastUpdated=null} (OptionSet).

OptionSet

Gendersm

Unknown reference to IdentifiableObject{id=0, uid='mJsHyly4UoH', code='Male1', name='Male1', created=Fri Dec 19 21:34:19 CAT 2014, lastUpdated=Fri Dec 19 21:34:19 CAT 2014} (Option) on object IdentifiableObject{id=0, uid='IxdnSBPyo9y', code='4m', name='Gendersm', created=null, lastUpdated=null} (OptionSet).

OptionSet

Colorsm

Unknown reference to IdentifiableObject{id=0, uid='pSmhTjz0yIS', code='Blue1', name='Blue1', created=Fri Dec 19 21:34:19 CAT 2014, lastUpdated=Fri Dec 19 21:34:19 CAT 2014} (Option) on object IdentifiableObject{id=0, uid='MVHPvOLojgd', code='1b', name='Colorsm', created=null, lastUpdated=null} (OptionSet).



Regards
Simon Muyambo


From: Knut Staring [mailto:knutst@xxxxxxxxx<mailto:knutst@xxxxxxxxx>]
Sent: 19 December 2014 07:03
To: Simon Muyambo
Cc: Jason Pickering; dhis2-users@xxxxxxxxxxxxxxxxxxx<mailto:dhis2-users@xxxxxxxxxxxxxxxxxxx>
Subject: Re: [Dhis2-users] inport meta data to option set


I think maybe you have to have something in the code field
On 19 Dec 2014 02:54, "Simon Muyambo" <smmuyambo@xxxxxxxxx<mailto:smmuyambo@xxxxxxxxx>> wrote:
Dear Jason

Please advise on the import of option set. I find option set is important but the members are not when I use CSV import. Any ideas? See example below
name,uid,code,option

Color,,,"Blue"

Color,,,"Green"

Gender,,,"Female"

Gender,,,"Male"



Regards
Simon Muyambo



__________ Information from ESET NOD32 Antivirus, version of virus signature database 10900 (20141218) __________

The message was checked by ESET NOD32 Antivirus.

http://www.eset.com

_______________________________________________
Mailing list: https://launchpad.net/~dhis2-users
Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx<mailto:dhis2-users@xxxxxxxxxxxxxxxxxxx>
Unsubscribe : https://launchpad.net/~dhis2-users
More help   : https://help.launchpad.net/ListHelp


__________ Information from ESET NOD32 Antivirus, version of virus signature database 10900 (20141218) __________


The message was checked by ESET NOD32 Antivirus.


http://www.eset.com


__________ Information from ESET NOD32 Antivirus, version of virus signature database 10905 (20141219) __________

The message was checked by ESET NOD32 Antivirus.

http://www.eset.com



--
Knut Staring
Dept. of Informatics, University of Oslo
Liberia: +231 770 496 123<tel:%2B231%20770%20496%20123> or +231 886 146 381
Norway: +4791880522<tel:%2B4791880522>
Skype: knutstar
http://dhis2.org


__________ Information from ESET NOD32 Antivirus, version of virus signature database 10905 (20141219) __________


The message was checked by ESET NOD32 Antivirus.


http://www.eset.com


__________ Information from ESET NOD32 Antivirus, version of virus signature database 10909 (20141220) __________

The message was checked by ESET NOD32 Antivirus.

http://www.eset.com

_______________________________________________
Mailing list: https://launchpad.net/~dhis2-users
Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx<mailto:dhis2-users@xxxxxxxxxxxxxxxxxxx>
Unsubscribe : https://launchpad.net/~dhis2-users
More help   : https://help.launchpad.net/ListHelp




--
Knut Staring
Dept. of Informatics, University of Oslo
Norway: +4791880522
Skype: knutstar
http://dhis2.org
--- Begin Message --- 
Thanks a lot Bob.

My site is back up; my webapps directory was not completely clear as you mentioned.


Thanks,

Ese


On Sun, Apr 12, 2015 at 9:04 PM, Bob Jolliffe <bobjolliffe@xxxxxxxxx<mailto:bobjolliffe@xxxxxxxxx>> wrote:


   Hi Ese

   I wonder did you end up with bits of the old and the new after
   upgrade.  Make sure you (i) completely clear out webapps directory
   before deploying 2.18 war file and (ii) make sure you clear cache
   completely in browser and also proxy cache if you are using proxy.

   Bob


   On 12 April 2015 at 20:58, Ese Egerega <ese@xxxxxxxxxx<mailto:ese@xxxxxxxxxx>> wrote:
   > Thanks for your response Alex.
   >
   > I actually added the column "active" and it fixed the initial error of
   > "column organisati1_.active" but I still cannot login. Below is the new
   > error am getting. Am wondering if it has anything to do with the version of
   > tomcat am using (7.0.57).
   >
   > * INFO  2015-04-12 19:47:55,227 Login success for user: 'admin', ip:
   > '41.71.158.221' (DefaultUserAuditService.java [ajp-bio-8009-exec-21])
   > * ERROR 2015-04-12 19:47:55,876 Error while executing action
   > (ExceptionInterceptor.java [ajp-bio-8009-exec-21])
   > org.hibernate.PropertyAccessException: Null value was assigned to a property
   > of primitive type setter of
   > org.hisp.dhis.organisationunit.OrganisationUnit.active
   >     at
   > org.hibernate.property.BasicPropertyAccessor$BasicSetter.set(BasicPropertyAccessor.java:108)
   >     at
   > org.hibernate.tuple.entity.AbstractEntityTuplizer.setPropertyValues(AbstractEntityTuplizer.java:710)
   >     at
   > org.hibernate.tuple.entity.PojoEntityTuplizer.setPropertyValues(PojoEntityTuplizer.java:371)
   >     at
   > org.hibernate.persister.entity.AbstractEntityPersister.setPropertyValues(AbstractEntityPersister.java:4499)
   >     at
   > org.hibernate.engine.internal.TwoPhaseLoad.doInitializeEntity(TwoPhaseLoad.java:185)
   >     at
   > org.hibernate.engine.internal.TwoPhaseLoad.initializeEntity(TwoPhaseLoad.java:137)
   >     at
   > org.hibernate.loader.Loader.initializeEntitiesAndCollections(Loader.java:1103)
   >     at org.hibernate.loader.Loader.processResultSet(Loader.java:960)
   >
   >     at
   > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
   >     at
   > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
   >     at
   > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
   >     at
   > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421)
   >     at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:190)
   >     at
   > org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611)
   >     at
   > org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314)
   >     at
   > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   >     at
   > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   >     at
   > org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
   >     at java.lang.Thread.run(Thread.java:745)
   > Caused by: java.lang.IllegalArgumentException
   >     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   >     at
   > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   >     at
   > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   >     at java.lang.reflect.Method.invoke(Method.java:606)
   >     at
   > org.hibernate.property.BasicPropertyAccessor$BasicSetter.set(BasicPropertyAccessor.java:65)
   >     ... 109 more
   > * WARN  2015-04-12 19:47:55,879 HHH000100: Fail-safe cleanup (collections) :
   > org.hibernate.engine.loading.internal.CollectionLoadContext@4b562655<rs=com.mchange.v2.c3p0.impl.NewProxyResultSet@4daf242c>
   > (LoadContexts.java [ajp-bio-8009-exec-21])
   > * WARN  2015-04-12 19:47:55,879 HHH000160: On CollectionLoadContext#cleanup,
   > localLoadingCollectionKeys contained [1] entries (CollectionLoadContext.java
   > [ajp-bio-8009-exec-21])
   >
   >
   > On Sun, Apr 12, 2015 at 8:29 PM, Alex Tumwesigye <atumwesigye@xxxxxxxxx<mailto:atumwesigye@xxxxxxxxx>>
   > wrote:
   >>
   >> Dear Ese,
   >>
   >> Please drop the column 'active' on table organisationunit. It was dropped
   >> in 2.18. See upgrade notes: https://www.dhis2.org/218-upgrade
   >>
   >> Alex
   >>
   >> On Sun, Apr 12, 2015 at 7:17 PM, Ese Egerega <ese@xxxxxxxxxx<mailto:ese@xxxxxxxxxx>> wrote:
   >>>
   >>> Dear All,
   >>>
   >>> Am having trouble with my dhis2 instance after upgrading from dhis2.17 to
   >>> 2.18. After loading the 2.18 war file, I no longer have access to the
   >>> system. The home page loads quite alright but when I try to login I get the
   >>> following error message:
   >>>
   >>> An exception occurred
   >>>
   >>> Sorry! The system failed to execute the operation. Usually, no data is
   >>> lost and you can continue working by going back to the previous page.
   >>>
   >>> My tomcat error log shows something like column organisati1_.active does
   >>> not exist.
   >>>
   >>> See details below:
   >>>
   >>> * WARN  2015-04-12 15:39:57,998 SQL Error: 0, SQLState: 42703
   >>> (SqlExceptionHelper.java [ajp-bio-8009-exec-18])
   >>> * ERROR 2015-04-12 15:39:57,998 ERROR: column organisati1_.active does
   >>> not exist
   >>>   Position: 608 (SqlExceptionHelper.java [ajp-bio-8009-exec-18])
   >>> * ERROR 2015-04-12 15:39:57,998 Error while executing action
   >>> (ExceptionInterceptor.java [ajp-bio-8009-exec-18])
   >>> org.hibernate.exception.SQLGrammarException: could not extract ResultSet
   >>>
   >>> I will appreciate any advice
   >>> .
   >>>
   >>> Thanks,
   >>> Ese
   >>>
   >>> _______________________________________________
   >>> Mailing list: https://launchpad.net/~dhis2-users
   >>> Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx<mailto:dhis2-users@xxxxxxxxxxxxxxxxxxx>
   >>> Unsubscribe : https://launchpad.net/~dhis2-users
   >>> More help   : https://help.launchpad.net/ListHelp
   >>>
   >>
   >>
   >>
   >> --
   >> Alex Tumwesigye
   >>
   >> Technical Advisor - DHIS2 (Consultant),
   >> Ministry of Health/AFENET
   >> Kampala
   >> Uganda
   >>
   >> IT Consultant - BarefootPower Uganda Ltd, SmartSolar, Kenya
   >>
   >> IT Specialist (Servers, Networks and Security, Health Information Systems
   >> - DHIS2 ) & Solar Consultant
   >>
   >> +256 774149 775, + 256 759 800161
   >>
   >> "I don't want to be anything other than what I have been - one tree hill "
   >
   >
   >

   > _______________________________________________
   > Mailing list: https://launchpad.net/~dhis2-devs
   > Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx<mailto:dhis2-devs@xxxxxxxxxxxxxxxxxxx>
   > Unsubscribe : https://launchpad.net/~dhis2-devs
   > More help   : https://help.launchpad.net/ListHelp
   >




_______________________________________________
Mailing list: https://launchpad.net/~dhis2-users
Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
Unsubscribe : https://launchpad.net/~dhis2-users
More help   : https://help.launchpad.net/ListHelp

--- End Message ---

References

  Thread Previous • Date Previous • • Thread Next