← Back to team overview

dhis2-devs team mailing list archive

Re: Aggregating values

 

I am saying the truth.

This is what I have on my catalina.out:
* INFO  16:31:50,284 User login success: 'admin'
(DefaultUserAuditService.java [http-8081-3])
Jul 14, 2010 4:31:50 PM org.apache.velocity.runtime.log.JdkLogChute log
SEVERE: #parse() null argument

Caveman


On Wed, Jul 14, 2010 at 5:35 PM, Jason Pickering <
jason.p.pickering@xxxxxxxxx> wrote:

> Hi Caveman,
> There must be something being logged in apache-tomcat-X.X.XX/logs
> directory. Open up a terminal, and then do tail -f catalina.out and in a
> separate terminal, start Tomcat. Post the result and we may be able to help.
>
>
> Let me know if something is not clear.
>
> Regards,
> Jason
>
>
>
> On Wed, Jul 14, 2010 at 4:40 PM, Orvalho Augusto <orvaquim@xxxxxxxxx>wrote:
>
>> There is nothing being logged.
>>
>> There are no views.
>>
>> Caveman
>>
>>
>>
>> On Wed, Jul 14, 2010 at 9:03 AM, Jason Pickering <
>> jason.p.pickering@xxxxxxxxx> wrote:
>>
>>> Hi Caveman,
>>>
>>> Can you please post a stack trace of your application (use
>>> pastebin.com)  from startup until you start the datamart/report table
>>> process?
>>>
>>> Do you have any external views that are linked to the report table you
>>> are attempting to aggregate?
>>>
>>> Regards,
>>> Jason
>>>
>>>
>>> On Wed, Jul 14, 2010 at 8:57 AM, Orvalho Augusto <orvaquim@xxxxxxxxx>
>>> wrote:
>>> > And even do those resource tables on Postgres it does not aggregate.
>>> >
>>> > Caveman
>>> >
>>> > On Wed, Jul 14, 2010 at 6:35 AM, Orvalho Augusto <orvaquim@xxxxxxxxx>
>>> wrote:
>>> >>
>>> >> I have changed the database (from MySQL to Postgres)... and used DHIS
>>> >> tools to export and import the data.
>>> >>
>>> >> Well, those resource tables are being built in seconds on Postgres
>>> now. So
>>> >> I think there is a bug with MySQL. And I want to help to correct it.
>>> >>
>>> >> One verry bad consequence of not importing the dataelementids is: The
>>> >> entering forms does not work no more. We have to redesign the forms
>>> after an
>>> >> import.
>>> >>
>>> >> Caveman
>>> >>
>>> >>
>>> >> On Tue, Jul 13, 2010 at 10:28 PM, Orvalho Augusto <orvaquim@xxxxxxxxx
>>> >
>>> >> wrote:
>>> >>>
>>> >>> Ok.
>>> >>>
>>> >>>
>>> >>> On Tue, Jul 13, 2010 at 11:55 AM, Bob Jolliffe <
>>> bobjolliffe@xxxxxxxxx>
>>> >>> wrote:
>>> >>>>
>>> >>>> On 13 July 2010 07:17, Orvalho Augusto <orvaquim@xxxxxxxxx> wrote:
>>> >>>> > I recompiled DHIS2.05 yesterday with help of a magician Jason
>>> >>>> > yesterday. And
>>> >>>> > the first thing I tried was this aggregation thing.
>>> >>>> >
>>> >>>> > So I started redoing the orgunitstructure. It gave the same error.
>>> >>>> >
>>> >>>> > While it was running I used the
>>> >>>> > show full processlist
>>> >>>> > to find out that it is a verry long insert to be run at once. I
>>> have
>>> >>>> > 1664
>>> >>>> > orgunits. So it sounds like indexing trouble.
>>> >>>> >
>>> >>>> > As there is a faith and "antimysql fight" I am moving my data to
>>> >>>> > postgres. I
>>> >>>> > have to do it manually.
>>> >>>>
>>> >>>> There is no mysql fight. Just more people using postgres I guess.
>>> >>>>
>>> >>>> >
>>> >>>> > I have to do it manually because DHIS does not keep de ids fixed
>>> >>>> > (organisationunid, dataelementid etc). That affects our mapping
>>> data
>>> >>>> > for
>>> >>>> > data importation from other systems. So if someone could suggest a
>>> >>>> > good
>>> >>>> > strategy is welcomed.
>>> >>>>
>>> >>>> How are you doing mapping data from other systems?  If you tell me a
>>> >>>> bit about these systems and the data they have maybe we can make a
>>> >>>> plan :-)
>>> >>>
>>> >>> Basically these systems are:
>>> >>> The dataset (the DHIS  concept) is represented by a table (in EpiInfo
>>> >>> 6.06, DBF or MSAccess) where each record should have a month (or week
>>> or
>>> >>> quarter), year, province, district and health center columns -
>>> sometimes
>>> >>> they place the name of the association or a special sector of the
>>> health
>>> >>> center - then the other columns define a dataelement for DHIS.
>>> >>>
>>> >>> So what we were doing is:
>>> >>> 1. Migrate the data from anywhere (EpiInfo, MSAccess and DBF) to
>>> MySQL.
>>> >>>
>>> >>> 2. On MySQL we have tables where the columns names of those tables
>>> >>> mapping to the dataelementid (which Lars recommend not be like this)
>>> >>>
>>> >>> 3. There is a script on Python made to run routinely to process this
>>> data
>>> >>> and insert the new one on DHIS 2 tables.
>>> >>> When I say routinely, I mean each month or whenever I want it.
>>> >>>
>>> >>> The objective were to use solely DHIS for report and one day for data
>>> >>> entry.
>>> >>>
>>> >>> Caveman
>>> >>>
>>> >>>>
>>> >>>> Cheers
>>> >>>> Bob
>>> >>>>
>>> >>>> >
>>> >>>> > Thanks for your patience guys
>>> >>>> > Caveman
>>> >>>> >
>>> >>>> >
>>> >>>> >
>>> >>>> >
>>> >>>> >
>>> >>>> > 2010/7/12 Jason Pickering <jason.p.pickering@xxxxxxxxx>
>>> >>>> >>
>>> >>>> >> Hi Lars,
>>> >>>> >> Well, yes and no. If the resource tables (namely
>>> orgunitstructure) is
>>> >>>> >> not generated, it would not be possible to assign any orgunits to
>>> a
>>> >>>> >> given data mart/report table? Likewise, if the orgunit levels
>>> have
>>> >>>> >> not
>>> >>>> >> been defined, it is also not possible to assign any orgunits to a
>>> >>>> >> given report table/datamart, or perhaps I am just hallucinating ?
>>> I
>>> >>>> >> have run into this problem several times actually when creating
>>> new
>>> >>>> >> databases. Ideally, the orgunitstructure table should be
>>> regenerated
>>> >>>> >> when there have been changes to the orgunit hierarchy and users
>>> >>>> >> should
>>> >>>> >> be warned that it is not possible to create a report
>>> table/datamart
>>> >>>> >> without first defining orgunit level names.
>>> >>>> >>
>>> >>>> >> Regards,
>>> >>>> >> Jason
>>> >>>> >>
>>> >>>> >>
>>> >>>> >> 2010/7/12 Lars Helge Øverland <larshelge@xxxxxxxxx>:
>>> >>>> >> > Generating any of the resource tables don't affect the
>>> aggregation
>>> >>>> >> > /
>>> >>>> >> > report tables...
>>> >>>> >> >
>>> >>>> >> >
>>> >>>> >> >
>>> >>>> >> > On Sun, Jul 11, 2010 at 5:36 PM, Orvalho Augusto
>>> >>>> >> > <orvaquim@xxxxxxxxx>
>>> >>>> >> > wrote:
>>> >>>> >> >> Thanks!!!
>>> >>>> >> >>
>>> >>>> >> >> Caveman
>>> >>>> >> >>
>>> >>>> >> >>
>>> >>>> >> >> On Sun, Jul 11, 2010 at 5:03 PM, Jason Pickering
>>> >>>> >> >> <jason.p.pickering@xxxxxxxxx> wrote:
>>> >>>> >> >>>
>>> >>>> >> >>> A very common mistake is not having generated the resource
>>> >>>> >> >>> tables,
>>> >>>> >> >>> namely "orgunitstructure". Consult the documentation for more
>>> >>>> >> >>> details
>>> >>>> >> >>> on how to generate these tables.
>>> >>>> >> >>>
>>> >>>> >> >>> Regards,
>>> >>>> >> >>> Jason
>>> >>>> >> >>>
>>> >>>> >> >>>
>>> >>>> >> >>> On Sun, Jul 11, 2010 at 4:27 PM, Orvalho Augusto
>>> >>>> >> >>> <orvaquim@xxxxxxxxx>
>>> >>>> >> >>> wrote:
>>> >>>> >> >>> > I have a dataelement collected at a Health Centre level
>>> (eg:
>>> >>>> >> >>> > ats_posfem)
>>> >>>> >> >>> > in
>>> >>>> >> >>> > one province and only in some particular districts. In
>>> >>>> >> >>> > unpredictible
>>> >>>> >> >>> > future
>>> >>>> >> >>> > at National level we will be collecting on every health
>>> centre.
>>> >>>> >> >>> >
>>> >>>> >> >>> > Now, my ignorance: How can I obtain the provincial total on
>>> >>>> >> >>> > Report
>>> >>>> >> >>> > Table? I
>>> >>>> >> >>> > have been battled with DHIS so far and nothing.
>>> >>>> >> >>> >
>>> >>>> >> >>> > Thanks guys
>>> >>>> >> >>> > Caveman
>>> >>>> >> >>> >
>>> >>>> >> >>> >
>>> >>>> >> >>> > _______________________________________________
>>> >>>> >> >>> > Mailing list: https://launchpad.net/~dhis2-devs<https://launchpad.net/%7Edhis2-devs>
>>> >>>> >> >>> > Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>> >>>> >> >>> > Unsubscribe : https://launchpad.net/~dhis2-devs<https://launchpad.net/%7Edhis2-devs>
>>> >>>> >> >>> > More help   : https://help.launchpad.net/ListHelp
>>> >>>> >> >>> >
>>> >>>> >> >>> >
>>> >>>> >> >>>
>>> >>>> >> >>>
>>> >>>> >> >>>
>>> >>>> >> >>> --
>>> >>>> >> >>> --
>>> >>>> >> >>> Jason P. Pickering
>>> >>>> >> >>> email: jason.p.pickering@xxxxxxxxx
>>> >>>> >> >>> tel:+260968395190
>>> >>>> >> >>
>>> >>>> >> >>
>>> >>>> >> >>
>>> >>>> >> >>
>>> >>>> >> >> _______________________________________________
>>> >>>> >> >> Mailing list: https://launchpad.net/~dhis2-devs<https://launchpad.net/%7Edhis2-devs>
>>> >>>> >> >> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>> >>>> >> >> Unsubscribe : https://launchpad.net/~dhis2-devs<https://launchpad.net/%7Edhis2-devs>
>>> >>>> >> >> More help   : https://help.launchpad.net/ListHelp
>>> >>>> >> >>
>>> >>>> >> >>
>>> >>>> >> >
>>> >>>> >>
>>> >>>> >>
>>> >>>> >>
>>> >>>> >> --
>>> >>>> >> --
>>> >>>> >> Jason P. Pickering
>>> >>>> >> email: jason.p.pickering@xxxxxxxxx
>>> >>>> >> tel:+260968395190
>>> >>>> >
>>> >>>> >
>>> >>>> >
>>> >>>> > _______________________________________________
>>> >>>> > Mailing list: https://launchpad.net/~dhis2-devs<https://launchpad.net/%7Edhis2-devs>
>>> >>>> > Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>> >>>> > Unsubscribe : https://launchpad.net/~dhis2-devs<https://launchpad.net/%7Edhis2-devs>
>>> >>>> > More help   : https://help.launchpad.net/ListHelp
>>> >>>> >
>>> >>>> >
>>> >>>
>>> >>>
>>> >>
>>> >>
>>> >
>>> >
>>> >
>>> >
>>>
>>>
>>>
>>> --
>>> --
>>> Jason P. Pickering
>>> email: jason.p.pickering@xxxxxxxxx
>>> tel:+260968395190
>>>
>>
>>
>
>
> --
> --
> Jason P. Pickering
> email: jason.p.pickering@xxxxxxxxx
> tel:+260968395190
>

Follow ups

References