← Back to team overview

dhis2-devs team mailing list archive

Re: Problem creating datamart

 

Dear Knut

Actually I never use PostgreSQL. So I asked  few of friends and they
suggests me that if the application are tuned and used in MySQL then stay
with that.

As you are suggesting to use PostgreSQL can you suggest how I could I shift
database from MySQL to PostgreSQL?

Regards

Hannan

On Tue, Sep 11, 2012 at 4:29 PM, Knut Staring <knutst@xxxxxxxxx> wrote:

> Perhaps this is a case where it would make sense to switch to
> PostgreSQL and possibly also make use of more of the internal
> crunching capabilities of the database?
>
> Knut
>
> On Tue, Sep 11, 2012 at 10:09 AM, Hannan Khan <hannank@xxxxxxxxx> wrote:
> > I might found an alternative solution. Yesterday we run data mart for 9
> > month and it was completed successfully but when we run for 5 years, the
> > errors arrived.
> >
> > Today and trying with again and get back to you all with the result. But
> > when we run datamart for 9 moth it takes 34 minutes, which is long is not
> > it?
> >
> > Regards
> >
> > Hannan
> >
> >
> > On Tue, Sep 11, 2012 at 2:01 PM, Morten Olav Hansen <mortenoh@xxxxxxxxx>
> > wrote:
> >>
> >> That's a really big database... I think maybe Lars should answer if the
> >> memory of the server is large enough (I'm sure he can compare it with
> what
> >> he has been doing to tune the Kenya server).
> >>
> >> --
> >> Morten
> >>
> >>
> >>
> >> On Tue, Sep 11, 2012 at 2:59 PM, Hannan Khan <hannank@xxxxxxxxx> wrote:
> >>>
> >>> Hi Morten
> >>>
> >>> Server have 8GB memory and database is 3159MB. We are using MySQL.
> >>>
> >>> Regards
> >>>
> >>> Hannan
> >>>
> >>>
> >>> On Tue, Sep 11, 2012 at 1:53 PM, Morten Olav Hansen <
> mortenoh@xxxxxxxxx>
> >>> wrote:
> >>>>
> >>>> Hi
> >>>>
> >>>> How much memory does your server have? and what is the size of your
> >>>> database?
> >>>>
> >>>> --
> >>>> Morten
> >>>>
> >>>>
> >>>>
> >>>> On Tue, Sep 11, 2012 at 2:51 PM, Hannan Khan <hannank@xxxxxxxxx>
> wrote:
> >>>>>
> >>>>> -XX:MaxPermSize=2048m and -XX:PermSize=512m
> >>>>>
> >>>>>
> >>>>> On Tue, Sep 11, 2012 at 11:27 AM, Jason Pickering
> >>>>> <jason.p.pickering@xxxxxxxxx> wrote:
> >>>>>>
> >>>>>> The PergGen space is the important option.  What are you JAVA_OPTS.?
> >>>>>>
> >>>>>> --
> >>>>>> Sent from my mobile
> >>>>>>
> >>>>>> On Sep 11, 2012 7:25 AM, "Hannan Khan" <hannank@xxxxxxxxx> wrote:
> >>>>>>>
> >>>>>>> Thanks Jason.
> >>>>>>>
> >>>>>>> I put the Java heap space 1GB and Maximum 6GB (Xmx6144m and
> >>>>>>> Xms1024m). But still getting 'java.lang.OutOfMemoryError: Java
> heap space'
> >>>>>>> error. That why I asked that whether that is a bug or not.
> >>>>>>>
> >>>>>>> I am trying to other options and contact you soon.
> >>>>>>>
> >>>>>>> Regards
> >>>>>>>
> >>>>>>> Hannan
> >>>>>>>
> >>>>>>> On Mon, Sep 10, 2012 at 11:40 AM, Jason Pickering
> >>>>>>> <jason.p.pickering@xxxxxxxxx> wrote:
> >>>>>>>>
> >>>>>>>> Hi Hannan,
> >>>>>>>>
> >>>>>>>> You configuration may have worked previously, but some things were
> >>>>>>>> changed in the datamart and you will need to allocate more memory
> to
> >>>>>>>> it after 2.7.
> >>>>>>>>
> >>>>>>>> The details are here of the server setup are here
> >>>>>>>>
> >>>>>>>>
> http://dhis2.org/doc/snapshot/en/implementer/html/ch08.html#d5e413
> >>>>>>>>
> >>>>>>>> and the important part is the JAVA_OPTS environment variable, for
> >>>>>>>> instance...
> >>>>>>>>
> >>>>>>>> export JAVA_OPTS='-Xmx6000m -Xms3000m -XX:MaxPermSize=800m
> >>>>>>>> -XX:PermSize=400m'
> >>>>>>>>
> >>>>>>>> I am not really sure what your settings are, but you should at
> least
> >>>>>>>> specify these options, of course adjusting them for the amount of
> >>>>>>>> memory which can be allocated to DHIS2 .
> >>>>>>>>
> >>>>>>>> Best regards,
> >>>>>>>> Jason
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Mon, Sep 10, 2012 at 10:49 AM, Hannan Khan <hannank@xxxxxxxxx>
> >>>>>>>> wrote:
> >>>>>>>> > Dear Jason
> >>>>>>>> >
> >>>>>>>> > Thanks for your advice. We are running dhis2 from 2010. In
> earlier
> >>>>>>>> > version
> >>>>>>>> > we dont have any problem creating datamart. After updating to
> >>>>>>>> > version 2.7 we
> >>>>>>>> > imported a big chuck of data from two data souce and after that
> we
> >>>>>>>> > have
> >>>>>>>> > problem creating datamart.
> >>>>>>>> >
> >>>>>>>> > So far I know the configuration is ok, however I will check
> again
> >>>>>>>> > and get
> >>>>>>>> > back to you.
> >>>>>>>> >
> >>>>>>>> > Regards
> >>>>>>>> >
> >>>>>>>> > Hannan
> >>>>>>>> > from Samsung Galaxy
> >>>>>>>> >
> >>>>>>>> > On Sep 9, 2012 5:39 PM, "Jason Pickering"
> >>>>>>>> > <jason.p.pickering@xxxxxxxxx>
> >>>>>>>> > wrote:
> >>>>>>>> >>
> >>>>>>>> >> This is not a bug. You must increase the heap size. Please
> >>>>>>>> >> consult the
> >>>>>>>> >> implementation manual for details.
> >>>>>>>> >>
> >>>>>>>> >> Regards,
> >>>>>>>> >> Jason
> >>>>>>>> >>
> >>>>>>>> >> --
> >>>>>>>> >> Sent from my mobile
> >>>>>>>> >>
> >>>>>>>> >> On Sep 9, 2012 1:12 PM, "Hannan Khan" <hannank@xxxxxxxxx>
> wrote:
> >>>>>>>> >>>
> >>>>>>>> >>> We are using dhis2 version 2.7. Whenever we generating
> datamart
> >>>>>>>> >>> are it
> >>>>>>>> >>> taking long time and it never ends.The error message is (in
> the
> >>>>>>>> >>> tomcat log
> >>>>>>>> >>> file)
> >>>>>>>> >>>
> >>>>>>>> >>>
> 'dhis2-org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor
> >>>>>>>> >>> processChildren SEVERE: Exception invoking periodic operation:
> >>>>>>>> >>> java.lang.OutOfMemoryError: Java heap space'.
> >>>>>>>> >>>
> >>>>>>>> >>> I attached the log file.
> >>>>>>>> >>>
> >>>>>>>> >>> Regards
> >>>>>>>> >>>
> >>>>>>>> >>> Hannan Khan
> >>>>>>>> >>>
> >>>>>>>> >>> National Data Warehouse
> >>>>>>>> >>>
> >>>>>>>> >>>
> >>>>>>>> >>>
> >>>>>>>> >>>
> >>>>>>>> >>> _______________________________________________
> >>>>>>>> >>> Mailing list: https://launchpad.net/~dhis2-devs
> >>>>>>>> >>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> >>>>>>>> >>> Unsubscribe : https://launchpad.net/~dhis2-devs
> >>>>>>>> >>> More help   : https://help.launchpad.net/ListHelp
> >>>>>>>> >>>
> >>>>>>>> >
> >>>>>>>
> >>>>>>>
> >>>>>
> >>>>>
> >>>>> _______________________________________________
> >>>>> Mailing list: https://launchpad.net/~dhis2-devs
> >>>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> >>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
> >>>>> More help   : https://help.launchpad.net/ListHelp
> >>>>>
> >>>>
> >>>
> >>
> >
> >
> > _______________________________________________
> > Mailing list: https://launchpad.net/~dhis2-devs
> > Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> > Unsubscribe : https://launchpad.net/~dhis2-devs
> > More help   : https://help.launchpad.net/ListHelp
> >
>
>
>
> --
> Knut Staring
> Dept. of Informatics, University of Oslo
> +4791880522
> http://dhis2.org
>

Follow ups

References