dhis2-users team mailing list archive

Thread
Date

Re: Trouble running Analytics for 2.29

To: Bob Jolliffe <bobjolliffe@xxxxxxxxx>
From: Knut Staring <knutst@xxxxxxxxx>
Date: Fri, 11 May 2018 09:21:44 +0200
Cc: dhis2-users <dhis2-users@xxxxxxxxxxxxxxxxxxx>
In-reply-to: <CACd=f9d=2sLrNSstRn7wtHYZ3K8qB2VcbmZ60KYwSKvPAM8Riw@mail.gmail.com>

Enclosing parts of the log - I see a hint  to increase max_wal_size

And also these lines:

2018-05-11 07:03:59 UTC [16449-2] LOG:  server process (PID 18016) was
terminated by signal 9: Killed
2018-05-11 07:03:59 UTC [16449-3] DETAIL:  Failed process was running:
create index "in_uidlevel4_ax_2015_r7jVa" on analytics_temp_2015
("uidlevel4")


On Fri, May 11, 2018 at 1:57 AM, Bob Jolliffe <bobjolliffe@xxxxxxxxx> wrote:

> It will most likely be a memory issue which gets exhausted at the same
> place each time. The log should be in standard place of
> /var/log/postgreswl/ etc. Not really anything to do with tools.
>
> On Thu, 10 May 2018, 22:00 Knut Staring, <knutst@xxxxxxxxx> wrote:
>
>> Thanks Bob, I will try, but where is the postgresql log file in your
>> setup (with dhis2-tools)?
>>
>> The strange thing is that everything else works up until the last part of
>> aggregation. In other words, I can do this again and again,
>>
>> Knut
>>
>> On Thu, May 10, 2018 at 6:19 PM, Bob Jolliffe <bobjolliffe@xxxxxxxxx>
>> wrote:
>>
>>> Your database has popped 😕
>>>
>>> To find out why take a look in the postgresql log file around the same
>>> time.
>>>
>>> On Thu, 10 May 2018, 13:51 Knut Staring, <knutst@xxxxxxxxx> wrote:
>>>
>>>> Hi all,
>>>>
>>>> Wondering if anyone has a tip for what is going wrong on my serer. I
>>>> just tested running Analytics in 2.29 with the same db on my laptop and
>>>> there were no issues, but everytime I try to do this on the server, it
>>>> gives me the following:
>>>>
>>>>  INFO  2018-05-10 08:06:51,987 [Level: INFO, category: ANALYTICS_TABLE,
>>>> time: Thu May 10 08:06:51 UTC 2018, message: Applying aggregation levels]
>>>> (InMemoryNotifier.java [taskScheduler-14])
>>>> * INFO  2018-05-10 08:07:10,890 Vacuumed tables
>>>> (DefaultAnalyticsTableService.java [taskScheduler-14])
>>>> * INFO  2018-05-10 08:07:10,890 Applied aggregation levels:
>>>> 00:01:54.496 (Clock.java [taskScheduler-14])
>>>> * INFO  2018-05-10 08:07:10,890 [Level: INFO, category:
>>>> ANALYTICS_TABLE, time: Thu May 10 08:07:10 UTC 2018, message: Creating
>>>> indexes] (InMemoryNotifier.java [taskScheduler-14])
>>>> * INFO  2018-05-10 08:07:10,891 No of analytics table indexes: 204
>>>> (DefaultAnalyticsTableService.java [taskScheduler-14])
>>>> May 10, 2018 8:07:25 AM org.postgresql.core.v3.ConnectionFactoryImpl
>>>> log
>>>> WARNING: SQLException occurred while connecting to localhost:5432
>>>> org.postgresql.util.PSQLException: FATAL: the database system is in
>>>> recovery mode
>>>>         at org.postgresql.core.v3.ConnectionFactoryImpl.
>>>> doAuthentication(ConnectionFactoryImpl.java:475)
>>>>         at org.postgresql.core.v3.ConnectionFactoryImpl.
>>>> openConnectionImpl(ConnectionFactoryImpl.java:207)
>>>>         at org.postgresql.core.ConnectionFactory.openConnection(
>>>> ConnectionFactory.java:49)
>>>>         at org.postgresql.jdbc.PgConnection.<init>(
>>>> PgConnection.java:195)
>>>>         at org.postgresql.Driver.makeConnection(Driver.java:452)
>>>>         at org.postgresql.Driver.connect(Driver.java:254)
>>>>         at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(
>>>> DriverManagerDataSource.java:175)
>>>>         at com.mchange.v2.c3p0.WrapperConnectionPoolDataSourc
>>>> e.getPooledConnection(WrapperConnectionPoolDataSource.java:220)
>>>>         at com.mchange.v2.c3p0.WrapperConnectionPoolDataSourc
>>>> e.getPooledConnection(WrapperConnectionPoolDataSource.java:206)
>>>>         at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$
>>>> 1PooledConnectionResourcePoolManager.acquireResource(
>>>> C3P0PooledConnectionPool.java:203)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(
>>>> BasicResourcePool.java:1138)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool.
>>>> doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(
>>>> BasicResourcePool.java:1125)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool.access$700(
>>>> BasicResourcePool.java:44)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool$
>>>> ScatteredAcquireTask.run(BasicResourcePool.java:1870)
>>>>         at com.mchange.v2.async.ThreadPoolAsynchronousRunner$
>>>> PoolThread.run(ThreadPoolAsynchronousRunner.java:696)
>>>>
>>>> May 10, 2018 8:07:25 AM org.postgresql.Driver connect
>>>> SEVERE: Connection error:
>>>> org.postgresql.util.PSQLException: FATAL: the database system is in
>>>> recovery mode
>>>>         at org.postgresql.core.v3.ConnectionFactoryImpl.
>>>> doAuthentication(ConnectionFactoryImpl.java:475)
>>>>         at org.postgresql.core.v3.ConnectionFactoryImpl.
>>>> openConnectionImpl(ConnectionFactoryImpl.java:207)
>>>>         at org.postgresql.core.ConnectionFactory.openConnection(
>>>> ConnectionFactory.java:49)
>>>>         at org.postgresql.jdbc.PgConnection.<init>(
>>>> PgConnection.java:195)
>>>>         at org.postgresql.Driver.makeConnection(Driver.java:452)
>>>>         at org.postgresql.Driver.connect(Driver.java:254)
>>>>         at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(
>>>> DriverManagerDataSource.java:175)
>>>>         at com.mchange.v2.c3p0.WrapperConnectionPoolDataSourc
>>>> e.getPooledConnection(WrapperConnectionPoolDataSource.java:220)
>>>>         at com.mchange.v2.c3p0.WrapperConnectionPoolDataSourc
>>>> e.getPooledConnection(WrapperConnectionPoolDataSource.java:206)
>>>>         at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$
>>>> 1PooledConnectionResourcePoolManager.acquireResource(
>>>> C3P0PooledConnectionPool.java:203)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(
>>>> BasicResourcePool.java:1138)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool.
>>>> doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(
>>>> BasicResourcePool.java:1125)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool.access$700(
>>>> BasicResourcePool.java:44)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool$
>>>> ScatteredAcquireTask.run(BasicResourcePool.java:1870)
>>>>         at com.mchange.v2.async.ThreadPoolAsynchronousRunner$
>>>> PoolThread.run(ThreadPoolAsynchronousRunner.java:696)
>>>>
>>>> May 10, 2018 8:07:25 AM org.postgresql.core.v3.ConnectionFactoryImpl
>>>> log
>>>> WARNING: SQLException occurred while connecting to localhost:5432
>>>> org.postgresql.util.PSQLException: FATAL: the database system is in
>>>> recovery mode
>>>>         at org.postgresql.core.v3.ConnectionFactoryImpl.
>>>> doAuthentication(ConnectionFactoryImpl.java:475)
>>>>         at org.postgresql.core.v3.ConnectionFactoryImpl.
>>>> openConnectionImpl(ConnectionFactoryImpl.java:207)
>>>>         at org.postgresql.core.ConnectionFactory.openConnection(
>>>> ConnectionFactory.java:49)
>>>>         at org.postgresql.jdbc.PgConnection.<init>(
>>>> PgConnection.java:195)
>>>>         at org.postgresql.Driver.makeConnection(Driver.java:452)
>>>>         at org.postgresql.Driver.connect(Driver.java:254)
>>>>         at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(
>>>> DriverManagerDataSource.java:175)
>>>>         at com.mchange.v2.c3p0.WrapperConnectionPoolDataSourc
>>>> e.getPooledConnection(WrapperConnectionPoolDataSource.java:220)
>>>>         at com.mchange.v2.c3p0.WrapperConnectionPoolDataSourc
>>>> e.getPooledConnection(WrapperConnectionPoolDataSource.java:206)
>>>>         at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$
>>>> 1PooledConnectionResourcePoolManager.acquireResource(
>>>> C3P0PooledConnectionPool.java:203)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(
>>>> BasicResourcePool.java:1138)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool.
>>>> doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(
>>>> BasicResourcePool.java:1125)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool.access$700(
>>>> BasicResourcePool.java:44)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool$
>>>> ScatteredAcquireTask.run(BasicResourcePool.java:1870)
>>>>         at com.mchange.v2.async.ThreadPoolAsynchronousRunner$
>>>> PoolThread.run(ThreadPoolAsynchronousRunner.java:696)
>>>>
>>>> May 10, 2018 8:07:25 AM org.postgresql.Driver connect
>>>> SEVERE: Connection error:
>>>> org.postgresql.util.PSQLException: FATAL: the database system is in
>>>> recovery mode
>>>>         at org.postgresql.core.v3.ConnectionFactoryImpl.
>>>> doAuthentication(ConnectionFactoryImpl.java:475)
>>>>         at org.postgresql.core.v3.ConnectionFactoryImpl.
>>>> openConnectionImpl(ConnectionFactoryImpl.java:207)
>>>>         at org.postgresql.core.ConnectionFactory.openConnection(
>>>> ConnectionFactory.java:49)
>>>>         at org.postgresql.jdbc.PgConnection.<init>(
>>>> PgConnection.java:195)
>>>>         at org.postgresql.Driver.makeConnection(Driver.java:452)
>>>>         at org.postgresql.Driver.connect(Driver.java:254)
>>>>         at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(
>>>> DriverManagerDataSource.java:175)
>>>>         at com.mchange.v2.c3p0.WrapperConnectionPoolDataSourc
>>>> e.getPooledConnection(WrapperConnectionPoolDataSource.java:220)
>>>>         at com.mchange.v2.c3p0.WrapperConnectionPoolDataSourc
>>>> e.getPooledConnection(WrapperConnectionPoolDataSource.java:206)
>>>>         at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$
>>>> 1PooledConnectionResourcePoolManager.acquireResource(
>>>> C3P0PooledConnectionPool.java:203)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(
>>>> BasicResourcePool.java:1138)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool.
>>>> doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(
>>>> BasicResourcePool.java:1125)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool.access$700(
>>>> BasicResourcePool.java:44)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool$
>>>> ScatteredAcquireTask.run(BasicResourcePool.java:1870)
>>>>         at com.mchange.v2.async.ThreadPoolAsynchronousRunner$
>>>> PoolThread.run(ThreadPoolAsynchronousRunner.java:696)
>>>>
>>>> May 10, 2018 8:07:25 AM org.postgresql.core.v3.ConnectionFactoryImpl
>>>> log
>>>> WARNING: SQLException occurred while connecting to localhost:5432
>>>> org.postgresql.util.PSQLException: FATAL: the database system is in
>>>> recovery mode
>>>>         at org.postgresql.core.v3.ConnectionFactoryImpl.
>>>> doAuthentication(ConnectionFactoryImpl.java:475)
>>>>         at org.postgresql.core.v3.ConnectionFactoryImpl.
>>>> openConnectionImpl(ConnectionFactoryImpl.java:207)
>>>>         at org.postgresql.core.ConnectionFactory.openConnection(
>>>> ConnectionFactory.java:49)
>>>>         at org.postgresql.jdbc.PgConnection.<init>(
>>>> PgConnection.java:195)
>>>>         at org.postgresql.Driver.makeConnection(Driver.java:452)
>>>>         at org.postgresql.Driver.connect(Driver.java:254)
>>>>         at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(
>>>> DriverManagerDataSource.java:175)
>>>>         at com.mchange.v2.c3p0.WrapperConnectionPoolDataSourc
>>>> e.getPooledConnection(WrapperConnectionPoolDataSource.java:220)
>>>>         at com.mchange.v2.c3p0.WrapperConnectionPoolDataSourc
>>>> e.getPooledConnection(WrapperConnectionPoolDataSource.java:206)
>>>>         at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$
>>>> 1PooledConnectionResourcePoolManager.acquireResource(
>>>> C3P0PooledConnectionPool.java:203)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(
>>>> BasicResourcePool.java:1138)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool.
>>>> doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(
>>>> BasicResourcePool.java:1125)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool.access$700(
>>>> BasicResourcePool.java:44)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool$
>>>> ScatteredAcquireTask.run(BasicResourcePool.java:1870)
>>>>         at com.mchange.v2.async.ThreadPoolAsynchronousRunner$
>>>> PoolThread.run(ThreadPoolAsynchronousRunner.java:696)
>>>>
>>>> May 10, 2018 8:07:26 AM org.postgresql.Driver connect
>>>> SEVERE: Connection error:
>>>> org.postgresql.util.PSQLException: FATAL: the database system is in
>>>> recovery mode
>>>>         at org.postgresql.core.v3.ConnectionFactoryImpl.
>>>> doAuthentication(ConnectionFactoryImpl.java:475)
>>>>         at org.postgresql.core.v3.ConnectionFactoryImpl.
>>>> openConnectionImpl(ConnectionFactoryImpl.java:207)
>>>>         at org.postgresql.core.ConnectionFactory.openConnection(
>>>> ConnectionFactory.java:49)
>>>>         at org.postgresql.jdbc.PgConnection.<init>(
>>>> PgConnection.java:195)
>>>>         at org.postgresql.Driver.makeConnection(Driver.java:452)
>>>>         at org.postgresql.Driver.connect(Driver.java:254)
>>>>         at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(
>>>> DriverManagerDataSource.java:175)
>>>>         at com.mchange.v2.c3p0.WrapperConnectionPoolDataSourc
>>>> e.getPooledConnection(WrapperConnectionPoolDataSource.java:220)
>>>>         at com.mchange.v2.c3p0.WrapperConnectionPoolDataSourc
>>>> e.getPooledConnection(WrapperConnectionPoolDataSource.java:206)
>>>>         at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$
>>>> 1PooledConnectionResourcePoolManager.acquireResource(
>>>> C3P0PooledConnectionPool.java:203)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(
>>>> BasicResourcePool.java:1138)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool.
>>>> doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(
>>>> BasicResourcePool.java:1125)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool.access$700(
>>>> BasicResourcePool.java:44)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool$
>>>> ScatteredAcquireTask.run(BasicResourcePool.java:1870)
>>>>         at com.mchange.v2.async.ThreadPoolAsynchronousRunner$
>>>> PoolThread.run(ThreadPoolAsynchronousRunner.java:696)
>>>>
>>>> May 10, 2018 8:07:26 AM org.postgresql.core.v3.ConnectionFactoryImpl
>>>> log
>>>> WARNING: SQLException occurred while connecting to localhost:5432
>>>> org.postgresql.util.PSQLException: FATAL: the database system is in
>>>> recovery mode
>>>>         at org.postgresql.core.v3.ConnectionFactoryImpl.
>>>> doAuthentication(ConnectionFactoryImpl.java:475)
>>>>         at org.postgresql.core.v3.ConnectionFactoryImpl.
>>>> openConnectionImpl(ConnectionFactoryImpl.java:207)
>>>>         at org.postgresql.core.ConnectionFactory.openConnection(
>>>> ConnectionFactory.java:49)
>>>>         at org.postgresql.jdbc.PgConnection.<init>(
>>>> PgConnection.java:195)
>>>>         at org.postgresql.Driver.makeConnection(Driver.java:452)
>>>>         at org.postgresql.Driver.connect(Driver.java:254)
>>>>         at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(
>>>> DriverManagerDataSource.java:175)
>>>>         at com.mchange.v2.c3p0.WrapperConnectionPoolDataSourc
>>>> e.getPooledConnection(WrapperConnectionPoolDataSource.java:220)
>>>>         at com.mchange.v2.c3p0.WrapperConnectionPoolDataSourc
>>>> e.getPooledConnection(WrapperConnectionPoolDataSource.java:206)
>>>>         at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$
>>>> 1PooledConnectionResourcePoolManager.acquireResource(
>>>> C3P0PooledConnectionPool.java:203)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(
>>>> BasicResourcePool.java:1138)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool.
>>>> doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(
>>>> BasicResourcePool.java:1125)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool.access$700(
>>>> BasicResourcePool.java:44)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool$
>>>> ScatteredAcquireTask.run(BasicResourcePool.java:1870)
>>>>         at com.mchange.v2.async.ThreadPoolAsynchronousRunner$
>>>> PoolThread.run(ThreadPoolAsynchronousRunner.java:696)
>>>>
>>>> May 10, 2018 8:07:26 AM org.postgresql.Driver connect
>>>> SEVERE: Connection error:
>>>> org.postgresql.util.PSQLException: FATAL: the database system is in
>>>> recovery mode
>>>>         at org.postgresql.core.v3.ConnectionFactoryImpl.
>>>> doAuthentication(ConnectionFactoryImpl.java:475)
>>>>         at org.postgresql.core.v3.ConnectionFactoryImpl.
>>>> openConnectionImpl(ConnectionFactoryImpl.java:207)
>>>>         at org.postgresql.core.ConnectionFactory.openConnection(
>>>> ConnectionFactory.java:49)
>>>>         at org.postgresql.jdbc.PgConnection.<init>(
>>>> PgConnection.java:195)
>>>>         at org.postgresql.Driver.makeConnection(Driver.java:452)
>>>>         at org.postgresql.Driver.connect(Driver.java:254)
>>>>         at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(
>>>> DriverManagerDataSource.java:175)
>>>>         at com.mchange.v2.c3p0.WrapperConnectionPoolDataSourc
>>>> e.getPooledConnection(WrapperConnectionPoolDataSource.java:220)
>>>>         at com.mchange.v2.c3p0.WrapperConnectionPoolDataSourc
>>>> e.getPooledConnection(WrapperConnectionPoolDataSource.java:206)
>>>>         at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$
>>>> 1PooledConnectionResourcePoolManager.acquireResource(
>>>> C3P0PooledConnectionPool.java:203)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(
>>>> BasicResourcePool.java:1138)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool.
>>>> doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(
>>>> BasicResourcePool.java:1125)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool.access$700(
>>>> BasicResourcePool.java:44)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool$
>>>> ScatteredAcquireTask.run(BasicResourcePool.java:1870)
>>>>         at com.mchange.v2.async.ThreadPoolAsynchronousRunner$
>>>> PoolThread.run(ThreadPoolAsynchronousRunner.java:696)
>>>>
>>>> May 10, 2018 8:07:25 AM org.postgresql.core.v3.ConnectionFactoryImpl
>>>> log
>>>> WARNING: SQLException occurred while connecting to localhost:5432
>>>> org.postgresql.util.PSQLException: FATAL: the database system is in
>>>> recovery mode
>>>>         at org.postgresql.core.v3.ConnectionFactoryImpl.
>>>> doAuthentication(ConnectionFactoryImpl.java:475)
>>>>         at org.postgresql.core.v3.ConnectionFactoryImpl.
>>>> openConnectionImpl(ConnectionFactoryImpl.java:207)
>>>>         at org.postgresql.core.ConnectionFactory.openConnection(
>>>> ConnectionFactory.java:49)
>>>>         at org.postgresql.jdbc.PgConnection.<init>(
>>>> PgConnection.java:195)
>>>>         at org.postgresql.Driver.makeConnection(Driver.java:452)
>>>>         at org.postgresql.Driver.connect(Driver.java:254)
>>>>         at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(
>>>> DriverManagerDataSource.java:175)
>>>>         at com.mchange.v2.c3p0.WrapperConnectionPoolDataSourc
>>>> e.getPooledConnection(WrapperConnectionPoolDataSource.java:220)
>>>>         at com.mchange.v2.c3p0.WrapperConnectionPoolDataSourc
>>>> e.getPooledConnection(WrapperConnectionPoolDataSource.java:206)
>>>>         at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$
>>>> 1PooledConnectionResourcePoolManager.acquireResource(
>>>> C3P0PooledConnectionPool.java:203)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(
>>>> BasicResourcePool.java:1138)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool.
>>>> doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(
>>>> BasicResourcePool.java:1125)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool.access$700(
>>>> BasicResourcePool.java:44)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool$
>>>> ScatteredAcquireTask.run(BasicResourcePool.java:1870)
>>>>         at com.mchange.v2.async.ThreadPoolAsynchronousRunner$
>>>> PoolThread.run(ThreadPoolAsynchronousRunner.java:696)
>>>>
>>>> May 10, 2018 8:07:26 AM org.postgresql.Driver connect
>>>> SEVERE: Connection error:
>>>> org.postgresql.util.PSQLException: FATAL: the database system is in
>>>> recovery mode
>>>>         at org.postgresql.core.v3.ConnectionFactoryImpl.
>>>> doAuthentication(ConnectionFactoryImpl.java:475)
>>>>         at org.postgresql.core.v3.ConnectionFactoryImpl.
>>>> openConnectionImpl(ConnectionFactoryImpl.java:207)
>>>>         at org.postgresql.core.ConnectionFactory.openConnection(
>>>> ConnectionFactory.java:49)
>>>>         at org.postgresql.jdbc.PgConnection.<init>(
>>>> PgConnection.java:195)
>>>>         at org.postgresql.Driver.makeConnection(Driver.java:452)
>>>>         at org.postgresql.Driver.connect(Driver.java:254)
>>>>         at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(
>>>> DriverManagerDataSource.java:175)
>>>>         at com.mchange.v2.c3p0.WrapperConnectionPoolDataSourc
>>>> e.getPooledConnection(WrapperConnectionPoolDataSource.java:220)
>>>>         at com.mchange.v2.c3p0.WrapperConnectionPoolDataSourc
>>>> e.getPooledConnection(WrapperConnectionPoolDataSource.java:206)
>>>>         at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$
>>>> 1PooledConnectionResourcePoolManager.acquireResource(
>>>> C3P0PooledConnectionPool.java:203)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(
>>>> BasicResourcePool.java:1138)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool.
>>>> doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(
>>>> BasicResourcePool.java:1125)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool.access$700(
>>>> BasicResourcePool.java:44)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool$
>>>> ScatteredAcquireTask.run(BasicResourcePool.java:1870)
>>>>         at com.mchange.v2.async.ThreadPoolAsynchronousRunner$
>>>> PoolThread.run(ThreadPoolAsynchronousRunner.java:696)
>>>>
>>>> May 10, 2018 8:07:26 AM org.postgresql.core.v3.ConnectionFactoryImpl
>>>> log
>>>> WARNING: SQLException occurred while connecting to localhost:5432
>>>> org.postgresql.util.PSQLException: FATAL: the database system is in
>>>> recovery mode
>>>>         at org.postgresql.core.v3.ConnectionFactoryImpl.
>>>> doAuthentication(ConnectionFactoryImpl.java:475)
>>>>         at org.postgresql.core.v3.ConnectionFactoryImpl.
>>>> openConnectionImpl(ConnectionFactoryImpl.java:207)
>>>>         at org.postgresql.core.ConnectionFactory.openConnection(
>>>> ConnectionFactory.java:49)
>>>>         at org.postgresql.jdbc.PgConnection.<init>(
>>>> PgConnection.java:195)
>>>>         at org.postgresql.Driver.makeConnection(Driver.java:452)
>>>>         at org.postgresql.Driver.connect(Driver.java:254)
>>>>         at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(
>>>> DriverManagerDataSource.java:175)
>>>>         at com.mchange.v2.c3p0.WrapperConnectionPoolDataSourc
>>>> e.getPooledConnection(WrapperConnectionPoolDataSource.java:220)
>>>>         at com.mchange.v2.c3p0.WrapperConnectionPoolDataSourc
>>>> e.getPooledConnection(WrapperConnectionPoolDataSource.java:206)
>>>>         at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$
>>>> 1PooledConnectionResourcePoolManager.acquireResource(
>>>> C3P0PooledConnectionPool.java:203)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(
>>>> BasicResourcePool.java:1138)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool.
>>>> doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(
>>>> BasicResourcePool.java:1125)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool.access$700(
>>>> BasicResourcePool.java:44)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool$
>>>> ScatteredAcquireTask.run(BasicResourcePool.java:1870)
>>>>         at com.mchange.v2.async.ThreadPoolAsynchronousRunner$
>>>> PoolThread.run(ThreadPoolAsynchronousRunner.java:696)
>>>>
>>>> May 10, 2018 8:07:26 AM org.postgresql.Driver connect
>>>> SEVERE: Connection error:
>>>> org.postgresql.util.PSQLException: FATAL: the database system is in
>>>> recovery mode
>>>>         at org.postgresql.core.v3.ConnectionFactoryImpl.
>>>> doAuthentication(ConnectionFactoryImpl.java:475)
>>>>         at org.postgresql.core.v3.ConnectionFactoryImpl.
>>>> openConnectionImpl(ConnectionFactoryImpl.java:207)
>>>>         at org.postgresql.core.ConnectionFactory.openConnection(
>>>> ConnectionFactory.java:49)
>>>>         at org.postgresql.jdbc.PgConnection.<init>(
>>>> PgConnection.java:195)
>>>>         at org.postgresql.Driver.makeConnection(Driver.java:452)
>>>>         at org.postgresql.Driver.connect(Driver.java:254)
>>>>         at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(
>>>> DriverManagerDataSource.java:175)
>>>>         at com.mchange.v2.c3p0.WrapperConnectionPoolDataSourc
>>>> e.getPooledConnection(WrapperConnectionPoolDataSource.java:220)
>>>>         at com.mchange.v2.c3p0.WrapperConnectionPoolDataSourc
>>>> e.getPooledConnection(WrapperConnectionPoolDataSource.java:206)
>>>>         at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$
>>>> 1PooledConnectionResourcePoolManager.acquireResource(
>>>> C3P0PooledConnectionPool.java:203)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(
>>>> BasicResourcePool.java:1138)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool.
>>>> doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(
>>>> BasicResourcePool.java:1125)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool.access$700(
>>>> BasicResourcePool.java:44)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool$
>>>> ScatteredAcquireTask.run(BasicResourcePool.java:1870)
>>>>         at com.mchange.v2.async.ThreadPoolAsynchronousRunner$
>>>> PoolThread.run(ThreadPoolAsynchronousRunner.java:696)
>>>>
>>>> May 10, 2018 8:07:26 AM org.postgresql.core.v3.ConnectionFactoryImpl
>>>> log
>>>> WARNING: SQLException occurred while connecting to localhost:5432
>>>> org.postgresql.util.PSQLException: FATAL: the database system is in
>>>> recovery mode
>>>>         at org.postgresql.core.v3.ConnectionFactoryImpl.
>>>> doAuthentication(ConnectionFactoryImpl.java:475)
>>>>         at org.postgresql.core.v3.ConnectionFactoryImpl.
>>>> openConnectionImpl(ConnectionFactoryImpl.java:207)
>>>>         at org.postgresql.core.ConnectionFactory.openConnection(
>>>> ConnectionFactory.java:49)
>>>>         at org.postgresql.jdbc.PgConnection.<init>(
>>>> PgConnection.java:195)
>>>>         at org.postgresql.Driver.makeConnection(Driver.java:452)
>>>>         at org.postgresql.Driver.connect(Driver.java:254)
>>>>         at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(
>>>> DriverManagerDataSource.java:175)
>>>>         at com.mchange.v2.c3p0.WrapperConnectionPoolDataSourc
>>>> e.getPooledConnection(WrapperConnectionPoolDataSource.java:220)
>>>>         at com.mchange.v2.c3p0.WrapperConnectionPoolDataSourc
>>>> e.getPooledConnection(WrapperConnectionPoolDataSource.java:206)
>>>>         at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$
>>>> 1PooledConnectionResourcePoolManager.acquireResource(
>>>> C3P0PooledConnectionPool.java:203)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(
>>>> BasicResourcePool.java:1138)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool.
>>>> doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(
>>>> BasicResourcePool.java:1125)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool.access$700(
>>>> BasicResourcePool.java:44)
>>>>         at com.mchange.v2.resourcepool.BasicResourcePool$
>>>> ScatteredAcquireTask.run(BasicResourcePool.java:1870)
>>>>         at com.mchange.v2.async.ThreadPoolAsynchronousRunner$
>>>> PoolThread.run(ThreadPoolAsynchronousRunner.java:696)
>>>>
>>>> May 10, 2018 8:07:26 AM org.postgresql.Driver connect
>>>> SEVERE: Connection error:
>>>>
>>>> _______________________________________________
>>>> Mailing list: https://launchpad.net/~dhis2-users
>>>> Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
>>>> Unsubscribe : https://launchpad.net/~dhis2-users
>>>> More help   : https://help.launchpad.net/ListHelp
>>>>
>>>
>>
>>
>> --
>> Knut Staring
>>
>> Department of Information, Evidence and Research
>> World Health Organization, Geneva, Switzerland
>> Office: +41 22 791 3683 Mob1: +33 6 4434 2931 Mob2: +47 9188 0522
>> Skype:     knutstar
>>
>


-- 
Knut Staring

Department of Information, Evidence and Research
World Health Organization, Geneva, Switzerland
Office: +41 22 791 3683 Mob1: +33 6 4434 2931 Mob2: +47 9188 0522
Skype:     knutstar

2018-05-11 06:59:50 UTC [18016-36] dhis@dhis STATEMENT:  select "dx","uidlevel2", sum(value) as value from analytics_completenesstarget where "dx" in ('kAofV66isvC') and "uidlevel2" in ('qKzosKQPl6G', 'ptWVfaCIdVx', 'pZqQRRW7PHP', 'ob6SxuRcqU4', 'sANMZ3lpqGs', 'sPkRcDvhGWA', 'tAbBVBbueqD', 'u4t9H8XyU9P', 'uepLTG8wGWJ', 'vvOK1BxTbet', 'yhCUgGcCcOo') and (("ouopeningdate" <= '2017-05-01' or "ouopeningdate" is null) and ("oucloseddate" >= '2017-05-31' or "oucloseddate" is null)) and (("costartdate" <= '2017-05-01' or "costartdate" is null) and ("coenddate" >= '2017-05-31' or "coenddate" is null)) group by "dx","uidlevel2"
2018-05-11 06:59:50 UTC [18021-17] dhis@dhis ERROR:  relation "analytics_completenesstarget" does not exist at character 51
2018-05-11 06:59:50 UTC [18021-18] dhis@dhis STATEMENT:  select "dx","uidlevel5", sum(value) as value from analytics_completenesstarget where "dx" in ('kAofV66isvC') and "uidlevel5" in ('PttV9DcrqDB') and (("ouopeningdate" <= '2017-05-01' or "ouopeningdate" is null) and ("oucloseddate" >= '2017-05-31' or "oucloseddate" is null)) and (("costartdate" <= '2017-05-01' or "costartdate" is null) and ("coenddate" >= '2017-05-31' or "coenddate" is null)) group by "dx","uidlevel5"
2018-05-11 07:02:46 UTC [16451-1] LOG:  checkpoints are occurring too frequently (26 seconds apart)
2018-05-11 07:02:46 UTC [16451-2] HINT:  Consider increasing the configuration parameter "max_wal_size".
2018-05-11 07:03:41 UTC [18056-1] ERROR:  canceling autovacuum task
2018-05-11 07:03:41 UTC [18056-2] CONTEXT:  automatic vacuum of table "dhis.public.analytics_temp_2015"
2018-05-11 07:03:41 UTC [18056-3] ERROR:  canceling autovacuum task
TopMemoryContext: 195496 total in 24 blocks; 28152 free (137 chunks); 167344 used
  TopTransactionContext: 8192 total in 1 blocks; 5176 free (1 chunks); 3016 used
  RI compare cache: 24576 total in 2 blocks; 15968 free (5 chunks); 8608 used
  RI query cache: 8192 total in 1 blocks; 2696 free (0 chunks); 5496 used
  RI constraint cache: 40888 total in 2 blocks; 3736 free (0 chunks); 37152 used
  Type information cache: 24472 total in 2 blocks; 3736 free (0 chunks); 20736 used
  Prepared Queries: 24576 total in 2 blocks; 15968 free (5 chunks); 8608 used
  TableSpace cache: 8192 total in 1 blocks; 3208 free (0 chunks); 4984 used
  Operator lookup cache: 24576 total in 2 blocks; 11872 free (5 chunks); 12704 used
  MessageContext: 8192 total in 1 blocks; 6968 free (0 chunks); 1224 used
  Operator class cache: 8192 total in 1 blocks; 1672 free (0 chunks); 6520 used
  smgr relation table: 57344 total in 3 blocks; 21960 free (10 chunks); 35384 used
  TransactionAbortContext: 32768 total in 1 blocks; 32728 free (0 chunks); 40 used
  Portal hash: 8192 total in 1 blocks; 1672 free (0 chunks); 6520 used
  PortalMemory: 8192 total in 1 blocks; 7880 free (0 chunks); 312 used
    PortalHeapMemory: 15360 total in 4 blocks; 3176 free (11 chunks); 12184 used
      ExecutorState: 8192 total in 1 blocks; 7656 free (0 chunks); 536 used
        ExprContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used
      TupleSort: 150986808 total in 22 blocks; 29384 free (7 chunks); 150957424 used
  Relcache by OID: 24576 total in 2 blocks; 6576 free (3 chunks); 18000 used
  CacheMemoryContext: 4228472 total in 12 blocks; 2084952 free (25 chunks); 2143520 used
    in_sixmonthlyapril_ax_2015_TYD75: 1024 total in 1 blocks; 192 free (0 chunks); 832 used
    CachedPlan: 1024 total in 1 blocks; 256 free (0 chunks); 768 used
    CachedPlanSource: 1024 total in 1 blocks; 0 free (0 chunks); 1024 used
      unnamed prepared statement: 8192 total in 1 blocks; 5408 free (2 chunks); 2784 used
    pg_depend_depender_index: 3072 total in 2 blocks; 1720 free (1 chunks); 1352 used
    pg_depend_reference_index: 3072 total in 2 blocks; 1720 free (1 chunks); 1352 used
    pg_class_tblspc_relfilenode_index: 1024 total in 1 blocks; 8 free (0 chunks); 1016 used
    EventTriggerCache: 8192 total in 1 blocks; 8152 free (2 chunks); 40 used
      Event Trigger Cache: 8192 total in 1 blocks; 3736 free (0 chunks); 4456 used
    _categoryoptioncomboname_temp_pkey: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    _categorystructure_temp_pkey: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    in_categoryoptioncombo_name: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    categoryoptioncombo_uid_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    categoryoptioncombo_pkey: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    categoryoptioncombo_code_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    in_orgunitgroupsetstructure_null_hd9lc: 3072 total in 2 blocks; 1808 free (0 chunks); 1264 used
    in_orgunitgroupsetstructure_not_null_fu1pd: 3072 total in 2 blocks; 1856 free (0 chunks); 1216 used
    in_orgunitstructure_organisationunituid_lk3ih: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    _orgunitstructure_temp_pkey1: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    _dataelementgroupsetstructure_temp_pkey: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    in_dataelementstructure_workflowid_qqb3b: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    in_dataelementstructure_periodtypeid_kni76: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    in_dataelementstructure_datasetuid_nr60n: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    in_dataelementstructure_datasetid_vhphc: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    in_dataelementstructure_dataelementuid_ifqfb: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    _dataelementstructure_temp_pkey: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    dataelement_shortname_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    dataelement_uid_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    dataelement_pkey: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    dataelement_name_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    dataelement_code_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    in_periodstructure_iso_ijztj: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    _periodstructure_temp_pkey: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    in_datavalue_lastupdated: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    in_datavalue_deleted: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    datavalue_pkey: 3072 total in 2 blocks; 1632 free (1 chunks); 1440 used
    pg_constraint_conrelid_index: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    CachedPlan: 1024 total in 1 blocks; 904 free (0 chunks); 120 used
    CachedPlanSource: 1024 total in 1 blocks; 688 free (0 chunks); 336 used
      CachedPlanQuery: 1024 total in 1 blocks; 872 free (1 chunks); 152 used
    indicatorlegendsets_pkey: 1024 total in 1 blocks; 8 free (0 chunks); 1016 used
    systemsetting_pkey: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    systemsetting_name_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    indicator_code_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    indicator_uid_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    indicator_pkey: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    CachedPlanSource: 3072 total in 2 blocks; 664 free (0 chunks); 2408 used
      CachedPlanQuery: 7168 total in 3 blocks; 3272 free (1 chunks); 3896 used
    SPI Plan: 1024 total in 1 blocks; 800 free (0 chunks); 224 used
    key_deleted_object_klass_uid: 1024 total in 1 blocks; 8 free (0 chunks); 1016 used
    key_deleted_object_klass_code: 1024 total in 1 blocks; 8 free (0 chunks); 1016 used
    deletedobject_pkey: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    chart_uid_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    chart_relativeperiodsid_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    chart_pkey: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    reporttable_filters_pkey: 1024 total in 1 blocks; 8 free (0 chunks); 1016 used
    reporttable_orgunitlevels_pkey: 1024 total in 1 blocks; 8 free (0 chunks); 1016 used
    reporttable_rows_pkey: 1024 total in 1 blocks; 8 free (0 chunks); 1016 used
    reporttable_datadimensionitems_pkey: 1024 total in 1 blocks; 8 free (0 chunks); 1016 used
    reporttable_columns_pkey: 1024 total in 1 blocks; 8 free (0 chunks); 1016 used
    pg_toast_545934_index: 1024 total in 1 blocks; 8 free (0 chunks); 1016 used
    reporttable_organisationunits_pkey: 1024 total in 1 blocks; 8 free (0 chunks); 1016 used
    reporttable_periods_pkey: 1024 total in 1 blocks; 8 free (0 chunks); 1016 used
    reporttable_itemorgunitgroups_pkey: 1024 total in 1 blocks; 56 free (0 chunks); 968 used
    orgunitgroupsetdimension_pkey: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    chart_orgunitgroupsetdimensions_pkey: 1024 total in 1 blocks; 56 free (0 chunks); 968 used
    categoryoptiongroupsetdimension_pkey: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    chart_categoryoptiongroupsetdimensions_pkey: 1024 total in 1 blocks; 8 free (0 chunks); 1016 used
    uk_g54lkp9a8q5qp4wsfwxg4c7xc: 1024 total in 1 blocks; 192 free (0 chunks); 832 used
    chartuseraccesses_pkey: 1024 total in 1 blocks; 56 free (0 chunks); 968 used
    categorydimension_pkey: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    chart_categorydimensions_pkey: 1024 total in 1 blocks; 56 free (0 chunks); 968 used
    chartusergroupaccesses_usergroupaccessid_key: 1024 total in 1 blocks; 192 free (0 chunks); 832 used
    chartusergroupaccesses_pkey: 1024 total in 1 blocks; 56 free (0 chunks); 968 used
    objecttranslation_pkey: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    charttranslations_pkey: 1024 total in 1 blocks; 8 free (0 chunks); 1016 used
    dataelementgroupsetdimension_pkey: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    chart_dataelementgroupsetdimensions_pkey: 1024 total in 1 blocks; 8 free (0 chunks); 1016 used
    datadimensionitem_pkey: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    chart_datadimensionitems_pkey: 1024 total in 1 blocks; 8 free (0 chunks); 1016 used
    chart_filters_pkey: 1024 total in 1 blocks; 8 free (0 chunks); 1016 used
    relativeperiods_pkey: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    chart_orgunitlevels_pkey: 1024 total in 1 blocks; 8 free (0 chunks); 1016 used
    chart_organisationunits_pkey: 1024 total in 1 blocks; 8 free (0 chunks); 1016 used
    uk_tbkbjga8h4j5u33d7hbcuk66t: 3072 total in 2 blocks; 1720 free (1 chunks); 1352 used
    period_pkey: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    period_periodtypeid_key: 3072 total in 2 blocks; 1720 free (1 chunks); 1352 used
    chart_periods_pkey: 1024 total in 1 blocks; 8 free (0 chunks); 1016 used
    orgunitgroup_name_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    orgunitgroup_uid_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    orgunitgroup_shortname_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    orgunitgroup_pkey: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    orgunitgroup_code_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    chart_itemorgunitgroups_pkey: 1024 total in 1 blocks; 56 free (0 chunks); 968 used
    CachedPlan: 7168 total in 3 blocks; 1120 free (0 chunks); 6048 used
    CachedPlanSource: 15360 total in 4 blocks; 4544 free (0 chunks); 10816 used
      CachedPlanQuery: 7168 total in 3 blocks; 1032 free (2 chunks); 6136 used
    maplegend_startvalue: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    maplegend_endvalue: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    maplegend_uid_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    maplegend_pkey: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    maplegend_code_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    maplegendset_uid_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    maplegendset_pkey: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    maplegendset_name_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    maplegendset_code_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    orgunitgroupset_name_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    orgunitgroupset_uid_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    orgunitgroupset_pkey: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    orgunitgroupset_code_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    CachedPlan: 31744 total in 5 blocks; 14216 free (0 chunks); 17528 used
    CachedPlanSource: 31744 total in 5 blocks; 13512 free (3 chunks); 18232 used
      CachedPlanQuery: 15360 total in 4 blocks; 624 free (1 chunks); 14736 used
    dashboarditem_uid_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    dashboarditem_pkey: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    dashboard_items_pkey: 1024 total in 1 blocks; 8 free (0 chunks); 1016 used
    useraccess_pkey: 1024 total in 1 blocks; 192 free (0 chunks); 832 used
    uk_ow5b3iks1dgs1hqimo59fgegl: 1024 total in 1 blocks; 192 free (0 chunks); 832 used
    dashboarduseraccesses_pkey: 1024 total in 1 blocks; 56 free (0 chunks); 968 used
    userinfo_uid_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    userinfo_pkey: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    userinfo_code_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    usergroupaccess_pkey: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    dashboardusergroupaccesses_pkey: 1024 total in 1 blocks; 8 free (0 chunks); 1016 used
    dashboard_uid_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    dashboard_pkey: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    interpretation_lastupdated: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    interpretation_uid_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    interpretation_pkey: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    usermessage_userid: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    usermessage_isread: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    usermessage_pkey: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    messageconversation_usermessages_usermessageid_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    messageconversation_usermessages_pkey: 1024 total in 1 blocks; 8 free (0 chunks); 1016 used
    messageconversation_lastmessage: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    messageconversation_uid_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    messageconversation_pkey: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    uk_n18s4feicujvngv2ajoesdgio: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    uk_by4pqq1ans00ffmrgqqh9ehog: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    uk_3svwn20y9qda34bmatesg5c0j: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    dataapprovalworkflow_pkey: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    dataset_uid_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    dataset_pkey: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    dataset_code_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    CachedPlan: 1024 total in 1 blocks; 776 free (0 chunks); 248 used
    CachedPlanSource: 1024 total in 1 blocks; 640 free (0 chunks); 384 used
      CachedPlanQuery: 1024 total in 1 blocks; 584 free (0 chunks); 440 used
    program_name_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    uk_1tcaydw2p91wi8ib0qqa1jcfs: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    program_uid_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    program_pkey: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    program_code_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    usergroup_uid_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    usergroup_pkey: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    usergroup_code_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    usergroupmembers_pkey: 1024 total in 1 blocks; 8 free (0 chunks); 1016 used
    attributevalue_pkey: 1024 total in 1 blocks; 192 free (0 chunks); 832 used
    userattributevalues_pkey: 1024 total in 1 blocks; 56 free (0 chunks); 968 used
    userattributevalues_attributevalueid_key: 1024 total in 1 blocks; 192 free (0 chunks); 832 used
    dataelementcategory_name_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    dataelementcategory_uid_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    dataelementcategory_pkey: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    dataelementcategory_code_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    users_catdimensionconstraints_pkey: 1024 total in 1 blocks; 56 free (0 chunks); 968 used
    categoryoptiongroupset_pkey: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    users_cogsdimensionconstraints_pkey: 1024 total in 1 blocks; 56 free (0 chunks); 968 used
    pg_toast_2619_index: 1024 total in 1 blocks; 8 free (0 chunks); 1016 used
    organisationunit_code_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    in_parentid: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    in_organisationunit_path: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    in_organisationunit_hierarchylevel: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    uk_pw2bgc9ykjad2obefeqha28t4: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    organisationunit_uid_key: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    organisationunit_pkey: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    userteisearchorgunits_pkey: 1024 total in 1 blocks; 56 free (0 chunks); 968 used
    usersetting_pkey: 1024 total in 1 blocks; 8 free (0 chunks); 1016 used
    pg_index_indrelid_index: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    pg_db_role_setting_databaseid_rol_index: 1024 total in 1 blocks; 56 free (0 chunks); 968 used
    pg_user_mapping_user_server_index: 1024 total in 1 blocks; 56 free (0 chunks); 968 used
    pg_user_mapping_oid_index: 1024 total in 1 blocks; 192 free (0 chunks); 832 used
    pg_type_oid_index: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    pg_type_typname_nsp_index: 1024 total in 1 blocks; 8 free (0 chunks); 1016 used
    pg_ts_template_oid_index: 1024 total in 1 blocks; 192 free (0 chunks); 832 used
    pg_ts_template_tmplname_index: 1024 total in 1 blocks; 56 free (0 chunks); 968 used
    pg_ts_parser_oid_index: 1024 total in 1 blocks; 192 free (0 chunks); 832 used
    pg_ts_parser_prsname_index: 1024 total in 1 blocks; 56 free (0 chunks); 968 used
    pg_ts_dict_oid_index: 1024 total in 1 blocks; 192 free (0 chunks); 832 used
    pg_ts_dict_dictname_index: 1024 total in 1 blocks; 56 free (0 chunks); 968 used
    pg_ts_config_oid_index: 1024 total in 1 blocks; 192 free (0 chunks); 832 used
    pg_ts_config_cfgname_index: 1024 total in 1 blocks; 56 free (0 chunks); 968 used
    pg_ts_config_map_index: 3072 total in 2 blocks; 1768 free (2 chunks); 1304 used
    pg_transform_type_lang_index: 1024 total in 1 blocks; 56 free (0 chunks); 968 used
    pg_transform_oid_index: 1024 total in 1 blocks; 192 free (0 chunks); 832 used
    pg_class_relname_nsp_index: 1024 total in 1 blocks; 8 free (0 chunks); 1016 used
    pg_range_rngtypid_index: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    pg_proc_oid_index: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    pg_proc_proname_args_nsp_index: 3072 total in 2 blocks; 1720 free (1 chunks); 1352 used
    pg_opfamily_oid_index: 1024 total in 1 blocks; 192 free (0 chunks); 832 used
    pg_opfamily_am_name_nsp_index: 3072 total in 2 blocks; 1768 free (2 chunks); 1304 used
    pg_operator_oid_index: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    pg_operator_oprname_l_r_n_index: 3072 total in 2 blocks; 1720 free (1 chunks); 1352 used
    pg_namespace_oid_index: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    pg_namespace_nspname_index: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    pg_language_oid_index: 1024 total in 1 blocks; 192 free (0 chunks); 832 used
    pg_language_name_index: 1024 total in 1 blocks; 192 free (0 chunks); 832 used
    pg_foreign_table_relid_index: 1024 total in 1 blocks; 192 free (0 chunks); 832 used
    pg_foreign_server_oid_index: 1024 total in 1 blocks; 192 free (0 chunks); 832 used
    pg_foreign_server_name_index: 1024 total in 1 blocks; 192 free (0 chunks); 832 used
    pg_foreign_data_wrapper_oid_index: 1024 total in 1 blocks; 192 free (0 chunks); 832 used
    pg_foreign_data_wrapper_name_index: 1024 total in 1 blocks; 192 free (0 chunks); 832 used
    pg_event_trigger_oid_index: 1024 total in 1 blocks; 192 free (0 chunks); 832 used
    pg_event_trigger_evtname_index: 1024 total in 1 blocks; 192 free (0 chunks); 832 used
    pg_enum_typid_label_index: 1024 total in 1 blocks; 56 free (0 chunks); 968 used
    pg_enum_oid_index: 1024 total in 1 blocks; 192 free (0 chunks); 832 used
    pg_default_acl_role_nsp_obj_index: 3072 total in 2 blocks; 1768 free (2 chunks); 1304 used
    pg_conversion_oid_index: 1024 total in 1 blocks; 192 free (0 chunks); 832 used
    pg_constraint_oid_index: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    pg_conversion_name_nsp_index: 1024 total in 1 blocks; 56 free (0 chunks); 968 used
    pg_conversion_default_index: 3072 total in 2 blocks; 1768 free (2 chunks); 1304 used
    pg_collation_oid_index: 1024 total in 1 blocks; 192 free (0 chunks); 832 used
    pg_collation_name_enc_nsp_index: 3072 total in 2 blocks; 1768 free (2 chunks); 1304 used
    pg_opclass_am_name_nsp_index: 3072 total in 2 blocks; 1720 free (1 chunks); 1352 used
    pg_cast_source_target_index: 1024 total in 1 blocks; 8 free (0 chunks); 1016 used
    pg_attribute_relid_attnam_index: 1024 total in 1 blocks; 8 free (0 chunks); 1016 used
    pg_amop_fam_strat_index: 3072 total in 2 blocks; 1720 free (1 chunks); 1352 used
    pg_amop_opr_fam_index: 3072 total in 2 blocks; 1720 free (1 chunks); 1352 used
    pg_aggregate_fnoid_index: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    pg_trigger_tgrelid_tgname_index: 1024 total in 1 blocks; 8 free (0 chunks); 1016 used
    pg_rewrite_rel_rulename_index: 1024 total in 1 blocks; 56 free (0 chunks); 968 used
    pg_amproc_fam_proc_index: 3072 total in 2 blocks; 1720 free (1 chunks); 1352 used
    pg_opclass_oid_index: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    pg_index_indexrelid_index: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    pg_attribute_relid_attnum_index: 1024 total in 1 blocks; 8 free (0 chunks); 1016 used
    pg_class_oid_index: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    pg_authid_oid_index: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    pg_auth_members_member_role_index: 1024 total in 1 blocks; 56 free (0 chunks); 968 used
    pg_tablespace_oid_index: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    pg_replication_origin_roname_index: 1024 total in 1 blocks; 192 free (0 chunks); 832 used
    pg_database_datname_index: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    pg_replication_origin_roiident_index: 1024 total in 1 blocks; 192 free (0 chunks); 832 used
    pg_auth_members_role_member_index: 1024 total in 1 blocks; 56 free (0 chunks); 968 used
    pg_database_oid_index: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
    pg_authid_rolname_index: 1024 total in 1 blocks; 144 free (0 chunks); 880 used
  WAL record construction: 49768 total in 2 blocks; 6584 free (0 chunks); 43184 used
  PrivateRefCount: 8192 total in 1 blocks; 2696 free (0 chunks); 5496 used
  MdSmgr: 8192 total in 1 blocks; 504 free (1 chunks); 7688 used
  ident parser context: 0 total in 0 blocks; 0 free (0 chunks); 0 used
  hba parser context: 7168 total in 3 blocks; 3160 free (1 chunks); 4008 used
  LOCALLOCK hash: 24576 total in 2 blocks; 13904 free (4 chunks); 10672 used
  Timezones: 104120 total in 2 blocks; 3736 free (0 chunks); 100384 used
  ErrorContext: 8192 total in 1 blocks; 8152 free (0 chunks); 40 used
2018-05-11 07:03:59 UTC [16449-2] LOG:  server process (PID 18016) was terminated by signal 9: Killed
2018-05-11 07:03:59 UTC [16449-3] DETAIL:  Failed process was running: create index "in_uidlevel4_ax_2015_r7jVa" on analytics_temp_2015 ("uidlevel4")
2018-05-11 07:03:59 UTC [16449-4] LOG:  terminating any other active server processes
2018-05-11 07:03:59 UTC [18025-11] dhis@dhis WARNING:  terminating connection because of crash of another server process
2018-05-11 07:03:59 UTC [18025-12] dhis@dhis DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2018-05-11 07:03:59 UTC [18025-13] dhis@dhis HINT:  In a moment you should be able to reconnect to the database and repeat your command.
2018-05-11 07:03:59 UTC [18024-11] dhis@dhis WARNING:  terminating connection because of crash of another server process
2018-05-11 07:03:59 UTC [18024-12] dhis@dhis DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2018-05-11 07:03:59 UTC [18024-13] dhis@dhis HINT:  In a moment you should be able to reconnect to the database and repeat your command.
2018-05-11 07:03:59 UTC [18061-1] dhis@dhis WARNING:  terminating connection because of crash of another server process
2018-05-11 07:03:59 UTC [18061-2] dhis@dhis DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2018-05-11 07:03:59 UTC [18061-3] dhis@dhis HINT:  In a moment you should be able to reconnect to the database and repeat your command.
2018-05-11 07:03:59 UTC [18023-13] dhis@dhis WARNING:  terminating connection because of crash of another server process
2018-05-11 07:03:59 UTC [18023-14] dhis@dhis DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2018-05-11 07:03:59 UTC [18023-15] dhis@dhis HINT:  In a moment you should be able to reconnect to the database and repeat your command.
2018-05-11 07:03:59 UTC [18044-1] dhis@dhis WARNING:  terminating connection because of crash of another server process
2018-05-11 07:03:59 UTC [18044-2] dhis@dhis DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2018-05-11 07:03:59 UTC [18044-3] dhis@dhis HINT:  In a moment you should be able to reconnect to the database and repeat your command.
2018-05-11 07:03:59 UTC [16454-2] WARNING:  terminating connection because of crash of another server process
2018-05-11 07:03:59 UTC [16454-3] DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2018-05-11 07:03:59 UTC [16454-4] HINT:  In a moment you should be able to reconnect to the database and repeat your command.
2018-05-11 07:03:59 UTC [18048-1] dhis@dhis WARNING:  terminating connection because of crash of another server process
2018-05-11 07:03:59 UTC [18048-2] dhis@dhis DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2018-05-11 07:03:59 UTC [18048-3] dhis@dhis HINT:  In a moment you should be able to reconnect to the database and repeat your command.
2018-05-11 07:03:59 UTC [18022-13] dhis@dhis WARNING:  terminating connection because of crash of another server process
2018-05-11 07:03:59 UTC [18022-14] dhis@dhis DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2018-05-11 07:03:59 UTC [18022-15] dhis@dhis HINT:  In a moment you should be able to reconnect to the database and repeat your command.
2018-05-11 07:03:59 UTC [18018-1] dhis@dhis WARNING:  terminating connection because of crash of another server process
2018-05-11 07:03:59 UTC [18018-2] dhis@dhis DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2018-05-11 07:03:59 UTC [18018-3] dhis@dhis HINT:  In a moment you should be able to reconnect to the database and repeat your command.
2018-05-11 07:03:59 UTC [18017-1] dhis@dhis WARNING:  terminating connection because of crash of another server process
2018-05-11 07:03:59 UTC [18017-2] dhis@dhis DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2018-05-11 07:03:59 UTC [18017-3] dhis@dhis HINT:  In a moment you should be able to reconnect to the database and repeat your command.
2018-05-11 07:03:59 UTC [18043-1] dhis@dhis WARNING:  terminating connection because of crash of another server process
2018-05-11 07:03:59 UTC [18043-2] dhis@dhis DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2018-05-11 07:03:59 UTC [18043-3] dhis@dhis HINT:  In a moment you should be able to reconnect to the database and repeat your command.
2018-05-11 07:03:59 UTC [18047-1] dhis@dhis WARNING:  terminating connection because of crash of another server process
2018-05-11 07:03:59 UTC [18047-2] dhis@dhis DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2018-05-11 07:03:59 UTC [18047-3] dhis@dhis HINT:  In a moment you should be able to reconnect to the database and repeat your command.
2018-05-11 07:03:59 UTC [18015-13] dhis@dhis WARNING:  terminating connection because of crash of another server process
2018-05-11 07:03:59 UTC [18015-14] dhis@dhis DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2018-05-11 07:03:59 UTC [18015-15] dhis@dhis HINT:  In a moment you should be able to reconnect to the database and repeat your command.
2018-05-11 07:03:59 UTC [16534-25] dhis@dhis WARNING:  terminating connection because of crash of another server process
2018-05-11 07:03:59 UTC [16534-26] dhis@dhis DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2018-05-11 07:03:59 UTC [16534-27] dhis@dhis HINT:  In a moment you should be able to reconnect to the database and repeat your command.
2018-05-11 07:03:59 UTC [18058-1] dhis@dhis WARNING:  terminating connection because of crash of another server process
2018-05-11 07:03:59 UTC [18058-2] dhis@dhis DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2018-05-11 07:03:59 UTC [18058-3] dhis@dhis HINT:  In a moment you should be able to reconnect to the database and repeat your command.
2018-05-11 07:03:59 UTC [18020-9] dhis@dhis WARNING:  terminating connection because of crash of another server process
2018-05-11 07:03:59 UTC [18020-10] dhis@dhis DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2018-05-11 07:03:59 UTC [18020-11] dhis@dhis HINT:  In a moment you should be able to reconnect to the database and repeat your command.
2018-05-11 07:03:59 UTC [18045-1] dhis@dhis WARNING:  terminating connection because of crash of another server process
2018-05-11 07:03:59 UTC [18045-2] dhis@dhis DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2018-05-11 07:03:59 UTC [18045-3] dhis@dhis HINT:  In a moment you should be able to reconnect to the database and repeat your command.
2018-05-11 07:04:00 UTC [18059-1] dhis@dhis WARNING:  terminating connection because of crash of another server process
2018-05-11 07:04:00 UTC [18059-2] dhis@dhis DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2018-05-11 07:04:00 UTC [18059-3] dhis@dhis HINT:  In a moment you should be able to reconnect to the database and repeat your command.
2018-05-11 07:04:00 UTC [18062-1] dhis@dhis WARNING:  terminating connection because of crash of another server process
2018-05-11 07:04:00 UTC [18062-2] dhis@dhis DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2018-05-11 07:04:00 UTC [18062-3] dhis@dhis HINT:  In a moment you should be able to reconnect to the database and repeat your command.
2018-05-11 07:04:00 UTC [18057-1] dhis@dhis WARNING:  terminating connection because of crash of another server process
2018-05-11 07:04:00 UTC [18057-2] dhis@dhis DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2018-05-11 07:04:00 UTC [18057-3] dhis@dhis HINT:  In a moment you should be able to reconnect to the database and repeat your command.
2018-05-11 07:04:00 UTC [18021-19] dhis@dhis WARNING:  terminating connection because of crash of another server process
2018-05-11 07:04:00 UTC [18021-20] dhis@dhis DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2018-05-11 07:04:00 UTC [18021-21] dhis@dhis HINT:  In a moment you should be able to reconnect to the database and repeat your command.
2018-05-11 07:04:00 UTC [18019-23] dhis@dhis WARNING:  terminating connection because of crash of another server process
2018-05-11 07:04:00 UTC [18019-24] dhis@dhis DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2018-05-11 07:04:00 UTC [18019-25] dhis@dhis HINT:  In a moment you should be able to reconnect to the database and repeat your command.
2018-05-11 07:04:00 UTC [16533-37] dhis@dhis WARNING:  terminating connection because of crash of another server process
2018-05-11 07:04:00 UTC [16533-38] dhis@dhis DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2018-05-11 07:04:00 UTC [16533-39] dhis@dhis HINT:  In a moment you should be able to reconnect to the database and repeat your command.
2018-05-11 07:04:00 UTC [18060-1] dhis@dhis WARNING:  terminating connection because of crash of another server process
2018-05-11 07:04:00 UTC [18060-2] dhis@dhis DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2018-05-11 07:04:00 UTC [18060-3] dhis@dhis HINT:  In a moment you should be able to reconnect to the database and repeat your command.
2018-05-11 07:04:00 UTC [16532-24] dhis@dhis WARNING:  terminating connection because of crash of another server process
2018-05-11 07:04:00 UTC [16532-25] dhis@dhis DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2018-05-11 07:04:00 UTC [16532-26] dhis@dhis HINT:  In a moment you should be able to reconnect to the database and repeat your command.
2018-05-11 07:04:00 UTC [18046-1] dhis@dhis WARNING:  terminating connection because of crash of another server process
2018-05-11 07:04:00 UTC [18046-2] dhis@dhis DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2018-05-11 07:04:00 UTC [18046-3] dhis@dhis HINT:  In a moment you should be able to reconnect to the database and repeat your command.
2018-05-11 07:04:00 UTC [18014-25] dhis@dhis WARNING:  terminating connection because of crash of another server process
2018-05-11 07:04:00 UTC [18014-26] dhis@dhis DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2018-05-11 07:04:00 UTC [18014-27] dhis@dhis HINT:  In a moment you should be able to reconnect to the database and repeat your command.
2018-05-11 07:04:00 UTC [18063-1] dhis@dhis FATAL:  the database system is in recovery mode
2018-05-11 07:04:00 UTC [18064-1] dhis@dhis FATAL:  the database system is in recovery mode
2018-05-11 07:04:00 UTC [18065-1] dhis@dhis FATAL:  the database system is in recovery mode
2018-05-11 07:04:00 UTC [18066-1] dhis@dhis FATAL:  the database system is in recovery mode
2018-05-11 07:04:00 UTC [18067-1] dhis@dhis FATAL:  the database system is in recovery mode
2018-05-11 07:04:00 UTC [18068-1] dhis@dhis FATAL:  the database system is in recovery mode
2018-05-11 07:04:00 UTC [18069-1] dhis@dhis FATAL:  the database system is in recovery mode
2018-05-11 07:04:00 UTC [18070-1] dhis@dhis FATAL:  the database system is in recovery mode
2018-05-11 07:04:00 UTC [18071-1] dhis@dhis FATAL:  the database system is in recovery mode
2018-05-11 07:04:00 UTC [18072-1] dhis@dhis FATAL:  the database system is in recovery mode
2018-05-11 07:04:00 UTC [18073-1] dhis@dhis FATAL:  the database system is in recovery mode
2018-05-11 07:04:00 UTC [18074-1] dhis@dhis FATAL:  the database system is in recovery mode
2018-05-11 07:04:00 UTC [18075-1] dhis@dhis FATAL:  the database system is in recovery mode
2018-05-11 07:04:00 UTC [18076-1] dhis@dhis FATAL:  the database system is in recovery mode
2018-05-11 07:04:00 UTC [18077-1] dhis@dhis FATAL:  the database system is in recovery mode
2018-05-11 07:04:00 UTC [18078-1] dhis@dhis FATAL:  the database system is in recovery mode
2018-05-11 07:04:00 UTC [18079-1] dhis@dhis FATAL:  the database system is in recovery mode
2018-05-11 07:04:00 UTC [18080-1] dhis@dhis FATAL:  the database system is in recovery mode
2018-05-11 07:04:00 UTC [18081-1] dhis@dhis FATAL:  the database system is in recovery mode
2018-05-11 07:04:00 UTC [18082-1] dhis@dhis FATAL:  the database system is in recovery mode
2018-05-11 07:04:00 UTC [18083-1] dhis@dhis FATAL:  the database system is in recovery mode
2018-05-11 07:04:00 UTC [18084-1] dhis@dhis FATAL:  the database system is in recovery mode
2018-05-11 07:04:00 UTC [18085-1] dhis@dhis FATAL:  the database system is in recovery mode
2018-05-11 07:04:00 UTC [18086-1] dhis@dhis FATAL:  the database system is in recovery mode
2018-05-11 07:04:00 UTC [18087-1] dhis@dhis FATAL:  the database system is in recovery mode
2018-05-11 07:04:00 UTC [18088-1] dhis@dhis FATAL:  the database system is in recovery mode
2018-05-11 07:04:00 UTC [16449-5] LOG:  all server processes terminated; reinitializing
2018-05-11 07:04:01 UTC [18090-1] dhis@dhis FATAL:  the database system is in recovery mode
2018-05-11 07:04:01 UTC [18091-1] dhis@dhis FATAL:  the database system is in recovery mode
2018-05-11 07:04:01 UTC [18092-1] dhis@dhis FATAL:  the database system is in recovery mode
2018-05-11 07:04:01 UTC [18089-1] LOG:  database system was interrupted; last known up at 2018-05-11 07:03:28 UTC
2018-05-11 07:04:01 UTC [18093-1] dhis@dhis FATAL:  the database system is in recovery mode
2018-05-11 07:04:01 UTC [18094-1] dhis@dhis FATAL:  the database system is in recovery mode
2018-05-11 07:04:01 UTC [18095-1] dhis@dhis FATAL:  the database system is in recovery mode
2018-05-11 07:04:01 UTC [18096-1] dhis@dhis FATAL:  the database system is in recovery mode
2018-05-11 07:04:01 UTC [18098-1] dhis@dhis FATAL:  the database system is in recovery mode
2018-05-11 07:04:01 UTC [18097-1] dhis@dhis FATAL:  the database system is in recovery mode
2018-05-11 07:04:01 UTC [18099-1] dhis@dhis FATAL:  the database system is in recovery mode
2018-05-11 07:04:01 UTC [18100-1] dhis@dhis FATAL:  the database system is in recovery mode
2018-05-11 07:04:01 UTC [18102-1] dhis@dhis FATAL:  the database system is in recovery mode
2018-05-11 07:04:01 UTC [18101-1] dhis@dhis FATAL:  the database system is in recovery mode
2018-05-11 07:04:01 UTC [18103-1] dhis@dhis FATAL:  the database system is in recovery mode
2018-05-11 07:04:01 UTC [18104-1] dhis@dhis FATAL:  the database system is in recovery mode

References

Trouble running Analytics for 2.29
From: Knut Staring, 2018-05-10
Re: Trouble running Analytics for 2.29
From: Bob Jolliffe, 2018-05-10
Re: Trouble running Analytics for 2.29
From: Knut Staring, 2018-05-10
Re: Trouble running Analytics for 2.29
From: Bob Jolliffe, 2018-05-10