← Back to team overview

dhis2-users team mailing list archive

Re: [Dhis2-devs] Amazon EC2 memory configuration

 

Thanks all for your suggestions. I got an instance from a simpler VPS service (DigitalOcean) with 2GB/2CPU and it's running well without any complex configurations needed.

Initially I followed the Implementation Guides' minimal requirements from https://www.dhis2.org/doc/snapshot/en/implementer/html/dhis2_implementation_guide_en_full.html#d3606e413 and suggest adding the mentioned implications to it, may help other implementers, too (or should I add it to https://github.com/dhis2/dhis2-docs/blob/master/src/docbkx/en/dhis2_implementation_guide_installation.xml#L17 myself?)

Best
David


On 02.09.2015 23:38, Bob Jolliffe wrote:
Hi David

The log file does seem to indicate that the startup was successful.
It does look like an orderly shutdown rather than any sign of
out-of-memory exceptions.  Which happens about 1 minute after the
successful startup.  Did you shut it down yourself or it just
collapsed without you doing anything?

if you started it then couldn't access it, so you shut it down, it
might well be related to a firewall issue as per Lars's suggestion.

Don't pay the SEVERE memory leak exceptions too much mind.  These are
important if you are starting and stopping individual web applications
in a (continuous) running tomcat.  They don't mean much at all if you
are bringing down the whole tomcat server.

Running in 1G is a challenge and you might be able to get dhis up and
running but barely functional.  There are a few things to try:

1.  Don't try to optimize postgres settings at all.  Let it run in the
bare minimum out-of-the-box default.  If I add up all your java and
postgres allocations you are already over 1G without even considering
kernel memory and the like.

2.  You could (in server.xml) reduce the max size of tomcat threadpool
to maybe 4 or 5.

3.  There are some dhis internal threadpool configs you can also
reduce but its a bit more fiddly (and I don't have the details at
hand)

The sensible thing (depending on what you are trying to achieve) is to
increase your available RAM to 2G or more.  getting it to run in 1G is
an interesting challenge which I've achieved once or twice but really
not all that useful.

Bob

On 2 September 2015 at 17:52, Lars Helge Øverland<larshelge@xxxxxxxxx>  wrote:
Hi David,

INFO: Pausing ProtocolHandler ["http-bio-8080"]

you are running on port 8080, this might be related to firewalls on aws. Try
running on port 80 (conf/server.xml).

Alternatively check if firewalls are active and blocks 8080.

regards,

Lars





On Wed, Sep 2, 2015 at 3:47 PM, Dan Cocos<dcocos@xxxxxxxxxxxxxx>  wrote:
At BAO we've found we could only reliably support DHIS 2 with a minimum of
2GB.


Dan Cocos
BAO Systems
www.baosystems.com
T: +1 202-569-8751 | gchat:dcocos@xxxxxxxxxxxxxx

On Sep 2, 2015, at 7:33 AM, Jason Pickering<jason.p.pickering@xxxxxxxxx>
wrote:


Are you sure that Tomcat actually shut down ?

Be sure Tomcat is dead by invoking ps -ef | grep tomcat and killing any
processes(es) which may be running, and then start it again. If you get that
memory leak error again, then something else is not right.

Having said that, I agree with Knut. 1GB will really be pushing it, but
you might be able to get it to work.

Regards,
Jason




On Wed, Sep 2, 2015 at 12:56 PM, Knut Staring<knutst@xxxxxxxxx>  wrote:
Hi David,

1 GB is probably pushing it a bit, AFAIK. Possible, but not too easy....2
GB should be fine.

Knut

On Wed, Sep 2, 2015 at 12:11 PM, David Huser<david.huser@xxxxxxxxxxxx>
wrote:
Hi

I registered for a free tier EC2 instance (t2.micro) on Amazon with
Ubuntu 14.04 AMD64http://aws.amazon.com/ec2/instance-types/
I have set it up according to the DHIS2 implementation manual.
I'm aware that this low hardware specifications are not ready for
national use, I'd like to use it as prototyping resource. However, the
server responds to pings but is not accessible through the browser. I
suspect the memory configuration. Please find the relevant configurations /
error messages in red below.

Hardware:
RAM: 1GB memory
CPU: 1 CPU, spike balancing (t2.micro tier from amazon)
Disk: 20GB

Basically, I just divided the RAM specs from the implementation manual
(2x 8GB) for each attribute by 16 (I have 1GB).


$ cat /etc/postgresql/9.3/main/postgresql.conf
shared_buffers = 128MB
work_mem = 1MB
maintenance_work_mem = 32MB
effective_cache_size = 128MB

$ java -version
java version "1.7.0_79"
OpenJDK Runtime Environment (IcedTea 2.5.6)
(7u79-2.5.6-0ubuntu1.14.04.1)
OpenJDK 64-Bit Server VM (build 24.79-b02, mixed mode)



$ cat /tomcat-dhis/bin/setenv.sh
#!/bin/sh
#

CATALINA_HOME=/usr/share/tomcat7

# The first existing directory is used for JAVA_HOME (if JAVA_HOME is
not
# defined in $DEFAULT)
JDK_DIRS="/usr/lib/jvm/java-6-openjdk /usr/lib/jvm/java-6-sun
/usr/lib/jvm/java-1.5.0-sun /usr/lib/j2sdk1.5-sun /usr/lib/j2sdk1.5-ibm"

# Look for the right JVM to use
for jdir in $JDK_DIRS; do
     if [ -r "$jdir/bin/java" -a -z "${JAVA_HOME}" ]; then
         JAVA_HOME_TMP="$jdir"
         # checks for a real JDK like environment, needed to check if
         # really the java-gcj-compat-dev package is installed
         if [ -r "$jdir/bin/jdb" ]; then
             JAVA_HOME="$JAVA_HOME_TMP"
         fi
     fi
done

# Default Java options
if [ -z "$JAVA_OPTS" ]; then
         JAVA_OPTS="-Djava.awt.headless=true -Xmx128M"
fi

export JAVA_HOME='/usr/lib/jvm/java-7-openjdk-amd64'
export JAVA_OPTS='-Xmx768m -Xms256m -XX:MaxPermSize=128m
-XX:PermSize=32m'
export DHIS2_HOME='/home/dhis/config'




after restarting tomcat:
$ cat catalina.out

Sep 02, 2015 11:59:08 AM org.apache.catalina.startup.ClassLoaderFactory
validateFile
WARNING: Problem with directory [/usr/share/tomcat7/common/classes],
exists: [false], isDirectory: [false], canRead: [false]
Sep 02, 2015 11:59:08 AM org.apache.catalina.startup.ClassLoaderFactory
validateFile
WARNING: Problem with directory [/usr/share/tomcat7/common], exists:
[false], isDirectory: [false], canRead: [false]
Sep 02, 2015 11:59:08 AM org.apache.catalina.startup.ClassLoaderFactory
validateFile
WARNING: Problem with directory [/usr/share/tomcat7/server/classes],
exists: [false], isDirectory: [false], canRead: [false]
Sep 02, 2015 11:59:08 AM org.apache.catalina.startup.ClassLoaderFactory
validateFile
WARNING: Problem with directory [/usr/share/tomcat7/server], exists:
[false], isDirectory: [false], canRead: [false]
Sep 02, 2015 11:59:08 AM org.apache.catalina.startup.ClassLoaderFactory
validateFile
WARNING: Problem with directory [/usr/share/tomcat7/shared/classes],
exists: [false], isDirectory: [false], canRead: [false]
Sep 02, 2015 11:59:08 AM org.apache.catalina.startup.ClassLoaderFactory
validateFile
WARNING: Problem with directory [/usr/share/tomcat7/shared], exists:
[false], isDirectory: [false], canRead: [false]
Sep 02, 2015 11:59:09 AM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["http-bio-8080"]
Sep 02, 2015 11:59:09 AM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 1007 ms
Sep 02, 2015 11:59:09 AM org.apache.catalina.core.StandardService
startInternal
INFO: Starting service Catalina
Sep 02, 2015 11:59:09 AM org.apache.catalina.core.StandardEngine
startInternal
INFO: Starting Servlet Engine: Apache Tomcat/7.0.52 (Ubuntu)
Sep 02, 2015 11:59:09 AM org.apache.catalina.startup.HostConfig
deployWAR
INFO: Deploying web application archive
/home/dhis/tomcat-dhis/webapps/ROOT.war
* INFO  2015-09-02 11:59:35,400 System property dhis2.home not set
(DefaultLocationManager.java [localhost-startStop-1])
* INFO  2015-09-02 11:59:35,402 Environment variable DHIS2_HOME points
to /home/dhis/config (DefaultLocationManager.java [localhost-startStop-1])
* INFO  2015-09-02 11:59:37,945 Hibernate configuration loaded, using
dialect: org.hibernate.dialect.PostgreSQLDialect
(DefaultHibernateConfigurationProvider.java [localhost-startStop-1])
* INFO  2015-09-02 11:59:57,732 Found the following message senders:
[org.hisp.dhis.message.EmailMessageSender@68b059dc,
org.hisp.dhis.sms.SmsMessageSender@5e5f2c88] (DefaultMessageService.java
[localhost-startStop-1])
* INFO  2015-09-02 12:00:09,471 Detected apps: []
(DefaultAppManager.java [localhost-startStop-1])
* INFO  2015-09-02 12:00:13,236 Added logger:
org.hisp.dhis.resourcetable using file:
/home/dhis/config/logs/dhis-analytics-table.log
(Log4JLogConfigInitializer.java [localhost-startStop-1])
* INFO  2015-09-02 12:00:13,236 Added logger:
org.hisp.dhis.analytics.table using file:
/home/dhis/config/logs/dhis-analytics-table.log
(Log4JLogConfigInitializer.java [localhost-startStop-1])
* INFO  2015-09-02 12:00:13,237 Added logger: org.hisp.dhis.dxf2 using
file: /home/dhis/config/logs/dhis-data-exchange.log
(Log4JLogConfigInitializer.java [localhost-startStop-1])
* INFO  2015-09-02 12:00:13,237 Added logger: org.hisp.dhis.dxf2.synch
using file: /home/dhis/config/logs/dhis-data-sync.log
(Log4JLogConfigInitializer.java [localhost-startStop-1])
* INFO  2015-09-02 12:00:13,237 Added root logger using file:
/home/dhis/config/logs/dhis.log (Log4JLogConfigInitializer.java
[localhost-startStop-1])
* WARN  2015-09-02 12:00:14,817 RP discovery / realm validation
disabled;  (RealmVerifier.java [localhost-startStop-1])
* INFO  2015-09-02 12:00:15,654 Executing startup routine [1 of 15,
runlevel 1]: InitTableAlteror (DefaultStartupRoutineExecutor.java
[localhost-startStop-1])
* INFO  2015-09-02 12:00:15,774 Executing startup routine [2 of 15,
runlevel 2]: PeriodTypePopulator (DefaultStartupRoutineExecutor.java
[localhost-startStop-1])
* INFO  2015-09-02 12:00:15,811 Executing startup routine [3 of 15,
runlevel 2]: DataElementDefaultDimensionPopulator
(DefaultStartupRoutineExecutor.java [localhost-startStop-1])
* INFO  2015-09-02 12:00:15,860 Linked default category with default
concept (DataElementDefaultDimensionPopulator.java [localhost-startStop-1])
* INFO  2015-09-02 12:00:16,005 Executing startup routine [4 of 15,
runlevel 3]: ReportingIdentityPopulator (DefaultStartupRoutineExecutor.java
[localhost-startStop-1])
* INFO  2015-09-02 12:00:16,180 Executing startup routine [5 of 15,
runlevel 3]: TableAlteror (DefaultStartupRoutineExecutor.java
[localhost-startStop-1])
* INFO  2015-09-02 12:00:20,284 Tables updated (TableAlteror.java
[localhost-startStop-1])
* INFO  2015-09-02 12:00:20,285 Executing startup routine [6 of 15,
runlevel 3]: TrackerIdentityPopulator (DefaultStartupRoutineExecutor.java
[localhost-startStop-1])
* INFO  2015-09-02 12:00:20,294 1 last updated set on trackedentity
(IdentityPopulator.java [localhost-startStop-1])
* INFO  2015-09-02 12:00:20,296 1 created timestamps set on
trackedentity (IdentityPopulator.java [localhost-startStop-1])
* INFO  2015-09-02 12:00:20,424 Executing startup routine [7 of 15,
runlevel 4]: TrackedEntityTableAlteror (DefaultStartupRoutineExecutor.java
[localhost-startStop-1])
* INFO  2015-09-02 12:00:21,370 Executing startup routine [8 of 15,
runlevel 4]: IdentityPopulator (DefaultStartupRoutineExecutor.java
[localhost-startStop-1])
* INFO  2015-09-02 12:00:21,488 Executing startup routine [9 of 15,
runlevel 5]: TableCreator (DefaultStartupRoutineExecutor.java
[localhost-startStop-1])
* INFO  2015-09-02 12:00:21,582 Executing startup routine [10 of 15,
runlevel 7]: MapViewUpgrader (DefaultStartupRoutineExecutor.java
[localhost-startStop-1])
* INFO  2015-09-02 12:00:21,690 Executing startup routine [11 of 15,
runlevel 7]: FavoriteDataItemUpgrader (DefaultStartupRoutineExecutor.java
[localhost-startStop-1])
* INFO  2015-09-02 12:00:21,698 Executing startup routine [12 of 15,
runlevel 7]: FavoriteDataItemUpgrader (DefaultStartupRoutineExecutor.java
[localhost-startStop-1])
* INFO  2015-09-02 12:00:21,707 Upgraded custom case entry form
identifiers (DataEntryFormUpgrader.java [localhost-startStop-1])
* INFO  2015-09-02 12:00:21,708 Executing startup routine [13 of 15,
runlevel 8]: ExpressionUpgrader (DefaultStartupRoutineExecutor.java
[localhost-startStop-1])
* INFO  2015-09-02 12:00:21,725 Executing startup routine [14 of 15,
runlevel 9]: ConfigurationPopulator (DefaultStartupRoutineExecutor.java
[localhost-startStop-1])
* INFO  2015-09-02 12:00:21,729 Executing startup routine [15 of 15,
runlevel 9]: I18nLocalePopulator (DefaultStartupRoutineExecutor.java
[localhost-startStop-1])
* INFO  2015-09-02 12:00:21,743 All startup routines done
(DefaultStartupRoutineExecutor.java [localhost-startStop-1])
Sep 02, 2015 12:00:31 PM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["http-bio-8080"]
Sep 02, 2015 12:00:31 PM org.apache.catalina.startup.Catalina start
INFO: Server startup in 82000 ms
Sep 02, 2015 12:01:37 PM org.apache.coyote.AbstractProtocol pause
INFO: Pausing ProtocolHandler ["http-bio-8080"]
Sep 02, 2015 12:01:37 PM org.apache.catalina.core.StandardService
stopInternal
INFO: Stopping service Catalina
* INFO  2015-09-02 12:01:37,676 De-registering jdbc driver:
org.postgresql.Driver@7e306ed7 (StartupListener.java
[localhost-startStop-2])
Sep 02, 2015 12:01:37 PM org.apache.catalina.loader.WebappClassLoader
clearReferencesThreads
SEVERE: The web application [] appears to have started a thread named
[GT authority factory disposer] but has failed to stop it. This is very
likely to create a memory leak.
Sep 02, 2015 12:01:37 PM org.apache.catalina.loader.WebappClassLoader
clearReferencesThreads
SEVERE: The web application [] appears to have started a thread named
[WeakCollectionCleaner] but has failed to stop it. This is very likely to
create a memory leak.
Sep 02, 2015 12:01:37 PM org.apache.coyote.AbstractProtocol stop
INFO: Stopping ProtocolHandler ["http-bio-8080"]
Sep 02, 2015 12:01:37 PM org.apache.coyote.AbstractProtocol destroy
INFO: Destroying ProtocolHandler ["http-bio-8080"]

_______________________________________________
Mailing list:https://launchpad.net/~dhis2-users
Post to     :dhis2-users@xxxxxxxxxxxxxxxxxxx
Unsubscribe :https://launchpad.net/~dhis2-users
More help   :https://help.launchpad.net/ListHelp


--
Knut Staring
Dept. of Informatics, University of Oslo
Norway: +4791880522
Skype: knutstar
http://dhis2.org

_______________________________________________
Mailing list:https://launchpad.net/~dhis2-devs
Post to     :dhis2-devs@xxxxxxxxxxxxxxxxxxx
Unsubscribe :https://launchpad.net/~dhis2-devs
More help   :https://help.launchpad.net/ListHelp


--
Jason P. Pickering
email:jason.p.pickering@xxxxxxxxx
tel:+46764147049
_______________________________________________
Mailing list:https://launchpad.net/~dhis2-devs
Post to     :dhis2-devs@xxxxxxxxxxxxxxxxxxx
Unsubscribe :https://launchpad.net/~dhis2-devs
More help   :https://help.launchpad.net/ListHelp



_______________________________________________
Mailing list:https://launchpad.net/~dhis2-users
Post to     :dhis2-users@xxxxxxxxxxxxxxxxxxx
Unsubscribe :https://launchpad.net/~dhis2-users
More help   :https://help.launchpad.net/ListHelp


--
Lars Helge Øverland
Lead developer, DHIS 2
University of Oslo
Skype: larshelgeoverland
http://www.dhis2.org


_______________________________________________
Mailing list:https://launchpad.net/~dhis2-devs
Post to     :dhis2-devs@xxxxxxxxxxxxxxxxxxx
Unsubscribe :https://launchpad.net/~dhis2-devs
More help   :https://help.launchpad.net/ListHelp

_______________________________________________
Mailing list:https://launchpad.net/~dhis2-devs
Post to     :dhis2-devs@xxxxxxxxxxxxxxxxxxx
Unsubscribe :https://launchpad.net/~dhis2-devs
More help   :https://help.launchpad.net/ListHelp

--
*David Huser*
Informatics Specialist
Swiss Centre for International Health
Tel: +41 79 740 63 41
Skype: dafhus
david.huser@xxxxxxxxxxxx

*Swiss Tropical and Public Health Institute*
Socinstrasse 57, P.O. Box
4002 Basel, Switzerland
www.swisstph.ch

References