dhis2-users team mailing list archive

[Jason Phillips <jason@xxxxxxxx>]

Valid points, Bob, from the sounds of it.

I appreciate the brief catchup – very much.



I shall urgently investigate!



Kind Regards,

*Jason Phillips*

[image: hisp]
*Information Systems / Infrastructure*

*Health Information Systems Program____________________________________*

eMail:               jason@xxxxxxxx
Tel/Fax:            +27 21 712 0170
Cell:                 +27 72 973 7250
Skype:             jason.n.phillips

This message and any attachments are subject to a disclaimer published at
http://www.hisp.org/policies.html#comms_disclaimer.  Please read the
disclaimer before opening any attachment or taking any other action in
terms of this electronic transmission.  If you cannot access the
disclaimer, kindly send an email to disclaimer@xxxxxxxx and a copy will be
provided to you. By replying to this e-mail or opening any attachment you
agree to be bound by the provisions of the disclaimer.







*From:* Bob Jolliffe [mailto:bobjolliffe@xxxxxxxxx]
*Sent:* Wednesday, 16 March 2016 4:43 PM
*To:* Jason Phillips
*Cc:* Lars Helge Øverland; DHIS 2 Users list
*Subject:* Re: [Dhis2-users] Load-balancing DHIS2 Webservers



Jason,



Put simply, when users interact with the application they deal with
information from the underlying database.  But the dhis2 java application
doesn't keep going backwards and forwards to the database for everything
all the time.  To be efficient it also maintains a cached copy of what is
in the database.  So at any given moment in time what is seen by the
application may not be exactly what is in the database.  This is not a
problem when there is only one dhis2 instance as differences get resolved,
cache is flushed to database, refreshed from database etc.



But with more than one dhis2 instance feeding off the same database they
can (and inevitably) will develop inconsistent views.  Which might have
some disastrous result.  So something else needs to be configured to ensure
these multiple cache's remain consistent.



One way to do that is to use distributed caching as described here (
http://www.ehcache.org/documentation/2.8/get-started/cache-topologies.html#distributed-caching)
- though that does seem to require an additional non-open source
component.  There might be other approaches as well.



I don't really have experience of doing this, just pointing out that I
think you need a strategy to address it in order to do the sort of load
scaling you talk about.



Bob







On 16 March 2016 at 15:13, Jason Phillips <jason@xxxxxxxx> wrote:

Hibernate cache...?

I’m sorry, I don’t speak Norwegian.  The what, now?



I’ll Google when I get a chance, but for now, have no idea what you’re
talking about…  J

It does sound important, though…



Kind Regards,

*Jason Phillips*

[image: hisp]
*Information Systems / Infrastructure*

*Health Information Systems Program____________________________________*

eMail:               jason@xxxxxxxx
Tel/Fax:            +27 21 712 0170
Cell:                 +27 72 973 7250
Skype:             jason.n.phillips

This message and any attachments are subject to a disclaimer published at
http://www.hisp.org/policies.html#comms_disclaimer.  Please read the
disclaimer before opening any attachment or taking any other action in
terms of this electronic transmission.  If you cannot access the
disclaimer, kindly send an email to disclaimer@xxxxxxxx and a copy will be
provided to you. By replying to this e-mail or opening any attachment you
agree to be bound by the provisions of the disclaimer.







*From:* Bob Jolliffe [mailto:bobjolliffe@xxxxxxxxx]
*Sent:* Wednesday, 16 March 2016 2:24 PM
*To:* Lars Helge Øverland
*Cc:* Jason Phillips; DHIS 2 Users list
*Subject:* Re: [Dhis2-users] Load-balancing DHIS2 Webservers



I have not done this, but I imagine that hibernate cache replication
between instances is something that really needs to be done to have this
sort of scaling configuration work.



I do remember there was a brief flirtation with hazelcast before reverting
to ehcache.  Is anybody using ehcache in this way ie. for replication in a
clustering setup?



On 16 March 2016 at 12:22, Lars Helge Øverland <lars@xxxxxxxxx> wrote:

Hi Jason,



there is the concept of "sticky sessions" which should ensure that a user
is always sent to the same server throughout his session:



https://www.nginx.com/products/session-persistence/

http://nginx.org/en/docs/http/load_balancing.html



There are other issues with load balancing (session persistence in case a
server goes down, hibernate cache replication) but this could help.



regards,



Lars













On Tue, Mar 15, 2016 at 8:20 PM, Jason Phillips <jason@xxxxxxxx> wrote:

Aha!



A little more reading goes a long way…  ip_hash persistence appears to work
– I can log in, anyway.

For those of you interested, here’s the URL:
http://nginx.org/en/docs/http/ngx_http_upstream_module.html#ip_hash



If I have anything more relevant to share, I will…



Kind Regards,

*Jason Phillips*

[image: hisp]
*Information Systems / Infrastructure*

*Health Information Systems Program____________________________________*

eMail:               jason@xxxxxxxx
Landline:          +27 21 712 0170
Mobile:             +27 72 973 7250
Skype:             jason.n.phillips

This message and any attachments are subject to a disclaimer published at
http://www.hisp.org/policies.html#comms_disclaimer.  Please read the
disclaimer before opening any attachment or taking any other action in
terms of this electronic transmission.  If you cannot access the
disclaimer, kindly send an email to disclaimer@xxxxxxxx and a copy will be
provided to you. By replying to this e-mail or opening any attachment you
agree to be bound by the provisions of the disclaimer.







*From:* Jason Phillips [mailto:jason@xxxxxxxx]
*Sent:* Tuesday, 15 March 2016 8:34 PM
*To:* DHIS 2 Users list
*Subject:* Load-balancing DHIS2 Webservers



Hi, community (and Oslo team),



I am trying to set up a load-balanced DHIS2 web-server system; we have a
web server that’s getting slaughtered with incoming data, and we want to
distribute the load to other physical servers.  I’ve done some
experimentation, a fair amount of web-trawling, and using a dedicated nginx
virtual server, have (semi)successfully set up a load-balancing model like
so:





My problem is that going through the load-balancer, I appear to need some
kind of setting that will “keep” me at the Web-Server I first started with
– the load-balancer is set to round-robin, currently, with the following
settings:



upstream dhis2 {

        server webserver1.dhis.hisp.org;

        server webserver2.dhis.hisp.org;

        }



server {

        listen 80;

        server_name www.hisp.org hisp.org;



        location / {

                include /etc/nginx/proxy_params;

                proxy_pass http://dhis2;

        }

        location /staging {

                include /etc/nginx/proxy_params;

                proxy_pass http://dhis2;

        }

        location /training {

                include /etc/nginx/proxy_params;

                proxy_pass http://dhis2;

        }

}



What’s happening is that I attempt to log in, and get immediately switched
“round-robin” style to the second server for the post (promptly getting the
“forgot password?” prompt).
Nett result: I can’t log in…

If I log in to either one of the webservers directly, everything is fine
and dandy, but going through the LB is a no-go, so far.  (I have noticed
some interesting behaviour from two web servers connected to the same Db,
and would be keen to hear what input anyone has on that score – bad idea?
If so, why?  And what can be done to address the issues? – but I shall save
that for a separate post…)



Has anyone done any research/experimentation/development with a
load-balancing model of DHIS2 in a web-server context?  I would be most
interested to hear input.



Kind Regards,

*Jason Phillips*

[image: hisp]
*Information Systems / Infrastructure*

*Health Information Systems Program____________________________________*

eMail:               jason@xxxxxxxx
Landline:          +27 21 712 0170
Mobile:             +27 72 973 7250
Skype:             jason.n.phillips

This message and any attachments are subject to a disclaimer published at
http://www.hisp.org/policies.html#comms_disclaimer.  Please read the
disclaimer before opening any attachment or taking any other action in
terms of this electronic transmission.  If you cannot access the
disclaimer, kindly send an email to disclaimer@xxxxxxxx and a copy will be
provided to you. By replying to this e-mail or opening any attachment you
agree to be bound by the provisions of the disclaimer.





*This message and any attachments are subject to a disclaimer published at
http://www.hisp.org/policies.html#comms_disclaimer
<http://www.hisp.org/policies.html#comms_disclaimer>.  Please read the
disclaimer before opening any attachment or taking any other action in
terms of this electronic transmission.  If you cannot access the
disclaimer, kindly send an email to disclaimer@xxxxxxxx
<disclaimer@xxxxxxxx> and a copy will be provided to you. By replying to
this e-mail or opening any attachment you agree to be bound by the
provisions of the disclaimer.*



_______________________________________________
Mailing list: https://launchpad.net/~dhis2-users
Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
Unsubscribe : https://launchpad.net/~dhis2-users
More help   : https://help.launchpad.net/ListHelp





-- 

Lars Helge Øverland

Lead developer, DHIS 2

University of Oslo

Skype: larshelgeoverland

http://www.dhis2.org <https://www.dhis2.org/>




_______________________________________________
Mailing list: https://launchpad.net/~dhis2-users
Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
Unsubscribe : https://launchpad.net/~dhis2-users
More help   : https://help.launchpad.net/ListHelp





*This message and any attachments are subject to a disclaimer published at
http://www.hisp.org/policies.html#comms_disclaimer
<http://www.hisp.org/policies.html#comms_disclaimer>.  Please read the
disclaimer before opening any attachment or taking any other action in
terms of this electronic transmission.  If you cannot access the
disclaimer, kindly send an email to disclaimer@xxxxxxxx
<disclaimer@xxxxxxxx> and a copy will be provided to you. By replying to
this e-mail or opening any attachment you agree to be bound by the
provisions of the disclaimer.*

-- 


*This message and any attachments are subject to a disclaimer published at 
http://www.hisp.org/policies.html#comms_disclaimer 
<http://www.hisp.org/policies.html#comms_disclaimer>.  Please read the 
disclaimer before opening any attachment or taking any other action in 
terms of this electronic transmission.  If you cannot access the 
disclaimer, kindly send an email to disclaimer@xxxxxxxx 
<disclaimer@xxxxxxxx> and a copy will be provided to you. By replying to 
this e-mail or opening any attachment you agree to be bound by the 
provisions of the disclaimer.*