dhis2-devs team mailing list archive

[Ngoc Thanh Nguyen <thanh.hispvietnam@xxxxxxxxx>]

ok, Jason. I have some quick note but will detail it and send back to you

Thanh

*Nguyễn Ngọc Thành*



On Mon, Sep 23, 2013 at 10:42 PM, Jason Pickering <
jason.p.pickering@xxxxxxxxx> wrote:

> Hi Thanh,
> Lets put it back on the list with your findings.I wanted to be sure we had
> some consensus before proceeding too much with that line.  I think the
> configuration steps you took would be great to have in the documents. Why
> don't you summarize what you find out and i can help you write everything
> up into the docs as an alternative, which is perhaps better. The example I
> gave was complete overkill, but maybe you found a good balance between
> having a highly available cluster, versus a complete overkill (which I
> think was my problem).
>
> Regards,
> Jason
>
>
>
>
>
>
>
>
> On Mon, Sep 23, 2013 at 5:28 PM, Ngoc Thanh Nguyen <
> thanh.hispvietnam@xxxxxxxxx> wrote:
>
>> Jason, thanks for the discussion. I eventually can get dhis2 work with
>> Tomcat clustering. Persisting sessions in jdbc works like a charm with very
>> less configuration. Aggregation data entry could happen quite rare, every
>> month or every quarter. However, individual data capture can happen daily
>> in real time so being down for 1 hour could be catastrophe. My argument is
>> dhis2 can be even better if it is configured followed strict standards of
>> commercial systems such as High Availability, High Scalability. Instead of
>> having 1 server with 16G RAM, why don't we have 2 separate ones with 8GB
>> each?
>>
>> I don't know why our discussion goes off the list?
>>
>> Thanh
>>
>>
>> *Nguyễn Ngọc Thành*
>>
>>
>>
>> On Mon, Sep 23, 2013 at 10:37 AM, Jason Pickering <
>> jason.p.pickering@xxxxxxxxx> wrote:
>>
>>> Of course it depends on the use case, my point here was what is the use
>>> case?? I just did not want you trying to prove to some corporate people
>>> that just because we are using an OpenSource  product like DHIS2/Tomcat,
>>> that it is somehow less robust than some commercial equivalent, for the
>>> vast majority of use cases. But now it seems the requirement is different.
>>>
>>> I think for the case you mention, that has never been tested or
>>> confirmed I think and would need you to implement session sharing across
>>> multiple tomcat instances. That sounds a lot more difficult to do than a
>>> simple cluster. Even if it did work, it would require users  (usually or
>>> often) to clear their browser cache anyway. IMHO, Upgrades should something
>>> which should be done at night, with few users active. Normally, they should
>>> take less than two minutes, again depending on the size of your server.
>>> Upgrades to a production server should be tested in a testing environment
>>> and confirmed to work, as usually, they are things you cannot easily
>>> roll-.back. So, if you are upgrading a production instance every day or
>>> multiple times a day, it would seem to me you are asking for trouble.
>>> Again, just the way I feel about it, but trying to implement a clustering
>>> solution to solve this "requirement" sounds pretty risky to me.
>>>
>>> Would certainly be worth to investigate and see if you can get it to
>>> work, as it would certainly help increase redundancy of the deployments.
>>>
>>> Regards,
>>> Jason
>>>
>>>
>>>
>>>
>>>
>>> On Mon, Sep 23, 2013 at 1:48 AM, Ngoc Thanh Nguyen <
>>> thanh.hispvietnam@xxxxxxxxx> wrote:
>>>
>>>> It is, Jason. I think we will need a cluster in case of upgrading
>>>> dhis2. When you want to change a jar file, you have to shutdown the whole
>>>> application but if you have 2 servers you can do one by one and there is no
>>>> pause in the service.
>>>> I think it depends on the use case.
>>>>
>>>>
>>>> *Nguyễn Ngọc Thành*
>>>>
>>>>
>>>>
>>>> On Sun, Sep 22, 2013 at 9:47 PM, Jason Pickering <
>>>> jason.p.pickering@xxxxxxxxx> wrote:
>>>>
>>>>> Hi Thanh,
>>>>>
>>>>> Going to take this off the list for the moment, as it may not be so
>>>>> constructive.
>>>>>
>>>>> I would  take a step back and ask what the use case is. Why do you
>>>>> need a cluster? Do you know you need a cluster?
>>>>>
>>>>> I think doing it could be a fun way to spend a Sunday afternoon, but
>>>>> my wife would probably think it better to spend time with the kids. My
>>>>> point is that unless you know you need to do it, don't bother, because it
>>>>> has been done before, and it is nothing exotic. It may be good experience
>>>>> for you, so could be worth it. However, setting up a DB cluster, Tomcat
>>>>> cluster, foo cluster has been done by many others, and the process is well
>>>>> documented and proven, so I see no particular need to prove we can setup
>>>>> DHIS2 in this environment. We know that it can be done, because it has been
>>>>> done with others with very similar types of Java applications. I have setup
>>>>> a similar type of architecture with Amazon RDS in the background, fleets of
>>>>> application servers, fronted by  load balancers (Apache). Nothing really so
>>>>> fancy. But it turned out to be complete overkill, and we went back to a
>>>>> single server with 16 GB of memory, and it works just fine for out needs.
>>>>>
>>>>> Regards,
>>>>> Jason
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Sun, Sep 22, 2013 at 3:45 PM, Ngoc Thanh Nguyen <
>>>>> thanh.hispvietnam@xxxxxxxxx> wrote:
>>>>>
>>>>>> Thanks Jason. The reply is insightful.
>>>>>>
>>>>>> I now try to set up dhis2 on a Tomcat cluster with nginx and mysql
>>>>>> cluster altogether. Will let you know the result soon
>>>>>>
>>>>>> *Nguyễn Ngọc Thành*
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Sun, Sep 22, 2013 at 12:26 PM, Knut Staring <knutst@xxxxxxxxx>wrote:
>>>>>>
>>>>>>> I think we should get DHIS2  added to that list of systems
>>>>>>> (typically) running on Tomcat. It is a MoinMoin wiki, so I thought I could
>>>>>>> just add us, but it seems this is an "immutable" page..
>>>>>>>
>>>>>>>
>>>>>>> On Sun, Sep 22, 2013 at 7:19 AM, Jason Pickering <
>>>>>>> jason.p.pickering@xxxxxxxxx> wrote:
>>>>>>>
>>>>>>>> Hi Thanh,
>>>>>>>>
>>>>>>>> First off, be sure you are aware of who you are dealing with in
>>>>>>>> such a debate. These "commercial companies" are probably a lot better at
>>>>>>>> you in selling products to people who do not need them than you are. This
>>>>>>>> is after all, what they do. So, keeping that in mind, lets pick apart what
>>>>>>>> they are saying.
>>>>>>>>
>>>>>>>>  "DHIS2 is not built on and use well-know reliable solutions such
>>>>>>>> as Microsoft Sharepoint, IBM Websphere,  Oracle Bea Weblogic"
>>>>>>>>
>>>>>>>> No reason why you could not used DHIS2 with Websphere or Weblogic
>>>>>>>> in theory. Of course, it would be a huge waste of money to plonk down money
>>>>>>>> for these applications, when Apache Tomcat (which is free) will do the job
>>>>>>>> just fine. DHIS2 is using Spring and not EJB, so there is simply no need
>>>>>>>> for these products. As for Sharepoint, well, it is a totally different
>>>>>>>> application stack, and not really interesting to debate about. Point is
>>>>>>>> DHIS2 is built using Java and lots of standard open-source libraries used
>>>>>>>> by lots of other projects (including commercial ones). So which application
>>>>>>>> server it uses is really a mater of choice. You could take Tomcat, and it
>>>>>>>> would work fine. You could also pay a company a lot of money for
>>>>>>>> a licence for one of the products you mention.  Here is a partial list of
>>>>>>>> companies/projects using Tomcat, and the number is not small (
>>>>>>>> http://wiki.apache.org/tomcat/PoweredBy)
>>>>>>>>
>>>>>>>> >1. When multiple DHIS2 instances can be deployed in multiple
>>>>>>>> servers (virtual or physical) and load balanced by a hard or soft balancer
>>>>>>>> (nginxs), this can not solve the real >problem of distributed HTTP sessions
>>>>>>>> like what can be done in Application Server, i.e. a logged in users
>>>>>>>> connected to a instance and this instance die, how can they continue
>>>>>>>> >without relogin.
>>>>>>>> >Can using Application Server solve the problems without having to
>>>>>>>> rewrite dhis2?
>>>>>>>>
>>>>>>>>
>>>>>>>> Which Application Server? I guess my question here would be what is
>>>>>>>> the impact of a clustered Tomcat instance crashing? OK, maybe the user has
>>>>>>>> to login again. Maybe they loose a little bit of work at worst. So what?
>>>>>>>> Not that this is not an impact, but would it justify even an attempt to
>>>>>>>> implement a load-balanced clustered solution with some sort of failover
>>>>>>>> protection transparent to the user? It would not really seem to be
>>>>>>>> justified.
>>>>>>>>
>>>>>>>> >How are we going to solve this? Will we need to use commercial
>>>>>>>> solutions like Terracotta BigMemory or IBM Data Grid?
>>>>>>>>
>>>>>>>> What is the point? This again sounds like a bit of a snake-oil
>>>>>>>> argument here. These products are not going to help with the issue of an
>>>>>>>> application server failing. .
>>>>>>>>
>>>>>>>> >3. I would like to know the typical deployment of dhis2 in other
>>>>>>>> countries. Is that on one or multiple server, with or without database
>>>>>>>> clusters? I heard that many countries go for >cloud but I am not sure if it
>>>>>>>> is a single or multiple cloud servers?
>>>>>>>>
>>>>>>>> Most countries which I know of are using a rather modest server.
>>>>>>>> Many are using the cloud, some are not. Many countries have everything on
>>>>>>>> the same machine (database, application server, nginx) and separating the
>>>>>>>> database and application server is fairly typical as well.  But I could
>>>>>>>> imagine there are some installations out there which are using clusters,
>>>>>>>> but I would say this is probably not typical (yet) of most DHIS2
>>>>>>>> installations.
>>>>>>>>
>>>>>>>>
>>>>>>>> >Can cloud provide us a single virtual but unlimited RAM and CPUs
>>>>>>>> server? As if yes, then we don't have to worry about question 1, 2 because
>>>>>>>> we can simple add 10000000 >GB RAM and 100000 CPUs to that cloud server.
>>>>>>>>
>>>>>>>> Well, for all intents and purposes, you could continue to increase
>>>>>>>> the size of the server up to a certain size as required but I think the
>>>>>>>> metrics you give there are pretty ridiculous.Why would you need that much
>>>>>>>> RAM or CPU power?  Even if it were physically possible which seems
>>>>>>>> reasonable (although likely very very expensive) , it would generally not
>>>>>>>> make much sense to do this. Probably better to break the system up into
>>>>>>>> smaller pieces instead. Using many small servers to distribute the process
>>>>>>>> is typically the favored approach, instead of having one big massive
>>>>>>>> machine.
>>>>>>>>
>>>>>>>> Anyway, there are a few thoughts off the top of my head. In
>>>>>>>> summary, there is simply no need for the commercial products you mention in
>>>>>>>> the vast majority of use cases we have come across. Not to say that they do
>>>>>>>> not exist, but I would press these "companies" for more details on how
>>>>>>>> these are actual requirements, and not simply invented hypothetical cases
>>>>>>>> which could potentially be used to make a sale of unneeded licences.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Jason
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Sun, Sep 22, 2013 at 3:14 AM, Ngoc Thanh Nguyen <
>>>>>>>> thanh.hispvietnam@xxxxxxxxx> wrote:
>>>>>>>>
>>>>>>>>> Hi all,
>>>>>>>>>
>>>>>>>>> Some commercial affiliated companies want to kick dhis2 out so
>>>>>>>>> they made the following arguments: "DHIS2 is not built on and use well-know
>>>>>>>>> reliable solutions such as Microsoft Sharepoint, IBM Websphere,  Oracle Bea
>>>>>>>>> Weblogic"
>>>>>>>>>
>>>>>>>>> When most of this is wrong they have some valid points that we
>>>>>>>>> need to be prepared in order to fight back. I would like to get your ideas
>>>>>>>>> on
>>>>>>>>>
>>>>>>>>> 1. When multiple DHIS2 instances can be deployed in multiple
>>>>>>>>> servers (virtual or physical) and load balanced by a hard or soft balancer
>>>>>>>>> (nginxs), this can not solve the real problem of distributed HTTP sessions
>>>>>>>>> like what can be done in Application Server, i.e. a logged in users
>>>>>>>>> connected to a instance and this instance die, how can they continue
>>>>>>>>> without relogin.
>>>>>>>>>
>>>>>>>>> Can using Application Server solve the problems without having to
>>>>>>>>> rewrite dhis2?
>>>>>>>>>
>>>>>>>>> How are we going to solve this? Will we need to use commercial
>>>>>>>>> solutions like Terracotta BigMemory or IBM Data Grid?
>>>>>>>>>
>>>>>>>>> 2. For database scale, with one database instances installed on
>>>>>>>>> single server. it has the risk of data loss. We can use database clusters
>>>>>>>>> to address this issue, and data on clusters mostly rely on memory. My
>>>>>>>>> question is how this cluster will work with distributed sessions described
>>>>>>>>> in question 1.
>>>>>>>>>
>>>>>>>>> 3. I would like to know the typical deployment of dhis2 in other
>>>>>>>>> countries. Is that on one or multiple server, with or without database
>>>>>>>>> clusters? I heard that many countries go for cloud but I am not sure if it
>>>>>>>>> is a single or multiple cloud servers?
>>>>>>>>>
>>>>>>>>> Can cloud provide us a single virtual but unlimited RAM and CPUs
>>>>>>>>> server? As if yes, then we don't have to worry about question 1, 2 because
>>>>>>>>> we can simple add 10000000 GB RAM and 100000 CPUs to that cloud server.
>>>>>>>>>
>>>>>>>>> Thanks in advanced.
>>>>>>>>>
>>>>>>>>> Thanh
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Mailing list: https://launchpad.net/~dhis2-devs
>>>>>>>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>>>>>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>>>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Mailing list: https://launchpad.net/~dhis2-devs
>>>>>>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>>>>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Knut Staring
>>>>>>> Dept. of Informatics, University of Oslo
>>>>>>> +4791880522
>>>>>>>  http://dhis2.org
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>