dhis2-devs team mailing list archive

[Lars Helge Øverland <larshelge@xxxxxxxxx>]

Hi Caveman,

thanks for the input. The memory recommendations are based on the size of
the indexes which are created for analytics tables - ideally the database
indexes on the analytics tables for the last 2 years should fit in memory
for optimal performance. But this depends on the number of dynamic
dimensions you have set up for your system - hence they should be
considered a rule-of-thumb as I said. If you want to do testing
specifically for your system then you can measure the size of the db
indexes with SQL:

*

select pg_size_pretty(pg_total_relation_size('analytics_2012'));

minus this value

select pg_size_pretty(pg_relation_size('analytics_2012'));

Lars

*




On Thu, May 2, 2013 at 2:01 PM, Bob Jolliffe <bobjolliffe@xxxxxxxxx> wrote:

> Hi Orvalho
>
> On 1 May 2013 15:07, Orvalho Augusto <orvaquim@xxxxxxxxx> wrote:
>
>> It is true that hardware prices have decreased a lot.
>>
>> But for me it seems too excessive these recommendations. So I am
>> wondering how we arrived to these recommendations. Would be nice if we
>> share that.
>>
>> And for professional solutions it is not only one big machine you buy.
>> You must have two. So when one dies you have the other ready.
>>
>> Some other things to be tried are:
>> Instead of using one big muscular server you might need to consider to
>> use one machine for webapplication and other database.
>>
>
> I agree that having a separate machine for the database can be a good
> thing.  As long as they are connected with a good Gbs backbone.  This also
> allows you to optimize the two machines differently.  I know in Rwanda they
> have got away with fairly humble hardware so far by doing this.  Though
> moving to 2.11 could still prove challenging.
>
> Bob
>
>
>
>>
>> And the webapplication server actually can be a group of machines doing
>> it; And for databases you can have just one machine for writing queries
>> which is master to other servers that are only queried for reading only
>> (this for MySQL have been done) or other high availability solution.
>>
>> For the tools to study your server, let me add some more:
>> - top - The basic to see the processes and CPU usage
>>
>> - vmstat - This is verry important to see how your server is using the
>> memory. Repeat it and check our the blocks are being used.
>>
>> - plotting tools based on snmp may help to profile your server; To see
>> correlations between traffic/connections and CPU or memory. So you can
>> consider what to change.
>>
>> Hope it helps.
>>
>> Caveman
>>
>>
>>
>>
>>
>> On Wed, May 1, 2013 at 12:21 PM, Dayo Adeyomoye <deemoyes@xxxxxxxxx>wrote:
>>
>>> Yes you are right, hardware prices are coming down plus a very good
>>> server can run VMWare to properly allocate resources
>>>
>>> BlackBerry: 3114F90C
>>> Phone: 08028851441, 08035560463
>>> emails: deemoyes@xxxxxxxxx, addyraph@xxxxxxxxx
>>> Yahoo IM: deemoyes
>>> Facebook: pharael
>>> twitter:pharaell
>>>   ------------------------------
>>>  *From:* Lars Helge Øverland <larshelge@xxxxxxxxx>
>>> *To:* DHIS 2 developers <dhis2-devs@xxxxxxxxxxxxxxxxxxx>; "
>>> dhis2-users@xxxxxxxxxxxxxxxxxxx" <dhis2-users@xxxxxxxxxxxxxxxxxxx>
>>> *Sent:* Wednesday, May 1, 2013 11:16 AM
>>> *Subject:* [Dhis2-users] Server specifications
>>>
>>> Hi all,
>>>
>>> in version 2.11 and later DHIS performs aggregation on-the-fly as
>>> opposed to pre-aggregating data into data marts.
>>>
>>> This raises the bar a little on your server's specifications, mostly in
>>> terms of available RAM and disk read speeds.
>>>
>>>
>>> Some rules-of-thumb server recommendations:
>>>
>>>
>>> - RAM: At least 1 GB memory per 1 million captured data records per
>>> month or per 1000 concurrent users. At least 4 GB for a small instance, 12
>>> GB for a medium instance.
>>>
>>> - CPU cores: 4 CPU cores for a small instance, 8 CPU cores for a medium
>>> or large instance.
>>>
>>> - Disk: Ideally use an SSD. Otherwise use a 7200 rpm disk. Minimum read
>>> speed is 150 Mb/s, 220 Mb/s is good, 350 Mb/s or better is ideal.
>>>
>>>
>>> So we encourage you to invest in appropriate hardware to continue to
>>> benefit from all the new features in DHIS. Hardware prices are coming down
>>> and value for money is increasing at vps' like linode<http://linode.com/>
>>>  and digitalocean <http://digitalocean.com/>.
>>>
>>>
>>> regards,
>>>
>>> Lars
>>>
>>> -----
>>>
>>> PS. For server admins, here are some tips to measure hard disk/drive
>>> read speeds on Linux:
>>>
>>> - Use the hdparm utility which will give you approximate read speed:
>>>
>>> sudo hdparm -Tt /dev/sda
>>>
>>>
>>> - If you are on a virtual server image without /dev/sda you can generate
>>> a random file and test write and read speed like this:
>>>
>>> dd if=/dev/zero of=testfile bs=4k count=2000000
>>>
>>> dd if=testfile of=/dev/null bs=4k count=2000000
>>>
>>>
>>> - To do a real-life test, you can install iotop (sudo apt-get install
>>> iotop) and run it with
>>>
>>> sudo iotop
>>>
>>> then start DHIS in a new terminal and load a very large pivot table,
>>> then monitor read speed from iotop. You should be seeing read speeds around
>>> 12 - 25 Mb/s. Be aware that database queries will be cached after some time.
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~dhis2-users
>>> Post to    : dhis2-users@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~dhis2-users
>>> More help  : https://help.launchpad.net/ListHelp
>>>
>>>
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~dhis2-devs
>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>>
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~dhis2-devs
>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~dhis2-devs
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~dhis2-devs
> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~dhis2-devs
> More help   : https://help.launchpad.net/ListHelp
>
>