dhis2-devs team mailing list archive

[Steffen Tengesdal <steffen@xxxxxxxxxxxxx>]

Hi Gerald,

As Bob pointed out, filezilla is a GUI tool and it does not support scheduling of downloads.  Your local server should not have a GUI on it if it is a production system.  If your local host a Linux system? If so, you can create a simple bash script on the localhost system that uses sftp or scp command line to connect and download a backup.  A script for that would not be very complicated.  

Steffen

> On Dec 18, 2014, at 7:47 AM, gerald thomas <gerald17006@xxxxxxxxx> wrote:
> 
> Bob,
> My Suggestion:
> All local servers must be on 2.15 war file then we create a SFTP
> account on cloud server then we can use filezilla from the local
> server to download the backup from the cloud server.
> I know it is crude but that help for now.
> What is your take Bob.
> 
> On 12/18/14, Bob Jolliffe <bobjolliffe@xxxxxxxxx <mailto:bobjolliffe@xxxxxxxxx>> wrote:
>> Hi Gerald
>> 
>> We tested this when I was in Sierra Leone and we were finding serious
>> problems with bandwidth getting the data back to Sierra Leone.
>> 
>> So you are going to have to think carefully about when and how often to
>> synch.  Currently your database files are very small as you don't have much
>> data on your cloud server, but it will soon grow.  I suspect "at least
>> twice a day" sounds unrealistic.
>> 
>> The way I typically do it is to first create an account on the backup
>> server.  Make sure that the account running your dhis instance can login to
>> the backup server without a password by creating an ssh key pair and
>> installing the public key on the backup server account.  Then you can
>> simply the rsync the backups directory (eg /var/lib/dhis2/dhis/backups) to
>> a directory on the backup server using cron.   In fact if you look in
>> /usr/bin/dhis2-backup you will see that the commands are already there to
>> do this, just commented out.  This would synch with the backup server after
>> taking the nightly backup.
>> 
>> This simple (and slightly lazy) setup has worked fine, and continues to
>> work, in a number of places.  But there are a number of reasons you might
>> want to do something different.
>> 
>> (i)  you might want to pull from the backup server rather than push to it.
>> Particularly as the backup server might not be as reliably always online as
>> the production server.  This would require a slightly different variation
>> on the above, but using the same principle of creating an ssh keypair and
>> letting rsync do the work.
>> 
>> (ii) rsync is a really great and simple tool, but it is sadly quite slow.
>> If you are bandwidth stressed and your database is growing it might not be
>> the best solution.  Works fine when bandwidth is not a critical issue.  The
>> trouble is it doesn't really take into account the incremental nature of
>> the data ie. you backup everything every time (besides the ephemeral tables
>> like analytics, aggregated etc).  In which case you need to start thinking
>> smarter and maybe a little bit more complicated.  One approach I have been
>> considering, (but not yet tried) is to make a copy of the metadata export
>> every night and then just pull all the datavalues with a lastupdated
>> greater than the last time you pulled.  That is going to reduce the size of
>> the backup quite considerably.  In theory this is probably even possible to
>> do through the api rather than directly through psql which might be fine if
>> you choose the time of day/night carefully.  I'd probably do it with psql
>> at the backed,
>> 
>> So there are a few options.  The first being the simplest and also the
>> crudest.  Any other thoughts?
>> 
>> Cheers
>> Bob
>> 
>> On 18 December 2014 at 05:07, gerald thomas <gerald17006@xxxxxxxxx> wrote:
>>> 
>>> Dear All,
>>> Sierra Leone wants to finally migrate to an online server (External
>>> server hosted outside the Ministry) but we will like to create a daily
>>> backup of that server locally in case anything goes wrong.
>>> My questions:
>>> 
>>> 1.  We need a help with a script that can create a sync between the
>>> External Server and the Local Server (at least twice a day)
>>> 
>>> 2. Is there something we should know from past experiences about
>>> hosting servers on the cloud
>>> 
>>> Please feel free to share anything and I will be grateful to learn new
>>> things about dhis2
>>> 
>>> --
>>> Regards,
>>> 
>>> Gerald
>>> 
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~dhis2-devs
>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>> More help   : https://help.launchpad.net/ListHelp
>>> 
>> 
> 
> 
> -- 
> Regards,
> 
> Gerald
> 
> _______________________________________________
> Mailing list: https://launchpad.net/~dhis2-devs <https://launchpad.net/~dhis2-devs>
> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx <mailto:dhis2-devs@xxxxxxxxxxxxxxxxxxx>
> Unsubscribe : https://launchpad.net/~dhis2-devs <https://launchpad.net/~dhis2-devs>
> More help   : https://help.launchpad.net/ListHelp <https://help.launchpad.net/ListHelp>