dhis2-devs team mailing list archive

[Chameera Mirihella <chameera9019@xxxxxxxxx>]

Hi Json,

Thanks for the information. We are doing the same thing as you suggested.
after db migration to postgres, we go version to version and treat the
errors come on the way. there were issues in boolean type and null values
as you said and fixed.

one thing i wanted to ask from you.

we are upgrading one instance from 2.13 to 2.23. first we tried with meta
data and few datavalues.
while the up-gradation, till 2.18 analytics works fine, analytic tables are
generated in the db and data is populated, pivot tables can be generated
with data. but from 2.19, analytics tables are generated but no data in the
tables, there is no errors in the db side and the tomcat log.

Are there any major differences in analytics module after 2.18 ?


On Sat, Jul 30, 2016 at 11:52 AM Chameera Mirihella <chameera9019@xxxxxxxxx>
wrote:

>
> Hi Halvdan, yes it is a good collection, we use one tool suggested by it
> to do the initial migration. but that is not enough, have to do some
> modifications in the schema manually to bring it back to the dhis schema.
>
> Hi Knut, I agree with your suggestion but as you said it is one-time job.
> In our case we have two instances in 2.7, one in 2.12, another in 2.13, and
> all these instances use mysql. we are planning to migrate and upgrade all
> these instances to 2.23. Which is quite a big challenge, some databases
> are more than 100GB. have explored some ways and made some progress. just
> wanted to hear any suggestions from you guys.
>
> Hi Gerald, yes earlier we have upgrade few instances but that was not 100%
> successful. So this time we started with more care and more focus on the
> approach, issues arisen in between the process. Will let you know when we
> got success.
>
> Thanks,
> Chameera.
>
> On Fri, Jul 29, 2016 at 6:13 PM Jason Pickering <
> jason.p.pickering@xxxxxxxxx> wrote:
>
>> Hi Chameera,
>>
>> I did not really use any scripts for this process when I upgraded a very
>> old version (although it was not 2.6) . Like I said, it went pretty well. I
>> monitored both the Tomcat log as well as the Postgresql database log during
>> startup, and resolved the  problems as they arose.
>>
>>
>>
>> In general, I would just try and go version to version and get DHIS2 to
>> start successfully. I am sure you will encounter some issues, but the
>> startup scripts do a pretty good job of converting the database changes
>> from one version to the other. When that does not work, then you may have
>> to craft some custom SQL or potentially delete some things. At a certain
>> point, duplicate organisation unit groups were allowed, but in one version,
>> this was "feature" was removed and it caused a startup error. In other
>> cases the startup scripts do not properly remove or add required columns. I
>> have also see many problems with boolean nulls, which should be set to
>> FALSE, otherwise, you will get errors.
>>
>> You will also need to consider that all users passwords need to be
>> upgraded past 2.20. We stopped there for a bit and let people reset their
>> passwords, at least those that really need access. The rest may need to be
>> recovered or reset.
>>
>> I suggest you give it a try and then ask specific questions as they arise
>> if you encounter problems.
>>
>> Regards,
>> Jason
>>
>>
>>
>> On Fri, Jul 29, 2016 at 2:00 PM, Knut Staring <knutst@xxxxxxxxx> wrote:
>>
>>> Switching databases is another use case for trying export and import.
>>> But the emphasis is on trying, not sure how much exists in terms of ready
>>> scripts that take care of everything. Jason does a lot of R scripting.
>>> Using Node and the web api is yet another approach. Ideally we should build
>>> up a library of scrpts both for upgrading, loading from existing sources
>>> and integrating. There is doubtlessly a whole lot out there, but most of it
>>> probably tailored to one-off jobs.
>>>
>>> Sorry to not offer something more concrete..
>>>
>>> On Jul 29, 2016 6:11 PM, "Halvdan Hoem Grelland" <halvdan@xxxxxxxxx>
>>> wrote:
>>>
>>> There is a fairly in-depth collection of resources on migrating to
>>> PostgreSQL on the official PostgreSQL wiki
>>> <https://wiki.postgresql.org/wiki/Converting_from_other_Databases_to_PostgreSQL#MySQL>
>>>
>>> On Fri, Jul 29, 2016 at 12:03 PM, Chameera Mirihella <
>>> chameera9019@xxxxxxxxx> wrote:
>>>
>>>> Hi Jason,
>>>>
>>>> HISP India is also in the process of upgrading dhis from 2.7 to 2.23.
>>>> are there any upgrade scripts available for earlier versions (2.7 - 2.16) ?
>>>>
>>>> and also the biggest challenge for us right now is that 2.7 instance
>>>> run with mysql db, we need to convert it to postgres somewhere in this
>>>> process. any suggestion is appreciated.
>>>>
>>>> Thanks.
>>>>
>>>> On Wed, Jul 27, 2016 at 6:37 PM Jason Pickering <
>>>> jason.p.pickering@xxxxxxxxx> wrote:
>>>>
>>>>> Hi Gerald,
>>>>>
>>>>> I upgraded a system recently from 2.10 to 2.22, and it went pretty
>>>>> well actually. There are some things you need to look out for along the
>>>>> way, particular changes in the model, like duplication in orgunit groups,
>>>>> unique (or non-unique codes) etc. I found it useful to turn on the
>>>>> Postgresql logging during the process as certain errors may only be clearly
>>>>> visible in the logs of the database, and not always through DHIS2.
>>>>>
>>>>> Most versions seem to be availble from
>>>>>
>>>>> https://www.dhis2.org/download/releases/2.XX/dhis.war
>>>>>
>>>>> where XX is the version which you need. Otherwise, you may need to
>>>>> build it from source code.
>>>>>
>>>>> I do agree with Knut, that attempting an export /import to something
>>>>> like 2.12 might be a good start,but this has lots of limitations, so a
>>>>> step-by-step upgrade would be the best way.
>>>>>
>>>>> Regards and good luck,
>>>>> Jason
>>>>>
>>>>>
>>>>> On Wed, Jul 27, 2016 at 2:54 PM, gerald thomas <gerald17006@xxxxxxxxx>
>>>>> wrote:
>>>>>
>>>>>> Thanks Knut, they are currently doing data entry till August 15. I
>>>>>> will start on the 16 or 17 and i will keep you informed.
>>>>>>
>>>>>> On Jul 27, 2016 12:49 PM, "Knut Staring" <knutst@xxxxxxxxx> wrote:
>>>>>>
>>>>>>> I suggest you try with 2.16 and let us know how it goes. If
>>>>>>> problematic, then try a few versions earlier.
>>>>>>> Knut
>>>>>>>
>>>>>>> On 27 Jul 2016 19:12, "gerald thomas" <gerald17006@xxxxxxxxx> wrote:
>>>>>>>
>>>>>>>> Dear Knut,
>>>>>>>> Noted!!!
>>>>>>>> I noticed that upgrade procedures' starts from version 2.16 and the
>>>>>>>> oldest war file available is 2.7; which means I will have to work blindly
>>>>>>>> till 2.16 by the grace of God.
>>>>>>>> HISP India, please can you help because I heard you had done
>>>>>>>> something like this before.
>>>>>>>>
>>>>>>>> On Jul 27, 2016 12:31 PM, "Knut Staring" <knutst@xxxxxxxxx> wrote:
>>>>>>>>
>>>>>>>>> Gi Gerald,
>>>>>>>>>
>>>>>>>>> Please do not take my suggestion as a fully formed and fool-proof
>>>>>>>>> procedure. But since you are exporting and then importing into an empty
>>>>>>>>> instance of a newer version, you will not be messing up your existing data
>>>>>>>>> (but should still take a backup first).
>>>>>>>>>
>>>>>>>>> Piecewise going through every version with the necessary database
>>>>>>>>> updates is the safest route and to always be recommended.
>>>>>>>>>
>>>>>>>>> However, this might be difficult in this case, and certainly very
>>>>>>>>> cumbersome. So I  suggest you give it a try. 2.16 was just a rough number,
>>>>>>>>> I am not sure it will work,  but this method can help you skip a few
>>>>>>>>> versions. However, you will have to test the results thoroughly, and
>>>>>>>>> probably will run into a few obstacles. So it is not a silver bullet,  I'm
>>>>>>>>> afraid.
>>>>>>>>>
>>>>>>>>> Knut
>>>>>>>>>
>>>>>>>>> On 27 Jul 2016 14:04, "gerald thomas" <gerald17006@xxxxxxxxx>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Knut,
>>>>>>>>>> That is a very big jump and if it works, it will make my day.
>>>>>>>>>> As per your procedure, I should deploy the 2.16 war file on the
>>>>>>>>>> newly installed DHIS2 with empty database "hmis" and do metadata export of
>>>>>>>>>> everything from the old system which is running on 2.5 war file and finally
>>>>>>>>>> import to the new instance?
>>>>>>>>>> My question again Knut, what is the best method to do this export
>>>>>>>>>> & import: piecewise or everything; if it is piecewise what should I import
>>>>>>>>>> first, second,...... and last?
>>>>>>>>>>
>>>>>>>>>> Thanks for your help
>>>>>>>>>>
>>>>>>>>>> On Jul 26, 2016 15:40, "Knut Staring" <knutst@xxxxxxxxx> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Gerald,
>>>>>>>>>>> I would try to skip a few versions by exporting metadata and
>>>>>>>>>>> data and probably try to import to an intermediate version such as 2.13 or
>>>>>>>>>>> 2.16.
>>>>>>>>>>>
>>>>>>>>>>> On 26 Jul 2016 21:02, "gerald thomas" <gerald17006@xxxxxxxxx>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Dear All,
>>>>>>>>>>>> I am inquiring for both technical guidance and materials to
>>>>>>>>>>>> upgrade a
>>>>>>>>>>>> DHIS2 instance from version 2.5 to 2.22.
>>>>>>>>>>>>
>>>>>>>>>>>> I will be grateful for any help anyone can render for me to
>>>>>>>>>>>> successfully achieve this task. Also, i will like to know where
>>>>>>>>>>>> i can
>>>>>>>>>>>> get all the old war files.
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Regards,
>>>>>>>>>>>>
>>>>>>>>>>>> Gerald
>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> Mailing list: https://launchpad.net/~dhis2-users
>>>>>>>>>>>> Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
>>>>>>>>>>>> Unsubscribe : https://launchpad.net/~dhis2-users
>>>>>>>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>> _______________________________________________
>>>>>> Mailing list: https://launchpad.net/~dhis2-users
>>>>>> Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
>>>>>> Unsubscribe : https://launchpad.net/~dhis2-users
>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Jason P. Pickering
>>>>> email: jason.p.pickering@xxxxxxxxx
>>>>> tel:+46764147049
>>>>> _______________________________________________
>>>>> Mailing list: https://launchpad.net/~dhis2-users
>>>>> Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
>>>>> Unsubscribe : https://launchpad.net/~dhis2-users
>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>
>>>>
>>>> _______________________________________________
>>>> Mailing list: https://launchpad.net/~dhis2-users
>>>> Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
>>>> Unsubscribe : https://launchpad.net/~dhis2-users
>>>> More help   : https://help.launchpad.net/ListHelp
>>>>
>>>>
>>>
>>>
>>> --
>>> Halvdan Hoem Grelland
>>> Software developer, DHIS 2
>>> University of Oslo
>>> http://www.dhis2.org <https://www.dhis2.org/>
>>>
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~dhis2-users
>>> Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~dhis2-users
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>>
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~dhis2-devs
>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>>
>> More help   : https://help.launchpad.net/ListHelp
>>>
>>>
>>
>>
>> --
>> Jason P. Pickering
>> email: jason.p.pickering@xxxxxxxxx
>> tel:+46764147049
>> _______________________________________________
>> Mailing list: https://launchpad.net/~dhis2-users
>> Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~dhis2-users
>> More help   : https://help.launchpad.net/ListHelp
>>
>