dhis2-devs team mailing list archive

[Stian Sandvold <stian@xxxxxxxxx>]

The convention today is that resource tables are prefixed with an
underscore, while analytics tables start with analytics*. As previously
mentioned in this thread, changing this now will probably cause a lot of
issues for existing scripts, so I think for now we should just avoid using
tables starting with analytics*.

On Thu, Jul 12, 2018 at 3:26 PM, Bob Jolliffe <bobjolliffe@xxxxxxxxx> wrote:

> Agree that might have made sense at the start.  But I think the
> priority now should be to rename that one table so that all the backup
> scripts in the wild get unbroken.  Not to break them even more :-)
>
> On 12 July 2018 at 13:02, Knut Staring <knutst@xxxxxxxxx> wrote:
> > Would be great if the convention could be that ALL generated tables
> > (including analytics) started with an underscore. This convention is
> sort of
> > already in place, just not applied consistently.
> >
> > Knut
> >
> > On Thu, Jul 12, 2018 at 6:05 PM Calle Hedberg <calle.hedberg@xxxxxxxxx>
> > wrote:
> >>
> >> Hi
> >>
> >> I'm glad it's being addressed - but I am less happy to hear you are
> aware
> >> of it but haven't said anything to the community...  These type of bugs
> >> causes havoc, and waste a lot of time for users affected while they try
> to
> >> figure out what's gone wrong.
> >>
> >> There are two major lessons here:
> >>
> >> Firstly, to stick to a standard and manageable naming convention for
> both
> >> tables and fields (and interfaces).
> >>
> >> Secondly, to inform the user community promptly when you become aware of
> >> major, "showstopper", bugs.
> >>
> >> Best regards
> >> calle
> >>
> >> On 12 July 2018 at 13:15, Morten Olav Hansen <morten@xxxxxxxxx> wrote:
> >>>
> >>> Ok, thanks Stian (no need to work on this then Viet)
> >>>
> >>> (knut, using pg_dump we normally filter away analytic* which means no
> >>> period boundaries..)
> >>>
> >>> --
> >>> Morten Olav Hansen
> >>> Senior Engineer, DHIS 2
> >>> University of Oslo
> >>> http://www.dhis2.org
> >>>
> >>> On Thu, Jul 12, 2018 at 6:14 PM, Stian Sandvold <stian@xxxxxxxxx>
> wrote:
> >>>>
> >>>> I have started the work on renaming the table. I will update this
> thread
> >>>> as soon as I make progress.
> >>>>
> >>>> On Thu, Jul 12, 2018 at 12:17 PM, Morten Olav Hansen <
> morten@xxxxxxxxx>
> >>>> wrote:
> >>>>>
> >>>>> Hi Knut
> >>>>>
> >>>>> I mean the main issue the thread was about... using maintenance =>
> >>>>> clear analytic tables, it will delete analytics* which includes the
> >>>>> analytical boundaries.
> >>>>>
> >>>>>
> >>>>> --
> >>>>> Morten Olav Hansen
> >>>>> Senior Engineer, DHIS 2
> >>>>> University of Oslo
> >>>>> http://www.dhis2.org
> >>>>>
> >>>>> On Thu, Jul 12, 2018 at 5:11 PM, Knut Staring <knutst@xxxxxxxxx>
> wrote:
> >>>>>>
> >>>>>> Hi Morten, sorry but which functionality are you suggesting should
> not
> >>>>>> be used? What do you mean by manually deleting?
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Knut
> >>>>>>
> >>>>>> On Thu, Jul 12, 2018, 4:23 PM Morten Olav Hansen <morten@xxxxxxxxx>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>> Hi Calle
> >>>>>>>
> >>>>>>> We are aware of this issue (actually it caused a problem with us in
> >>>>>>> Lao), for now.. I would also say don't use this functionality, its
> better to
> >>>>>>> manually delete the analytics_* tables if you need it. We will
> rename that
> >>>>>>> table soon we hope, and that should fix it (this also causes
> potential
> >>>>>>> issues with your backups...)
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> Morten Olav Hansen
> >>>>>>> Senior Engineer, DHIS 2
> >>>>>>> University of Oslo
> >>>>>>> http://www.dhis2.org
> >>>>>>>
> >>>>>>> On Thu, Jul 12, 2018 at 3:58 PM, Calle Hedberg
> >>>>>>> <calle.hedberg@xxxxxxxxx> wrote:
> >>>>>>>>
> >>>>>>>> Bob,
> >>>>>>>>
> >>>>>>>> No response/action on the JIRA bug report yet - I guess most
> >>>>>>>> developers are on leave (wonderful summer here in Norway this
> year).
> >>>>>>>>
> >>>>>>>> Otherwise I agree, the name of that table does not fit the general
> >>>>>>>> naming convention as far as I can see. It would make more sense
> to call it
> >>>>>>>> e.g. "programindicator_periodboundary". The name then provides
> an intuitive
> >>>>>>>> description of the content and it sorts together with the group of
> >>>>>>>> programindicator tables.
> >>>>>>>>
> >>>>>>>> Regards
> >>>>>>>> calle
> >>>>>>>>
> >>>>>>>> On 12 July 2018 at 08:57, Bob Jolliffe <bobjolliffe@xxxxxxxxx>
> >>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>> Thats nasty alright.  I guess using "-T anlytics_*" instead would
> >>>>>>>>> help.  But there are so many backup scripts out there broken by
> >>>>>>>>> this
> >>>>>>>>> that it will be better to rename the table.
> >>>>>>>>>
> >>>>>>>>> On 11 July 2018 at 22:39, Calle Hedberg <calle.hedberg@xxxxxxxxx
> >
> >>>>>>>>> wrote:
> >>>>>>>>> > Hi
> >>>>>>>>> >
> >>>>>>>>> > For as long as I can remember, we have used the standard
> >>>>>>>>> > parameter "-T
> >>>>>>>>> > analytics*" when dumping a DHIS2 database into e.g. a backup or
> >>>>>>>>> > similar.
> >>>>>>>>> >
> >>>>>>>>> > The purpose of the parameter was to exclude all analytics
> tables
> >>>>>>>>> > from the
> >>>>>>>>> > dump, since it is significantly faster to restore a dump
> without
> >>>>>>>>> > analytics
> >>>>>>>>> > tables and then run analytics to re-create them (due to the use
> >>>>>>>>> > of
> >>>>>>>>> > multi-threading), compared to dumping and restoring a database
> >>>>>>>>> > instance with
> >>>>>>>>> > all the analytics table (restore is NOT using multi-threading).
> >>>>>>>>> >
> >>>>>>>>> > For some reason, in 2.29 a new table that stores periodboundary
> >>>>>>>>> > data for
> >>>>>>>>> > Program Indicators was called "analyticsperiodboundary" - which
> >>>>>>>>> > means the
> >>>>>>>>> > standard pg_dump parameter will leave that table behind
> together
> >>>>>>>>> > with all
> >>>>>>>>> > other "analytics*" tables.
> >>>>>>>>> >
> >>>>>>>>> > Furthermore, the routine called "Clear Analytics Tables" found
> >>>>>>>>> > under Data
> >>>>>>>>> > Administration -> Maintenance is as before deleting all tables
> >>>>>>>>> > named
> >>>>>>>>> > Analytics* - THE PROBLEM IS THAT IT ALSO DELETES THE NEW
> >>>>>>>>> > ANALYTICSPERIODBOUNDARY TABLE (same in both 2.29 and 2.30)
> >>>>>>>>> >
> >>>>>>>>> > Which will crash your system in the sense that you won't see
> any
> >>>>>>>>> > program
> >>>>>>>>> > indicator data in dashboards etc.
> >>>>>>>>> >
> >>>>>>>>> > The "analyticsperiodboundary" table will be re-created and
> >>>>>>>>> > re-populated with
> >>>>>>>>> > DEFAULT (boundless) Program Indicator Period boundaries when
> you
> >>>>>>>>> > re-start
> >>>>>>>>> > the system (it's part of the TableAlteror routine during
> >>>>>>>>> > startup), but
> >>>>>>>>> > - you have to re-start the system
> >>>>>>>>> > - you will lose any non-default boundary settings used for any
> >>>>>>>>> > program
> >>>>>>>>> > indicator.
> >>>>>>>>> >
> >>>>>>>>> > This has also been reported as a high-priority bug on JIRA
> >>>>>>>>> > (DHIS2-4260).
> >>>>>>>>> >
> >>>>>>>>> > Regards
> >>>>>>>>> > Calle
> >>>>>>>>> >
> >>>>>>>>> > *******************************************
> >>>>>>>>> >
> >>>>>>>>> > Calle Hedberg
> >>>>>>>>> >
> >>>>>>>>> > 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
> >>>>>>>>> >
> >>>>>>>>> > Tel/fax (home): +27-21-685-6472
> >>>>>>>>> >
> >>>>>>>>> > Cell: +27-82-853-5352
> >>>>>>>>> >
> >>>>>>>>> > Iridium SatPhone: +8816-315-19119
> >>>>>>>>> >
> >>>>>>>>> > Email: calle.hedberg@xxxxxxxxx
> >>>>>>>>> >
> >>>>>>>>> > Skype: calle_hedberg
> >>>>>>>>> >
> >>>>>>>>> > *******************************************
> >>>>>>>>> >
> >>>>>>>>> >
> >>>>>>>>> >
> >>>>>>>>> > _______________________________________________
> >>>>>>>>> > Mailing list: https://launchpad.net/~dhis2-devs
> >>>>>>>>> > Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> >>>>>>>>> > Unsubscribe : https://launchpad.net/~dhis2-devs
> >>>>>>>>> > More help   : https://help.launchpad.net/ListHelp
> >>>>>>>>> >
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>>
> >>>>>>>> *******************************************
> >>>>>>>>
> >>>>>>>> Calle Hedberg
> >>>>>>>>
> >>>>>>>> 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
> >>>>>>>>
> >>>>>>>> Tel/fax (home): +27-21-685-6472
> >>>>>>>>
> >>>>>>>> Cell: +27-82-853-5352
> >>>>>>>>
> >>>>>>>> Iridium SatPhone: +8816-315-19119
> >>>>>>>>
> >>>>>>>> Email: calle.hedberg@xxxxxxxxx
> >>>>>>>>
> >>>>>>>> Skype: calle_hedberg
> >>>>>>>>
> >>>>>>>> *******************************************
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> _______________________________________________
> >>>>>>>> Mailing list: https://launchpad.net/~dhis2-devs
> >>>>>>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> >>>>>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
> >>>>>>>> More help   : https://help.launchpad.net/ListHelp
> >>>>>>>>
> >>>>>>>
> >>>>>>> _______________________________________________
> >>>>>>> Mailing list: https://launchpad.net/~dhis2-devs
> >>>>>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> >>>>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
> >>>>>>> More help   : https://help.launchpad.net/ListHelp
> >>>>>
> >>>>>
> >>>>>
> >>>>> _______________________________________________
> >>>>> Mailing list: https://launchpad.net/~dhis2-devs
> >>>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> >>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
> >>>>> More help   : https://help.launchpad.net/ListHelp
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Stian Sandvold
> >>>> Software developer, DHIS2
> >>>> University of Oslo
> >>>> http://www.dhis2.org
> >>>
> >>>
> >>>
> >>> _______________________________________________
> >>> Mailing list: https://launchpad.net/~dhis2-devs
> >>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> >>> Unsubscribe : https://launchpad.net/~dhis2-devs
> >>> More help   : https://help.launchpad.net/ListHelp
> >>>
> >>
> >>
> >>
> >> --
> >>
> >> *******************************************
> >>
> >> Calle Hedberg
> >>
> >> 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
> >>
> >> Tel/fax (home): +27-21-685-6472
> >>
> >> Cell: +27-82-853-5352
> >>
> >> Iridium SatPhone: +8816-315-19119
> >>
> >> Email: calle.hedberg@xxxxxxxxx
> >>
> >> Skype: calle_hedberg
> >>
> >> *******************************************
> >>
> >>
> >> _______________________________________________
> >> Mailing list: https://launchpad.net/~dhis2-devs
> >> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> >> Unsubscribe : https://launchpad.net/~dhis2-devs
> >> More help   : https://help.launchpad.net/ListHelp
> >
> >
> >
> > --
> > Knut Staring
> >
> > Department of Information, Evidence and Research
> > World Health Organization, Geneva, Switzerland
> > Office: +41 22 791 3683 Mob1: +33 6 4434 2931 Mob2: +47 9188 0522
> > Skype:     knutstar
> >
> > _______________________________________________
> > Mailing list: https://launchpad.net/~dhis2-devs
> > Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> > Unsubscribe : https://launchpad.net/~dhis2-devs
> > More help   : https://help.launchpad.net/ListHelp
> >
>
> _______________________________________________
> Mailing list: https://launchpad.net/~dhis2-devs
> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~dhis2-devs
> More help   : https://help.launchpad.net/ListHelp
>



-- 
Stian Sandvold
Software developer, DHIS2
University of Oslo
http://www.dhis2.org