dhis2-devs team mailing list archive

[Uwe Wahser <uwe@xxxxxxxxx>]

Hi Lars,

thanks for taking this over. If there was an api for clearing the analytics
table, which is giving back a result (unlike the one for updating the analytics
tables), that would already be a sufficient solution for nightly runs. Then I
could always delete the analytics tables before generating them afresh. Whatever
is easier for you to implement ...

You have interesting working hours :-)

Regards, Uwe


> Lars Helge Øverland <lars@xxxxxxxxx> hat am 19. Februar 2016 um 04:38
> geschrieben:
> 
> 
> Hi Uwe,
> 
> will work on this.
> 
> https://bugs.launchpad.net/dhis2/+bug/1408096
> 
> Short-term workaround is to go to data admin > maintenance and click Clear
> analytics tables.
> 
> regards,
> 
> Lars
> 
> 
> On Thu, Feb 18, 2016 at 3:32 AM, Olav Poppe <olav.poppe@xxxxxx> wrote:
> 
> > Hi, I can only support/confirm what Uwe is saying here - I just experieced
> > the same thing this morning: deleted data for one year, ran analytics, and
> > the deleted data is still displayed.
> >
> > Olav
> >
> >
> >
> > 18. feb. 2016 kl. 07.38 skrev Uwe Wahser <uwe@xxxxxxxxx>:
> >
> > Dear devs,
> >
> > it seems like the analytics table update does not remove data/tables of
> > years,
> > that are no longer valid.
> >
> > Since dataValues can be deleted via api in batch, it can happen that
> > dataValues
> > for a whole year get deleted. Of course these years should not show up in
> > the
> > analytics (pivot, chart etc...) any more. However, those obsolete tables
> > seem to
> > remain in the database after running the analytics update. Of course they
> > could
> > be remove manually, but this is not applicable in a nightly load scenario.
> >
> > I did empty the cache and checked catalina.out, but I can't find an error.
> > In
> > the database the dataValues table remains with only the years, that were
> > not
> > deleted, so the deletion obviously worked. But the obsolete analytics
> > tables
> > (e.g. analytics_2012) are still there and populated. You can verify in the
> > attached excerpt from catalina.out: only years >= 2016 were (correctly)
> > updated,
> > deleted years <= 2015 remained untouched.
> >
> > This is quite serious as it results into wrong statistics, if not
> > detected. I
> > just discovered this because of a major clean-up activity - if there are
> > only
> > very few years affected, users might not even notice that they are
> > reporting
> > deleted data.
> >
> > Currently running on 2.22, build 21765, Standard Ubuntu 14.04 & OracleJava
> > 8.
> >
> > Thanks for checking on that,
> >
> > Uwe<catalina.out.2016-02-18>
> > _______________________________________________
> > Mailing list: https://launchpad.net/~dhis2-devs
> > Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> > Unsubscribe : https://launchpad.net/~dhis2-devs
> > More help   : https://help.launchpad.net/ListHelp
> >
> >
> >
> > _______________________________________________
> > Mailing list: https://launchpad.net/~dhis2-devs
> > Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> > Unsubscribe : https://launchpad.net/~dhis2-devs
> > More help   : https://help.launchpad.net/ListHelp
> >
> >
> 
> 
> -- 
> Lars Helge Øverland
> Lead developer, DHIS 2
> University of Oslo
> Skype: larshelgeoverland
> http://www.dhis2.org <https://www.dhis2.org/>