← Back to team overview

dhis2-devs team mailing list archive

Re: Inconsistency in generating analytics

 

Thanks - this was on a Windows (training) server


On Wed, May 28, 2014 at 6:14 PM, Jason Pickering <
jason.p.pickering@xxxxxxxxx> wrote:

> Hi Knut,
>
> Well, in this case, if the analytics result was already stored in the
> browser and cached until 6 AM, then the browser would not request a new
> copy from the server, even if you had changed it ex post facto (from the
> first request). What you did not detail in your first mail, was clearing
> the browser cache, which would explain why the result were the same if you
> had not done this. But if you did clear your browser cache, refresh
> analytics, and got the same result (when you expected something different),
> then indeed it seems strange.
>
> Of course, "DELETE FROM DATAVALUES" will have no effect ,but "DELETE FROM
> datavalues" should , and you would expect of course, nothing to be returned
> in any analytics, and that could definitely explain why no analytics tables
> were generated and the error, as there would be nothing to aggregate.
>
> Regards,
> Jason
>
>
>
> On Wed, May 28, 2014 at 6:07 PM, Lars Helge Øverland <larshelge@xxxxxxxxx>wrote:
>
>> Okay. The data admin cache has no effect on analytics. If there is no
>> server side cache then I am not sure why the data is not updated.
>>
>>
>> On Wed, May 28, 2014 at 6:04 PM, Knut Staring <knutst@xxxxxxxxx> wrote:
>>
>>> It is certainly easy for me to get lost in the number of things referred
>>> to as "cache" - but in this case there is no Nginx, just plain Tomcat, i.e.
>>> nothing "in front of it"
>>>
>>> I just didn't understand why the analytics tables did not get
>>> deleted/updated until I manually had to get rid of them. But probably this
>>> was caused by the setting which Jason referred to?
>>>
>>> I assumed I could do the following:
>>> 1) Run "DELETE FROM DATAVALUES"
>>> 2) Clear the "Cache" that is referred to under dhis-web-maintenance-
>>> dataadmin/showCache.action
>>> 3) Clear my own browser cache
>>> 4) Run Analytics
>>>
>>> But it seems my algorithm missed the step of switching to "No cache" ?
>>>
>>>
>>> On Wed, May 28, 2014 at 5:10 PM, Lars Helge Øverland <
>>> larshelge@xxxxxxxxx> wrote:
>>>
>>>> Hi,
>>>>
>>>> I think its quite logical that if you set up a server cache and ask the
>>>> system to cache until next morning, it will return the same result even if
>>>> the upstream server content changes.
>>>>
>>>> The cache under data admin is referring to the hibernate dhis
>>>> application cache, not the nginx server cache. There is no way that dhis
>>>> running in tomcat could clear all caches in front of it.
>>>>
>>>> The reason for  "Table not valid, aborted update" probably happened
>>>> for the event analytics, and just means that there is no event data so we
>>>> cannot generate event analytics tables.
>>>>
>>>> regards,
>>>>
>>>> Lars
>>>>
>>>>
>>>>
>>>> On Wed, May 28, 2014 at 3:30 PM, Knut Staring <knutst@xxxxxxxxx> wrote:
>>>>
>>>>> The thing I found confusing was that I got the same results also in
>>>>> the browser on the server, which I did not think had been used to look at
>>>>> pivot tables at all before. I admit the setting was probably wrong, and the
>>>>> situation was finally resolved after clearing both browser and server side
>>>>> caches BUT also manually deleting the analytics tables. So not exactly sure
>>>>> what the issue was exactly.
>>>>>
>>>>> But I still find it very confusing that the Analytics generation
>>>>> process first provides something that looks like (but is not particularly
>>>>> marked out, e.g. in BOLD or RED or something) as an error message, AND THEN
>>>>> proceeds to indicate that everything is honkey-dorey....
>>>>>
>>>>>
>>>>>
>>>>> On Wed, May 28, 2014 at 3:25 PM, Jason Pickering <
>>>>> jason.p.pickering@xxxxxxxxx> wrote:
>>>>>
>>>>>> Well, I think that is the isssue. If you regenerate the analytics,
>>>>>> and the system cache is set to 6 AM, nothing is going to happen, unless you
>>>>>> clear your browser cache, as the results for the same analytics resource,
>>>>>> have been cached there, right?
>>>>>>
>>>>>>
>>>>>> On Wed, May 28, 2014 at 3:24 PM, Knut Staring <knutst@xxxxxxxxx>wrote:
>>>>>>
>>>>>>> It is set to "Cache until 6 am next morning", which I think is the
>>>>>>> default.
>>>>>>>
>>>>>>> But when I said I cleared the cache, I meant on the server, as
>>>>>>> in dhis-web-maintenance-dataadmin/showCache.action
>>>>>>>
>>>>>>>
>>>>>>> On Wed, May 28, 2014 at 2:45 PM, Jason Pickering <
>>>>>>> jason.p.pickering@xxxxxxxxx> wrote:
>>>>>>>
>>>>>>>> I mean the system cache settings..
>>>>>>>>
>>>>>>>> dhis-web-maintenance-settings/systemGeneralSettings.action
>>>>>>>>
>>>>>>>> What is this set to?
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Jason
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, May 28, 2014 at 2:37 PM, Knut Staring <knutst@xxxxxxxxx>wrote:
>>>>>>>>
>>>>>>>>> Yes I had no event data. But the analytics tables remained the
>>>>>>>>> same after deleting all data and then clearing cache. Had to delete them
>>>>>>>>> directly in the db (which admittedly is also how I got rid of all the
>>>>>>>>> testdata in the first place)
>>>>>>>>>
>>>>>>>>> On May 28, 2014 6:59 PM, "Jason Pickering" <
>>>>>>>>> jason.p.pickering@xxxxxxxxx> wrote:
>>>>>>>>> >
>>>>>>>>> > Check the cache settings. That message may not be fatal, and may
>>>>>>>>> be related to systems which have no event data.
>>>>>>>>> >
>>>>>>>>> > Regards,
>>>>>>>>> > Jason
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> > On Wed, May 28, 2014 at 2:21 PM, Knut Staring <knutst@xxxxxxxxx>
>>>>>>>>> wrote:
>>>>>>>>> >>
>>>>>>>>> >> When running the Analytics (2.15) everything looked fine at the
>>>>>>>>> first glance, since the top message had the green tick icon and said
>>>>>>>>> "Analytics tables updated"
>>>>>>>>> >>
>>>>>>>>> >> However, everything remained the same, so I looked closer and
>>>>>>>>> saw that the penultimate message (line 2) indicated the process had failed:
>>>>>>>>> >>
>>>>>>>>> >> 2014-05-28 09:03:37 Analytics tables updated
>>>>>>>>> >> 2014-05-28 09:03:37 Table not valid, aborted update
>>>>>>>>> >>
>>>>>>>>> >> --
>>>>>>>>> >> Knut Staring
>>>>>>>>> >> Dept. of Informatics, University of Oslo
>>>>>>>>> >> +4791880522
>>>>>>>>> >> http://dhis2.org
>>>>>>>>> >>
>>>>>>>>> >> _______________________________________________
>>>>>>>>> >> Mailing list: https://launchpad.net/~dhis2-devs
>>>>>>>>> >> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>>>>>>>> >> Unsubscribe : https://launchpad.net/~dhis2-devs
>>>>>>>>> >> More help   : https://help.launchpad.net/ListHelp
>>>>>>>>> >>
>>>>>>>>> >
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Knut Staring
>>>>>>> Dept. of Informatics, University of Oslo
>>>>>>> +4791880522
>>>>>>> http://dhis2.org
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Knut Staring
>>>>> Dept. of Informatics, University of Oslo
>>>>> +4791880522
>>>>> http://dhis2.org
>>>>>
>>>>> _______________________________________________
>>>>> Mailing list: https://launchpad.net/~dhis2-devs
>>>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> Knut Staring
>>> Dept. of Informatics, University of Oslo
>>> +4791880522
>>> http://dhis2.org
>>>
>>
>>
>


-- 
Knut Staring
Dept. of Informatics, University of Oslo
+4791880522
http://dhis2.org

References