dhis2-devs team mailing list archive

[Calle Hedberg <calle.hedberg@xxxxxxxxx>]

Knut,

No, I didn't try another browser - we are trying to standardise on chrome
for DHIS2. Remember that this is not some kind of "crisis" issue where we
are stuck with something per se - I'm just trying to track down the root
cause of (several) seemingly cache-related issues we have encountered.

I don't know anything about "incognito mode", I'm afraid. So I obviously
did not try that...

Regards
Calle

On 2 September 2015 at 13:58, Knut Staring <knutst@xxxxxxxxx> wrote:

> Probably a stupid question, but did you try incognito mode or another
> browser?
>
> On Wed, Sep 2, 2015 at 12:44 PM, Calle Hedberg <calle.hedberg@xxxxxxxxx>
> wrote:
>
>> Halvdan,
>>
>> Noted, thanks for clarification - and yes, this issue is not related to
>> that (I'm still 98% sure it's somehow related to caching, I just don't know
>> how - my reason for pursuing this is obviously that since we are regularly
>> updating databases directly or via the web api, we want to be sure that all
>> cache refreshing is working and thus ensure that such updates are pushed to
>> users immediately.
>>
>> Regards
>> Calle
>>
>> On 2 September 2015 at 13:35, Halvdan Grelland <halvdanhg@xxxxxxxxx>
>> wrote:
>>
>>> Ok. I'm sorry the api query was lacking a parameter (should be
>>> api/dataSet.json&fields=name,shortName,displayName) to show all the fields.
>>> DisplayNames are generated (i.e. they're either the translation if one
>>> exists or they're simply the name). If you're not using translations that's
>>> obviously a dead end though.
>>>
>>> 2015-09-02 12:21 GMT+02:00 Calle Hedberg <calle.hedberg@xxxxxxxxx>:
>>>
>>>> Halvdan,
>>>>
>>>> We do not use translations for any South African instance - all English
>>>> (i.e. the translation table has 0 rows)
>>>>
>>>> We are using either 2.19 or 2.20 (mostly 2.19) - actually a slightly
>>>> modified version of 2.19 in most cases. That version contains additional
>>>> code related to DHIS 1.4->DHIS2 conversions etc, but none of that code is
>>>> in any way linked to this problem.
>>>>
>>>> <instance>/api/dataSets gives the result below, or in other words it
>>>> only show names - and all those names are the correct/updated ones.
>>>>
>>>> I'm not sure where "displayNames" would be coming from - I am not aware
>>>> that data sets HAVE a field called "displayName" or similar. In which table
>>>> can I find that?
>>>>
>>>> Regards
>>>>
>>>> On 2 September 2015 at 13:05, Halvdan Grelland <halvdanhg@xxxxxxxxx>
>>>> wrote:
>>>>
>>>>> Could you go to <instance>/api/dataSets and confirm that the names,
>>>>> shortNames and displayNames are correct? The names displayed should be the
>>>>> displayNames; which reminds me: do you have translations for the given
>>>>> dataSets/current locale? If you do they will be displayed, and they will
>>>>> not have been updated.
>>>>>
>>>>> Q:
>>>>> Is the problem true for all objects in the integrity checks, or just
>>>>> DataSets? What about DataElements?
>>>>>
>>>>> Q:
>>>>> Also, what version are you on?
>>>>>
>>>>> Try running ...
>>>>> select * from translation where objectclass like '%DataSet%';
>>>>> ...and inspect the results (might be a large query if you have a lot
>>>>> of dataSets).
>>>>>
>>>>> Also find the uid of one of the affected DataSets and try to run:
>>>>> select * from translation where objectuid like '<insert uid of object
>>>>> here>';
>>>>> .. any hints there?
>>>>>
>>>>> 2015-09-02 10:40 GMT+02:00 Calle Hedberg <calle.hedberg@xxxxxxxxx>:
>>>>>
>>>>>> Jason,
>>>>>>
>>>>>> 1. Yes
>>>>>> 2. Yes, I changed both name and shortname (identical) -  but I do not
>>>>>> know if the Data Integrity checks display name or short name (settings =
>>>>>> short name for that instance).
>>>>>> 3. We are not using any caching proxy
>>>>>>
>>>>>> We did restart Tomcat - which should clear the server cache (?) - and
>>>>>> I did clear all my local caches.
>>>>>>
>>>>>> BUT we had some other strange behaviour with regard to dashboard
>>>>>> privileges yesterday, and it was seemingly fixed by clearing the DHIS2
>>>>>> cache MULTIPLE times (not sure how many, but I only cleared the cache one
>>>>>> time...)
>>>>>>
>>>>>> Regards
>>>>>> Calle
>>>>>>
>>>>>> On 2 September 2015 at 10:10, Jason Pickering <
>>>>>> jason.p.pickering@xxxxxxxxx> wrote:
>>>>>>
>>>>>>> So, coming back to Lars's point.
>>>>>>>
>>>>>>> 1) Did you check the "dataset" table to confirm that your intended
>>>>>>> changes were persisted?
>>>>>>>
>>>>>>> 2) Did you alter the dataset "shortname" and is that the same which
>>>>>>> you are seeing in your integrity checks?
>>>>>>>
>>>>>>> 3) You indicated you cleared the DHIS 2 server cache. Can you
>>>>>>> confirm that you have no other upstream caching proxy which might be
>>>>>>> caching some response?
>>>>>>>
>>>>>>> The data integrity checks are a series of SQL (actually generated by
>>>>>>> Hibernate) queries, so as is usually the case, if you make direct database
>>>>>>> manipulations (and do not clear the server cache and possibly your local
>>>>>>> browser cache) you may see inconsistent results. For this type of
>>>>>>> operation, use of the UI is probably better, or alternative with the
>>>>>>> WebAPI.
>>>>>>>
>>>>>>> Regards,
>>>>>>> Jason
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Sep 1, 2015 at 11:57 PM, Calle Hedberg <
>>>>>>> calle.hedberg@xxxxxxxxx> wrote:
>>>>>>>
>>>>>>>> Halvdan,
>>>>>>>>
>>>>>>>> Ah, OK, and not - I was not using any queries, I simply updated the
>>>>>>>> Data Set names directly in the table.
>>>>>>>>
>>>>>>>> As I've said - the weird thing is that all those new names
>>>>>>>> immediately showed up in the UI data set list, but the old names are still
>>>>>>>> appearing wherever the Data Integrity process throws up integrity violations
>>>>>>>>
>>>>>>>> Regards
>>>>>>>> Calle
>>>>>>>>
>>>>>>>> On 1 September 2015 at 18:01, Halvdan Grelland <halvdanhg@xxxxxxxxx
>>>>>>>> > wrote:
>>>>>>>>
>>>>>>>>> Calle, I'm pretty sure Jason was referring to whatever you were
>>>>>>>>> doing to update the records (your point #1 in original email) which was
>>>>>>>>> not, as far as I can understand, using the UI or the web-api. I am not
>>>>>>>>> familiar with "pgremote" but am assuming it's some kind of postgres
>>>>>>>>> connector allowing you to run (sql) queries. We'd need to see those queries
>>>>>>>>> in order to figure out what could have gone wrong.
>>>>>>>>>
>>>>>>>>> 2015-09-01 14:50 GMT+02:00 Calle Hedberg <calle.hedberg@xxxxxxxxx>
>>>>>>>>> :
>>>>>>>>>
>>>>>>>>>> Jason
>>>>>>>>>>
>>>>>>>>>> I was not executing any custom sql - I'm running the Data
>>>>>>>>>> Administration -> Data Integrity process checks.
>>>>>>>>>>
>>>>>>>>>> My reference to the sql queries being run relates to my
>>>>>>>>>> ASSUMPTION that the Data Integrity checks are a series of SQL queries - but
>>>>>>>>>> maybe not, maybe the integrity checks are a series of web api calls
>>>>>>>>>> followed by some kind of front-end processing? I just don't know, and I'm
>>>>>>>>>> not a Java developer and I don't have the code base on my laptop so I
>>>>>>>>>> cannot find out myself...
>>>>>>>>>>
>>>>>>>>>> Seems my initial post was not clear enough on this - apologies
>>>>>>>>>> for that.
>>>>>>>>>>
>>>>>>>>>> Regards
>>>>>>>>>> Calle
>>>>>>>>>>
>>>>>>>>>> On 1 September 2015 at 14:31, Jason Pickering <
>>>>>>>>>> jason.p.pickering@xxxxxxxxx> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Calle,
>>>>>>>>>>> It is very difficult to say much without knowing what SQL it was
>>>>>>>>>>> you executed. In general, use of the API or UI is usually much safer.
>>>>>>>>>>>
>>>>>>>>>>> Do you have any sort of cache in front of your DHIS2 instance?
>>>>>>>>>>>
>>>>>>>>>>> Can you confirm from the database that the names were actually
>>>>>>>>>>> changed?
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>> Jason
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Sep 1, 2015 at 12:49 PM, Calle Hedberg <
>>>>>>>>>>> calle.hedberg@xxxxxxxxx> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Lars,
>>>>>>>>>>>>
>>>>>>>>>>>> Nope - all data set names and short names are the same.
>>>>>>>>>>>>
>>>>>>>>>>>> It's just weird, and I don't like inexplicable things happening
>>>>>>>>>>>> to a database....
>>>>>>>>>>>>
>>>>>>>>>>>> When you run data integrity - WHERE are those queries fetching
>>>>>>>>>>>> data from? Are the integrity queries running against the database itself,
>>>>>>>>>>>> or some kind of cache repository?
>>>>>>>>>>>>
>>>>>>>>>>>> Regards
>>>>>>>>>>>> Calle
>>>>>>>>>>>>
>>>>>>>>>>>> On 1 September 2015 at 13:00, Lars Helge Øverland <
>>>>>>>>>>>> larshelge@xxxxxxxxx> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Calle,
>>>>>>>>>>>>>
>>>>>>>>>>>>> perhaps you did not update the data set short names?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Lars
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Sep 1, 2015 at 11:50 AM, Calle Hedberg <
>>>>>>>>>>>>> calle.hedberg@xxxxxxxxx> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I've got a baffling problem (presumably related to some kind
>>>>>>>>>>>>>> of caching):
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 1. yesterday I updated various data records using pgremote,
>>>>>>>>>>>>>> including data set names. We restarted tomcat after the updates.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2. I've also cleared my browser cache, the DHIS2 cache,
>>>>>>>>>>>>>> updated resource tables, etc.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 3. When I now open the list of data sets, all the changed/new
>>>>>>>>>>>>>> names are showing as expected.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 4. BUT when I run Data Integrity, a few violations related to
>>>>>>>>>>>>>> data sets are coming up AND THE DATA SET NAMES ARE THE OLD ONES.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Any ideas what could cause this weird behaviour?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Regards
>>>>>>>>>>>>>> Calle
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> *******************************************
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Calle Hedberg
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Tel/fax (home): +27-21-685-6472
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Cell: +27-82-853-5352
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Iridium SatPhone: +8816-315-19274
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Email: calle.hedberg@xxxxxxxxx
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Skype: calle_hedberg
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> *******************************************
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>> Mailing list: https://launchpad.net/~dhis2-devs
>>>>>>>>>>>>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>>>>>>>>>>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>>>>>>>>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Lars Helge Øverland
>>>>>>>>>>>>> Lead developer, DHIS 2
>>>>>>>>>>>>> University of Oslo
>>>>>>>>>>>>> Skype: larshelgeoverland
>>>>>>>>>>>>> http://www.dhis2.org <https://www.dhis2.org>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>>
>>>>>>>>>>>> *******************************************
>>>>>>>>>>>>
>>>>>>>>>>>> Calle Hedberg
>>>>>>>>>>>>
>>>>>>>>>>>> 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
>>>>>>>>>>>>
>>>>>>>>>>>> Tel/fax (home): +27-21-685-6472
>>>>>>>>>>>>
>>>>>>>>>>>> Cell: +27-82-853-5352
>>>>>>>>>>>>
>>>>>>>>>>>> Iridium SatPhone: +8816-315-19274
>>>>>>>>>>>>
>>>>>>>>>>>> Email: calle.hedberg@xxxxxxxxx
>>>>>>>>>>>>
>>>>>>>>>>>> Skype: calle_hedberg
>>>>>>>>>>>>
>>>>>>>>>>>> *******************************************
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> Mailing list: https://launchpad.net/~dhis2-devs
>>>>>>>>>>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>>>>>>>>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>>>>>>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Jason P. Pickering
>>>>>>>>>>> email: jason.p.pickering@xxxxxxxxx
>>>>>>>>>>> tel:+46764147049
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>>
>>>>>>>>>> *******************************************
>>>>>>>>>>
>>>>>>>>>> Calle Hedberg
>>>>>>>>>>
>>>>>>>>>> 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
>>>>>>>>>>
>>>>>>>>>> Tel/fax (home): +27-21-685-6472
>>>>>>>>>>
>>>>>>>>>> Cell: +27-82-853-5352
>>>>>>>>>>
>>>>>>>>>> Iridium SatPhone: +8816-315-19274
>>>>>>>>>>
>>>>>>>>>> Email: calle.hedberg@xxxxxxxxx
>>>>>>>>>>
>>>>>>>>>> Skype: calle_hedberg
>>>>>>>>>>
>>>>>>>>>> *******************************************
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Mailing list: https://launchpad.net/~dhis2-devs
>>>>>>>>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>>>>>>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>>>>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>>
>>>>>>>> *******************************************
>>>>>>>>
>>>>>>>> Calle Hedberg
>>>>>>>>
>>>>>>>> 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
>>>>>>>>
>>>>>>>> Tel/fax (home): +27-21-685-6472
>>>>>>>>
>>>>>>>> Cell: +27-82-853-5352
>>>>>>>>
>>>>>>>> Iridium SatPhone: +8816-315-19274
>>>>>>>>
>>>>>>>> Email: calle.hedberg@xxxxxxxxx
>>>>>>>>
>>>>>>>> Skype: calle_hedberg
>>>>>>>>
>>>>>>>> *******************************************
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Jason P. Pickering
>>>>>>> email: jason.p.pickering@xxxxxxxxx
>>>>>>> tel:+46764147049
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> *******************************************
>>>>>>
>>>>>> Calle Hedberg
>>>>>>
>>>>>> 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
>>>>>>
>>>>>> Tel/fax (home): +27-21-685-6472
>>>>>>
>>>>>> Cell: +27-82-853-5352
>>>>>>
>>>>>> Iridium SatPhone: +8816-315-19119
>>>>>>
>>>>>> Email: calle.hedberg@xxxxxxxxx
>>>>>>
>>>>>> Skype: calle_hedberg
>>>>>>
>>>>>> *******************************************
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> *******************************************
>>>>
>>>> Calle Hedberg
>>>>
>>>> 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
>>>>
>>>> Tel/fax (home): +27-21-685-6472
>>>>
>>>> Cell: +27-82-853-5352
>>>>
>>>> Iridium SatPhone: +8816-315-19119
>>>>
>>>> Email: calle.hedberg@xxxxxxxxx
>>>>
>>>> Skype: calle_hedberg
>>>>
>>>> *******************************************
>>>>
>>>>
>>>
>>
>>
>> --
>>
>> *******************************************
>>
>> Calle Hedberg
>>
>> 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
>>
>> Tel/fax (home): +27-21-685-6472
>>
>> Cell: +27-82-853-5352
>>
>> Iridium SatPhone: +8816-315-19119
>>
>> Email: calle.hedberg@xxxxxxxxx
>>
>> Skype: calle_hedberg
>>
>> *******************************************
>>
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~dhis2-devs
>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~dhis2-devs
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>
>
> --
> Knut Staring
> Dept. of Informatics, University of Oslo
> Norway: +4791880522
> Skype: knutstar
> http://dhis2.org
>



-- 

*******************************************

Calle Hedberg

46D Alma Road, 7700 Rosebank, SOUTH AFRICA

Tel/fax (home): +27-21-685-6472

Cell: +27-82-853-5352

Iridium SatPhone: +8816-315-19119

Email: calle.hedberg@xxxxxxxxx

Skype: calle_hedberg

*******************************************