dhis2-users team mailing list archive

[Chris van Hasselt <CVanHasselt@xxxxxxxxxx>]

This is more or less the conclusion we came to – recommend that if you want that much data, do an API query from Power BI (which we are having a good deal of success with).

Thanks
Chris

From: Jason Pickering [mailto:jason.p.pickering@xxxxxxxxx]
Sent: Friday, June 23, 2017 7:43 AM
To: Chris van Hasselt <CVanHasselt@xxxxxxxxxx>
Cc: dhis2-users@xxxxxxxxxxxxxxxxxxx; Noah Evens <NEvens@xxxxxxxxxx>; Amina Abba Gana (SIDHAS - Abuja) <aabbagana@xxxxxxxxxxxxx>
Subject: Re: [Dhis2-users] Problem rendering a pivot table

Hi Chris,

Yes, this sounds pretty much as we have seen in other cases when trying to render large pivots in the app. If the downloaded CSV file is 212 MB, that would certainly qualify as a "huge pivot" I think.

The dev team is working on a new implementation of the pivots, which may deal with this issue, but there are always going to be limitations in rendering very large data tables inside a browser window. I think if you are dealing with a CSV file that large, then you may need tools with higher horsepower (R, SAS, SPSS, etc), as opposed to the web browser.

I think the only immediate way forward would be for you to use other tools, or better yet, break the table down into component pieces if at all possible. Smaller more focused tables assembled on a dashboard might allow you to see all of the data in chunks as opposed to one very large table.

Regards,
Jason


On Thu, Jun 22, 2017 at 4:41 PM, Chris van Hasselt <CVanHasselt@xxxxxxxxxx<mailto:CVanHasselt@xxxxxxxxxx>> wrote:
Thanks Jason – Yes, it is the “Aw, snap!” message in Chrome.
There are no browser console messages when this happens.

As far as the size of the pivot table, the “all elements” option for the data has many, many data elements.  There are six data element groups, and when I break it down group by group, there is only one group that fails, with 73 data elements.  When this happens, I’m selecting data at the  “District” level, and there are many districts.

Yesterday, I started breaking down the 73 elements into smaller groups, selecting 10 at a time.  When I did this, the table rendered, but with no data, in all cases.  But no “Aw Snap” message.

The downloaded CSV file, however, has data.  The file is 212 MB, and takes an exceedingly long time to process, even though I don’t think of his as a huge file.

From: Jason Pickering [mailto:jason.p.pickering@xxxxxxxxx<mailto:jason.p.pickering@xxxxxxxxx>]
Sent: Tuesday, June 20, 2017 2:50 PM
To: Chris van Hasselt <CVanHasselt@xxxxxxxxxx<mailto:CVanHasselt@xxxxxxxxxx>>
Cc: dhis2-users@xxxxxxxxxxxxxxxxxxx<mailto:dhis2-users@xxxxxxxxxxxxxxxxxxx>
Subject: Re: [Dhis2-users] Problem rendering a pivot table

Hi Chris,

This normally is a result of loading a pivot with "too much" data. What too much is is not entirely clear and can depend on a number of factors. Do you get an "Aww snap!" message in the browser? Are you trying to load a "big" pivot with lots of data?

If possible,  I would suggest to load smaller subsets of data or use the "Download CSV" function if you are experiencing this issue.

[cid:image001.png@01D2EC07.B36DA800]
​

Also, any browser console and/or server messages when executing the pivot would be helpful to diagnose the problem.

Regards,
Jason


On Tue, Jun 20, 2017 at 4:46 PM, Chris van Hasselt <CVanHasselt@xxxxxxxxxx<mailto:CVanHasselt@xxxxxxxxxx>> wrote:
We are experiencing a problem running pivot tables on  version 2.26 of DHIS 2.   When the data is requested via the API, the data is returned very quickly. However when we  launch a pivot table, specifically with the all the data elements, the pivot table hangs.

From what we can tell via the Chrome developer tools (and other tools), there is a lot of Javascript garbage collection.  The same sort of behavior happens in IE and Firefox, but the upshot is that the pivot table hangs as it is trying to render, and eventually the browser crashes.

Has anyone seen this type of thing before?  Any suggestions on what to look for?

Thanks in advance,
CvH

Chris van Hasselt | IT Project Manager
359 Blackwell Street, Suite 200 | Durham, NC  27701
O: 919.544.7040 x11271 | cvanhasselt@xxxxxxxxxx<http://cvanhasselt@xxxxxxxxxx> |

[Description: Description: fhi 360 email.jpg]
Visit us at www.fhi360.org<http://www.fhi360.org/>



_______________________________________________
Mailing list: https://launchpad.net/~dhis2-users
Post to     : dhis2-users@xxxxxxxxxxxxxxxxxxx<mailto:dhis2-users@xxxxxxxxxxxxxxxxxxx>
Unsubscribe : https://launchpad.net/~dhis2-users
More help   : https://help.launchpad.net/ListHelp



--
Jason P. Pickering
email: jason.p.pickering@xxxxxxxxx<mailto:jason.p.pickering@xxxxxxxxx>
tel:+46764147049



--
Jason P. Pickering
email: jason.p.pickering@xxxxxxxxx<mailto:jason.p.pickering@xxxxxxxxx>
tel:+46764147049