dhis2-devs team mailing list archive

[Morten Olav Hansen <mortenoh@xxxxxxxxx>]

This was just a quick-fix done at the airport. I will look at it later.

--
Morten



On Mon, Dec 17, 2012 at 7:13 PM, Brajesh Murari <brajesh2murari@xxxxxxxxx>wrote:

> Hi Morten,
>
> Pardon to put you in little bit more work around, although my build
> process is working fine in my local system ...I have three quires which i
> want to share with you and others on list before I lose them...
>
> 1. I just want to know which one is the best !!!
>
> public class OrganisationUnitByLevelComparator
>     implements Comparator<OrganisationUnit>
>  {
>     @Override
>     public int compare( OrganisationUnit o1, OrganisationUnit o2 )
>     {
>         return *new Integer*( o1.getOrganisationUnitLevel() ).compareTo(
> o2.getOrganisationUnitLevel() );
>     }
> }
>
> Note: In each comparison, creating a new wrapper Integer object, ie. if
> this could be called approx 40,000 times to compare two different
> Organisation Units might be more in some case ( they might be name of
> households of a certain West African country ) while loading a Tree than
> creating 40,000 new wrapper Integer objects just for doing comparison, and
> that might be treated as bottle neck in such large scale operation as an
> assumption.
>
> *or  *
> *
> *
> Light weighted code like given below
>
> public class OrganisationUnitByLevelComparator
>     implements Comparator<OrganisationUnit>
> {
>     @Override
>     public int compare( OrganisationUnit o1, OrganisationUnit o2 )
>     {
>         return o1.getOrgaonisationUnitLevel() -
> o2.getOrganisationUnitLevel();
>     }
> }
>
>
> 2. Would you like to review your commit in revision 9329 !!!
>
> 3. Are you in a favor to configure Crucible with DHIS2 root trunk like
> other tools eg. CI (Jinkins/Atlassian Bamboo etc..) so that other can put
> their code review comments on dhis2 committed code by great commiter like
> you  !!!
>
> http://www.atlassian.com/software/crucible/overview
>
> If yes in second and third query, than i am strongly recommend, just do it
> in DHIS2 before December 24th this year.
>
>   Regards,
> ~Brajesh~
>
>
> On Sun, Dec 16, 2012 at 1:33 AM, Morten Olav Hansen <mortenoh@xxxxxxxxx>wrote:
>
>> Ok
>> On Dec 15, 2012 10:12 PM, "Brajesh Murari" <brajesh2murari@xxxxxxxxx>
>> wrote:
>>
>>> Hi Morten,
>>>
>>> I am using java version with info below
>>>
>>> brajesh.murari@mohlapbrajesh /cygdrive/d/src/dhis2/dhis-2
>>> $ java -version
>>> java version "1.6.0_26"
>>> Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
>>> Java HotSpot(TM) Client VM (build 20.1-b02, mixed mode)
>>>
>>> After your fix, its now working, with same above java version. If you do
>>> any
>>> java version up-gradation in scm-dhis2, don't forget to update its info
>>> here as well on
>>>
>>> http://www.dhis2.org/development
>>>
>>> I will upgrade my java version on 24th December in morning at 7:30 AM
>>> IST after
>>> taking fresh shower with cold/child water. I thought, may be there would
>>> be any real time idea implementation with your commit 9242 without
>>> blueprint or any bug fixing after Bob's universal declaration ie. "Been
>>> looking at the case of a certain West African country with *40000
>>> orgunits*. *Which is actually not a huge number of objects and we could
>>> very likely be looking at orgunit tables bigger than this as villages and even
>>> households start finding their way into the tree*."
>>>
>>> On Sat, Dec 15, 2012 at 10:58 PM, Morten Olav Hansen <mortenoh@xxxxxxxxx
>>> > wrote:
>>>
>>>> If you have an old version of Java, the commit I just did should
>>>> probably fix it. But I would suggest that you upgrade.
>>>>
>>>> --
>>>> Morten
>>>>
>>>>
>>>>
>>>> On Sat, Dec 15, 2012 at 8:12 PM, Morten Olav Hansen <mortenoh@xxxxxxxxx
>>>> > wrote:
>>>>
>>>>> There should not be any issues with that code, and as bob said our ci
>>>>> server seems to compile it fine (also, no other dev has complained) .
>>>>> Update, clean, install and if it fails update java.
>>>>>
>>>> *>>>*Developers usually don't take fresh checkout nor they like to do
>>> some crazy white box testing all the time, that's why perhaps they may
>>> not complain usually, but there are some developers who don't even
>>> relay on CI nightly build status update, that's why they complain about
>>> build failure...which was perhaps true.
>>> *:-)*
>>>
>>>
>>>>  On Dec 15, 2012 2:50 AM, "Bob Jolliffe" <bobjolliffe@xxxxxxxxx> wrote:
>>>>>
>>>>>> Brajesh I am not sure I understand the tone of your email nor the
>>>>>> impressive cc list.  But anyway ...
>>>>>>
>>>>>> If you have a build failure then please describe that simply on the
>>>>>> list and perhaps people can help you.  It is worth noting that the
>>>>>> continuous integration server http://apps.dhis2.org/ci/ is building
>>>>>> fine so at least the current trunk is fine.
>>>>>>
>>>>>> In general if there was a compilation failure on trunk (and I am not
>>>>>> sure I can verify because I don't think I built 9242 but you can look back
>>>>>> through the emails and see) an alert is sent to the devs list and it is
>>>>>> resolved very quickly.  Never more than a few hours.  So I think you are
>>>>>> safe regarding the issue being resolved before December 24.
>>>>>>
>>>>>> Please verify that you have updated to latest trunk and see if the
>>>>>> problem persists.  If so then give a detailed report to the list.  But
>>>>>> please do so in a separate thread.
>>>>>>
>>>>>> Regards
>>>>>> Bob
>>>>>>
>>>>>>
>>>>>> On 14 December 2012 21:31, Brajesh Murari <brajesh2murari@xxxxxxxxx>wrote:
>>>>>>
>>>>>>> Hello Bob,
>>>>>>>
>>>>>>> Wishing you and everyone on dhis2-dev list, a
>>>>>>> joyous Christmas season.
>>>>>>>
>>>>>>> I am getting build failure in main trunk build with info as given
>>>>>>> below,
>>>>>>>
>>>>>>> [INFO] -------------------------------------------------------------
>>>>>>> [ERROR] COMPILATION ERROR :
>>>>>>> [INFO] -------------------------------------------------------------
>>>>>>> [ERROR]
>>>>>>> \src\dhis2\dhis-2\dhis-api\src\main\java\org\hisp\dhis\organisationunit\comparator\OrganisationUnitByLevelComparator.java:[43,22]
>>>>>>> cannot find symbol
>>>>>>> symbol  : method compare(int,int)
>>>>>>> location: class java.lang.Integer
>>>>>>> [INFO] 1 error
>>>>>>>
>>>>>>> On Sat, Dec 8, 2012 at 9:45 PM, Bob Jolliffe <bobjolliffe@xxxxxxxxx>wrote:
>>>>>>>
>>>>>>>> Just sharing some thoughts before I lose them ...
>>>>>>>>
>>>>>>>> Been looking at the case of a certain West African country
>>>>>>>> with 40000 orgunits.   Which is actually not a huge number of objects and
>>>>>>>> we could very likely be looking at orgunit tables bigger than this as
>>>>>>>> villages and even households start finding their way into the tree.
>>>>>>>>
>>>>>>>>
>>>>>>> One day later after your this email, Morten Olav Hansen did
>>>>>>> a miner change
>>>>>>> in one custom comparator ie. OrganisationUnitByLevelComparator.java<http://bazaar.launchpad.net/~dhis2-devs-core/dhis2/trunk/revision/9242/dhis-2/dhis-api/src/main/java/org/hisp/dhis/organisationunit/comparator/OrganisationUnitByLevelComparator.java>
>>>>>>>  .
>>>>>>> in lp:dhis2 branch with revision number 9242. Before 9242, this
>>>>>>> comparator
>>>>>>> was looks like this given below
>>>>>>>
>>>>>>> @Override
>>>>>>> public int compare( OrganisationUnit o1, OrganisationUnit o2 ){
>>>>>>>          return o1.getLevel() - o2.getLevel();
>>>>>>>     }
>>>>>>> }
>>>>>>>
>>>>>>> Now, after 9242, it has been like
>>>>>>>
>>>>>>> @Override
>>>>>>> public int compare( OrganisationUnit o1, OrganisationUnit o2 ){
>>>>>>>          return Integer.compare( o1.getOrganisationUnitLevel(),
>>>>>>> o2.getOrganisationUnitLevel() );
>>>>>>>     }
>>>>>>> }
>>>>>>>
>>>>>>> After doing this change, committer left a log as "Updated
>>>>>>> OrgUnitByLevelComparator to not rely on pre-populated level field, Added
>>>>>>> 'level Sorted' parameter to OrgUnitController, set to true if you want
>>>>>>> orgUnits sored by level". Could you let us know why
>>>>>>> this pretty change results with DHIS2 build failure !!! I am not sure but
>>>>>>> is there any link in between pretty cool commit 9242 and your this email
>>>>>>> !!!  Pls elaborate and provide us your expert view on this issue, i am
>>>>>>> expecting, we should do pretty cool hand around this issue and you let us
>>>>>>> know how we should comes out from this
>>>>>>> "Build Failure" !!! I wish this issue should be closed before 23rd
>>>>>>> December.
>>>>>>>
>>>>>>>  Our orgunit tree (the one-true-tree) is maintained using parentid
>>>>>>>> pointers.  This is simple enough to maintain and updates and insertions are
>>>>>>>> efficient.
>>>>>>>>
>>>>>>>> Of course updates and insertions are relatively rare.  What we need
>>>>>>>> to do much more frequently is selecting various subtrees.  All the
>>>>>>>> facilities in a district, all the districts in a province etc.
>>>>>>>>
>>>>>>>> Also the depth of our trees are relatively shallow.  Most places
>>>>>>>> seem to have around 5 levels.  There is some trend that this starts to
>>>>>>>> increase, but not exponentially. We might conceive one day of 8 or even 10
>>>>>>>> levels but not 100s or 1000s of levels.  Its this shallowness which makes
>>>>>>>> the _orgunitstructure table viable as the number of columns in that table
>>>>>>>> will always be within a practical limit..
>>>>>>>>
>>>>>>>> Selecting for tree structures (traversal) which are built using
>>>>>>>> 'parentid' is not very efficient.  Postgres offers 'WITH RECURSIVE' which
>>>>>>>> is pretty cool if you can get your head around it, but not supported in
>>>>>>>> mysql (and not necessarily fast either).
>>>>>>>>
>>>>>>>> So when selecting subtrees the _orgunitstructure table is our best
>>>>>>>> friend.  I use it for the mydatamart aggregatedXXvalue queries and it looks
>>>>>>>> like Lars uses it in the scheduled datamart job as well as well.   And it
>>>>>>>> is the obvious table to use when implementing filtering as I am now looking
>>>>>>>> at re mydatamart export.
>>>>>>>>
>>>>>>>> The problem is (with this and the other resource tables) how to
>>>>>>>> maintain integrity.  Currently we generate this table on demand (from the
>>>>>>>> user) from the parentid pointers.  If users forget to do it then all sorts
>>>>>>>> of things fail.  There has been some discussion on list of generating this,
>>>>>>>> at least nightly which would be an improvement.  There would be some
>>>>>>>> benefit in maintaining it dynamically, ie "triggered" during those
>>>>>>>> relatively rare updates and insertions of new orgunits.  One possible
>>>>>>>> consequence of this approach would be that the parentid on the orgunit
>>>>>>>> becomes effectively redundant. It could be argued that its *only* current
>>>>>>>> use is to generate the _orgunitstructure table.
>>>>>>>>
>>>>>>>> A consequence of the parentid being redundant is that hierarchy is
>>>>>>>> maintained in a separate table to the orgunits themselves.  And there is no
>>>>>>>> reason why there should be only one _orgunitstructure table.  There could
>>>>>>>> be any number, limited only by the number of hierarchies we needed to
>>>>>>>> maintain.  There's a thought ...
>>>>>>>>
>>>>>>>> Of course the other use for the parentid is when you are exporting
>>>>>>>> a bundle of orgunits in a way which reflects the (or a) hierarchy.  In
>>>>>>>> which case it is really kind to clients to serialize this tree as a
>>>>>>>> breadth-first traversal ie. they come out in order which is quick and easy
>>>>>>>> to rebuild on the client.  That is what I am looking at doing now to try
>>>>>>>> and help out our mydatamart metadata export.  And I will of course use the
>>>>>>>> _orgunitstructure table to make this trivial on the server side.
>>>>>>>>
>>>>>>>> But some principles derived from the above discussion:
>>>>>>>> 1.  try to avoid using parentid directly in code ... always link to
>>>>>>>> the orgunitstructuretable. This might ease its eventual demise
>>>>>>>> 2.  all places where we export collections of orgunits should be in
>>>>>>>> a  proper traversal - breadth first or depth firt shouldn't matter really
>>>>>>>> but I'm going to do the latter.
>>>>>>>>
>>>>>>>> Bob
>>>>>>>>
>>>>>>>
>>>>>>> Merry Christmas
>>>>>>> :-)
>>>>>>>
>>>>>>> Regards,
>>>>>>> Brajesh
>>>>>>> --
>>>>>>>
>>>>>>
>>> Any way, build procedure is working fine now in my system.
>>> thanks for informing us.
>>> :-)
>>>
>>> --
>>> Regards,
>>> Brajesh
>>>
>>>
>>>
>
>
> --
> Regards,
> Brajesh
>
>
>