dhis2-devs team mailing list archive

[Brajesh Murari <brajesh2murari@xxxxxxxxx>]

Hi Morten,

Pardon to put you in little bit more work around, although my build process
is working fine in my local system ...I have three quires which i want to
share with you and others on list before I lose them...

1. I just want to know which one is the best !!!

public class OrganisationUnitByLevelComparator
    implements Comparator<OrganisationUnit>
{
    @Override
    public int compare( OrganisationUnit o1, OrganisationUnit o2 )
    {
        return *new Integer*( o1.getOrganisationUnitLevel() ).compareTo(
o2.getOrganisationUnitLevel() );
    }
}

Note: In each comparison, creating a new wrapper Integer object, ie. if
this could be called approx 40,000 times to compare two different
Organisation Units might be more in some case ( they might be name of
households of a certain West African country ) while loading a Tree than
creating 40,000 new wrapper Integer objects just for doing comparison, and
that might be treated as bottle neck in such large scale operation as an
assumption.

*or  *
*
*
Light weighted code like given below

public class OrganisationUnitByLevelComparator
    implements Comparator<OrganisationUnit>
{
    @Override
    public int compare( OrganisationUnit o1, OrganisationUnit o2 )
    {
        return o1.getOrgaonisationUnitLevel() -
o2.getOrganisationUnitLevel();
    }
}


2. Would you like to review your commit in revision 9329 !!!

3. Are you in a favor to configure Crucible with DHIS2 root trunk like
other tools eg. CI (Jinkins/Atlassian Bamboo etc..) so that other can put
their code review comments on dhis2 committed code by great commiter like
you  !!!

http://www.atlassian.com/software/crucible/overview

If yes in second and third query, than i am strongly recommend, just do it
in DHIS2 before December 24th this year.

  Regards,
~Brajesh~


On Sun, Dec 16, 2012 at 1:33 AM, Morten Olav Hansen <mortenoh@xxxxxxxxx>wrote:

> Ok
> On Dec 15, 2012 10:12 PM, "Brajesh Murari" <brajesh2murari@xxxxxxxxx>
> wrote:
>
>> Hi Morten,
>>
>> I am using java version with info below
>>
>> brajesh.murari@mohlapbrajesh /cygdrive/d/src/dhis2/dhis-2
>> $ java -version
>> java version "1.6.0_26"
>> Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
>> Java HotSpot(TM) Client VM (build 20.1-b02, mixed mode)
>>
>> After your fix, its now working, with same above java version. If you do
>> any
>> java version up-gradation in scm-dhis2, don't forget to update its info
>> here as well on
>>
>> http://www.dhis2.org/development
>>
>> I will upgrade my java version on 24th December in morning at 7:30 AM IST
>> after
>> taking fresh shower with cold/child water. I thought, may be there would
>> be any real time idea implementation with your commit 9242 without
>> blueprint or any bug fixing after Bob's universal declaration ie. "Been
>> looking at the case of a certain West African country with *40000
>> orgunits*. *Which is actually not a huge number of objects and we could
>> very likely be looking at orgunit tables bigger than this as villages and even
>> households start finding their way into the tree*."
>>
>> On Sat, Dec 15, 2012 at 10:58 PM, Morten Olav Hansen <mortenoh@xxxxxxxxx>wrote:
>>
>>> If you have an old version of Java, the commit I just did should
>>> probably fix it. But I would suggest that you upgrade.
>>>
>>> --
>>> Morten
>>>
>>>
>>>
>>> On Sat, Dec 15, 2012 at 8:12 PM, Morten Olav Hansen <mortenoh@xxxxxxxxx>wrote:
>>>
>>>> There should not be any issues with that code, and as bob said our ci
>>>> server seems to compile it fine (also, no other dev has complained) .
>>>> Update, clean, install and if it fails update java.
>>>>
>>> *>>>*Developers usually don't take fresh checkout nor they like to do
>> some crazy white box testing all the time, that's why perhaps they may
>> not complain usually, but there are some developers who don't even relay
>> on CI nightly build status update, that's why they complain about build
>> failure...which was perhaps true.
>> *:-)*
>>
>>
>>>  On Dec 15, 2012 2:50 AM, "Bob Jolliffe" <bobjolliffe@xxxxxxxxx> wrote:
>>>>
>>>>> Brajesh I am not sure I understand the tone of your email nor the
>>>>> impressive cc list.  But anyway ...
>>>>>
>>>>> If you have a build failure then please describe that simply on the
>>>>> list and perhaps people can help you.  It is worth noting that the
>>>>> continuous integration server http://apps.dhis2.org/ci/ is building
>>>>> fine so at least the current trunk is fine.
>>>>>
>>>>> In general if there was a compilation failure on trunk (and I am not
>>>>> sure I can verify because I don't think I built 9242 but you can look back
>>>>> through the emails and see) an alert is sent to the devs list and it is
>>>>> resolved very quickly.  Never more than a few hours.  So I think you are
>>>>> safe regarding the issue being resolved before December 24.
>>>>>
>>>>> Please verify that you have updated to latest trunk and see if the
>>>>> problem persists.  If so then give a detailed report to the list.  But
>>>>> please do so in a separate thread.
>>>>>
>>>>> Regards
>>>>> Bob
>>>>>
>>>>>
>>>>> On 14 December 2012 21:31, Brajesh Murari <brajesh2murari@xxxxxxxxx>wrote:
>>>>>
>>>>>> Hello Bob,
>>>>>>
>>>>>> Wishing you and everyone on dhis2-dev list, a
>>>>>> joyous Christmas season.
>>>>>>
>>>>>> I am getting build failure in main trunk build with info as given
>>>>>> below,
>>>>>>
>>>>>> [INFO] -------------------------------------------------------------
>>>>>> [ERROR] COMPILATION ERROR :
>>>>>> [INFO] -------------------------------------------------------------
>>>>>> [ERROR]
>>>>>> \src\dhis2\dhis-2\dhis-api\src\main\java\org\hisp\dhis\organisationunit\comparator\OrganisationUnitByLevelComparator.java:[43,22]
>>>>>> cannot find symbol
>>>>>> symbol  : method compare(int,int)
>>>>>> location: class java.lang.Integer
>>>>>> [INFO] 1 error
>>>>>>
>>>>>> On Sat, Dec 8, 2012 at 9:45 PM, Bob Jolliffe <bobjolliffe@xxxxxxxxx>wrote:
>>>>>>
>>>>>>> Just sharing some thoughts before I lose them ...
>>>>>>>
>>>>>>> Been looking at the case of a certain West African country
>>>>>>> with 40000 orgunits.   Which is actually not a huge number of objects and
>>>>>>> we could very likely be looking at orgunit tables bigger than this as
>>>>>>> villages and even households start finding their way into the tree.
>>>>>>>
>>>>>>>
>>>>>> One day later after your this email, Morten Olav Hansen did
>>>>>> a miner change
>>>>>> in one custom comparator ie. OrganisationUnitByLevelComparator.java<http://bazaar.launchpad.net/~dhis2-devs-core/dhis2/trunk/revision/9242/dhis-2/dhis-api/src/main/java/org/hisp/dhis/organisationunit/comparator/OrganisationUnitByLevelComparator.java>
>>>>>>  .
>>>>>> in lp:dhis2 branch with revision number 9242. Before 9242, this
>>>>>> comparator
>>>>>> was looks like this given below
>>>>>>
>>>>>> @Override
>>>>>> public int compare( OrganisationUnit o1, OrganisationUnit o2 ){
>>>>>>          return o1.getLevel() - o2.getLevel();
>>>>>>     }
>>>>>> }
>>>>>>
>>>>>> Now, after 9242, it has been like
>>>>>>
>>>>>> @Override
>>>>>> public int compare( OrganisationUnit o1, OrganisationUnit o2 ){
>>>>>>          return Integer.compare( o1.getOrganisationUnitLevel(),
>>>>>> o2.getOrganisationUnitLevel() );
>>>>>>     }
>>>>>> }
>>>>>>
>>>>>> After doing this change, committer left a log as "Updated
>>>>>> OrgUnitByLevelComparator to not rely on pre-populated level field, Added
>>>>>> 'level Sorted' parameter to OrgUnitController, set to true if you want
>>>>>> orgUnits sored by level". Could you let us know why
>>>>>> this pretty change results with DHIS2 build failure !!! I am not sure but
>>>>>> is there any link in between pretty cool commit 9242 and your this email
>>>>>> !!!  Pls elaborate and provide us your expert view on this issue, i am
>>>>>> expecting, we should do pretty cool hand around this issue and you let us
>>>>>> know how we should comes out from this
>>>>>> "Build Failure" !!! I wish this issue should be closed before 23rd
>>>>>> December.
>>>>>>
>>>>>>  Our orgunit tree (the one-true-tree) is maintained using parentid
>>>>>>> pointers.  This is simple enough to maintain and updates and insertions are
>>>>>>> efficient.
>>>>>>>
>>>>>>> Of course updates and insertions are relatively rare.  What we need
>>>>>>> to do much more frequently is selecting various subtrees.  All the
>>>>>>> facilities in a district, all the districts in a province etc.
>>>>>>>
>>>>>>> Also the depth of our trees are relatively shallow.  Most places
>>>>>>> seem to have around 5 levels.  There is some trend that this starts to
>>>>>>> increase, but not exponentially. We might conceive one day of 8 or even 10
>>>>>>> levels but not 100s or 1000s of levels.  Its this shallowness which makes
>>>>>>> the _orgunitstructure table viable as the number of columns in that table
>>>>>>> will always be within a practical limit..
>>>>>>>
>>>>>>> Selecting for tree structures (traversal) which are built using
>>>>>>> 'parentid' is not very efficient.  Postgres offers 'WITH RECURSIVE' which
>>>>>>> is pretty cool if you can get your head around it, but not supported in
>>>>>>> mysql (and not necessarily fast either).
>>>>>>>
>>>>>>> So when selecting subtrees the _orgunitstructure table is our best
>>>>>>> friend.  I use it for the mydatamart aggregatedXXvalue queries and it looks
>>>>>>> like Lars uses it in the scheduled datamart job as well as well.   And it
>>>>>>> is the obvious table to use when implementing filtering as I am now looking
>>>>>>> at re mydatamart export.
>>>>>>>
>>>>>>> The problem is (with this and the other resource tables) how to
>>>>>>> maintain integrity.  Currently we generate this table on demand (from the
>>>>>>> user) from the parentid pointers.  If users forget to do it then all sorts
>>>>>>> of things fail.  There has been some discussion on list of generating this,
>>>>>>> at least nightly which would be an improvement.  There would be some
>>>>>>> benefit in maintaining it dynamically, ie "triggered" during those
>>>>>>> relatively rare updates and insertions of new orgunits.  One possible
>>>>>>> consequence of this approach would be that the parentid on the orgunit
>>>>>>> becomes effectively redundant. It could be argued that its *only* current
>>>>>>> use is to generate the _orgunitstructure table.
>>>>>>>
>>>>>>> A consequence of the parentid being redundant is that hierarchy is
>>>>>>> maintained in a separate table to the orgunits themselves.  And there is no
>>>>>>> reason why there should be only one _orgunitstructure table.  There could
>>>>>>> be any number, limited only by the number of hierarchies we needed to
>>>>>>> maintain.  There's a thought ...
>>>>>>>
>>>>>>> Of course the other use for the parentid is when you are exporting a
>>>>>>> bundle of orgunits in a way which reflects the (or a) hierarchy.  In which
>>>>>>> case it is really kind to clients to serialize this tree as a breadth-first
>>>>>>> traversal ie. they come out in order which is quick and easy to rebuild on
>>>>>>> the client.  That is what I am looking at doing now to try and help out our
>>>>>>> mydatamart metadata export.  And I will of course use the _orgunitstructure
>>>>>>> table to make this trivial on the server side.
>>>>>>>
>>>>>>> But some principles derived from the above discussion:
>>>>>>> 1.  try to avoid using parentid directly in code ... always link to
>>>>>>> the orgunitstructuretable. This might ease its eventual demise
>>>>>>> 2.  all places where we export collections of orgunits should be in
>>>>>>> a  proper traversal - breadth first or depth firt shouldn't matter really
>>>>>>> but I'm going to do the latter.
>>>>>>>
>>>>>>> Bob
>>>>>>>
>>>>>>
>>>>>> Merry Christmas
>>>>>> :-)
>>>>>>
>>>>>> Regards,
>>>>>> Brajesh
>>>>>> --
>>>>>>
>>>>>
>> Any way, build procedure is working fine now in my system.
>> thanks for informing us.
>> :-)
>>
>> --
>> Regards,
>> Brajesh
>>
>>
>>


-- 
Regards,
Brajesh