← Back to team overview

dhis2-devs team mailing list archive

Re: "Atomic" data validation rules

 

Hi Jim,

I also like the simpler option better (surprise). If the default option
clearly states that there is no change then we are not forcing the users
who do not need this feature to even understand it.

regards,

Lars




On Fri, May 23, 2014 at 11:05 PM, Jim Grace <jimgrace@xxxxxxxxx> wrote:

> I'm liking the simpler option of just adding one dropdown with category
> combinations, called something like "Repeat validation rule using". The
> default dropdown selection could be something like "[ Use validation rule
> as is ]" or "[ Don't repeat rule ]". The following choices are the option
> combos: "Age", "Sex", "Age/Sex", or whatever.
>
> On Fri, May 23, 2014 at 2:07 PM, Jim Grace <jimgrace@xxxxxxxxx> wrote:
>
>> Hi Jason,
>>
>> Here's my idea: Add one extra field to ValidationRule, a reference to
>> DataElementCategoryCombo. If this is null, it means no change from the
>> current validation rule. If it's not null, it means that the validation
>> rule should be applied to all option combos within the category combo, for
>> any data element(s) reference in the left and right side expressions.
>>
>> On the New Validation Rule screen, add an option right below "Rule type"
>> that selects whether or not this will be a "template" rule. (Except don't
>> use the word "template" -- too technical.) Maybe like what you said, or
>> "Apply rule to category combinations" (by default "No"). If "Yes" is
>> selected, another option pops into visibility just below to choose a
>> category combination. Once this is chosen, it is used to filter the list of
>> data elements that will be available to the editor in "Edit left side" and
>> "Edit right side". Perhaps the "Edit left side" and "Edit right side"
>> buttons should then be disabled until a category combination is chosen.
>> Otherwise the UI is left the same -- just the addition of these two fields
>> and the data element filter. The user is free to make any expressions they
>> want, but the data elements used on the left and right side must all have
>> the same category combo.
>>
>> From the software point of view this could be done much more simply in a
>> single dropdown of category option combos; it's a matter of the art of UI
>> design to make this all somewhat intelligible to the user. The semantics
>> would be that if you choose a combo, the data elements will be restricted
>> to those having the combo and the rule will be multiplied by every combo
>> option value. Using our pop-up contextual help feature on this screen (as
>> well as in many other areas of the product) would assist usability a lot!
>>
>> At validation rule execution time, some pre-processing is done
>> in ValidationRunContext.java (to avoid processing things that are common to
>> a validation run for each org unit / period). We could add an extra private
>> method to this class that we could take each validation rule with a
>> category option combo, and "multiply" it to make an in-memory validation
>> rule (not persisted) for each category combo. This would be done by taking
>> the left and right side expressions, looking for the data element tags
>> inside the expression, and adding each category option combo, respectively,
>> onto each data element tag. Then the validation run could proceed without
>> any further modification.
>>
>> Let me know what you think. I'm also happy to do the implementation if
>> that makes sense; I'm familiar with the code and it should be pretty quick.
>>
>> Cheers,
>> Jim
>>
>>
>>
>> On Fri, May 23, 2014 at 9:10 AM, Jason Pickering <
>> jason.p.pickering@xxxxxxxxx> wrote:
>>
>>> Hi Jim
>>> Yes, I think I agree with you, but a still a bit fuzzy how it could
>>> happen with the current UI.  I think the user would have an option like
>>> "Apply this rule to common category combo options", and they would then
>>> select a left side, and the right side would only show data elements with
>>> the same category combo. The user would not select individual data
>>> elements.  An operator would need to be selected as well, obviously.
>>>
>>> For the use case we are dealing with, it is always comparing two data
>>> elements, one which is typically a subset of another, which is quite common
>>> in HIV M&E. Number counselled <= Number tested <= Number positive <= Number
>>> receiving results.
>>>
>>> Regards,
>>> Jason
>>>
>>>
>>>
>>> On Fri, May 23, 2014 at 1:44 PM, Jim Grace <jimgrace@xxxxxxxxx> wrote:
>>>
>>>> I like it. A couple of ideas:
>>>>
>>>> - I suggest that this be a new property of validation rules, rather
>>>> than a third type in addition to "Validation" and "Surveillance". It could
>>>> be potentially useful for either of these two rule types.
>>>>
>>>> - When selecting this option, the user could choose from a list of
>>>> category combos that then appears on the add validation rule screen. (We
>>>> could even present a list of category combos that are actually *used* by
>>>> data elements.) This list could be used to filter the data elements
>>>> available when creating the left and right sides, to make sure they all
>>>> have the same combos.
>>>>
>>>> Cheers,
>>>> Jim
>>>>
>>>>
>>>>
>>>> On Fri, May 23, 2014 at 3:55 AM, Jason Pickering <
>>>> jason.p.pickering@xxxxxxxxx> wrote:
>>>>
>>>>> Hi Lars,
>>>>> Finally got around to creating the blueprint here
>>>>>
>>>>> https://blueprints.launchpad.net/dhis2/+spec/atomic-validation-rules
>>>>>
>>>>> Regards,
>>>>> Jason
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Jan 15, 2014 at 6:36 PM, Lars Helge Øverland <
>>>>> larshelge@xxxxxxxxx> wrote:
>>>>>
>>>>>> Hi Jason,
>>>>>>
>>>>>> yes I think this makes sense. One could make a special kind of
>>>>>> validation rule, with a constraint that all data elements on the left and
>>>>>> right side expressions share the same category combination, and then tell
>>>>>> the system to validate each and every category option combination. Feel
>>>>>> free to write a blueprint.
>>>>>>
>>>>>> regards,
>>>>>>
>>>>>> Lars
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, Jan 14, 2014 at 3:52 PM, Jason Pickering <
>>>>>> jason.p.pickering@xxxxxxxxx> wrote:
>>>>>>
>>>>>>> Hi Devs,
>>>>>>> I came up against an issue today when looking at implementing a
>>>>>>> legacy database in DHIS2. In the old system, there are about 75 rules. The
>>>>>>> rules in the past have been applied at the atomic level, meaning that each
>>>>>>> and every data element category combination value is compared with a
>>>>>>> counter part. As an example.
>>>>>>>
>>>>>>> Number of persons currently receiving ART (including pregnant women)
>>>>>>> is greater than or equal to
>>>>>>> Number of individuals NEWLY initiating ART (including pregnant women)
>>>>>>>
>>>>>>> Each of these data elements are disaggregated by Gender and three
>>>>>>> age groups, and the validation rules need to be applied to each respective
>>>>>>> category combination options (not the totals).
>>>>>>>
>>>>>>> Although I do not know exactly how many rules we would need to to
>>>>>>> this, it is surely at least several hundred. I have not really come up
>>>>>>> against this before, and need to see if anyone has expericience with how
>>>>>>> the system behaves with "large" numbers of rules.
>>>>>>>
>>>>>>> I think it would be good to think about how to we could make the
>>>>>>> rules be applied within category combo options , without having to
>>>>>>> explicitly define each and every rule. Thoughts?
>>>>>>>
>>>>>>> Regards,
>>>>>>> Jason
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Mailing list: https://launchpad.net/~dhis2-devs
>>>>>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>>>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Mailing list: https://launchpad.net/~dhis2-devs
>>>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>
>>>>>
>>>>
>>>
>>
>

Follow ups

References