dhis2-devs team mailing list archive

[Jason Pickering <jason.p.pickering@xxxxxxxxx>]

I thought I should forward this conversation Hieu and I have been having for
the benefit of the group. Perhaps there are others that have
suggestions/comments/rants about how we should go about field level
validation with regular expressions?

Regards,
Jason


---------- Forwarded message ----------
From: Jason Pickering <jason.p.pickering@xxxxxxxxx>
Date: Fri, Mar 26, 2010 at 10:31 AM
Subject: Re: About Regex-Validation blueprint
To: Hieu Dang Duy <hieu.hispvietnam@xxxxxxxxx>


Hi Hieu,
Well, we can always start simple, and go from there, but it is important
that we architect everything properly.

So, I suggest that for everything out of the translation table, we use
something like the data validation procedures. A user enter a rules, and
then runs a check. Eventually, this could be extended to either the UI
validation. But, my thinking was similar to that of DHIS 1.4, which allows
data validation rules to be defined in terms of SQL, and then run by the
user during a validation check.

So, lets try a a full example. I have a situation now, where I have
organisationunit short names, with trailing spaces.

These there would need to be a UI screen to define the rule itself.
a) 1 (or something from the hibernate_sequence) (regexid)
b) No trailing spaces (name)
c) \s+$ (regexexpression)
d) This field is not allowed to have trailing spaces (violation_message)
e) Please remove all trailing spaces from this field. This means any spaces
after the last character. (violation_resolution)

The UI, would contain a list of objects. The user would select the object
(organisationunit) and then the property (name) and the rule (foreign key
reference to the first table).


a) 1 (foreign key reference to the regex_objects table /object)
b) organisationunit (object/table)
c) name (property)
d)  FALSE (negate)
e)  TRUE (case_insensitive)
f)   NULL locale

I would guess for the translation table, the method would need to be
extended somehow, to deal with the locale, which is not present in other
objects/tables.

Now, a third UI screen would allow the user to choose the rule(s) they wish
to run, similar to the data validation screen. A list of objects that match
the rules that the user selects would then be returned to the user. I guess
having the ability to print/save this list would be useful. Perhaps it
should be brought up as a popup/seperate window to allow the user to resolve
each of the violations in turn.


Personally, I think we should start with everything other than the
translations table, but I do not actually see much difference. Basically, if
the user were to select the translation table, then the locale_code would
need to be set in the regex definition.

Making any more sense now? If I could code Java, I would help out, but I am
like an old dog, too old to learn new tricks. :)

Best regards,
jason





On Fri, Mar 26, 2010 at 10:16 AM, Hieu Dang Duy
<hieu.hispvietnam@xxxxxxxxx>wrote:

> Dear Jason,
>
> Actually, your password example it is an interesting point. So, I've
> thought quite simple but maybe not in fact. Especially with the locale
> field. So, I understand now that why you called this field is the extra
> field.
> The functionality of regex validation which is larger and scale than I
> thought.
>
> I am confusing now. Would u like to give me any suggestion about what
> should I have set up which key is it ?
>
> Thank you !
>
>
> On Fri, Mar 26, 2010 at 4:15 PM, Jason Pickering <
> jason.p.pickering@xxxxxxxxx> wrote:
>
>>
>>
>>
>> And the term* f )* ... How I can forget this one. I am really careless
>>> person. DHIS2 is a multilingual software. Thank you so much for reminding me
>>> on this issue.
>>>
>>>
>>>> Yes, I did not consider the translation table. I suppose we would need
>>>> to specify an additional field for the locale in't  this table. For
>>>> instance, certain regex may apply for English that do not apply for
>>>> Vietnamese. A good example would be that of the data format field. Different
>>>> countries have different ways of specifying the dates, so I guess the second
>>>> table would
>>>>
>>>>
>>> Yes, of course or absolutely. But may I ask you a sensitive question? Why
>>> don't you consider to the translation table. As forgot or you really don't
>>> want to?
>>>
>>
>> OK, here is the issue with the translation table as I see it. Again, this
>> may not be the same way as with Java. When I look at for instance certain
>> fields, lets say Organisationunit.shortname, there is no corresponding
>> translation in the translation table. As far as i can tell, these properties
>> are not translatable, yet. Additionally, we might want to use regex
>> validation on other properties that are not translatable, for instance the
>> user password, for example ^(?=.*\d)(?=.*[a-z])(?=.*[A-Z]).{4,8}$ to give
>> a password that must be at least 4 characters, no more than 8 characters,
>> and must include at least one upper case letter, one lower case letter, and
>> one numeric digit. Now, different implementers might have different
>> restrictions on the type of the type of password they would accept. Another
>> example might be the use of regular expressions to validate identifiers. For
>> instance ^\d{3}-\d{2}-\d{4}$ can be used to validate a Social Security
>> number used in the US. Other countries have different identifiers, so there
>> is a need to be able to validate different identifier types depending on the
>> country.
>>
>> So, as for the translation table, I would consider this as a separate
>> object really to apply a regex to, but I would not consider it to be the
>> ONLY table that regex validations should apply to.
>>
>> So, for the second table, I would see it being persisted like this..
>>
>> a) regexid- 1 (foreign key reference to the regex_objects) table
>> b) object/table-translation
>> c) property/field-value
>>
>> d)  FALSE (In this case, we will not negate the expression as we want all
>> fields that DO have trailing spaces.
>>  e)  TRUE (With this regex, it does not matter, but we will default to
>> true anyway)
>> f) locale (en_GB) I think this may be necessary to deal with your next
>> point about translations
>>
>> To validate a password, the object would look something like this..
>>
>> a) regexid- 1 (foreign key reference to the regex_objects) table
>> b) object/table-users
>> c) property/field-password
>>
>> d)  FALSE (In this case, we will not negate the expression as we want all
>> fields that DO have trailing spaces.
>>  e) FALSE (With this regex, it does, but we will default to true anyway)
>> f) locale NULL
>>
>> Now, the case of the password brings up an interesting point, as it is
>> persisted in the DB as a hash. Not really sure how to deal with this one....
>>
>> Anyway, maybe this is clearer?
>>
>> Regards,
>> Jason
>>
>>
>>
>>
>>
>>>
>>>
>>>>  I did not really consider the translations so much. I guess there is a
>>>> need for an additional field. By default, it would be the default locale.
>>>> The user would then need to construct different regex expressions for
>>>> different locales, or apply the same regex to several locales.  Maybe this
>>>> extra property would be enough for this? If it is NULL, then the regex would
>>>> use the base object (organisationunit.name), but if there is a locale
>>>> stored in the field, then it would use the
>>>> translation.organisationunit.name property for that particular locale.
>>>> Does this make sense?
>>>>
>>>>
>>> After your suggestion on adding new field is locale. So, I thought that
>>> the translation table's model data which are very very useful for the regex
>>> table's case. Beside, the locale filed I would like to suggest another field
>>> is that object's i likes the translation table.
>>>
>>> If so, we will have a composite key as same as the key one in Translation
>>> table.
>>> Please have a look at the example is below:
>>>
>>> *** Example ***
>>>
>>> With *regexId *: *01*
>>>
>>>
>>> 01- (001 - DataElement - name - en_GB) - false - true*    ===> SAFED*
>>>
>>> 01- (002 - DataElement - shortname - en_GB) - F - T*     ===> SAFED*
>>>
>>> 01- (003 - DataElement - name - vi_VN) - F - T*              ===> SAFED*
>>>
>>> 02 - (001 -DataElement - name - en_GB) - false - true*     ===> UN-SAFED
>>> ===> UN-INSERTED*
>>>
>>> In this example, with the composite key. We do not allow the action of
>>> inserting/updating an other regex for the same object.
>>>
>>>
>>> How do you think dear Jason?
>>>
>>>
>>>
>>>> Looking forward to working on this with you.
>>>>
>>>
>>> Thanks again !
>>>
>>>
>>>>
>>>> Best regards,
>>>> Jason
>>>>
>>>> --
>>>> --
>>>> Jason P. Pickering
>>>> email: jason.p.pickering@xxxxxxxxx
>>>> tel:+260968395190
>>>>
>>>
>>>
>>>
>>> --
>>> Hieu.HISPVietnam
>>> Good Health !
>>>
>>
>>
>>
>>
>> --
>> --
>> Jason P. Pickering
>> email: jason.p.pickering@xxxxxxxxx
>> tel:+260968395190
>>
>
>
>
> --
> Hieu.HISPVietnam
> Good Health !
>



-- 
--
Jason P. Pickering
email: jason.p.pickering@xxxxxxxxx
tel:+260968395190



-- 
--
Jason P. Pickering
email: jason.p.pickering@xxxxxxxxx
tel:+260968395190