dhis2-devs team mailing list archive

[Bob Jolliffe <bobjolliffe@xxxxxxxxx>]

2009/4/17 Knut Staring <knutst@xxxxxxxxx>:
> On Fri, Apr 17, 2009 at 6:10 PM, Bob Jolliffe <bobjolliffe@xxxxxxxxx> wrote:
>>
>> 2009/4/17 Knut Staring <knutst@xxxxxxxxx>:
>> > On Fri, Apr 17, 2009 at 5:09 PM, Saptarshi Purkayastha
>> > <sunbiz@xxxxxxxxx>
>> > wrote:
>> >>>
>> >>> When importing dataelements, it becomes a problem that shornames have
>> >>> to
>> >>> be both unique AND max 25 characters. I think we should loosen the
>> >>> second
>> >>> requirement in the db. It must be up to the users to make their
>> >>> shortnames
>> >>> sufficiently short, not a db constraint.
>> >>
>> >> I disagree, it makes more sense to keep it the way it is... a reason
>> >> its
>> >> called short name ;-)
>> >> and building boundaries on the data model is a best practice of sorts.
>> >
>> > Yes, that is exactly why we made it the way it is.
>> >>
>> >> Some other solution may just be able to solve the problem??
>> >
>> > Let me describe the issue in some more detail: We want to import a
>> > database
>> > which has 3000 data elements (survey questions), most of them
>> > considerably
>> > longer than 25 characters, and many very similar, often differing mainly
>> > in
>> > a few characters at the end. Shortname is restricted to be required,
>> > unique
>> > and 25 characters. One possibility is to just use the dataelementid as
>> > shortname, but that of course would mean it is completely useless, just
>> > a
>> > filler to enable data to be imported.
>> > We are now putting a high priority on import (of hierarchies, data
>> > elements,
>> > and data) from a spreadsheet, and the import process could perhaps
>> > contain
>> > intelligence suggesting possible shortnames, but it is quite hard to
>> > come up
>> > with something generic that will make sense in most cases. One strategy
>> > is
>> > to just take the 25 first characters of the name, and if the result
>> > exists
>> > already, substitute a number for the last two characters (alternatively
>> > do
>> > this for all). This is sort of workable, but it would in that case be
>> > desirable to also mark the data element so that someone can go through
>> > manually afterwards.
>> > Another option would be to the uniquenesscheck in the spreadsheet first
>> > - or
>> > go through all dataelements with length over 25 characters manually
>> > before
>> > importing. It is actually easier to do it in the spreadsheet than in
>> > DHIS 2
>> > - but we don't have that import functionality yet.
>> > Knut
>>
>> From fs/namei.c in linux kernel source:
>> /*
>>  * Given a valid longname, create a unique shortname.  Make sure the
>>  * shortname does not exist
>>  * Returns negative number on error, 0 for a normal
>>  * return, and 1 for valid shortname
>>  */
>> static int vfat_create_shortname(struct inode *dir, struct nls_table *nls,
>>                                 wchar_t *uname, int ulen,
>>                                 unsigned char *name_res, unsigned char
>> *lcase)
>>
>> The algorithm is pretty short and could probably be pretty easily
>> replicated in java or perhaps better still as a spreadsheet macro.
>
> A spreadsheet macro may be the best for coming up with plausible candidates,
> though I have no idea what this particular one does.
>
>>
>> Seriously though, I have to agree with Saptarshi here.  The point of a
>> short name is that it should be bounded.
>
> There is no disagreement on that, the issue is what to do when importing
> data from existing systems. From many perspectives, it does make sense to do
> most of the required processing before importing, at least the simple.
>
>>
>> (I'm not sure a survey question is a suitable candidate as a data element
>> name anyway but thats another issue).
>
> Survey data is a very high priority for us, but you are right that the bulk
> description will go in a different field. And one should probably take the
> required time to change both shortnames and names when importing into DHIS,
> but this has to be done by the owners of the data. After we have spreadsheet
> import implemented, it will be easier to give people instructions as to how
> to process their data in preparation for import. Probably, we have to create
> some macros to assist in this.
>>
>> I presume you won't lose the full name on import?
>
>
> The full name is no problem, issue is only what to do with the shortname
> field. But I do think you're right that the issue should be handled before
> importing. Still, we may want to extend to 30
>
>>
>>  We have
>> 'description' which holds an arbitrary amount of text.  So I would opt
>> to import the full question in here and do some sort of fat truncation
>> trick for the short names which are too long. How many of them go
>>
>> over the limit?
>
> 2341


Thats a lot more than you want to do by hand!  Dunno.  Will think some more.

Bob

>
> Knut
>>
>> 3000 is a lot but if you are talking 200 over the
>> limit it might be doable to manually edit them after the fact and make
>> them more meaningful.
>>
>> Bob
>>
>> >
>> >>
>> >> ---
>> >> Regards,
>> >> Saptarshi PURKAYASTHA
>> >> Director R & D, HISP India
>> >> Health Information Systems Programme
>> >>
>> >> My Tech Blog:  http://sunnytalkstech.blogspot.com
>> >> You Live by CHOICE, Not by CHANCE
>> >>
>> >>>
>> >>> Knut
>> >>>
>> >>> _______________________________________________
>> >>> Mailing list: https://launchpad.net/~dhis2-devs
>> >>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>> >>> Unsubscribe : https://launchpad.net/~dhis2-devs
>> >>> More help   : https://help.launchpad.net/ListHelp
>> >>>
>> >>
>> >>
>> >> _______________________________________________
>> >> Mailing list: https://launchpad.net/~dhis2-devs
>> >> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>> >> Unsubscribe : https://launchpad.net/~dhis2-devs
>> >> More help   : https://help.launchpad.net/ListHelp
>> >>
>> >
>> >
>> >
>> > --
>> > Cheers,
>> > Knut Staring
>> >
>> > _______________________________________________
>> > Mailing list: https://launchpad.net/~dhis2-devs
>> > Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>> > Unsubscribe : https://launchpad.net/~dhis2-devs
>> > More help   : https://help.launchpad.net/ListHelp
>> >
>> >
>
>
>
> --
> Cheers,
> Knut Staring
>