dhis2-devs team mailing list archive

[Knut Staring <knutst@xxxxxxxxx>]

On Fri, Apr 17, 2009 at 6:10 PM, Bob Jolliffe <bobjolliffe@xxxxxxxxx> wrote:

> 2009/4/17 Knut Staring <knutst@xxxxxxxxx>:
> > On Fri, Apr 17, 2009 at 5:09 PM, Saptarshi Purkayastha <sunbiz@xxxxxxxxx
> >
> > wrote:
> >>>
> >>> When importing dataelements, it becomes a problem that shornames have
> to
> >>> be both unique AND max 25 characters. I think we should loosen the
> second
> >>> requirement in the db. It must be up to the users to make their
> shortnames
> >>> sufficiently short, not a db constraint.
> >>
> >> I disagree, it makes more sense to keep it the way it is... a reason its
> >> called short name ;-)
> >> and building boundaries on the data model is a best practice of sorts.
> >
> > Yes, that is exactly why we made it the way it is.
> >>
> >> Some other solution may just be able to solve the problem??
> >
> > Let me describe the issue in some more detail: We want to import a
> database
> > which has 3000 data elements (survey questions), most of them
> considerably
> > longer than 25 characters, and many very similar, often differing mainly
> in
> > a few characters at the end. Shortname is restricted to be required,
> unique
> > and 25 characters. One possibility is to just use the dataelementid as
> > shortname, but that of course would mean it is completely useless, just a
> > filler to enable data to be imported.
> > We are now putting a high priority on import (of hierarchies, data
> elements,
> > and data) from a spreadsheet, and the import process could perhaps
> contain
> > intelligence suggesting possible shortnames, but it is quite hard to come
> up
> > with something generic that will make sense in most cases. One strategy
> is
> > to just take the 25 first characters of the name, and if the result
> exists
> > already, substitute a number for the last two characters (alternatively
> do
> > this for all). This is sort of workable, but it would in that case be
> > desirable to also mark the data element so that someone can go through
> > manually afterwards.
> > Another option would be to the uniquenesscheck in the spreadsheet first -
> or
> > go through all dataelements with length over 25 characters manually
> before
> > importing. It is actually easier to do it in the spreadsheet than in DHIS
> 2
> > - but we don't have that import functionality yet.
> > Knut
>
> From fs/namei.c in linux kernel source:
> /*
>  * Given a valid longname, create a unique shortname.  Make sure the
>  * shortname does not exist
>  * Returns negative number on error, 0 for a normal
>  * return, and 1 for valid shortname
>  */
> static int vfat_create_shortname(struct inode *dir, struct nls_table *nls,
>                                 wchar_t *uname, int ulen,
>                                 unsigned char *name_res, unsigned char
> *lcase)
>
> The algorithm is pretty short and could probably be pretty easily
> replicated in java or perhaps better still as a spreadsheet macro.


A spreadsheet macro may be the best for coming up with plausible candidates,
though I have no idea what this particular one does.


> Seriously though, I have to agree with Saptarshi here.  The point of a
> short name is that it should be bounded.


There is no disagreement on that, the issue is what to do when importing
data from existing systems. From many perspectives, it does make sense to do
most of the required processing before importing, at least the simple.


> (I'm not sure a survey question is a suitable candidate as a data element
> name anyway but thats another issue).
>

Survey data is a very high priority for us, but you are right that the bulk
description will go in a different field. And one should probably take the
required time to change both shortnames and names when importing into DHIS,
but this has to be done by the owners of the data. After we have spreadsheet
import implemented, it will be easier to give people instructions as to how
to process their data in preparation for import. Probably, we have to create
some macros to assist in this.

I presume you won't lose the full name on import?


The full name is no problem, issue is only what to do with the shortname
field. But I do think you're right that the issue should be handled before
importing. Still, we may want to extend to 30


>  We have
> 'description' which holds an arbitrary amount of text.  So I would opt
> to import the full question in here and do some sort of fat truncation
> trick for the short names which are too long. How many of them go

over the limit?


2341

Knut

3000 is a lot but if you are talking 200 over the
> limit it might be doable to manually edit them after the fact and make
> them more meaningful.
>
> Bob
>
> >
> >>
> >> ---
> >> Regards,
> >> Saptarshi PURKAYASTHA
> >> Director R & D, HISP India
> >> Health Information Systems Programme
> >>
> >> My Tech Blog:  http://sunnytalkstech.blogspot.com
> >> You Live by CHOICE, Not by CHANCE
> >>
> >>>
> >>> Knut
> >>>
> >>> _______________________________________________
> >>> Mailing list: https://launchpad.net/~dhis2-devs
> >>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> >>> Unsubscribe : https://launchpad.net/~dhis2-devs
> >>> More help   : https://help.launchpad.net/ListHelp
> >>>
> >>
> >>
> >> _______________________________________________
> >> Mailing list: https://launchpad.net/~dhis2-devs
> >> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> >> Unsubscribe : https://launchpad.net/~dhis2-devs
> >> More help   : https://help.launchpad.net/ListHelp
> >>
> >
> >
> >
> > --
> > Cheers,
> > Knut Staring
> >
> > _______________________________________________
> > Mailing list: https://launchpad.net/~dhis2-devs
> > Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> > Unsubscribe : https://launchpad.net/~dhis2-devs
> > More help   : https://help.launchpad.net/ListHelp
> >
> >
>



-- 
Cheers,
Knut Staring