dhis2-devs team mailing list archive

[Bob Jolliffe <bobjolliffe@xxxxxxxxx>]

Ola,

I am sorry I have set you off on this path of datasets with common
categorycombos but I am now having doubts .... see below.

 2009/12/10 Ola Hodne Titlestad <olatitle@xxxxxxxxx>

> Hi,
>
> I have brought up this discussion a couple of times before and good to see
> it moving now.
>
> I like the idea of allowing multiple catcombos in one data entry form, and
> I think this should also go for default non-custom forms, as in many cases
> it would be a matter of listing multiple tables on a page, one table per
> catcombo.
>
> We are facing exactly this challenge now that we are working on revising
> the Sierra Leone database for 2010. We want to use multidimensional data
> elements as much as possible and that typically means having 1-5 catcombos
> per paper form. At the same time we want this database to be compatible with
> the new SDMX-HD reporting so that it can easily work together with OpenMRS
> and other software. Then we need to make sure that our datasets only have 1
> catcombo, if not the sdmx export will have trouble finding and structuring
> the data. Our problem then becomes how to create data entry forms that look
> exactly like the paper forms when such a form typically will have more then
> 1 catcombo? So we are under a bit of pressure here to be able to come up
> with such forms before they start entering data for 2010 in beginning of
> February next year.
>
> I agree with Bob's view that a data entry form is the combined thing, what
> you see and where you enter data, typically the same as the paper form. In
> order to support multiple catcombos (which to me means multiple datasets),
> then I think it is natural to be able to link a data entry form to n data
> sets. And a dataset can be used in multiple forms, that still needs to be
> supported. The datasets become the crucial database design part, where we
> define how data is modelled in the DHIS. The forms as I see it are simply
> views on top of that which can make data entry easier and be more like the
> paper experience.
>

After considerable reflection I have softened in some respects on this on
hardened in others.  I do think we need to have a notion of a set of
reported data - whether that data has arrived through a data entry
screen/form or sucked in from the ether.  That is a particular set of
datavalues rather than a set of dataelements.  The Dataset has always been
approximately (but not quite) this abstraction - I guess I am talking about
a Dataset instance - a ReportedDataSet.  I think this is in some ways
similar to our completedDataSetRegistration.

But I have also come to understand the fact that SDMX-HD datasets are
obliged to be of the same dimensionality as being a limitation of SDMX and
not necessarily the shining example of best practice in our field.  When we
read in data (say an sdmx package from openmrs which might be a group of
SDMX datasets) its not really that important that we have to match 1-1 those
SDMX datasets onto DHIS datasets.  In fact there is no loss of meaningful
information if they all go into the same DHIS dataset.  As long as we can
pull back out values byCategoryCombo() then that is fine.

And I suspect the same thing is true in reverse - when we are reporting data
rather than receiving reported data.  That is a more complicated problem
which I haven't fully got my head around yet (its different because we would
be dealing mostly with aggregated indicator values with no categorycombos)
but again DHIS just needs to produce the reported data set.  If that data
has to be subdivided further to parcel into SDMX messages that is a
downstream issue which shouldn't necessarily affect constraints on our
model.

So what about the forms/screens?  Well I think we must cleanly separate
model-view issues.  The requirement to group dataentry elements in different
sized tables is primarily a UI issue which should be taken care of at the UI
layer.  The UI might have screens with forms or screens with sections or
what have you.  And those sections might have a common dimensionality
requirement but I don't think this has necessarily got to do with how we
model datasets.  Currently I think sections are a property of the DataSet -
the SectionManager has getSectionsByDataSet() for example.  This I am now
suspecting is the wrong approach.  One data set for reporting instrument
(whether we call that a screen or a form is immaterial as long as we agree
on something) makes sense.  And each section on that form should have a
requirement that only dataelements from the dataset which share the same
dimensionality are eligle for inclusion in that section.  But these are
sections on the form not anything structurally meaningful to the DataSet.  I
think this can be enforced without fragmenting our current model.

So in summary (my list):
1.  discount my earlier concerns about sdmx datasets - they are not the same
as dhis datasets and we can workaround the difference.
2.  think about something to represent an actual set of reported (electronic
or screen-entered) values.  There is metadata regarding this from sdmx which
we might currently have to discard.  It might be a refinement of
completedDataSetRegistartion.
3.  sections on a form should have the constraint that they only allow
dataelements of similar dimensionality.
4.  If the UI (the web forms) requires a breakdown of datasets by
categorycombo (or dimensionality) for the purpose of layout then keep that
mechanism in the view layer.  Sometimes the view needs its own micro-model
...

Bob.




> It seems we are thinking along the same lines, just use different names for
> the various parts.
>
> I think a group of data elements sharing the same catcombo should be a
> dataset, as this also best matches the terminology of SDMX-HD where one
> dataset will contain data that share the same dimensions.
>
> *Whatever the names we go for my main requirements are:
> *1) need for an artefact that groups together data elements that share the
> same category combo and also define the allowed periodicity (periodType) for
> these data elements
> 2) when entering data it should be possible to see multiple tables with
> different dimensions in the same form
> - these tables should be generated by default (like today for 1 catcombo)
> from catcombos without a need to design them manually
> 3) it should be able to make minor adjustments to what the default data
> entry form looks (not completely custom form) by e.g. adding headings for
> each table (catcombo), selecting which fields to grey out (make inactive),
> freely select whether to put data element or catoptions on columns/rows
> (pivot the table)
> 4) Like today offer a custom form designer with full freedom to link to
> data elements+catoptioncombos from multiple catcombos
> 5) When exporting data we again need easily to be able to pick data
> elements that share the same category combo ("sdmx datasets")
> 6) In report tables we need to filter by category combos and get options
> from these (already implemented)
>
> It also seems like this improvement would be equally important to both
> routine and patient data.
>
> Ola
> ----------
>
>
>
> I like the idea of separating forms (or screens) from data sets and to
> allow multiple data sets (or groups of data elements with the same catcombo)
> in one form.
>
>
> 2009/12/10 Bob Jolliffe <bobjolliffe@xxxxxxxxx>
>
>> Hi Abyot
>>
>> I like your suggestion very much but want (again I know!) to suggest a
>> slightly different slant. I tend to think of a form as being the composite
>> thing (like a paper form which might even have a reference number - form
>> ANC-B3).  And that form might have different screens and/or sections.
>> Rather than the screen having multiple forms.  Just a matter of naming I
>> know, but if we are going to refactor like this, we do need to think about
>> what is the least confusing to users.
>>
>> Otherwise I am fully behind the suggestion of subsetting by common
>> categorycombos - it does make sense.  Possibly we need to implement a
>> getDataElementsByCategoryCombo() method on DataSet to make it easier to
>> select appropriate dataelements.
>>
>> Regards
>> Bob.
>>
>>
>> 2009/12/10 Abyot Gizaw <abyota@xxxxxxxxx>
>>
>>>  Hi Bharath,
>>>
>>> What I wanted to say is - think of separating dataentry screens from
>>> dataentry forms they shouldn't be the same. A dataentry screen contains
>>> dataentryforms. And there shouldn't be any object called dataentryScreen but
>>> only dataentryForm. Dataentry screen is more of a presentation and something
>>> to be implemented using action classes at the web layer.
>>>
>>> When you have a dataset having dataelements with different category
>>> combos then difficult to display them using a single table. So what I am
>>> suggesting is to group these dataelements (those having the same catcombo)
>>> and create them dataentryform.... like this a dataset can have have multiple
>>> dataentry forms. DataEntry form is to have a group of dataelements all
>>> sharing same categorycombo. Finally the dataset is free to have a set of
>>> dataforms, but only one dataentryscreen. I have attached you a sample form
>>> ... we need to handle such a requirement.
>>>
>>> Thank you
>>> Abyot.
>>>
>>> On Thu, Dec 10, 2009 at 6:11 AM, bharath kumar <chbharathk@xxxxxxxxx>wrote:
>>>
>>>>
>>>> Hi Abyot,
>>>>
>>>> Sorry I didn't understand clearly.
>>>>
>>>> You are saying one dataset say dataset1 can have multiple dataentry
>>>> screens like dataentryscreen1,dataentryscreen2,dataentryscreen3.
>>>>
>>>> When we select dataset1 in the dataentry module we should display these
>>>> 3 screens for dataentry?
>>>>
>>>> These 3 screens can have same dataelements or different dataelements?
>>>>
>>>>
>>>>
>>>> On Wed, Dec 9, 2009 at 9:53 PM, Abyot Gizaw <abyota@xxxxxxxxx> wrote:
>>>>
>>>>> and also assigning multiple forms for datasets or programstages. For
>>>>> example a dataset might have dataelements having different
>>>>> categorycombos.... for such cases it makes sense to group those datalements
>>>>> having the same categorycombo together then create a form.... in the end
>>>>> such datasets will have a set of forms. So when displaying the dataentry
>>>>> screen we can display a list of forms and come up a nice looking dataentry
>>>>> screen.
>>>>>
>>>>> abyot.
>>>>>
>>>>>
>>>>> On Wed, Dec 9, 2009 at 5:59 AM, bharath kumar <chbharathk@xxxxxxxxx>wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>>
>>>>>> Regarding DataEntryScreen, as we (Abyot & Me) discussed on his last
>>>>>> visit we agreed to make CustomDataEntryScreen as Genric.
>>>>>>
>>>>>> For that what we started is:
>>>>>>
>>>>>> We changed DataEntryForm object to contain its id, name and html code
>>>>>> only. (before it was bound with datasetid).
>>>>>> All the CustomScreen associations we kept in another table *
>>>>>> dataentryformassociation* which contains associationtablename(
>>>>>> represents which object DataSet/ProgramStage etc), associationid (
>>>>>> represents datasetid/programstage id), dataentryformid.
>>>>>>
>>>>>> *From the GUI:*
>>>>>> If you goto dataset screen there will be icon to design custom screen
>>>>>> for dataset like before. only change is before it was saving in
>>>>>> dataentryform table. now it will save the design in dataentryform table and
>>>>>> association will be saved in dataentryformassociation table.
>>>>>>
>>>>>> Similarly for ProgramState also we can design screens.
>>>>>>
>>>>>> Breifly that is what we are doing regarding CustomScreens. Is it OK or
>>>>>> any changes?
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2009/12/9 Lars Helge Øverland <larshelge@xxxxxxxxx>
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> 2009/12/8 Viet Nguyen <phamquocviet@xxxxxxxxx>
>>>>>>>
>>>>>>> Hi Lars  and  Abyot,
>>>>>>>>
>>>>>>>> I am sorry, the message of that commit was missing some information.
>>>>>>>>
>>>>>>>>
>>>>>>>> I had some problem while committing the source. It was locked, and I
>>>>>>>> tried to use the break-lock command many times and finally it worked. That
>>>>>>>> is why I did not write the full commit message.
>>>>>>>>
>>>>>>>> So, that commit include
>>>>>>>>
>>>>>>>>    - Custom DataEntryForm design form for ProgramStage ( this is
>>>>>>>>    what the FCK editor is for ).
>>>>>>>>    - I Added class DataEntryFormAssociation. And yes Abyot, when I
>>>>>>>>    was creating that class, I also wanted to create another package for all the
>>>>>>>>    dataentryform things. But that was like just a few days I touch the dhis
>>>>>>>>    code and I did not want to change so much. I just tried to follow the old
>>>>>>>>    work flow. Please do not worry, after read your email, I changed it
>>>>>>>>    immediately (  really happy to do that :)  ) . You will see it in the next
>>>>>>>>    commit.
>>>>>>>>    -  Custom data entry screen for case ( patient ) .  This is not
>>>>>>>>    completely done, I am still working on it.
>>>>>>>>
>>>>>>>> The reason for this commit is to merge my local changes with your
>>>>>>>> trunk. Because I have been coding for about two weeks on this module with my
>>>>>>>> local source code, not on any branch...so it really need to be merged as
>>>>>>>> soon as possible.
>>>>>>>>
>>>>>>>> And actually , I have not been  in the
>>>>>>>> dhis2-devs@xxxxxxxxxxxxxxxxxxx util Abyot send me and Bharath the
>>>>>>>> email. I have just subscribed to the list yesterday. I did not know about
>>>>>>>> that mailing list. When you added me to the core-dev team, I looked for
>>>>>>>> mailing list of this team, but I could not.
>>>>>>>>
>>>>>>>> About Abyot 's question :
>>>>>>>>
>>>>>>>> The other thing, if you remember we even talked about having a set
>>>>>>>> of forms for datasets/programstages. Because sometimes you might have
>>>>>>>> multiple tables/sections. Do you think you can include these things?
>>>>>>>>
>>>>>>>> Sorry, I do not know about this... Please explain it again if you
>>>>>>>> have time.
>>>>>>>>
>>>>>>>> Thank you,
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> Thanks Viet for your reply.. Don't misunderstand, we're happy to see
>>>>>>> u contributing and I'm exited to see the new code/functionality. Its just
>>>>>>> important that Abyot understands whats going on and gets the change to
>>>>>>> review things as he is leading the patient system work... Now that you have
>>>>>>> enlisted for the dev list that's will be easier:-) Tell me if you there is
>>>>>>> other stuff u are wondering about.
>>>>>>>
>>>>>>> cheers Lars
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> Regards,
>>>>>> Bharath Kumar. Ch
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Regards,
>>>> Bharath Kumar. Ch
>>>>
>>>
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~dhis2-devs<https://launchpad.net/%7Edhis2-devs>
>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~dhis2-devs<https://launchpad.net/%7Edhis2-devs>
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>>
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~dhis2-devs<https://launchpad.net/%7Edhis2-devs>
>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~dhis2-devs<https://launchpad.net/%7Edhis2-devs>
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>