dhis2-devs team mailing list archive

[Bob Jolliffe <bobjolliffe@xxxxxxxxx>]

A few quick thoughts on document storage:

1.  once you have a lot of documents it becomes an interesting problem
finding them/searching them.  People have been doing this for a LONG
time and there are a couple of well understood metadata standards for
storing metadata about documents (for example Dublin Core comes to
mind).  Its a good idea to store the metadata in the database separate
from the file artefact itself - which can be on a (not necessarily
local) filesystem.  I am not sure how well this fits with the simple
document-as-special-kind-of-dataelement-value model, but it can be
streamlined.  For example, one can have a generic way of storing
document metadata (dublin core, tags etc).  Documents uploaded through
a particular route (eg photo via a dataset form element) can have its
associated metadata pre-populated.  Other documents, such as we make
available as resources, could have their metadata tagged more
flexibly.

2.  I don't like the idea of storing a value as relative file url.
That seems to paint us into a box.  Just a plain http(s) url will be
more flexible.

3.  This is maybe a deeper level of sophistication to have in mind,
and we should still consider the above two points without necessarily
considering this, but there might be some benefit to consider
integration with a document management system like alfresco
(http://docs.alfresco.com/community/concepts/welcome-infocenter_community.html).
Its a bit overkill for just the use cases described (document as
dataelement value) but as a general DMS it brings many benefits which
open new possibilities eg. google docs integration, builtin activitii
workflow engine (eg document approval but also potentially things like
care protocols), versioning etc.  And of course deals with the 2
points above by (i) providing metadata templates and (ii) identifying
documents by url.  And we wouldn't have to worry about document
storage in dhis2 - just storing the url would perhaps be fine.

There are also other systems.

In general I know programmers are smart, but it might also be good to
consult a librarian :-)  They seem to know most about these things.

On 28 July 2015 at 15:44, Morten Olav Hansen <mortenoh@xxxxxxxxx> wrote:
> I don't like that we at naming the bp document. Let's file it a file api.
>
> As much as possible I want to save the wording document until we implement a
> proper nosql document storage.
>
> We already have /api:documents which I want to remove (I doubt many external
> apps depend on it)
>
> On Tuesday, July 28, 2015, Elmarie Claasen <elmarie@xxxxxxxx> wrote:
>>
>> Hi all, there is a blueprint for this already
>>
>>
>> https://blueprints.launchpad.net/dhis2/+spec/document-dataelement-attribute-type
>>
>> Elmarie
>>
>> On 28 Jul 2015 11:27 AM, "Greg Rowles" <greg.rowles@xxxxxxxxx> wrote:
>>>
>>> Hi Halvdan
>>>
>>> Sounds great and a good alternative! I'd like to work through an example
>>> or two:
>>>
>>> We have annual performance plans made up of multiple quarterly reports so
>>> I would expect there to be 2 data sets:
>>>  - Annual Performance Plans (fYear)
>>>  - Quarterly Performance Reports (fQuarter)
>>>
>>> If we configure these data sets to include a single data element of type
>>> 'Text' what name would we give to this element? 'Document' or 'File'?
>>>
>>> We're currently experimenting with ownCloud for document storage as an
>>> interim solution. I guess nothing stops us from loading the ownCloud URL
>>> into datavalue.value?
>>>
>>> Regards,
>>> Greg
>>>
>>>
>>>
>>> On Tue, Jul 28, 2015 at 10:49 AM, Harsh Atal <harsh.atal@xxxxxxxxx>
>>> wrote:
>>>>
>>>> Hello Halvdan,
>>>>
>>>> We are planning on introducing files as a type for data values, as well
>>>> as data elements supporting this (document, images). You would then use the
>>>> datavalue model to implement these types of requirements.
>>>>
>>>> ^^This is exactly what is needed by us. I didn't know this was already
>>>> in the works!!
>>>>
>>>> May be its a bit early to ask but.....for which version is this planned
>>>> for? It will be of immense use in one of our projects.
>>>>
>>>> regards
>>>> harsh
>>>>
>>>> On 28 July 2015 at 14:10, Halvdan Grelland <halvdanhg@xxxxxxxxx> wrote:
>>>>>
>>>>> We are planning on introducing files as a type for data values, as well
>>>>> as data elements supporting this (document, images). You would then use the
>>>>> datavalue model to implement these types of requirements.
>>>>>
>>>>> The implementation is in the works but is a fairly complex beast as
>>>>> we're juggling actual files outside of the database (we consider in-db
>>>>> storage of large blobs uscalable and a generally bad idea). We're also
>>>>> planning to enable offloading this to third party cloud storage (eg amazon
>>>>> s3).
>>>>>
>>>>> I'm on holiday at the moment but please respond if you have any
>>>>> comments and I'll get back to you when I can.
>>>>>
>>>>> On 28 Jul 2015 10:02 am, "Greg Rowles" <greg.rowles@xxxxxxxxx> wrote:
>>>>>>
>>>>>> Hi All
>>>>>>
>>>>>> These are the concepts we've been thinking about:
>>>>>>
>>>>>> DocumentSet, DocumentSetLevel, Document.
>>>>>>
>>>>>>
>>>>>> [DocumentSet] would similar to data sets in that they are cyclical.
>>>>>> They may have different periodTypes (e.g. yearly, fYealy, quarterly,
>>>>>> fQuarterly, onChange) and may be compulsory. These include performance
>>>>>> plans, development plans, mandatory reporting requirements, policy documents
>>>>>> or content originating in other domains. A DocumentSet may have a design
>>>>>> template (or default document) from which users borrow layout for their
>>>>>> submissions.
>>>>>>
>>>>>> [DocumentSetLevel] is the orgunitlevel associated with DocumentSet
>>>>>> (and possibly a compulsory y/n field). This caters for mandatory reporting
>>>>>> requirements across different levels of the DHIS hierarchy.
>>>>>>
>>>>>> [Document] would be the actual object and associated meta data
>>>>>> (DocumentID, name, uid, description, owner, creationdate,
>>>>>> organisationUnitID, DocumentSetID, {startdate:enddate/periodid},
>>>>>> downloadURL, etc)
>>>>>>
>>>>>> Not sure how the actual document content would be hosted, stored or
>>>>>> even what solutions are available right now but if the following concepts
>>>>>> were available through API calls - it would make life really easy for us
>>>>>> here in SA...
>>>>>>
>>>>>> Regards,
>>>>>> Greg
>>>>>>
>>>>>>
>>>>>> On Tue, Jul 28, 2015 at 9:07 AM, Jason Pickering
>>>>>> <jason.p.pickering@xxxxxxxxx> wrote:
>>>>>>>
>>>>>>> Ah, I get the requirement now. This would be documents which would be
>>>>>>> uploaded as part of a data set submission. Yeah, this is not really
>>>>>>> currently supported as far as I can tell, but it would be a good idea.
>>>>>>>
>>>>>>> Regards,
>>>>>>> Jason
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Jul 28, 2015 at 8:37 AM, Harsh Atal <harsh.atal@xxxxxxxxx>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Hi
>>>>>>>>
>>>>>>>> Related to this we also have a similar requirement where the
>>>>>>>> functionality for document upload is required as part of normal data entry.
>>>>>>>>
>>>>>>>> Basically, a person has to visit a facility for inspection and
>>>>>>>> taking photos of the facility is part of that inspection. These photos then
>>>>>>>> have to be uploaded as a kind of appendix along with the routine data entry
>>>>>>>> of the inspection form.
>>>>>>>>
>>>>>>>>
>>>>>>>> regards
>>>>>>>> harsh
>>>>>>>>
>>>>>>>> On 28 July 2015 at 11:08, Jason Pickering
>>>>>>>> <jason.p.pickering@xxxxxxxxx> wrote:
>>>>>>>>>
>>>>>>>>> Hi Greg,
>>>>>>>>> You can always simply upload the supporting document as a resource
>>>>>>>>> and then make it available through a dashboard.
>>>>>>>>>
>>>>>>>>> Otherwise, maybe you could write a more specific blueprint?
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Jason
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Jul 27, 2015 at 8:59 PM, Calle Hedberg
>>>>>>>>> <calle.hedberg@xxxxxxxxx> wrote:
>>>>>>>>>>
>>>>>>>>>> Greg,
>>>>>>>>>>
>>>>>>>>>> As far as I know, this is just an idea that Lars and I have been
>>>>>>>>>> chatting about. I'll bring it up during the expert academy to get wider
>>>>>>>>>> inputs
>>>>>>>>>>
>>>>>>>>>> Regards
>>>>>>>>>> calle
>>>>>>>>>>
>>>>>>>>>> On 27 July 2015 at 12:33, Greg Rowles <greg.rowles@xxxxxxxxx>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Hi Devs
>>>>>>>>>>>
>>>>>>>>>>> I heard there is talk for supporting document storage as part of
>>>>>>>>>>> DHIS2 but I don't find any plans on launchpad. Can anyone confirm?
>>>>>>>>>>>
>>>>>>>>>>> Kind Regards,
>>>>>>>>>>> Greg
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Health Information Systems Program - South Africa
>>>>>>>>>>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>>>>>>>>>> Mobile  :    073 246 2992
>>>>>>>>>>> Landline:   021 554 3130
>>>>>>>>>>> Fax:          086 733 8432
>>>>>>>>>>> Skype:      gregory_rowles
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Mailing list: https://launchpad.net/~dhis2-devs
>>>>>>>>>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>>>>>>>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>>>>>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>>
>>>>>>>>>> *******************************************
>>>>>>>>>>
>>>>>>>>>> Calle Hedberg
>>>>>>>>>>
>>>>>>>>>> 46D Alma Road, 7700 Rosebank, SOUTH AFRICA
>>>>>>>>>>
>>>>>>>>>> Tel/fax (home): +27-21-685-6472
>>>>>>>>>>
>>>>>>>>>> Cell: +27-82-853-5352
>>>>>>>>>>
>>>>>>>>>> Iridium SatPhone: +8816-315-19274
>>>>>>>>>>
>>>>>>>>>> Email: calle.hedberg@xxxxxxxxx
>>>>>>>>>>
>>>>>>>>>> Skype: calle_hedberg
>>>>>>>>>>
>>>>>>>>>> *******************************************
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Mailing list: https://launchpad.net/~dhis2-devs
>>>>>>>>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>>>>>>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>>>>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Jason P. Pickering
>>>>>>>>> email: jason.p.pickering@xxxxxxxxx
>>>>>>>>> tel:+46764147049
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Mailing list: https://launchpad.net/~dhis2-devs
>>>>>>>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>>>>>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>>>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Jason P. Pickering
>>>>>>> email: jason.p.pickering@xxxxxxxxx
>>>>>>> tel:+46764147049
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Mailing list: https://launchpad.net/~dhis2-devs
>>>>>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>>>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Health Information Systems Program - South Africa
>>>>>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>>>>> Mobile  :    073 246 2992
>>>>>> Landline:   021 554 3130
>>>>>> Fax:          086 733 8432
>>>>>> Skype:      gregory_rowles
>>>>>>
>>>>>> _______________________________________________
>>>>>> Mailing list: https://launchpad.net/~dhis2-devs
>>>>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Mailing list: https://launchpad.net/~dhis2-devs
>>>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Health Information Systems Program - South Africa
>>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>> Mobile  :    073 246 2992
>>> Landline:   021 554 3130
>>> Fax:          086 733 8432
>>> Skype:      gregory_rowles
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~dhis2-devs
>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>
>> This message and any attachments are subject to a disclaimer published at
>> http://www.hisp.org/policies.html#comms_disclaimer.  Please read the
>> disclaimer before opening any attachment or taking any other action in terms
>> of this electronic transmission.  If you cannot access the disclaimer,
>> kindly send an email to disclaimer@xxxxxxxx and a copy will be provided to
>> you. By replying to this e-mail or opening any attachment you agree to be
>> bound by the provisions of the disclaimer.
>>
>>
>> This message and any attachments are subject to a disclaimer published at
>> http://www.hisp.org/policies.html#comms_disclaimer.  Please read the
>> disclaimer before opening any attachment or taking any other action in terms
>> of this electronic transmission.  If you cannot access the disclaimer,
>> kindly send an email to disclaimer@xxxxxxxx and a copy will be provided to
>> you. By replying to this e-mail or opening any attachment you agree to be
>> bound by the provisions of the disclaimer.
>
>
>
> --
> --
> Morten
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~dhis2-devs
> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~dhis2-devs
> More help   : https://help.launchpad.net/ListHelp
>