dhis2-devs-core team mailing list archive

[Bob Jolliffe <bobjolliffe@xxxxxxxxx>]

OK cross purposes resolved ...

I think Jason you start to get the heart of the matter over permission,
auditing and publication.

One of the problems I see with the current proposal is to do with
permission.  By allowing deletion via "importing" we cannot easily
distinguish between the authority to enter data and the authority to delete
data.  That I think is possibly problematic and is what I was thinking
would be solved with a DELETE method.  In principle the same could apply to
a csv payload.  Except ....

Whereas POST (and PUT) imply an enclosed payload, I believe DELETE simply
identifies a resource thru the url.  So one could delete a (slightly
mythical) datavalueset or a datavalue, but not very practical for a whole
bundle of values (not consisting of a dataset).  In which case we are
forced to look instead at a more SOAP-ish RPC type mechanism in combination
with the POST (like we do with import strategy - perhaps DELETE needs to be
an optional import strategy).

Thinking further, currently someone with authority to import already has
authority to overwrite data, so having additional authority to delete isn't
actually that different.  Though no less problematic.

It strikes me that we cannot easily escape the notion of publication.
Whereas our datavalue table (and related event table) maybe a slightly
wild, fluid and untamed space, once we "publish" data from that space (eg
donor or any other type of official reporting), something must get fixed
somewhere.  Either the published data itself gets preserved (eg in a
separate table or system) or we mark it somehow as unalterable in the
source.  Jim might have some thoughts on this as I recall he is addressing
similar problem with the PEPFAR reporting though I don't recall the detail
of the implementation (and I missed the presentation).

It is quite important that we get a reasonable and justifiable semantics in
place here as we are currently in the process of trying to shoehorn our dxf
and related api through IHE. So agree that we need to have a bit more
discussion on this.

Bob

On 4 October 2014 11:22, Jason Pickering <jason.p.pickering@xxxxxxxxx>
wrote:

> Hi Bob,
> Yes, I think we likely are, which I think shows we need more thinking on
> this issue.
>
>  I think I also do not really know what NULL in DXF means either. Lars, is
> it a ,"", or ,, or ,"NULL", ?
>
> While I see the utility of using "DELETE" to perform this operation, it
> does not solve the more urgent problem of how to delete existing values
> when importing through CSV from one DHIS2 instance to the other. It also
> raises some interesting questions about when a value would should be
> allowed to be UPDATED/DELETED. Coming back to our friend CRIS, they had
> implemented the concept of a soft delete, so that deletes could be
> transmitted explicitly in XML, without having to infer it from something
> like a NULL or rely on possibly unknown rules like zero significance in
> upstream systems to take care of this. The data values in their system were
> thus never deleted but rather marked as deleted.  Maybe DHIS 1.4 had
> something like this as well? But the other question here is that even if a
> downstream system sends you a NULL or a zero or a request to delete a
> record, do you choose to accept it? My thinking here is that I know for
> instance with PSI, they say that data cannot be changed after it has been
> submitted to the donor. Such restrictions may also exist with PEPFAR. So,
> considering if a downstream system wants to change a data value or delete
> it, do you as the upstream system choose to accept it? Right now, we have
> favored speed of import over security of the data, but it might need to be
> something we should look into.
>
> Drifting here Bob, but maybe I am no longer talking at cross-purposes? :)
>
> Regards,
> Jason
>
>
> On Sat, Oct 4, 2014 at 12:49 AM, Bob Jolliffe <bobjolliffe@xxxxxxxxx>
> wrote:
>
>> Jason maybe we talk at cross purposes. What you describe makes perfect
>> sense. If a server imports a zero or any other value it is reasonable to
>> expect that to replace the existing value. And if that server does not
>> store zeroes, then deleting what is there seems intuitively correct.
>>
>> But that is not what the blueprint refers to. It suggests the importing
>> of nulls as a general mechanism for deletion. I think that is counter
>> intuitive, a little dangerous and also unclear what is null wrt dxf.
>>
>
>
>
> --
> Jason P. Pickering
> email: jason.p.pickering@xxxxxxxxx
> tel:+46764147049
>