dhis2-devs team mailing list archive

Thread
Date
Re: Bulk Deletion of Data

To: Morten Olav Hansen <mortenoh@xxxxxxxxx>
From: Lorill Crees <lcrees@xxxxxxxxxx>
Date: Tue, 27 Oct 2015 12:35:52 -0700
Cc: DHIS 2 Developers list <dhis2-devs@xxxxxxxxxxxxxxxxxxx>
In-reply-to: <CAL+BWVv2cjLS6-Kj=FSd84L68Q4x1-VokB4z9UzdfSMUR_nnzA@mail.gmail.com>
Thanks for this Morten. Just to clarify the business need here, the desire
is to completely delete a dataset or a program from DHIS 2 which is not
possible when associated data value records are in the database.

With blueprint #1
https://blueprints.launchpad.net/dhis2/+spec/webapi-bulk-deletion-of-tracker-objects
it doesn't mention the deletion of the data values itself. Would the
deletion of events via the web api also cascade delete the underlying data
values?

Alternately, how would I delete single tracked entity data values via the
web API? I can't seem to find any API call specifically for these values.

And for datasets, I have tried sending a delete request for a specific data
value but I get a 406. Not sure if I am doing something wrong? I've pasted
my curl and results below for both GET and DELETE.

Thanks,

Lorill

curl "
http://localhost:8090/dhis/api/dataValues?de=YBl3JnHlR5F&pe=2015&ou=qubeTudDNSC";
-X GET -u admin:district -v

*   Trying ::1...

* Connected to localhost (::1) port 8090 (#0)

* Server auth using Basic with user 'admin'

> GET /dhis/api/dataValues?de=YBl3JnHlR5F&pe=2015&ou=qubeTudDNSC HTTP/1.1

> Host: localhost:8090

> Authorization: Basic YWRtaW46ZGlzdHJpY3Q=

> User-Agent: curl/7.43.0

> Accept: */*

>

< HTTP/1.1 200 OK

< Server: Apache-Coyote/1.1

< X-XSS-Protection: 1; mode=block

< X-Frame-Options: DENY

< X-Content-Type-Options: nosniff

< Set-Cookie: JSESSIONID=3F51C4A3F10135A648E45BAF42818AB7; Path=/dhis/;
HttpOnly

< ETag: "0ab3ca659aa5fc585fa72c6474d6b21df"

< Content-Type: application/json;charset=UTF-8

< Content-Language: en-US

< Content-Length: 10

< Date: Tue, 27 Oct 2015 19:16:29 GMT

<

* Connection #0 to host localhost left intact

["396411"]

-----------

curl "
http://localhost:8090/dhis/api/dataValues?de=YBl3JnHlR5F&pe=2015&ou=qubeTudDNSC";
-X DELETE -u admin:district -v

*   Trying ::1...

* Connected to localhost (::1) port 8090 (#0)

* Server auth using Basic with user 'admin'

> DELETE /dhis/api/dataValues?de=YBl3JnHlR5F&pe=2015&ou=qubeTudDNSC HTTP/1.1

> Host: localhost:8090

> Authorization: Basic YWRtaW46ZGlzdHJpY3Q=

> User-Agent: curl/7.43.0

> Accept: */*

>

< HTTP/1.1 406 Not Acceptable

< Server: Apache-Coyote/1.1

< X-XSS-Protection: 1; mode=block

< X-Frame-Options: DENY

< X-Content-Type-Options: nosniff

< Set-Cookie: JSESSIONID=10438B228DF6F9FEF289CA05A3E9CA5F; Path=/dhis/;
HttpOnly

< Content-Type: text/html;charset=utf-8

< Content-Language: en

< Content-Length: 1110

< Date: Tue, 27 Oct 2015 19:16:35 GMT

<

* Connection #0 to host localhost left intact

<!DOCTYPE html><html><head><title>Apache Tomcat/8.0.21 - Error
report</title><style type="text/css">H1
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;}
H2
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;}
H3
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;}
BODY
{font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;}
P
{font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A
{color : black;}A.name {color : black;}.line {height: 1px;
background-color: #525D76; border: none;}</style> </head><body><h1>HTTP
Status 406 - </h1><div class="line"></div><p><b>type</b> Status
report</p><p><b>message</b> <u></u></p><p><b>description</b> <u>The
resource identified by this request is only capable of generating responses
with characteristics not acceptable according to the request "accept"
headers.</u></p><hr class="line"><h3>Apache Tomcat/8.0.21</h3></body></html>



On Tue, Oct 27, 2015 at 1:07 AM, Morten Olav Hansen <mortenoh@xxxxxxxxx>
wrote:

> Just FYI, we have two blueprints related to this in 2.22, [1] is
> implemented already and [2] will hopefully come (but we are still
> discussing implementation details for that one)
>
> [1]
> https://blueprints.launchpad.net/dhis2/+spec/webapi-bulk-deletion-of-tracker-objects
> [2] https://blueprints.launchpad.net/dhis2/+spec/soft-event-delete
>
> --
> Morten
>
> On Tue, Oct 27, 2015 at 4:01 PM, Knut Staring <knutst@xxxxxxxxx> wrote:
>
>> My interpretation was that they would like a flag on each datavalue to
>> say whether it is still current - which I guess would require an extension
>> of that table. However, we do have something akin to this, namely the audit
>> functionality, which preserves the history of earlier (overwritten) values
>> - perhaps that could be leveraged?
>>
>> Knut
>>
>> On Tue, Oct 27, 2015 at 5:50 AM, Jason Pickering <
>> jason.p.pickering@xxxxxxxxx> wrote:
>>
>>> Hi Aaron,
>>>
>>> There is no "soft delete" function, if that is what you are after, but
>>> it has been discussed.
>>>
>>> Although there is no direct way as Lars points out, if you know what you
>>> wish to delete, it is still possible.  Namely, if you transmit a "" value
>>> for the value which you wish to delete, it will be deleted. Thus, if you
>>> wish to delete all values for a given dataset/OU/period, you can get all of
>>> the values from the API, then set them all to "", and upload the data
>>> again.  I think you can also use the DELETE verb as documented here (
>>> https://www.dhis2.org/doc/snapshot/en/developer/html/ch01s12.html) to
>>> delete a value.
>>>
>>>
>>> Not sure if that is the functionality you are after though. Typically,
>>> this would be a DB admin's role though and would be done with SQL, if you
>>> are say, going to delete ALL values for a large part of the hierarchy. The
>>> possibility for a whoopsie-moment could be large here.
>>>
>>> Regards,
>>> Jason
>>>
>>>
>>> On Mon, Oct 26, 2015 at 10:37 PM, Aaron Gladders <agladders@xxxxxxxxxx>
>>> wrote:
>>>
>>>> Hi Lars - is there a recommended approach to "deactive" data without
>>>> getting into the database itself?  If not is it worth setting up a
>>>> blueprint for it?  We may have a group interested in funding that
>>>> development.
>>>>
>>>> Thanks,
>>>>
>>>> Aaron
>>>>
>>>> On Thu, Sep 24, 2015 at 2:39 PM, Lars Helge Øverland <
>>>> larshelge@xxxxxxxxx> wrote:
>>>>
>>>>> Hi Lorill,
>>>>>
>>>>> sorry but at the moment there is no such option. So far we have
>>>>> decided not to open up for this to prevent mishaps.
>>>>>
>>>>> regards,
>>>>>
>>>>> Lars
>>>>>
>>>>>
>>>>> On Thu, Sep 24, 2015 at 11:37 PM, Lorill Crees <lcrees@xxxxxxxxxx>
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Just following up on this. Any ideas on the best way to
>>>>>> programatically bulk delete data values?
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Lorill
>>>>>>
>>>>>> On Tue, Sep 15, 2015 at 1:04 PM, Lorill Crees <lcrees@xxxxxxxxxx>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I seem to remember a prior thread on this but I can't seem to find
>>>>>>> it. We want to be able to programatically bulk delete data values (both
>>>>>>> aggregate and program/program stage data) via the Web API. Is there a way
>>>>>>> to achieve this?
>>>>>>>
>>>>>>> We don't want to have to do this on a sql level directly via the
>>>>>>> database, nor do we want to have to loop through individual data points in
>>>>>>> order to delete them.
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Lorill
>>>>>>>
>>>>>>
>>>>>>
>>>>
>>>> _______________________________________________
>>>> Mailing list: https://launchpad.net/~dhis2-devs
>>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>>> More help   : https://help.launchpad.net/ListHelp
>>>>
>>>>
>>>
>>>
>>> --
>>> Jason P. Pickering
>>> email: jason.p.pickering@xxxxxxxxx
>>> tel:+46764147049
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~dhis2-devs
>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>>
>>
>>
>> --
>> Knut Staring
>> Dept. of Informatics, University of Oslo
>> Norway: +4791880522
>> Skype: knutstar
>> http://dhis2.org
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~dhis2-devs
>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~dhis2-devs
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~dhis2-devs
> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~dhis2-devs
> More help   : https://help.launchpad.net/ListHelp
>
>
Follow ups

Re: Bulk Deletion of Data
From: Morten Olav Hansen, 2015-10-28
References

Bulk Deletion of Data
From: Lorill Crees, 2015-09-15
Re: Bulk Deletion of Data
From: Lorill Crees, 2015-09-24
Re: Bulk Deletion of Data
From: Lars Helge Øverland, 2015-09-24
Re: Bulk Deletion of Data
From: Aaron Gladders, 2015-10-26
Re: Bulk Deletion of Data
From: Jason Pickering, 2015-10-27
Re: Bulk Deletion of Data
From: Knut Staring, 2015-10-27
Re: Bulk Deletion of Data
From: Morten Olav Hansen, 2015-10-27