dhis2-devs team mailing list archive

[Lorill Crees <lcrees@xxxxxxxxxx>]

Hi,

Jason - thanks for heading me in the right direction for this. I'm now
looping through and deleting all related maps, charts, reportTables,
eventReports and eventCharts, and dashboard items. I know this sounds
dangerous, but the programs that the users are deleting are all either test
data or corrupt data, so keeping those artifacts around is counter
productive for them.

I'm now looking for guidance from the broader group as per my original
email of if there are other objects that I would need to take into
consideration. I'm doing this on a trial-and-error basis but there may be
some pieces of DHIS 2 functionality I haven't used that could potentially
cause deletion errors.

Please let me know if anyone has any advice.

Thanks,

Lorill

On Tue, May 30, 2017 at 12:29 PM, Lorill Crees <lcrees@xxxxxxxxxx> wrote:

> Hi Jason,
>
> Thanks for the tips. We have already developed a custom data import app in
> Grails, and have now already incorporated virtually all of the deletion
> functionality for both Data Sets and Programs via the Web API. The
> availability of this functionality via the web app is pretty key to the
> users of our system, and functionality introduced for event and tracked
> entity instance deletion in version 2.25 seemed to be the missing piece to
> make this possible.
>
> We have been told that it is not really viable for the users' workflow to
> request an administrator to delete via sql queries, although agreed it may
> be the more prudent option in regards to preventing mistakes. This approach
> was rejected by our users.
>
> That said, I'll take a look at the reportTables endpoint to see if that
> can help. If not, the users may just need to do some manual deletion steps
> via the DHIS 2 UI prior to deleting via our app, which would not be ideal.
>
> Thanks again for your help,
>
> Lorill
>
> On Tue, May 30, 2017 at 11:56 AM, Jason Pickering <
> jason.p.pickering@xxxxxxxxx> wrote:
>
>> Hi Lorill,
>> I am not too sure off-hand exactly how to do that. I  suppose maybe you
>> could loop through all of the available report tables or craft an API query
>> to determine which report tables and other artifacts need to be deleted.
>> These should be available through a report tables endpoint like
>> https://play.dhis2.org/dev/api/reportTables/qfMh2IjOxvw
>>
>> I adopted a slightly different route, and created an SQL function to
>> delete any trace of an organisation unit, simply because it was much
>> easier. This approach is of course also problematic, but you might want to
>> consider for efficiencies sake this approach. You can find an
>> implementation here for orgunits (https://github.com/dhis2/dhis
>> 2-utils/blob/master/resources/sql/delete_site_with_data.sql) but I think
>> something similar could be developed for a program.
>>
>> Of course, this probably should be easier, but there is also some
>> downside to making this operation too easy, as one false move could result
>> in the deletion of a lot of artifacts. Thus, having an SQL function which
>> can only be  run by an administrator, may be in fact an attractive option
>> to minimize the possibility of a mistake.
>>
>> Regards,
>> Jason
>>
>>
>> On Tue, May 30, 2017 at 5:42 PM, Lorill Crees <lcrees@xxxxxxxxxx> wrote:
>>
>>> Hi Jason,
>>>
>>> How can I programatically determine this via the Web API? Are
>>> DataDimensionItems related to favourites? If so, I had asked the dev list
>>> about the querying and deletion of favourites via the Web API back in
>>> February and had created these three JIRA tasks:
>>>
>>> https://jira.dhis2.org/browse/DHIS2-1174 - Show more meaningful message
>>> to user when a favourite cannot be loaded due to missing data elements
>>> https://jira.dhis2.org/browse/DHIS2-1175 - Add ability to query for
>>> Favourites by their corresponding data elements and indicators via the
>>> API
>>> https://jira.dhis2.org/browse/DHIS2-1176 - Add ability to delete
>>> Favourites via the API
>>>
>>> From my limited testing it looked like the inclusion in a favourite
>>> didn't prevent the deletion of data elements, but only made the favourite
>>> invalid.
>>>
>>> Any other help on this issue would be appreciated as I'm now blocked in
>>> being able to complete the functionality to delete programs.
>>>
>>> Thanks,
>>>
>>> Lorill
>>>
>>> On Tue, May 30, 2017 at 12:02 AM, Jason Pickering <
>>> jason.p.pickering@xxxxxxxxx> wrote:
>>>
>>>> Hi Lorill,
>>>>
>>>> Is the data element linked to a chart, pivot or map?
>>>>
>>>> Regards,
>>>> Jason
>>>>
>>>>
>>>> On Mon, May 29, 2017 at 9:58 PM, Lorill Crees <lcrees@xxxxxxxxxx>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Further to this, for some of our programs we are unable to delete
>>>>> programdataelements via the Web API due to the following error:
>>>>>
>>>>> ERROR: update or delete on table "programdataelement" violates foreign
>>>>> key constraint "fk_datadimensionitem_programdataelementid" on table
>>>>> "datadimensionitem" Detail: Key (programdataelementid)=(541878) is
>>>>> still referenced from table "datadimensionitem".
>>>>>
>>>>> What do I need to do to be able to delete these programdataelements? I
>>>>> have already deleted all the events and tracked entity instances prior to
>>>>> deleting programdataelements. I have even tried rerunning analytics prior
>>>>> to deleting.
>>>>>
>>>>> Please let me know. Thanks,
>>>>>
>>>>> Lorill
>>>>>
>>>>> On Fri, May 26, 2017 at 2:33 PM, Lorill Crees <lcrees@xxxxxxxxxx>
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> We have been working on functionality to delete full Programs and all
>>>>>> associated data and metadata via the Web API using:
>>>>>>
>>>>>> Version:
>>>>>> 2.25
>>>>>> Build revision:
>>>>>> ee7e61b
>>>>>> Build date:
>>>>>> 2017-05-19 12:47
>>>>>>
>>>>>> The instance being used is assuming that attributes and data elements
>>>>>> are specific to a particular program/program stage and will not be reused
>>>>>> across programs.
>>>>>>
>>>>>> I wanted to check to see if there are any other objects we need to
>>>>>> worry about for deletion that we may be missing.
>>>>>>
>>>>>> This is our methodology in order of execution, all via the Web API:
>>>>>>
>>>>>>    1. Delete Program and Program Stage Data
>>>>>>       1. delete all program stage events
>>>>>>       2. delete all tracked entity instances
>>>>>>    2. Delete Program and Program Stage Metadata
>>>>>>       1. delete all related program indicators
>>>>>>       2. delete all related program rules
>>>>>>       3. delete all related program rule variables (I'm not sure why
>>>>>>       these aren't deleted automatically via the program rule deletion)
>>>>>>       4. delete programDataElements (is it correct that these are
>>>>>>       generated by analytics?)
>>>>>>       5. delete all related indicators for programStage data elements
>>>>>>       6. delete underlying data elements
>>>>>>       7. delete related option sets and related options
>>>>>>       8. delete programTrackedEntityAttributes
>>>>>>       9. delete related trackedEntityAttributes
>>>>>>       10. delete related option sets and related options
>>>>>>    3. Delete Program and Program Stages
>>>>>>       1. delete program stages
>>>>>>       2. unassign program from all user roles (we note that the
>>>>>>       dataset deletion API call takes care of this step but the program deletion
>>>>>>       API call doesn't and throws an error if we miss this step)
>>>>>>       3. delete program
>>>>>>
>>>>>> I see some things in the program payload such as:
>>>>>>
>>>>>>    - "notificationTemplates": [],
>>>>>>    - "translations": [],
>>>>>>    - "userGroupAccesses": [],
>>>>>>    - "attributeValues": [],
>>>>>>    - "validationCriterias": [],
>>>>>>
>>>>>> Do any of these need to be explicitly deleted in order to delete a
>>>>>> program or will they cascade? Are there any other associated objects I
>>>>>> haven't mentioned that we need to delete? So far we've been implementing on
>>>>>> a trial and error basis.
>>>>>>
>>>>>> Please let me know if there are any other considerations we should be
>>>>>> incorporating into our logic.
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Lorill
>>>>>> --
>>>>>> Lorill Crees
>>>>>> Project Leader / Senior Developer
>>>>>> 2Paths Solutions Ltd. <http://www.2paths.com>
>>>>>>
>>>>>> lcrees@xxxxxxxxxx
>>>>>> skype: lorill2paths
>>>>>> (604) 689-4123 x 15 <(604)%20689-4123>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Lorill Crees
>>>>> Project Leader / Senior Developer
>>>>> 2Paths Solutions Ltd. <http://www.2paths.com>
>>>>>
>>>>> lcrees@xxxxxxxxxx
>>>>> skype: lorill2paths
>>>>> (604) 689-4123 x 15 <(604)%20689-4123>
>>>>>
>>>>> _______________________________________________
>>>>> Mailing list: https://launchpad.net/~dhis2-devs
>>>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Jason P. Pickering
>>>> email: jason.p.pickering@xxxxxxxxx
>>>> tel:+46764147049 <+46%2076%20414%2070%2049>
>>>>
>>>
>>>
>>>
>>> --
>>> Lorill Crees
>>> Project Leader / Senior Developer
>>> 2Paths Solutions Ltd. <http://www.2paths.com>
>>>
>>> lcrees@xxxxxxxxxx
>>> skype: lorill2paths
>>> (604) 689-4123 x 15 <(604)%20689-4123>
>>>
>>
>>
>>
>> --
>> Jason P. Pickering
>> email: jason.p.pickering@xxxxxxxxx
>> tel:+46764147049 <+46%2076%20414%2070%2049>
>>
>
>
>
> --
> Lorill Crees
> Project Leader / Senior Developer
> 2Paths Solutions Ltd. <http://www.2paths.com>
>
> lcrees@xxxxxxxxxx
> skype: lorill2paths
> (604) 689-4123 x 15 <(604)%20689-4123>
>



-- 
Lorill Crees
Project Leader / Senior Developer
2Paths Solutions Ltd. <http://www.2paths.com>

lcrees@xxxxxxxxxx
skype: lorill2paths
(604) 689-4123 x 15