dhis2-devs team mailing list archive

[Halvdan Hoem Grelland <halvdan@xxxxxxxxx>]

This should now be fixed in latest 2.25, 2.24 and 2.23 (currently building
on CI, war-file available within the hour).

Anyone using Tracker with File data values will need to upgrade. Please let
me know if you have any more issues.

On Mon, Jul 25, 2016 at 5:50 PM, Halvdan Hoem Grelland <halvdan@xxxxxxxxx>
wrote:

> To update everyone: this is definitely a bug with this stuff when using
> FileResource with Tracker (2.23 -> ). I'm currently working on a fix, but
> it will take some time.
>
> It hopefully won't take long, but as it's a pretty critical issue to have
> in production you will need to work around it in the mean time. The least
> intrusive way is to run a cronjob at just before the cleanup task triggers
> (2:00 server time) and update all fileresource rows in the DB with
> isassigned=TRUE. Alternately a patch could be applied (comments out a
> single line to disable the deletion/cleanup task). Let me know if you need
> any more details.
>
>
> On Mon, Jul 25, 2016 at 2:06 PM, Halvdan Hoem Grelland <halvdan@xxxxxxxxx>
> wrote:
>
>> Ok.
>>
>> That task *should* detect *unreferenced* fileResource objects and delete
>> them. If they are referenced, however (i.e. there is a data value
>> referencing their UID) they should not be deleted. If the uid in your
>> example is in fact stored in a data value with a FR data element there is a
>> bug, if not it works as intended (to prevent orphaned files).
>>
>> Could you check if the 'assigned' field of the FR is true after creating
>> a new event with a FR data elements? If not, there is a problem in the
>> server side event creation logic.
>>
>> There is no way to disable the task (also, you really shouldn't).
>>
>> On Mon, Jul 25, 2016 at 1:37 PM, Mike Nelushi <mikeevolution@xxxxxxxxx>
>> wrote:
>>
>>> Hi Halvdan,
>>>
>>> We want the user to upload the file and be able to retrieve it but the
>>> files are deleted(purged) at 2AM everyday.
>>> * INFO  2016-07-25 02:00:00,072 'system-process' delete
>>> org.hisp.dhis.fileresource.FileResource, name:
>>> PERSONAL_CONSIDERATION_SUBMISSION. Samuel Mur 7910025121081.pdf, uid:
>>> ko0TqodGcPh (AuditLogUtil.java [taskScheduler-4])
>>> * INFO  2016-07-25 02:00:00,082 'system-process' delete
>>> org.hisp.dhis.fileresource.FileResource, name: Craig Anders Personal
>>> Consideration.pdf, uid: ShiqhAsXc78 (AuditLogUtil.java [taskScheduler-4])
>>> * INFO  2016-07-25 02:00:00,096 'system-process' delete
>>> org.hisp.dhis.fileresource.FileResource, name: Motivation for Comm -
>>> Copy.pdf, uid: L1fLHOSqwkM (AuditLogUtil.java [taskScheduler-4])
>>> * INFO  2016-07-25 02:00:00,102 'system-process' delete
>>> org.hisp.dhis.fileresource.FileResource, name: SATA - TOUCH WORLD CUP 2015
>>> - SA MIXED TEAM CONFIRMATION.pdf, uid: EWQXKQsCunv (AuditLogUtil.java
>>> [taskScheduler-4])
>>> * INFO  2016-07-25 02:00:00,108 'system-process' delete
>>> org.hisp.dhis.fileresource.FileResource, name: N_Isaacs_WPTA_July2016.pdf,
>>> uid: AmoMiCsoSvD (AuditLogUtil.java [taskScheduler-4])
>>> *.....*
>>>
>>> How do we disable this task from running?
>>>
>>> Regards,
>>> Mike
>>>
>>> On Mon, Jul 25, 2016 at 1:27 PM, Halvdan Hoem Grelland <
>>> halvdan@xxxxxxxxx> wrote:
>>>
>>>> This should be fine. Are you experiencing that it doesn't work as
>>>> expected? (I didn't consider the tracker use case when I wrote 'dataset'.
>>>> My bad...)
>>>>
>>>> Calle: your assumptions are correct. DV to FR is one-to-one. You would
>>>> need a DE per specific file for your event/dataset, or alternately package
>>>> multiple files in some archive format and store that instead (you could
>>>> handle this on the client side).
>>>>
>>>> What if you want multiple data value records to reference the same
>>>>> document [...]
>>>>>
>>>>
>>>> A FR data value is just a data value with a weak reference to a file.
>>>> Grouping data elements is no different than for straight numeric or text
>>>> values. That is, there is no 'link' between the fields, but you could use a
>>>> construct such as a data set (aggr) or programstageinstance/event (tracker)
>>>> to group DEs.
>>>>
>>>> On Mon, Jul 25, 2016 at 1:02 PM, Mike Nelushi <mikeevolution@xxxxxxxxx>
>>>> wrote:
>>>>
>>>>> Hi Halvdan,
>>>>>
>>>>> The following are the steps I'm using:
>>>>> 1. Save the file on the api/fileResource endpoint, get the UID back,
>>>>> then save the uid as of a dataValue on the event. Example:
>>>>>
>>>>>     POST to api/fileResources, get id: “a11111bbbb” back.
>>>>> 2. PUT the event:
>>>>>
>>>>> {
>>>>>   "event" : "<some-event-uid>",
>>>>>   "dataValues" : [
>>>>>     {
>>>>>       "value" : "a11111bbbb",
>>>>>       "dataElement" : "<some-fileResource-dataelement-uid>"
>>>>>     }
>>>>>   ]
>>>>> }
>>>>>
>>>>> Are these steps correct?
>>>>> Are the datasets required in these case? If yes, how should I go about
>>>>> doing this in the tracker module.
>>>>>
>>>>> Regards,
>>>>> Mike
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Jul 25, 2016 at 11:53 AM, Halvdan Hoem Grelland <
>>>>> halvdan@xxxxxxxxx> wrote:
>>>>>
>>>>>> Hi Mike,
>>>>>>
>>>>>> As is explained in the docs, the FR must be referenced by a data
>>>>>> value. If the FR is referenced it will not be deleted.
>>>>>>
>>>>>> The FR object is not usable by itself, so the upload to the
>>>>>> api/*/fileResources is step 1 of 2 in the process of uploading a file (it
>>>>>> needs to be tied to a DV afterwards).
>>>>>>
>>>>>> In practice this means that in order to use FR you need to create
>>>>>> data elements (and datasets) to contain your files.
>>>>>>
>>>>>> Hope that answers your question. If not, please let me know.
>>>>>>
>>>>>> On Mon, Jul 25, 2016 at 7:57 AM, Mike Nelushi <
>>>>>> mikeevolution@xxxxxxxxx> wrote:
>>>>>>
>>>>>>> Hi Halvdan/Morten,
>>>>>>>
>>>>>>> There seem to be a cleanup task which is ran at 2AM every 24H which
>>>>>>> necessary maintenance which removes orphaned files (FileResources) which
>>>>>>> have been posted to the instance but are marked for deletion as Halvdan
>>>>>>> explained here
>>>>>>> <http://dhis-developers-list.1563109.n2.nabble.com/Re-Dhis2-users-Nightly-Batch-Jobs-td7604891.html>.
>>>>>>> And from File resource constraints explained *here
>>>>>>> <http://dhis2.github.io/dhis2-docs/2.23/en/developer/html/ch01s12.html#d0e2396> "Any
>>>>>>> file resources which are in this state and are older than two hours will be
>>>>>>> marked for deletion and will eventually be purged from the system." *
>>>>>>>
>>>>>>> Our new application allows users to upload files and would like to
>>>>>>> prevent this from happening.
>>>>>>> How do we stop the system from running this file deletion task or
>>>>>>> tag the files to not be deleted?
>>>>>>>
>>>>>>> Please advice.
>>>>>>>
>>>>>>> Regards,
>>>>>>> Mike
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Halvdan Hoem Grelland
>>>>>> Software developer, DHIS 2
>>>>>> University of Oslo
>>>>>> http://www.dhis2.org <https://www.dhis2.org/>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Halvdan Hoem Grelland
>>>> Software developer, DHIS 2
>>>> University of Oslo
>>>> http://www.dhis2.org <https://www.dhis2.org/>
>>>>
>>>>
>>>
>>
>>
>> --
>> Halvdan Hoem Grelland
>> Software developer, DHIS 2
>> University of Oslo
>> http://www.dhis2.org <https://www.dhis2.org/>
>>
>>
>
>
> --
> Halvdan Hoem Grelland
> Software developer, DHIS 2
> University of Oslo
> http://www.dhis2.org <https://www.dhis2.org/>
>
>


-- 
Halvdan Hoem Grelland
Software developer, DHIS 2
University of Oslo
http://www.dhis2.org <https://www.dhis2.org/>