dhis2-devs team mailing list archive

[Jason Pickering <jason.p.pickering@xxxxxxxxx>]

For me the issue here is choosing the right tool. I do not know much about
Camel, but it seems like a good tool. There are others which perform
similar things like Pentaho and Kettle. Personally (as you have probably
noted from my previous posts on this list) I tend to personally prefer a
combination of R and SQL. We run some small tasks, like trimming data
element and orgunit names on a regular basis. In my mind, there is no need
for a heavy hammer for this. It is a single SQL statement, and can be
easily scheduled on a regular basis to run with cron (or pgagent). Other
tasks are more suited to things that R is good at, such as more complex
data manipulation  Since R can work easily with both SQL as well as the web
API (and can be automated to execute from the shell) I have never seen a
need for Camel, but again, I think it is a matter of which tools you prefer
to get the job done and are most familiar with.

Personally, I think it makes sense to remove this from the core build. It
makes sense to continue to document how it Camel can be used with DHIS2,
but in the long run, I think the loose coupling of the two probably makes a
lot of sense.

Regards,
Jason



On Tue, Oct 1, 2013 at 4:24 PM, Bob Jolliffe <bobjolliffe@xxxxxxxxx> wrote:

> Note that the danger of using sql directly is not so much securing a cron
> executed script (that can be done reasonably well), but more that it is
> quite easy to mess up the metadata integrity.
>
>
> On 1 October 2013 15:19, Bob Jolliffe <bobjolliffe@xxxxxxxxx> wrote:
>
>> Well by taking it out of the jvm we do remove some functionality ..
>> mostly access to all the dhis2 beans and whatever is not immediately
>> available through the api.
>>
>> But it is quite straightforward to run a standalone camel instance which
>> talks to dhis2 via the api.
>>
>> Unfortunately there are always some times when you have to resort to the
>> database :-(  If you only want to schedule some sql script to run then I
>> agree with Jason that a cron job which pipes the script to psql (not
>> pgadmin) can often suffice.
>>
>> If you find yourself (in addition to just scheduling), also doing more
>> complex data transformations and the like then it can make sense to have an
>> integration engine like camel - which also has a jdbc endpoint.  But again,
>> if the only thing your camel context is doing is running the sql then it
>> really doesn't need to be in the dhis.war.  Thats really why we have
>> removed it.  You can do the same thing with camel (and/or other tools)
>> without having it loaded in the virtual machine.
>>
>>
>> On 1 October 2013 15:04, Jason Pickering <jason.p.pickering@xxxxxxxxx>wrote:
>>
>>> Nothing against Camel, but not sure what the issue with executing a task
>>> on the database might be. Just wrap it up as a shell script and schedule it
>>> with cron.
>>>
>>> Maybe I am missing some security issue there, but we use this approach
>>> extensively for this type of task.
>>>
>>> Regards,
>>> jason
>>>
>>>
>>>
>>> On Tue, Oct 1, 2013 at 3:59 PM, Rodolfo Melia <rmelia@xxxxxxxxxxxx>wrote:
>>>
>>>> Hi - we were thinking about using Camel for automating certain jobs
>>>> that we want to run every 24 hours (update some Data Elements based on some
>>>> custom calculations that cannot be run as indicators). We ruled out pgAdmin
>>>> because we feel is too insecure. Now that camel is out, how can we schedule
>>>> task to run every day? Do you plan to have a scheduler for an app, so
>>>> certain routines can run on the basis of a schedule?
>>>>
>>>> Rodolfo
>>>>
>>>> *Rodolfo Meliá*
>>>> *Principal*
>>>> rmelia@xxxxxxxxxxxx
>>>> www.knowming.com
>>>> +1 708 872 7636
>>>> +44 777 576 4090
>>>> Skype: rod.melia
>>>>
>>>>
>>>> On Tue, Oct 1, 2013 at 11:26 AM, Bob Jolliffe <bobjolliffe@xxxxxxxxx>wrote:
>>>>
>>>>> We have decided to remove the camel integration module from the dhis2
>>>>> war file in the next release.
>>>>>
>>>>> As the web api has matured it is now more feasible to run the likes of
>>>>> apache camel or mule or similar software as external clients of the api.
>>>>>
>>>>> Regards
>>>>> Bob
>>>>>
>>>>> _______________________________________________
>>>>> Mailing list: https://launchpad.net/~dhis2-devs
>>>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> Mailing list: https://launchpad.net/~dhis2-devs
>>>> Post to     : dhis2-devs@xxxxxxxxxxxxxxxxxxx
>>>> Unsubscribe : https://launchpad.net/~dhis2-devs
>>>> More help   : https://help.launchpad.net/ListHelp
>>>>
>>>>
>>>
>>
>