maria-discuss team mailing list archive

[Justin Swanhart <greenlion@xxxxxxxxx>]

HI,

Btw, you can do the above scenario with the external script, by making a
shell script that writes to a named pipe.  It will block until something
reads from the pipe, which would be the CONNECT engine.  Once the pipe
closes (because CONNECT has finished) just loop and write to the pipe again.

--Justin

On Sat, Jun 13, 2015 at 5:03 PM, Justin Swanhart <greenlion@xxxxxxxxx>
wrote:

> Hi,
>
> What would really be nice is a CONNECT storage type that could execute a
> program and read the output.  For security, it would only run executables
> owned by the user the database is running as, setuid the user the database
> is running as, or executable files could be restricted to those in the data
> directory, but it could never follow symlinks.  You could have something
> like:
> create table test (
>  ...
> )
> engine=CONNECT
> readonly=1  -- must be true
> header=1
> table_type=exec_program
> option_list='output=csv'
> FILE_NAME="process_flatfile.sh /path/to/flatfile.txt"
>
> This would execute the shell script, which would format the output, and
> return the data as CSV, as specified in option_list.
>
> Regards,
>
> On Sat, Jun 13, 2015 at 4:20 PM, Justin Swanhart <greenlion@xxxxxxxxx>
> wrote:
>
>> Hi,
>>
>> If it it read only, why don't you just import it with load data infile
>> periodically, such as every 30 seconds?  Load into new table then swap,
>> just like you would a materialized view.  LOAD DATA INFILE handles fixed
>> width files if you do:
>> LOAD DATA INFILE ... FIELDS DELIMITED BY '';
>>
>> If the fixed with LOAD DATA INFILE won't work, then just load the file
>> with a delimiter that does not exist in the file, so that all lines are
>> treated as a single value.  Use the LOAD DATA ... SET syntax combined with
>> SUBSTR() to extract the columns.  It isn't too difficult to make it work,
>> but it might require a little trial and error.
>>
>> Here is a very old blog post with a trick to get "top" like output in the
>> database without having to run the "top" tool.  It uses the LOAD DATA ...
>> SET technique, so it will give you something to go on:
>> http://swanhart.livejournal.com/131541.html and
>> http://swanhart.livejournal.com/131788.html
>>
>> Regards,
>>
>> --Justin
>>
>>
>>
>> On Sat, Jun 13, 2015 at 4:05 PM, Roberto Spadim <roberto@xxxxxxxxxxxxx>
>> wrote:
>>
>>> I was trying the HEADER=1 but don`t work, check what i`m trying to do:
>>>
>>> create table teste(
>>> ticker char(68) not null,
>>> ts timestamp not null field_format='II',   (this should be a time64_t
>>> value)
>>> bid double not null,
>>> ask double not null,
>>> last double not null,
>>> volume bigint not null,
>>> unknown char(2) not null
>>> )
>>> engine=CONNECT
>>> readonly=1
>>> header=1
>>> table_type=bin
>>> offset=432    (this don`t exists)
>>> option_list='eof=1'
>>> lrecl =110      (record size)
>>> FILE_NAME='/home/mysql/data/ticks.dat'
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~maria-discuss
>>> Post to     : maria-discuss@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~maria-discuss
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>
>>
>