pbxt-discuss team mailing list archive

[Paul McCullagh <paul.mccullagh@xxxxxxxxxxxxx>]

Hi Stewart,

I am sending this reply to the Drizzle list because we are discussing  
a significant change to Drizzle.

On May 26, 2010, at 5:26 AM, Stewart Smith wrote:
> > But, how would you then handle doInsertRow()?
>
> Server provides something using its format. e.g. when we have magical
> (and not sucky) prepared statements, we could send rows over the wire
> in a binary format and pass a Tuple to the engine that is natively in
> this format. Then the conversion happens straight to whatever the  
> engine
> wants. No intermediary server row format.
>
> > The natural way to handle this would be for the front-end to provide
> > the tuple to be inserted.
>
> yep.
>
> > Would you add a call: getTuple(), which returns an empty tuple, and
> > the call setField() for each column on the tuple and then call
> > doInsertRow(tuple)?
> >
> > Seems rather awkward to me. Or do you have a different idea?
>
> I'd imagine something like this:
>
> class Tuple
> {
> Tuple(Tuple &t)
> { for(i= 0; i< t.nr_columns(); i++) { set_column(i, t.column(i)); } }
> }

Currently, I can do this with one memcpy, in the cases when PBXT is  
using a fixed length record structure.

But, I guess this is a small price to pay for proper encapsulation.

> class OverTheWireTuple : class Tuple
>
> class PBXTTuple : class Tuple
> {
>        int set_column(int colnr, Value v)
>        {
>          /* convert value into pbxt format and store in this  
> PBXTTuple */
>        }
> }

Besides column(i) which returns a Value, I would need the following:

u_char *column_ptr(i)
size_t column_size(i)
bool is_column_null(i)

and also:

int set_column_data(int colnr, u_char *data, size_t len)
int set_column_null(int colnr)

Then I can just copy the data if I don't care about the contents. This  
would enable the engine to just pack (and later unpack) the data into  
a buffer for storage on disk, without having to understand that the  
data represents.

(although some thought needs to be given to the endian problem -  
currently byte order of data in the MySQL row buffer is in a processor  
independent format, and can therefore be stored on disk without  
further conversion).

All I then need is for Drizzle to provide comparison routines for each  
column.

In this way the engine does not need to know anything about the data,  
and the interpretation of the data is always in sync with the server.

These routines do not need to be methods on the Tuple. They can be  
methods on the TableShare, for example. This makes most sense because  
the comparing data depends on information stored in the data  
dictionary, which includes: data type and collation sequences for  
strings.

So on TableShare would could have a method:

int compare_column(int colnr, Tuple &ta, Tuple &tb)

and also:

int compare_column(int colnr, u_char *data_a, size_t len_a, u_char  
*data_b, size_t len_b)

This would be sufficient for engines to build and compare index key  
items (which are basically just Tuples with a mapping to the columns).

> int PBXTCursor::doInsertRecord(Tuple &tuple)
> {
>        PBXTTuple pbxt_tuple(tuple);
>        pbxt_write_row(pbxt_tuple);
> }

This is good. And it fits into what Brian is suggesting: turning the  
record array into something like this:

Tuple record[2]

> Where the upper layer has read off the wire a tuple, constructed a
> OverTheWireTuple, which then gets handed to PBXT. Because PBXT doesn't
> want it in that format, it converts it to its format.
>
> For reading a row, PBXT hands back a PBXTTuple, so for WHERE  
> conditions
> and the like the upper layer just checks the value in the
> PBXTTuple. Only if the Row is going to back to the user over the wire
> does it need to be converted into a OverTheWireTuple.
>
> A temp only engine could just use the OverTheWire format and *never*  
> do
> a conversion.

Although this may present a problem with regard to the scope of  
validity of Tuples returned by the engine.

The best for the moment would be that the Tuple is valid until the  
next call to the engine on the Cursor that returned the Tuple.

In your blog you discuss a nasty little exception to this rule. This  
would need to be corrected, so that the scoping rules are simple for  
the engine.

Best regards,

Paul


--
Paul McCullagh
PrimeBase Technologies
www.primebase.org
www.blobstreaming.org
pbxt.blogspot.com