pbxt-discuss team mailing list archive

[Brian Aker <brian@xxxxxxxxxxx>]

Hi!

On May 21, 2010, at 2:31 AM, Paul McCullagh wrote:

> My problem is that the val_ methods are hard-coded to reference record[0].

My plan is to add a second set of Field that will correspond to the second row (got any handy names!). That way we won't need a hack in order to access them.

The cost of generating those for user generated Tables is very small, so there is no reason not too. For "internal" tables it is a bit different, but then the pattern of access there is really much more confined.

> PBXT uses the MySQL record format internally if the record has a fixed length (similar to how MyISAM).

Are you though? MyISAM doesn't strictly use the format. It rereads for the format and packs in for cases where blobs, varchar, etc exist. Most of the time, assuming that most tables are dynamic,  you have to cycle through it so you are reading each bit. The only unaltered use is when you have a row that is entirely made up of constants (aka... numbers, etc). It MySQL the "byte" type is even more confusing since it is encoded differently in the raw format.

> Instead, I suggest a "Row" object, into which I can plug the start of the row buffer. Then we would have:
> doInsertRecord(Row *row_object)
> The Row object knows everything about the row and can be used to access the fields.

This sounds fine, and essentially what we have at the moment (but no knowledge of row2). 

I hate calling them row1 and row2 though. Any ideas on that? Do you want them handed to you? Should we encapsulate so that you can't access them unless they are passed in?

One thing I would disagree on though is the setBuffer(), that removes all encapsulation on the fields (which is bad, since over time those do change). We have one format for Decimal right now, if we change that and have no version information we take the chance we might push data back and forth which would not be compatible.

Cheers,
	-Brian


> 
> So the Row object is a "per thread" object. You call row_object->setBuffer(u_char *buf) to set the row buffer and then you use an array of fields to get and set the data in the buffer.
> 
> So then the record[] array would become an array of Row objects. And an engine could create Row object to manipulate row buffers internally (which is exactly what I am doing with the internal Table object - or TableShare object).
> 
> The record[0] object would be passed to doInsertRecord(buf), as it is today.
> 
> But now, the engine will not have to know that: buf == record[0]!
> 
> (Which is a correspondence that is far from obvious, but is almost essential to know if you are implementing a storage engine today!)
> 
> Stewart: would something like this also work for your implementation of the Embedded InnoDB?
> 
> On May 20, 2010, at 8:16 PM, Brian Aker wrote:
> 
>> Hi!
>> 
>> I'd unpack the row by using the Field object and not go with the offset (longterm we won't support touching the record[] directly since it creates a big problem for abstraction). Via val_ methods you can request the individual members. The offset was originally added so that two sets of field objects would be needed (aka one for recored[0] and one for record[1]). Our cost for building this stuff is pretty low so we can just give folks a set of fields for both images so that you aren't stuck trying to figure out the underlying contents.
>> 
>> Cheers,
>> 	-Brian
>> 
>> On May 20, 2010, at 1:04 AM, Paul McCullagh wrote:
>> 
>>> Hi Brian,
>>> 
>>> Just one problem maybe you have a quick suggestion:
>>> 
>>> How do I get the offset of a field into the row buffer?
>>> 
>>> When using the Table object I did this as follows:
>>> 
>>> field->offset(field->table->record[0])
>>> 
>>> But this does not work with TableShare.
>>> 
>>> On May 17, 2010, at 6:07 PM, Brian Aker wrote:
>>> 
>>>> Hi!
>>>> 
>>>> On May 17, 2010, at 4:21 AM, Paul McCullagh wrote:
>>>> 
>>>>> Are you suggesting I create a TableShare on the stack whenever I need it?
>>>>> I don't think this would work because AFAIK I have to call open_table_def(), which loads the table definition. So calling this each time I want to copy data in and out of the row would be too slow.
>>>>> 
>>>>> What may work is to use a TableShare object instead of a Table object.
>>>> 
>>>> That is what I was suggesting, just create an object and use it.
>>>> 
>>>>> Question is, can I do something like:
>>>>> share = new TableShare();
>>>>> share->init(db_name, 0, name, path);
>>>>> error = open_table_def(&thd, *ident, share);
>>>> 
>>>> Yes you can do this, hell, we can probably make it simpler then this as well.
>>>> 
>>>>> and then later simply:
>>>>> delete share;
>>>>> to remove.
>>>> 
>>>> Yep. If you look in createTable() you can even see how we do this during that operation.
>>>> 
>>>> Cheers,
>>>> 	-Brian
>>>> 
>>>>> 
>>>>> On May 14, 2010, at 9:03 PM, Brian Aker wrote:
>>>>> 
>>>>>> Hi!
>>>>>> 
>>>>>> On May 14, 2010, at 9:37 AM, Paul McCullagh wrote:
>>>>>> 
>>>>>>> Here the engine will follow the "table" pointer to the "field" array, where it uses the offsets of the data in a record, in order to copy data in and out of the record.
>>>>>> 
>>>>>> So what you need is the Field** that is in share (aka, you don't even need Table, you just need TableShare).
>>>>>> 
>>>>>> You could just do this:
>>>>>> 
>>>>>> TableShare my_share(<share_key>,....);
>>>>>> 
>>>>>> That way you have your own object and you never pass through any of the locking system/dealing with any of the object counting.
>>>>>> 
>>>>>> Cheers,
>>>>>> 	-Brian
>>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> Paul McCullagh
>>>>> PrimeBase Technologies
>>>>> www.primebase.org
>>>>> www.blobstreaming.org
>>>>> pbxt.blogspot.com
>>>>> 
>>>>> 
>>>>> 
>>>> 
>>> 
>>> 
>>> 
>>> --
>>> Paul McCullagh
>>> PrimeBase Technologies
>>> www.primebase.org
>>> www.blobstreaming.org
>>> pbxt.blogspot.com
>>> 
>>> 
>>> 
>> 
> 
> 
> 
> --
> Paul McCullagh
> PrimeBase Technologies
> www.primebase.org
> www.blobstreaming.org
> pbxt.blogspot.com
> 
> 
>