pbxt-discuss team mailing list archive

[Paul McCullagh <paul.mccullagh@xxxxxxxxxxxxx>]

Hi Martin,

(If possible it would be great if you could switch to text e-mails. I  
find quoting is strange and fonts are small in HTML e-mails.)

On Feb 12, 2010, at 12:44 AM, Martin Scholl wrote:
> - In which format shall we store the table/db definitions? protobuff  
> maybe? Afair Drizzle does so, so we could borrow some code there...
>
> protobuf may be an overkill for the initial implementation.
>
> How are you planning to do create table? The innodb API does it by  
> building a create table structure with various API calls.
>
> By submitting a CREATE TABLE statement as text, you can save a lot  
> of API routines.
>
> PBXT already has a parser for CREATE (and ALTER) table statements.  
> So, you could accept the text and feed the parser.
>
> Then, you could actually store the table definition as a CREATE  
> TABLE statement. When the table is loaded you just invoke the  
> parser. The CREATE TABLE text could be stored in a separate file,  
> like the .frm file, for each table.
>
> Alternatively the text could be stored in the header of the .xtd  
> file, where I already store the foreign key information (the foreign  
> key information is actually stored as SQL text).
>
> However, this may be going too far with the integration of the  
> embedded code and PBXT itself.
> I like this idea really really much -- its a nice separation and  
> self-hosting feature also.
> I'd guess, the lexer code of mysql would have to be ported over  
> then? If I remember things correctly, lexer and THD are coupled  
> which would complicate things even further.

No, PBXT can do everything:

XTDDTable *xt_ri_create_table(XTThreadPtr self, bool convert,  
XTPathStrPtr tab_path, char *sql, XTDDTable *my_tab, struct  
XTDictionary *source_dic);

This function takes a string of SQL and converts it to a table  
(XTDDTable) structure.

> Basically, what would be cool is if the embedded wrapper code  
> controls the following:
>
> 1. The types of data stored.
>   - We can start with a few very basic types.
> 2. The format of a record in RAM
>   - This is the same format that PBXT uses on disk, as long as the  
> records are fixed length
>   - For variable length records it uses a simple serialization  
> method (as I mentioned before)
> 3. The format of index records
>   - with an interface to get and set data in a row, the engine does  
> not need to actual format
> 4. The comparison of data types
>   - the wrapper provides routines to compare data types.
>   - These are mostly methods which are part of the data dictionary  
> in RAM
> 5. The format of the data dictionary on disk, and in RAM
>   - the wrapper reads and writes this data.
>
> This will give us great flexibility to add data types and other  
> complexities later.
> Sounds great. Especially when having a data type registry for  
> supplying custom data types at runtime.

Yes, that would then work.

> It is also pretty much the division of work between MySQL code and  
> PBXT today. However, the division is not so clear in the code.
>
>
> - else, should table serialization / deserialization be pluggable or  
> even be purely programmatic? I am fine with this, too, as it is an  
> _embedded_ library and I'd guess most people will control pbx  
> programmatically anyways
>
> Although I spoke mainly about the textual interface above, I am  
> really flexible on this. I think both solutions have there advantages.
>
> Use whichever is best and easiest for you at the moment, which may  
> be simply writing your own stuff! :)
> I guess hardcoding table definitions will be the most easy way for  
> now.

OK, so the wrapper class would just know the format of the tables?

Best regards,

Paul


--
Paul McCullagh
PrimeBase Technologies
www.primebase.org
www.blobstreaming.org
pbxt.blogspot.com