maria-developers team mailing list archive

[Sergei Petrunia <psergey@xxxxxxxxxxxx>]

Hello,

I've figured there may be an additional challenge in getting Cassandra
storage engine into a *MariaDB release*. The problem is packaging. 

<contents>
Cassandra SE packaging problems 
 Need Thrift library 
 Need recent gcc 
How did Sphinx SE manage to avoid a similar mess? 
Possible way out
</contents>

== Cassandra SE packaging problems ==

=== Need recent gcc ===
C++ code generated by Thrift uses boost, and so one needs a recent gcc 
to compile it with MySQL.  We have the same issue with oqgraph. See MDEV-501
for details. I am not aware of crashing problems that ograph+boost 
combination has. 

Still, recent-gcc requirement may limit the range of platforms we can build
for. For example, I never managed to compile on Ubuntu 9.04.

=== Need Thrift library ===
Cassandra SE uses Thrift protocol to talk to server. In order to do this, it
needs to link againist libthrift.so.  

There are no Thrift packages in either Fedora or Ubuntu (actually I haven't
seen Thrift packages for any OS). Thrift homepage only offers source tarballs. 

The source tarball uses automake and compiles into
1. thrift compiler binary (we dont actually need it. Source code generated by 
  thrift needs to be modified anyway.)

2. thrift libraries for a dozen of languages, including libthrift.so, which we
do need.

== How did Sphinx SE manage to avoid a similar mess? ==

I've asked myself a question how do we manage to include Sphinx SE without
having to mess with Sphinx packages.

The answer is: Sphinx SE doesn't need any sphinx libraries, either at compile
time, or at runtime. It implements its client protocol fully, except for 
INSERT commands, for which it uses the available mysql client library to talk 
to Sphinx server using mysql client-server protocol.

== An unlikely but possible way out ==

Cassandra 1.2 (the next version that is promised to be released "in Q4 2012" 
will introduce native network protocol. A week ago they've pushed first
documentation draft.  The native network protocol should allow to do 
everything that Thrift API allows, and more.  There are no C++ clients yet,
though.

If I am faced with a choice betwen 
1. implementing a relatively simple network protocol
2. Making Thrift packages for various platforms (and potentially convincing 
   downstream maintainers to accept them)

then I will definitely pick #1 (I have implemented network protocols in the
past).

There is a big risk, though: native protocol support is very new in Cassandra,
they pushed it about a week ago. It was pushed into their "trunk" which is 
under development. I don't know if the code in their trunk is sufficiently 
stable for us to develop againist.

BR
 Sergei
-- 
Sergei Petrunia, Software Developer
Monty Program AB, http://askmonty.org
Blog: http://s.petrunia.net/blog