maria-developers team mailing list archive

[Zardosht Kasheff <zardosht@xxxxxxxxx>]

My main interest is getting a feature that many think is very useful
properly implemented and integrated into 5.1. This is a feature that
was enthusiastically brought up during the 2010 storage engine summit.
Also, I think Mark's online schema change work has sparked interest in
this. This feature exists and is used in another GA product: MySQL
cluster, so I think the code should be good.

My reason for being interested in 5.1: that is the GA version of MySQL
and MariaDB. Having this available in 5.1 allows for storage engines
to take advantage of this feature more quickly, and in turn, get it in
the hands of customers more quickly.

That being said, I am wondering what the best approach for doing this
is. I would like to produce something that can be integrated into
MariaDB and MySQL (although I realize that actually getting it into
MySQL is highly highly unlikely). Is there anyway this can be done?

In the worst case, I will only be able to integrate it in our private
branches, and that just diverges our code even more.

As I said earlier, my work is not ready. There are a couple of bugs I
need to track down. Once I get it working, I will happily share what I
have done (which is really just moving code from one MySQL branch to
another, nothing special) and encourage feedback. But if I can get it
working, is there a possible way to get this into MariaDB 5.1 (even a
"safe" version)? If not, some near future version?

Thanks
-Zardosht

On Thu, Sep 23, 2010 at 10:29 AM, MARK CALLAGHAN <mdcallag@xxxxxxxxx> wrote:
> I agree with your intent but some of the motivation is wrong. The
> public Google patch has always been a gigantic diff except for a few
> large features that were extracted. The Facebook patch isn't really a
> patch. It is a launchpad branch. Percona maintained patches. That must
> have been a lot of work on their part. Maybe they won't spend as much
> time on it going forward now that there is Percona server.
>
> On Wed, Sep 22, 2010 at 11:27 PM, Kristian Nielsen
> <knielsen@xxxxxxxxxxxxxxx> wrote:
>> Zardosht Kasheff <zardosht@xxxxxxxxx> writes:
>>
>>> I have spent some of my spare time looking into this. It turns out
>>> that MySQL Cluster already has this ability. They have the following
>>> handler functions listed below. I spent a weekend trying to port MySQL
>>> Cluster's alter table function (mysql_alter_table) over to 5.1.46.
>>
>>>  - All storage engines use the new alter table, I want storage engines
>>> to opt in, to reduce the risk of bugs
>>>
>>> Any thoughts?
>>
>> Yes. I think you should use the new interface for everything. I remember when
>> we did this work in cluster (I did the low-end table change part), and the
>> alter table interface was designed as a general solution, not something to be
>> used only for some special engines.
>>
>> I understand of course why you want to "reduce risk". This has been the
>> approach taken with much community development in the last couple of years
>> (Percona patches, Google patches, Facebook patches, etc): maintain isolated
>> patches, trying to minimise the impact of each patch on the code base to keep
>> things manageable as individual patches and reduce risk and effort needed.
>>
>> But recently I see so much community development happening that I believe it
>> is necessary to move to the next phase. With your work, the Facebook group, my
>> group at Monty Program, Perconas work, and other stuff, the development effort
>> outside of MySQL@Oracle is on the same order of magnitude as that
>> inside. Maintaining individual patches does not scale to that level of effort,
>> in my opinion.
>>
>> Until MySQL@Oracle opens up their development to outside groups, we need
>> another solution. I spend the first year on MariaDB just working to make
>> everything we need available for full-scale development on the MySQL codebase
>> for just this reason.
>>
>> But I recognise that there is still something missing for MariaDB to be
>> useable for Facebook, Percona, you, etc. I think maybe it is the release
>> model, if so we can fix it. Or use something else, it does not have to be
>> MariaDB, but I strongly believe we need to coordinate our efforts. Within the
>> next 6 months or so we have the merge with all of our still into MySQL 5.5
>> coming up, that will take considerable effort and is not something we should
>> each repeat individually.
>>
>> So we need to find a model that works for all you people out in the trenches
>> needing to put things in production use ASAP. But you guys also need at some
>> point to start spending the extra effort to test, fix, and integrate new
>> things properly, otherwise we end up with a huge spaghetti of patches that
>> cannot be maintained.
>
> Define "properly"? I suspect you mean that you want us to spend the
> time to get our changes into MariaDB. While that would be a good thing
> to do, I don't think that makes it proper. Getting changes from my
> team into MariaDB is more about helping others use those changes than
> it is about making my work easier.
>
> I maintain a branch on launchpad, not a set of patches. It isn't that
> hard to maintain them although merging to 5.5 might be some work.
> Pushing patches from Facebook to anybody else is extra work. I don't
> have time to do it. I try to make it easy to pull them.
>
> --
> Mark Callaghan
> mdcallag@xxxxxxxxx
>