maria-developers team mailing list archive

[Jonas Oreland <jonaso@xxxxxxxxxx>]

Hi again,

> by "interfaces" I was looking for the Maria DB place/ function / hook...
where you are enhancing the MariaDB Code.

I'm not sure how to convey this in a digestible form, attaching diffstats
below. Not sure if it's helps :-(

There are many aspects of it.
And each of the sub-projects (innodb data, innodb log, maria, tempfiles,
binlog) has "interesting" details.

/Jonas


storage/innodb has this diffstat:
 CMakeLists.txt       |    2
 btr/btr0cur.cc       |    9
 buf/buf0buf.cc       |  213 +++++
 buf/buf0checksum.cc  |    8
 buf/buf0dblwr.cc     |   40 -
 buf/buf0flu.cc       |    6
 buf/buf0rea.cc       |    7
 dict/dict0load.cc    |    8
 fil/fil0crypt.cc     | 1986
+++++++++++++++++++++++++++++++++++++++++++++++++++
 fil/fil0fil.cc       |  280 ++++++-
 fsp/fsp0fsp.cc       |   36
 handler/ha_innodb.cc |  110 ++
 handler/i_s.cc       |  292 +++++++
 handler/i_s.h        |    1
 include/buf0buf.h    |   60 +
 include/buf0buf.ic   |   29
 include/fil0fil.h    |  266 ++++++
 include/fsp0fsp.h    |    9
 include/log0crypt.h  |   85 ++
 include/log0log.h    |   21
 include/log0recv.h   |    5
 include/mtr0log.ic   |    2
 include/mtr0mtr.h    |    8
 include/srv0srv.h    |    8
 log/log0crypt.cc     |  256 ++++++
 log/log0log.cc       |   93 ++
 log/log0recv.cc      |   35
 mtr/mtr0log.cc       |    4
 row/row0import.cc    |    3
 srv/srv0srv.cc       |   14
 srv/srv0start.cc     |   29
 31 files changed, 3853 insertions(+), 72 deletions(-)

storage/maria has this diffstat:
 CMakeLists.txt                            |   12
 ha_maria.cc                               |   12
 ma_bitmap.c                               |   63 ++--
 ma_blockrec.c                             |  222 ++++++++------
 ma_blockrec.h                             |   26 +
 ma_check.c                                |   49 +--
 ma_checkpoint.c                           |    4
 ma_close.c                                |    2
 ma_create.c                               |   56 +++
 ma_crypt.c                                |  464
++++++++++++++++++++++++++++++
 ma_crypt.h                                |   26 +
 ma_delete.c                               |    2
 ma_key_recover.c                          |   10
 ma_loghandler.c                           |   63 +---
 ma_open.c                                 |   48 ++-
 ma_pagecache.c                            |  154 ++++++---
 ma_pagecache.h                            |   34 +-
 ma_pagecrc.c                              |  118 ++++---
 ma_static.c                               |    1
 ma_write.c                                |   24 -
 maria_def.h                               |   81 ++---
 unittest/ma_pagecache_consist.c           |   28 -
 unittest/ma_pagecache_rwconsist.c         |   27 -
 unittest/ma_pagecache_rwconsist2.c        |   27 -
 unittest/ma_pagecache_single.c            |   27 -
 unittest/ma_test_loghandler_pagecache-t.c |   29 -
 26 files changed, 1102 insertions(+), 507 deletions(-)

A noticeable difference between innodb and maria is that we didn't
implement encryption of the log for maria,
as we only added support for temporary tables. For maria we also only added
encryption support for BLOCK format
but added all the features to this format so that it was usable for all
temp-table scenarios. maria also doesn't have
key-rotation feature like innodb has.

I couldn't (as) easily extract diffstats for binlog and tempfile encryption.
You have to wait for the code to get published...


On Tue, Jun 17, 2014 at 7:29 AM, Elmar Eperiesi-Beck <elmar@xxxxxxxxxxxxxxxx
> wrote:

> Hi,
> by "interfaces" I was looking for the Maria DB place/ function / hook...
> where you are enhancing the MariaDB Code.
> This would help me to understand what you are trying to do.
>
> Elmar
>
> Am 17.06.2014 um 07:02 schrieb Jonas Oreland <jonaso@xxxxxxxxxx>:
>
> Hi again,
>
> > What is the type of license of your code?
>
> I asked internally about license, and it seems like we releasing dual
> gpl2/apache licensed code.
>
> > I would like to know, which interfaces from maria-DB you are using.
>
> I don't 100% understand the question.
> We didn't write any actual encryption code, but used the one provided in
> openssl.
> Other than that, we didn't really "use interfaces", but rather
> added/modified functionality/interfaces here and there.
>
> Can you be more specific ?
>
> /Jonas
>
>
>
> On Sat, Jun 7, 2014 at 11:20 PM, Elmar Eperiesi-Beck <
> elmar@xxxxxxxxxxxxxxxx> wrote:
>
>> Hi!
>> We (eperi) would be glad to do a joined work with Google.
>> Our solution works with MS-SQL, Oracle and other DBs and we are currently
>> porting it to MariaDB - and - as Monty said - its never to late to put some
>> sources together and make the best for the open source community.
>>
>> What is the type of license of your code?
>>
>> Jonas, I am looking forward to connect to you directly.
>>
>> Regards
>> Elmar
>>
>> Hi!
>>
>> > Hi Jonas,
>> > (same Jonas we know from NDBCLUSTER? :-) Good to see you again)
>> >
>> > On 6 Jun 2014, at 02:31, Jonas Oreland <jonaso@xxxxxxxxxx> wrote:
>> >
>> >> Hi there,
>> >> I read this blog post
>> >>
>> http://monty-says.blogspot.com/2014/05/for-your-eyes-only-or-adding-better.html
>> >> and wanted to inform you that we at Google has developed
>> on-disk/block-level encryption for Innodb, aria (as used by temporary
>> tables), binlogs and temp-files.
>> >> The code is not yet published, but we expect it to be within a few
>> weeks or so.
>> >> We (of course?) think that it would be better if you instead of
>> developing new code
>> >> spent the time testing/reviewing ours.
>>
>> We are out course happy to do this!
>>
>> >> I'm happy to answer questions on the topic,
>> >> and will let you know once we've published it.
>>
>> The main question I have about the Innodb encryption is if it based on
>> the compression code we did for fusion-io?
>> The idea we had on our side was that by using the new compression hooks
>> we could add encryption with very little changes to the Innodb code.
>> Looking forward to when you are ready to publish the code so we can
>> discuss your changes in detail.
>>
>> > This is great news!
>> >
>> > From what I gather, from Monty's blog post (and a 1:1 we had some time
>> back), this is something done by a partner/external company that has a
>> mostly OSS solution, that we should integrate into 10.1
>>
>> Yes,  that's correct. It I would have known that Google was working on
>> encryption I would have included them in my discussions with eperi.
>> Fortunately it's not yet too late to do this.
>> I am sure eperi would like to work on the Google code as a base!
>>
>> > That said, Google's release of something that works for InnoDB, Aria,
>> binlogs, temp files (and presumably not too hard to add for MyISAM) is
>> something we should definitely review and target for 10.1
>>
>> Yes!
>>
>> Regards,
>> Monty
>>
>>
>>
>