maria-developers team mailing list archive

[Sergei Golubchik <serg@xxxxxxxxxxx>]

Hi, Jean-Francois!

On Apr 17, Jean-Francois B. Gagne wrote:
> Hi Sergei,
> 
> (sorry for cross-posting to MariaDB.dev and MariaDB.discuss: I think this
> interest both audiences.)
> 
> Sergei Golubchik commented on MDEV-12499:
> > -----------------------------------------
> >
> > This only happens if you created an ARCHIVE table and never used it
> > before the upgrade. No INSERT, no SELECT, not even {{SELECT * FROM
> > INFORMATION_SCHEMA.TABLES}} — nothing.
> >
> > ARCHIVE engine supports table discovery, that is, the engine is
> > always the authoritative source of table metadata, while {{.frm}}
> > files are merely a metadata cache. When you do {{CREATE TABLE}} the
> > server does not know whether the engine wants the table metadata to
> > be cached. This happens when the table is opened for the first time.
> >
> > If there is no {{.frm}} file and the ARCHIVE engine is not loaded,
> > the server has no way of knowing that the ARCHIVE table exists.
> > That's why it's not shown anywhere.
> >
> > If you do {{SELECT * FROM test.a1}} before the upgrade, the engine
> > will tell the server to create the {{.frm}} file and on the upgrade
> > the plugin will be loaded. In a practical use case, it's enough to
> > have at least one ARCHIVE table that was accessed at least once
> > after it was created, and the ARCHIVE plugin will be loaded on
> > upgrade.
> 
> Thanks for the detailed info above.  In this, you write ".frm files
> are merely a metadata cache", is there any sort of documentation about
> when .frm files are useful, useless, and might get out of sync ?  This
> probably deserves a page in the KB (and Google does not know much on
> "mariadb frm file site:mariadb.com").

Good point, thanks. I've added a paragraph about it to the
https://mariadb.com/kb/en/mariadb/table-discovery/ page.

> However, your comment above raises the following question: if "ARCHIVE
> engine supports table discovery, that is, the engine is always the
> authoritative source of table metadata", where is this information
> stored?  I guess that unless the plugin is loaded, this information is
> not accessible.  This causes a chicken and egg problem which
> MDEV-12499 was trying to solve: how can mysql_upgrade knows if it
> should add BLACKHOLE and ARCHIVE to the list of plugin to load (see
> MDEV-11942).

Correct. The information is stored in the engine - for Archive tables,
in the .ARZ file - and it is not accessible when the plugin is not
loaded.

> Finally, you marked MDEV-12499 as "Won't Fix", do you have another
> solution for ARCHIVE and MDEV-11942 ?

I think it is not an issue in this particular case. This problem will
only show up when one created Archive tables and never ever accessed a
single one of them before the upgrade. I don't think it's a realistic
scenario. Note, that in my tests I simply started "mysql" client and
that was enough for .frm files to be created (because mysql client
queries table metadata on startup).

And it can hardly be called a compatibility issue :)
A "broken compatibility" refers to something that worked before the
upgrade, but stopped working after. If one never accessed Archive tables
before the upgrade, one can still complain that they are not accessible
after the upgrade, but one cannot claim it to be a compatibility
problem, I think.

Regards,
Sergei
Chief Architect MariaDB
and security@xxxxxxxxxxx