maria-discuss team mailing list archive

[Marko Mäkelä <marko.makela@xxxxxxxxxxx>]

Hi Michael,

Thanks! In the stack trace, I see fsp_flags_try_adjust(), a function that I
had to introduce to fix a file format compatibility problem that had been
introduced in MariaDB 10.1.0:
MDEV-11623 <https://jira.mariadb.org/browse/MDEV-11623>, commit ab1e6fefd
<https://github.com/MariaDB/server/commit/ab1e6fefd869242d962cb91a006f37bb9ad534a7>
in MariaDB 10.1.21, 10.2.4. The format compatibility problem mostly affects
a non-default innodb_page_size.

However, the "root cause" for this slowdown is the validate=true in the
following code in the function innobase_start_or_create_for_mysql():

            bool validate = recv_needed_recovery
                && srv_force_recovery == 0;

            dict_check_tablespaces_and_store_max_id(validate);

Normally, we should have validate=false, because no crash recovery should
be needed. If the server was not shut down cleanly, then there should be a
startup message "Starting crash recovery", and recv_needed_recovery should
hold.
The variable srv_force_recovery is tied to the parameter
innodb_force_recovery, which is 0 by default.

To me, this looks like it could be 'working as designed'. But, you did not
share the server error log from the very start. Was there a crash recovery
message? If not, then it is an interesting bug.

If you copied the data files from a file system snapshot from a running
server, then it is very likely that crash recovery was needed. You could
work around this by setting --innodb-force-recovery=1 when starting up. But
beware that this will cause InnoDB to ignore certain errors.
Maybe a better idea would be to invoke "mariabackup --prepare" and
"mariabackup --copy-back" in order to apply the redo log before starting up
the server on the copied files. And while you are at it, you could also use
"mariabackup --backup" and skip the file system snapshot.

Best regards,

Marko

On Tue, Jan 9, 2018 at 4:32 PM, Michael Caplan <michael@xxxxxxxxxxx> wrote:

> Hi Marko,
>
> Thanks for your patience with me.  I think I have better material for you
> to look at now.
> Best,
>
> Mike
>
>
>
> On 2018-01-09 03:17 AM, Marko Mäkelä wrote:
>
> Michael,
>
> Sorry, I now realize that you must likely have used a stripped executable,
> and therefore the symbols for the InnoDB functions are missing (displayed
> as ?? in the stack trace output from gdb). As far as I understand, gdb
> should be able to resolve the symbols if you install the separate debug
> symbol package:
>
> sudo apt install mariadb-server-core-10.2-dbgsym
>
> I tried to find the installation package corresponding to
> '10.2.11-MariaDB-10.2.11+maria~xenial-log' so that I would be able to
> resolve the stack trace manually. As a developer, normally I work with code
> that I build myself from the source code, and therefore I am a bit bad at
> giving advice that applies to the packaged code.
>
> I downloaded and extracted the files from the following packages:
> mariadb-server-10.2-dbgsym_10.2.11+maria~xenial_amd64.deb
> <http://mirror.netinch.com/pub/mariadb//mariadb-10.2.11/repo/ubuntu/pool/main/m/mariadb-10.2/mariadb-server-10.2-dbgsym_10.2.11+maria%7Exenial_amd64.deb>
> mariadb-server-10.2_10.2.11+maria~xenial_amd64.deb
> <http://mirror.netinch.com/pub/mariadb//mariadb-10.2.11/repo/ubuntu/pool/main/m/mariadb-10.2/mariadb-server-10.2_10.2.11+maria%7Exenial_amd64.deb>
> mariadb-server-core-10.2-dbgsym_10.2.11+maria~xenial_amd64.deb
> <http://mirror.netinch.com/pub/mariadb//mariadb-10.2.11/repo/ubuntu/pool/main/m/mariadb-10.2/mariadb-server-core-10.2-dbgsym_10.2.11+maria%7Exenial_amd64.deb>
> mariadb-server-core-10.2_10.2.11+maria~xenial_amd64.deb
> <http://mirror.netinch.com/pub/mariadb//mariadb-10.2.11/repo/ubuntu/pool/main/m/mariadb-10.2/mariadb-server-core-10.2_10.2.11+maria%7Exenial_amd64.deb>
>
> The server executable /usr/sbin/mysqld is included in the
> mysql-server-core package. Unfortunately I was unable to match that
> executable with the addresses from your gdb output. I did the following:
>
> file mysqld
> mysqld: ELF 64-bit LSB shared object, x86-64, version 1 (GNU/Linux),
> dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux
> 2.6.32, BuildID[sha1]=7649cc717c0ee820185a633fd27692f3b9160bed, stripped
>
> gdb mysqld
> symbol-file ../lib/debug/.build-id/76/49cc717c0ee820185a633fd27692f3
> b9160bed.debug
>
> followed by a number of gdb commands. I expected the least significant 12
> bits of the addresses to match in the disassembly of
> ha_initialize_handlerton() and plugin_init(), but that was not the case.
> Maybe I did something wrong.
>
> So, can you please install the debug symbol package and try again?
>
> Best regards,
>
> Marko
>
>
>


-- 
Marko Mäkelä, Lead Developer InnoDB
MariaDB Corporation

DON’T MISS

M|18

MariaDB User Conference

February 26 - 27, 2018

New York City

https://m18.mariadb.com/