← Back to team overview

maria-discuss team mailing list archive

Re: Horrendeous InnoDB crash

 



On 06.06.2016 17:39, Mark wrote:

Here it is; came pretty stock this way with XAMPP (made some minor adjustments for key cache and such, but crashed on pure stock too)

It’s really the scheduled task that makes it crash, but I can’t fathom why.


Now that's actually not that bad, in a QA sense - that it seems to be a genuine, reproducible bug. I suggest to file a bug with details schema etc . Unlike our distributions, it seems that xampp does not to package mysqld.pdb, which makes the backtrace less pretty, almost unusable. But if you can add --core-file to [mysqld] section of my.ini , and add a compressed mysqld.dmp (from the data directory) to your bug report, that would mostly likely help




*From:*Justin Swanhart [mailto:greenlion@xxxxxxxxx]
*Sent:* Monday, June 6, 2016 17:30
*To:* Mark <asarian@xxxxxxxxx>
*Cc:* maria-discuss@xxxxxxxxxxxxxxxxxxx
*Subject:* Re: [Maria-discuss] Horrendeous InnoDB crash

Please attach your my.cnf (perhaps it is my.ini on Windows)

Sent from my iPhone


On Jun 6, 2016, at 11:17 AM, Mark <asarian@xxxxxxxxx <mailto:asarian@xxxxxxxxx>> wrote:

    And yet the same 9*&^%^&*()_ crash (see below). I’m beginning to
    get real tired of this.

    The corruption starts almost immediately upon running a scheduled
    task that’s retrieving posts (at 16:47 hours). It’s the same task
    I ran manually first. There’s absolutely no reason why a simple
    scheduled post retrieval script should immediately corrupt MariaDB
    beyond all recognition. The scheduled task consists of nothing
    more than a script (that runs as me), like:

    c:\xampp\php\php.exe c:\xampp\htdocs\news\retrieve.php –force

    It runs every 30 mins.

    And no, no write-delays, or computer crashes that preceded this.

    2016-06-06 15:52:27 3552 [Note] C:\xampp\mysql\bin\mysqld.exe:
    Normal shutdown

    2016-06-06 15:52:27 3552 [Note] Event Scheduler: Purging the
    queue. 0 events

    2016-06-06 15:52:27 3504 [Note] InnoDB: FTS optimize thread exiting.

    2016-06-06 15:52:27 3552 [Note] InnoDB: Starting shutdown...

    2016-06-06 15:52:29 3552 [Note] InnoDB: Shutdown completed; log
    sequence number 6767908697

    2016-06-06 15:52:29 3552 [Note] C:\xampp\mysql\bin\mysqld.exe:
    Shutdown complete

    2016-06-06 16:39:16 1e24 InnoDB: Warning: Using
    innodb_additional_mem_pool_size is DEPRECATED. This option may be
    removed in future releases, together with the option
    innodb_use_sys_malloc and with the InnoDB's internal memory allocator.

    2016-06-06 16:39:16 7716 [Note] InnoDB: Using mutexes to ref count
    buffer pool pages

    2016-06-06 16:39:16 7716 [Note] InnoDB: The InnoDB memory heap is
    disabled

    2016-06-06 16:39:16 7716 [Note] InnoDB: Mutexes and rw_locks use
    Windows interlocked functions

    2016-06-06 16:39:16 7716 [Note] InnoDB: Memory barrier is not used

    2016-06-06 16:39:16 7716 [Note] InnoDB: Compressed tables use zlib
    1.2.3

    2016-06-06 16:39:16 7716 [Note] InnoDB: Using generic crc32
    instructions

    2016-06-06 16:39:16 7716 [Note] InnoDB: Initializing buffer pool,
    size = 16.0M

    2016-06-06 16:39:16 7716 [Note] InnoDB: Completed initialization
    of buffer pool

    2016-06-06 16:39:16 7716 [Note] InnoDB: Highest supported file
    format is Barracuda.

    2016-06-06 16:39:16 7716 [Note] InnoDB: 128 rollback segment(s)
    are active.

    2016-06-06 16:39:16 7716 [Note] InnoDB: Waiting for purge to start

    2016-06-06 16:39:16 7716 [Note] InnoDB:  Percona XtraDB
    (http://www.percona.com) 5.6.28-76.1 started; log sequence number
    6767908697

    2016-06-06 16:39:16 7100 [Note] InnoDB: Dumping buffer pool(s) not
    yet started

    2016-06-06 16:39:16 7716 [Note] Plugin 'FEEDBACK' is disabled.

    2016-06-06 16:39:16 7716 [Note] Server socket created on IP: '::'.

    2016-06-06 16:39:16 7716 [Note] C:\xampp\mysql\bin\mysqld.exe:
    ready for connections.

    Version: '10.1.13-MariaDB'  socket: ''  port: 3306 mariadb.org
    <http://mariadb.org> binary distribution

    InnoDB: Error: trying to access page number 250370 in space 69,

    InnoDB: space name spotweb/commentsxover,

    InnoDB: which is outside the tablespace bounds.

    InnoDB: Byte offset 0, len 16384, i/o type 10.

    InnoDB: If you get this error at mysqld startup, please check that

    InnoDB: your my.cnf matches the ibdata files that you have in the

    InnoDB: MySQL server.

    2016-06-06 16:47:38 a74  InnoDB: Assertion failure in thread 2676
    in file fil0fil.cc <http://fil0fil.cc> line 5866

    InnoDB: We intentionally generate a memory trap.

    InnoDB: Submit a detailed bug report to http://bugs.mysql.com.

    InnoDB: If you get repeated assertion failures or crashes, even

    InnoDB: immediately after the mysqld startup, there may be

    InnoDB: corruption in the InnoDB tablespace. Please refer to

    InnoDB:
    http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html

    InnoDB: about forcing recovery.

    160606 16:47:38 [ERROR] mysqld got exception 0x80000003 ;

    This could be because you hit a bug. It is also possible that this
    binary

    or one of the libraries it was linked against is corrupt,
    improperly built,

    or misconfigured. This error can also be caused by malfunctioning
    hardware.

    To report this bug, see https://mariadb.com/kb/en/reporting-bugs

    We will try our best to scrape up some info that will hopefully help

    diagnose the problem, but since we have already crashed,

    something is definitely wrong and this may fail.

    Server version: 10.1.13-MariaDB

    key_buffer_size=16777216

    read_buffer_size=262144

    max_used_connections=1

    max_threads=1001

    thread_count=1

    It is possible that mysqld could use up to

    key_buffer_size + (read_buffer_size +
    sort_buffer_size)*max_threads = 787100 K  bytes of memory

    Hope that's ok; if not, decrease some variables in the equation.

    Thread pointer: 0x0x4af7270

    Attempting backtrace. You can use the following information to
    find out

    where mysqld died. If you see no messages after this, something went

    terribly wrong...

    mysqld.exe!my_parameter_handler()

    mysqld.exe!my_mb_ctype_mb()

    mysqld.exe!?get_ctx@MDL_ticket@@QBEPAVMDL_context@@XZ()
    <mailto:mysqld.exe%21?get_ctx@MDL_ticket@@QBEPAVMDL_context@@XZ%28%29>

    mysqld.exe!?get_ctx@MDL_ticket@@QBEPAVMDL_context@@XZ()
    <mailto:mysqld.exe%21?get_ctx@MDL_ticket@@QBEPAVMDL_context@@XZ%28%29>

    mysqld.exe!?functype@Item_func_dyncol_create@@UBE?AW4Functype@Item_func@@XZ()
    <mailto:mysqld.exe%21?functype@Item_func_dyncol_create@@UBE?AW4Functype@Item_func@@XZ%28%29>

    mysqld.exe!?get_ctx@MDL_ticket@@QBEPAVMDL_context@@XZ()
    <mailto:mysqld.exe%21?get_ctx@MDL_ticket@@QBEPAVMDL_context@@XZ%28%29>

    mysqld.exe!??0Global_read_lock@@QAE@XZ()

    mysqld.exe!??0Global_read_lock@@QAE@XZ()

    mysqld.exe!?get_trg_event_map@Update_rows_log_event@@UAEEXZ()
    <mailto:mysqld.exe%21?get_trg_event_map@Update_rows_log_event@@UAEEXZ%28%29>

    mysqld.exe!?get_trg_event_map@Update_rows_log_event@@UAEEXZ()
    <mailto:mysqld.exe%21?get_trg_event_map@Update_rows_log_event@@UAEEXZ%28%29>

    mysqld.exe!?ha_open@handler@@QAEHPAUTABLE@@PBDHI@Z()
    <mailto:mysqld.exe%21?ha_open@handler@@QAEHPAUTABLE@@PBDHI@Z%28%29>

    mysqld.exe!?open_table_from_share@@YA?AW4open_frm_error@@PAVTHD@@PAUTABLE_SHARE@@PBDIIIPAUTABLE@@_N@Z()

    mysqld.exe!?open_table@@YA_NPAVTHD@@PAUTABLE_LIST@@PAVOpen_table_context@@@Z()

    mysqld.exe!?recover_from_failed_open@Open_table_context@@QAE_NXZ()
    <mailto:mysqld.exe%21?recover_from_failed_open@Open_table_context@@QAE_NXZ%28%29>

    mysqld.exe!?open_tables@@YA_NPAVTHD@@ABUDDL_options_st@@PAPAUTABLE_LIST@@PAIIPAVPrelocking_strategy@@@Z()

    mysqld.exe!?open_and_lock_tables@@YA_NPAVTHD@@ABUDDL_options_st@@PAUTABLE_LIST@@_NIPAVPrelocking_strategy@@@Z()

    mysqld.exe!??0Table_scope_and_contents_source_st@@QAE@ABU0@@Z()

    mysqld.exe!?mysql_execute_command@@YAHPAVTHD@@@Z()

    mysqld.exe!?mysql_parse@@YAXPAVTHD@@PADIPAVParser_state@@@Z()

    mysqld.exe!?dispatch_command@@YA_NW4enum_server_command@@PAVTHD@@PADI@Z()

    mysqld.exe!?do_command@@YA_NPAVTHD@@@Z()

    mysqld.exe!?threadpool_process_request@@YAHPAVTHD@@@Z()

    mysqld.exe!?tp_end@@YAXXZ()

    KERNEL32.DLL!SetUserGeoID()

    ntdll.dll!TpSimpleTryPost()

    ntdll.dll!EtwNotificationRegister()

    KERNEL32.DLL!BaseThreadInitThunk()

    ntdll.dll!RtlUnicodeStringToInteger()

    ntdll.dll!RtlUnicodeStringToInteger()

    Trying to get some variables.

    Some pointers may be invalid and cause the dump to abort.

    Query (0x4b01740): SELECT messageid FROM commentsxover ORDER BY id
    DESC LIMIT 5000

    Connection ID (thread ID): 2

    Status: NOT_KILLED

    Optimizer switch:
    index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on

    The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html
    contains

    information that should help you find out what is causing the crash.

    *From:*Justin Swanhart [mailto:greenlion@xxxxxxxxx]
    *Sent:* Monday, June 6, 2016 16:16
    *To:* Mark <asarian@xxxxxxxxx <mailto:asarian@xxxxxxxxx>>
    *Cc:* maria-discuss@xxxxxxxxxxxxxxxxxxx
    <mailto:maria-discuss@xxxxxxxxxxxxxxxxxxx>
    *Subject:* Re: [Maria-discuss] Horrendeous InnoDB crash

    Hi,

    Do you have write-behind caching turned on?  Do you have RAID card
    without BBU?  Did your computer crash?  If the OS or hardware
    (like IDE HDD cache) lies about actually writing to disk, InnoDB
    can become corrupted, and this is not the fault of innodb.  Be
    aware that if this is a VM, many VM lie about disk writes, a
    "feature" you must turn off.

    Sent from my iPhone


    On Jun 6, 2016, at 6:23 AM, Mark <asarian@xxxxxxxxx
    <mailto:asarian@xxxxxxxxx>> wrote:

        Sorry I wiped the database; but it got to the point where
        MariaDB immediately crashed upon startup (so the usual repair
        attempts were futile, as they all kinda assume a MariaDB
        server that’s up and running). I have years of experience
        (FreeBSD) with myisam tables; but a crash like this is a first
        for me, really, where the entire database was foobarred to the
        point MariaDB wouldn’t even start any more.

        Thanks for creating the bug report. I shall investigate myself
        too, in trying to find a way to integrate the MariaDB shutdown
        gracefully (in UNIX this is all much simpler and straightforward).

        -Mark

        *From:*Vladislav Vaintroub [mailto:vvaintroub@xxxxxxxxx]
        *Sent:* Monday, June 6, 2016 11:39
        *To:* Mark <asarian@xxxxxxxxx <mailto:asarian@xxxxxxxxx>>;
        maria-discuss@xxxxxxxxxxxxxxxxxxx
        <mailto:maria-discuss@xxxxxxxxxxxxxxxxxxx>
        *Subject:* Re: [Maria-discuss] Horrendeous InnoDB crash

        Yes, it is possible that Windows does not give enough time to
        MariaDB to shut down gracefully. I just created a task for
        this one here https://jira.mariadb.org/browse/MDEV-10183.

        Still, Innodb should handle this situation gracefully, and
        recover on startup, so what you see is a bug. It is a pity
        that you wiped down the database, it would be useful for a bug
        report.

        On 06.06.2016 11:08, Mark wrote:

            Got a huge crash today, right after initializing my first
            MariaDB database (see below). Got several more errors
            about tables having crashed later on, and MariaDB wouldn't
            even start up any more (I wound up wiping the entire
            database).

            So, my question is, what could cause MariaDB to fail so
            horrendeously?! I though InnoDB was supposed to be
            *better* than myisam!? Is is because maybe Windows 10
            doesn't give MariaDB enough time to shut down gracefully?
            Could it be because a Windows scheduler job (potentially)
            aborts the fetcher script when it's still running? (Again,
            I though InnoDB was supposed to be transaction-safe).

            In its current state, MariaDB is completely unusable for me.

            And no, I don't have hard disk errors. :)

            Seriously, though, I could use some major insight!

            Thanks.

            2016-06-06 10:32:15 2552 [Note] Server socket created on
            IP: '::'.

            2016-06-06 10:32:15 2552 [Note]
            C:\xampp\mysql\bin\mysqld.exe: ready for connections.

            Version: '10.1.13-MariaDB'  socket: ''  port: 3306
            mariadb.org <http://mariadb.org> binary distribution

            InnoDB: Error: trying to access page number 251010 in
            space 69,

            InnoDB: space name news/commentsxover,

            InnoDB: which is outside the tablespace bounds.

            InnoDB: Byte offset 0, len 16384, i/o type 10.

            InnoDB: If you get this error at mysqld startup, please
            check that

            InnoDB: your my.cnf matches the ibdata files that you have
            in the

            InnoDB: MySQL server.

            2016-06-06 08:47:46 e08  InnoDB: Assertion failure in
            thread 3592 in file fil0fil.cc <http://fil0fil.cc> line 5866

            InnoDB: We intentionally generate a memory trap.

            InnoDB: Submit a detailed bug report to http://bugs.mysql.com.

            InnoDB: If you get repeated assertion failures or crashes,
            even

            InnoDB: immediately after the mysqld startup, there may be

            InnoDB: corruption in the InnoDB tablespace. Please refer to

            InnoDB:
            http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html

            InnoDB: about forcing recovery.

            160606 8:47:46 [ERROR] mysqld got exception 0x80000003 ;

            This could be because you hit a bug. It is also possible
            that this binary

            or one of the libraries it was linked against is corrupt,
            improperly built,

            or misconfigured. This error can also be caused by
            malfunctioning hardware.






            _______________________________________________

            Mailing list:https://launchpad.net/~maria-discuss
            <https://launchpad.net/%7Emaria-discuss>

            Post to     :maria-discuss@xxxxxxxxxxxxxxxxxxx
            <mailto:maria-discuss@xxxxxxxxxxxxxxxxxxx>

            Unsubscribe :https://launchpad.net/~maria-discuss
            <https://launchpad.net/%7Emaria-discuss>

            More help   :https://help.launchpad.net/ListHelp

        _______________________________________________
        Mailing list: https://launchpad.net/~maria-discuss
        <https://launchpad.net/%7Emaria-discuss>
        Post to     : maria-discuss@xxxxxxxxxxxxxxxxxxx
        <mailto:maria-discuss@xxxxxxxxxxxxxxxxxxx>
        Unsubscribe : https://launchpad.net/~maria-discuss
        <https://launchpad.net/%7Emaria-discuss>
        More help   : https://help.launchpad.net/ListHelp

    _______________________________________________
    Mailing list: https://launchpad.net/~maria-discuss
    <https://launchpad.net/%7Emaria-discuss>
    Post to     : maria-discuss@xxxxxxxxxxxxxxxxxxx
    <mailto:maria-discuss@xxxxxxxxxxxxxxxxxxx>
    Unsubscribe : https://launchpad.net/~maria-discuss
    <https://launchpad.net/%7Emaria-discuss>
    More help   : https://help.launchpad.net/ListHelp



_______________________________________________
Mailing list: https://launchpad.net/~maria-discuss
Post to     : maria-discuss@xxxxxxxxxxxxxxxxxxx
Unsubscribe : https://launchpad.net/~maria-discuss
More help   : https://help.launchpad.net/ListHelp


References