← Back to team overview

maria-discuss team mailing list archive

Re: Troubleshooting Statement That Crashes MariaDB (mysqld got signal 11)

 

What hardware are you on, and does it have ECC memory?

On Thu, Nov 4, 2021 at 7:48 PM Michael Caplan <michael@xxxxxxxxxxx> wrote:
>
> Hello,
>
> I have a nothing fancy SQL statement that crashes MariaDB 10.3.31.  The
> error log reports "mysqld got signal 11" with the below included mini
> core dump.
>
> The statement caused no issues with my prior install of 10.2.xx. With
> 10.3, the statement runs the majority of the time.  The vast majority of
> the time.
>
> I have never dealt with such an issue with MariaDB, and am unsure where
> to start with troubleshooting this.  Any tips?
>
> FYI, there is a corresponding bug recorded here:
> https://jira.mariadb.org/browse/MDEV-26835
>
> Thanks,
>
> Mike
>
>
>
>
> 211014 16:13:53 [ERROR] mysqld got signal 11 ;
> This could be because you hit a bug. It is also possible that this binary
> or one of the libraries it was linked against is corrupt, improperly built,
> or misconfigured. This error can also be caused by malfunctioning hardware.
>
> To report this bug, see https://mariadb.com/kb/en/reporting-bugs
>
> We will try our best to scrape up some info that will hopefully help
> diagnose the problem, but since we have already crashed,
> something is definitely wrong and this may fail.
>
> Server version: 10.3.31-MariaDB-1:10.3.31+maria~focal-log
> key_buffer_size=134217728
> read_buffer_size=131072
> max_used_connections=587
> max_threads=752
> thread_count=81
> It is possible that mysqld could use up to
> key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads =
> 1784362 K  bytes of memory
> Hope that's ok; if not, decrease some variables in the equation.
>
> Thread pointer: 0x7f13e83b3768
> Attempting backtrace. You can use the following information to find out
> where mysqld died. If you see no messages after this, something went
> terribly wrong...
> stack_bottom = 0x7f1644e96dd8 thread_stack 0x49000
> /usr/sbin/mysqld(my_print_stacktrace+0x32)[0x55d5a2f6f3a2]
> /usr/sbin/mysqld(handle_fatal_signal+0x55d)[0x55d5a2a46e8d]
> /lib/x86_64-linux-gnu/libpthread.so.0(+0x153c0)[0x7f27569fc3c0]
> /usr/sbin/mysqld(+0x978f4f)[0x55d5a2bdbf4f]
> /usr/sbin/mysqld(_ZN7handler10ha_rnd_posEPhS0_+0x50)[0x55d5a2a4c6d0]
> /usr/sbin/mysqld(_Z16rr_from_pointersP11READ_RECORD+0x3e)[0x55d5a2b7cb2e]
> /usr/sbin/mysqld(_Z10sub_selectP4JOINP13st_join_tableb+0x18e)[0x55d5a28869be]
> /usr/sbin/mysqld(_ZN4JOIN10exec_innerEv+0xbee)[0x55d5a28b0dce]
> /usr/sbin/mysqld(_ZN4JOIN4execEv+0x37)[0x55d5a28b1137]
> /usr/sbin/mysqld(_Z12mysql_selectP3THDP10TABLE_LISTjR4ListI4ItemEPS4_jP8st_orderS9_S7_S9_yP13select_resultP18st_select_lex_unitP13st_select_lex+0xf2)[0x55d5a28b1282]
> /usr/sbin/mysqld(_Z13handle_selectP3THDP3LEXP13select_resultm+0x14b)[0x55d5a28b1bcb]
> /usr/sbin/mysqld(+0x5de9f1)[0x55d5a28419f1]
> /usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x5414)[0x55d5a284f6e4]
> /usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_statebb+0x1f3)[0x55d5a2852253]
> /usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcjbb+0x105d)[0x55d5a285462d]
> /usr/sbin/mysqld(_Z10do_commandP3THD+0x12d)[0x55d5a28565dd]
> /usr/sbin/mysqld(_Z24do_handle_one_connectionP7CONNECT+0x246)[0x55d5a293f186]
> /usr/sbin/mysqld(handle_one_connection+0x3f)[0x55d5a293f35f]
> /lib/x86_64-linux-gnu/libpthread.so.0(+0x9609)[0x7f27569f0609]
> /lib/x86_64-linux-gnu/libc.so.6(clone+0x43)[0x7f2756915293]
>
> Trying to get some variables.
> Some pointers may be invalid and cause the dump to abort.
> Query (0x7f13e84a8b70): SELECT * FROM ((
>              #
>              # get postings from last statement and postings not on
> statements
>              #
>              SELECT a.*, COALESCE(ci.id, i.id) as invoice_id
>              FROM ddx_lab_801317.postings a
>              LEFT JOIN ddx_lab_801317.cases_invoices ci
>              ON a.ref_type = 'CASE_INVOICE' AND a.ref_id = ci.id
>              LEFT JOIN ddx_lab_801317.invoices i
>              ON a.ref_type = 'INVOICE' AND a.ref_id = i.id
>              WHERE (
>                  a.statement_date IS NULL
>                  OR a.statement_date = '2021-09-27'
>              )
>           AND a.practice_id='19686'
>              ORDER BY post_date
>          ) UNION (
>              #
>              # get postings that have remaining balances
>              #
>              SELECT DISTINCT a.*, COALESCE(ci.id, i.id) as invoice_id
>              FROM ddx_lab_801317.postings a
>              LEFT JOIN ddx_lab_801317.cases_invoices ci
>              ON a.ref_type = 'CASE_INVOICE' AND a.ref_id = ci.id
>              LEFT JOIN ddx_lab_801317.invoices i
>              ON a.ref_type = 'INVOICE' AND a.ref_id = i.id
>              LEFT JOIN ddx_lab_801317.postings b
>                  ON a.ref_type = b.ref_type
>                  AND a.ref_id = b.ref_id
>                  AND a.id != b.id
>
>              WHERE NOT (a.type = 'CREDIT' OR a.type = 'PAYMENT')
>              AND ((SELECT SUM(b.amount)) IS NULL OR (SELECT
> SUM(a.amount)) + (SELECT SUM(b.amount)) > 0)
>           AND a.practice_id='19686'
>              GROUP BY a.id
>              ORDER BY post_date
>          ) UNION (
>              #
>              # get partial payments and credits from older statements
>              #
>              SELECT a.*, COALESCE(ci.id, i.id) as invoice_id
>              FROM ddx_lab_801317.postings a
>              LEFT JOIN ddx_lab_801317.cases_invoices ci
>              ON a.ref_type = 'CASE_INVOICE' AND a.ref_id = ci.id
>              LEFT JOIN ddx_lab_801317.invoices i
>              ON a.ref_type = 'INVOICE' AND a.ref_id = i.id
>              WHERE (
>                  a.statement_date < '2021-09-27'
>              )
>              AND (a.type='PAYMENT' OR a.type='CREDIT')
>           AND a.practice_id='19686' )) AS t1 ORDER BY post_date ASC,
> invoice_id ASC
>
> Connection ID (thread ID): 21014956
> Status: NOT_KILLED
>
> Optimizer switch:
> index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on
>
> The manual page at
> https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/
> contains
> information that should help you find out what is causing the crash.
> Writing a core file...
> Working directory at /var/lib/mysql
> Resource Limits:
> Limit                     Soft Limit           Hard Limit           Units
> Max cpu time              unlimited unlimited            seconds
> Max file size             unlimited unlimited            bytes
> Max data size             unlimited unlimited            bytes
> Max stack size            8388608 unlimited            bytes
> Max core file size        0 unlimited            bytes
> Max resident set          unlimited unlimited            bytes
> Max processes             unlimited unlimited            processes
> Max open files            1048576 1048576              files
> Max locked memory         unlimited unlimited            bytes
> Max address space         unlimited unlimited            bytes
> Max file locks            unlimited unlimited            locks
> Max pending signals       514974 514974               signals
> Max msgqueue size         819200 819200               bytes
> Max nice priority         0 0
> Max realtime priority     0 0
> Max realtime timeout      unlimited unlimited            us
> Core pattern: core
>
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~maria-discuss
> Post to     : maria-discuss@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~maria-discuss
> More help   : https://help.launchpad.net/ListHelp


Follow ups

References