linux-traipu team mailing list archive
-
linux-traipu team
-
Mailing list archive
-
Message #02396
[Bug 867866] Re: multi-master replication test failing - duplicate trx message error/ first master rpl failure.
I see this output when the slave crashes hard:
*** glibc detected *** /drizzle_mm_test/drizzled/.libs/lt-drizzled: double free or corruption (!prev): 0x0000000003991230 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x78a96)[0x7fdd3669ba96]
/lib/x86_64-linux-gnu/libc.so.6(cfree+0x6c)[0x7fdd3669fd7c]
/usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs6assignERKSs+0x76)[0x7fdd3711d5c6]
/drizzle_mm_test/drizzled/.libs/lt-drizzled[0x532868]
/drizzle_mm_test/drizzled/.libs/lt-drizzled[0x579d78]
/drizzle_mm_test/drizzled/.libs/lt-drizzled(_ZN8drizzled7Execute3runE14data_ref_basicIPKcERNS_3sql9ResultSetE+0xb5)[0x578a07]
/drizzle_mm_test/plugin/.libs/libslave_plugin.so(+0x4d6eb)[0x7fdd31c846eb]
/drizzle_mm_test/plugin/.libs/libslave_plugin.so(+0x4caaf)[0x7fdd31c83aaf]
/drizzle_mm_test/plugin/.libs/libslave_plugin.so(+0x4b060)[0x7fdd31c82060]
/drizzle_mm_test/plugin/.libs/libslave_plugin.so(+0x42d59)[0x7fdd31c79d59]
/drizzle_mm_test/plugin/.libs/libslave_plugin.so(+0x425c5)[0x7fdd31c795c5]
/drizzle_mm_test/plugin/.libs/libslave_plugin.so(+0x42029)[0x7fdd31c79029]
/drizzle_mm_test/plugin/.libs/libslave_plugin.so(+0x4181e)[0x7fdd31c7881e]
/drizzle_mm_test/plugin/.libs/libslave_plugin.so(+0x40ffa)[0x7fdd31c77ffa]
/drizzle_mm_test/plugin/.libs/libslave_plugin.so(+0x3f71e)[0x7fdd31c7671e]
/usr/lib/libboost_thread.so.1.42.0(thread_proxy+0x5e)[0x7fdd380e916e]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x7efc)[0x7fdd369c9efc]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7fdd3670489d]
======= Memory map: ========
00400000-00cb0000 r-xp 00000000 00:15 8915151 /drizzle_mm_test/drizzled/.libs/lt-drizzled
00eb0000-00ec9000 r--p 008b0000 00:15 8915151 /drizzle_mm_test/drizzled/.libs/lt-drizzled
00ec9000-00eff000 rw-p 008c9000 00:15 8915151 /drizzle_mm_test/drizzled/.libs/lt-drizzled
00eff000-00f34000 rw-p 00000000 00:00 0
02d2a000-039ad000 rw-p 00000000 00:00 0 [heap]
7fdd14000000-7fdd14021000 rw-p 00000000 00:00 0
7fdd14021000-7fdd18000000 ---p 00000000 00:00 0
7fdd197fb000-7fdd197fc000 ---p 00000000 00:00 0
7fdd197fc000-7fdd19ffc000 rw-p 00000000 00:00 0
7fdd19ffc000-7fdd19ffd000 ---p 00000000 00:00 0
7fdd19ffd000-7fdd1a7fd000 rw-p 00000000 00:00 0
7fdd1a7fd000-7fdd1a7fe000 ---p 00000000 00:00 0
7fdd1a7fe000-7fdd1affe000 rw-p 00000000 00:00 0
7fdd1affe000-7fdd1afff000 ---p 00000000 00:00 0
7fdd1afff000-7fdd1b7ff000 rw-p 00000000 00:00 0
7fdd1b7ff000-7fdd1b800000 ---p 00000000 00:00 0
7fdd1b800000-7fdd1c000000 rw-p 00000000 00:00 0
7fdd1c000000-7fdd1c021000 rw-p 00000000 00:00 0
7fdd1c021000-7fdd20000000 ---p 00000000 00:00 0
7fdd2014a000-7fdd2014b000 ---p 00000000 00:00 0
7fdd2014b000-7fdd2094b000 rw-p 00000000 00:00 0
7fdd2094b000-7fdd2094c000 ---p 00000000 00:00 0
7fdd2094c000-7fdd2114c000 rw-p 00000000 00:00 0
7fdd2114c000-7fdd2114d000 ---p 00000000 00:00 0
7fdd2114d000-7fdd2194d000 rw-p 00000000 00:00 0
7fdd2194d000-7fdd2194e000 ---p 00000000 00:00 0
7fdd2194e000-7fdd2214e000 rw-p 00000000 00:00 0
7fdd2214e000-7fdd2214f000 ---p 00000000 00:00 0
7fdd2214f000-7fdd2294f000 rw-p 00000000 00:00 0
7fdd2294f000-7fdd22950000 ---p 00000000 00:00 0
7fdd22950000-7fdd23356000 rw-p 00000000 00:00 0
7fdd23356000-7fdd23357000 ---p 00000000 00:00 0
7fdd23357000-7fdd23b57000 rw-p 00000000 00:00 0
7fdd23b57000-7fdd23b58000 ---p 00000000 00:00 0
7fdd23b58000-7fdd24358000 rw-p 00000000 00:00 0
7fdd24358000-7fdd24359000 ---p 00000000 00:00 0
7fdd24359000-7fdd24b59000 rw-p 00000000 00:00 0
7fdd24b59000-7fdd24b5a000 ---p 00000000 00:00 0
7fdd24b5a000-7fdd2535a000 rw-p 00000000 00:00 0
7fdd2535a000-7fdd2535b000 ---p 00000000 00:00 0
7fdd2535b000-7fdd25b5b000 rw-p 00000000 00:00 0
7fdd25b5b000-7fdd25b5c000 ---p 00000000 00:00 0
7fdd25b5c000-7fdd2635c000 rw-p 00000000 00:00 0
7fdd2635c000-7fdd2635d000 ---p 00000000 00:00 0
7fdd2635d000-7fdd26b5d000 rw-p 00000000 00:00 0
7fdd26b5d000-7fdd26b5e000 ---p 00000000 00:00 0
7fdd26b5e000-7fdd2735e000 rw-p 00000000 00:00 0
7fdd2735e000-7fdd2735f000 ---p 00000000 00:00 0
7fdd2735f000-7fdd27b5f000 rw-p 00000000 00:00 0
7fdd27b5f000-7fdd27b60000 ---p 00000000 00:00 0
7fdd27b60000-7fdd311b4000 rw-p 00000000 00:00 0
7fdd311b4000-7fdd311c1000 r-xp 00000000 00:15 9046329 /drizzle_mm_test/plugin/.libs/libversion_plugin.so
7fdd311c1000-7fdd313c1000 ---p 0000d000 00:15 9046329 /drizzle_mm_test/plugin/.libs/libversion_plugin.so
7fdd313c1000-7fdd313c2000 r--p 0000d000 00:15 9046329 /drizzle_mm_test/plugin/.libs/libversion_plugin.so
7fdd313c2000-7fdd313c3000 rw-p 0000e000 00:15 9046329 /drizzle_mm_test/plugin/.libs/libversion_plugin.so
7fdd313c3000-7fdd313d0000 r-xp 00000000 00:15 9046327 /drizzle_mm_test/plugin/.libs/libuuid_function_plugin.so
7fdd313d0000-7fdd315d0000 ---p 0000d000 00:15 9046327 /drizzle_mm_test/plugin/.libs/libuuid_function_plugin.so
7fdd315d0000-7fdd315d1000 r--p 0000d000 00:15 9046327 /drizzle_mm_test/plugin/.libs/libuuid_function_plugin.so
7fdd315d1000-7fdd315d2000 rw-p 0000e000 00:15 9046327 /drizzle_mm_test/plugin/.libs/libuuid_function_plugin.so
7fdd315d2000-7fdd31603000 r-xp 00000000 00:15 9046308 /drizzle_mm_test/plugin/.libs/libsyslog_plugin.so
7fdd31603000-7fdd31802000 ---p 00031000 00:15 9046308 /drizzle_mm_test/plugin/.libs/libsyslog_plugin.so
7fdd31802000-7fdd31805000 r--p 00030000 00:15 9046308 /drizzle_mm_test/plugin/.libs/libsyslog_plugin.so
7fdd31805000-7fdd31806000 rw-p 00033000 00:15 9046308 /drizzle_mm_test/plugin/.libs/libsyslog_plugin.so
7fdd31806000-7fdd31816000 r-xp 00000000 00:15 9046303 /drizzle_mm_test/plugin/.libs/libsubstr_functions_plugin.so
7fdd31816000-7fdd31a16000 ---p 00010000 00:15 9046303 /drizzle_mm_test/plugin/.libs/libsubstr_functions_plugin.so
7fdd31a16000-7fdd31a17000 r--p 00010000 00:15 9046303 /drizzle_mm_test/plugin/.libs/libsubstr_functions_plugin.so
7fdd31a17000-7fdd31a18000 rw-p 00011000 00:15 9046303 /drizzle_mm_test/plugin/.libs/libsubstr_functions_plugin.so
7fdd31a18000-7fdd31a36000 r-xp 00000000 00:15 8915107 /drizzle_mm_test/libdrizzle-2.0/libdrizzle/.libs/libdrizzle-2.0.so.3.0.0
7fdd31a36000-7fdd31c35000 ---p 0001e000 00:15 8915107 /drizzle_mm_test/libdrizzle-2.0/libdrizzle/.libs/libdrizzle-2.0.so.3.0.0
7fdd31c35000-7fdd31c36000 r--p 0001d000 00:15 8915107 /drizzle_mm_test/libdrizzle-2.0/libdrizzle/.libs/libdrizzle-2.0.so.3.0.0
7fdd31c36000-7fdd31c37000 rw-p 0001e000 00:15 8915107 /drizzle_mm_test/libdrizzle-2.0/libdrizzle/.libs/libdrizzle-2.0.so.3.0.0
7fdd31c37000-7fdd31ca3000 r-xp 00000000 00:15 9046299 /drizzle_mm_test/plugin/.libs/libslave_plugin.so
7fdd31ca3000-7fdd31ea3000 ---p 0006c000 00:15 9046299 /drizzle_mm_test/plugin/.libs/libslave_plugin.so
7fdd31ea3000-7fdd31ea6000 r--p 0006c000 00:15 9046299 /drizzle_mm_test/plugin/.libs/libslave_plugin.so
7fdd31ea6000-7fdd31ea8000 rw-p 0006f000 00:15 9046299 /drizzle_mm_test/plugin/.libs/libslave_plugin.so
7fdd31ea8000-7fdd31eb7000 r-xp 00000000 00:15 9046286 /drizzle_mm_test/plugin/.libs/libshow_schema_proto_plugin.so
7fdd31eb7000-7fdd320b7000 ---p 0000f000 00:15 9046286 /drizzle_mm_test/plugin/.libs/libshow_schema_proto_plugin.so
7fdd320b7000-7fdd320b8000 r--p 0000f000 00:15 9046286 /drizzle_mm_test/plugin/.libs/libshow_schema_proto_plugin.so
7fdd320b8000-7fdd320b9000 rw-p 00010000 00:15 9046286 /drizzle_mm_test/plugin/.libs/libshow_schema_proto_plugin.so
7fdd320b9000-7fdd320c7000 r-xp 00000000 00:15 9046284 /drizzle_mm_test/plugin/.libs/libreverse_function_plugin.so
7fdd320c7000-7fdd322c6000 ---p 0000e000 00:15 9046284 /drizzle_mm_test/plugin/.libs/libreverse_function_plugin.so
7fdd322c6000-7fdd322c7000 r--p 0000d000 00:15 9046284 /drizzle_mm_test/plugin/.libs/libreverse_function_plugin.so
7fdd322c7000-7fdd322c8000 rw-p 0000e000 00:15 9046284 /drizzle_mm_test/plugin/.libs/libreverse_function_plugin.so
7fdd322c8000-7fdd322d6000 r-xp 00000000 00:15 9046272 /drizzle_mm_test/plugin/.libs/librand_function_plugin.so
7fdd322d6000-7fdd324d5000 ---p 0000e000 00:15 9046272 /drizzle_mm_test/plugin/.libs/librand_function_plugin.so
7fdd324d5000-7fdd324d6000 r--p 0000d000 00:15 9046272 /drizzle_mm_test/plugin/.libs/librand_function_plugin.so
7fdd324d6000-7fdd324d7000 rw-p 0000e000 00:15 9046272 /drizzle_mm_test/plugin/.libs/librand_function_plugin.so
7fdd324d7000-7fdd32500000 r-xp 00000000 00:15 9046259 /drizzle_mm_test/plugin/.libs/libmulti_thread_plugin.so
7fdd32500000-7fdd326ff000 ---p 00029000 00:15 9046259 /drizzle_mm_test/plugin/.libs/libmulti_thread_plugin.so
7fdd326ff000-7fdd32701000 r--p 00028000 00:15 9046259 /drizzle_mm_test/plugin/.libs/libmulti_thread_plugin.so
7fdd32701000-7fdd32702000 rw-p 0002a000 00:15 9046259 /drizzle_mm_test/plugin/.libs/libmulti_thread_plugin.so
7fdd32702000-7fdd32705000 r-xp 00000000 08:01 917562 /lib/x86_64-linux-gnu/libgpg-error.so.0.8.0
7fdd32705000-7fdd32904000 ---p 00003000 08:01 917562 /lib/x86_64-linux-gnu/libgpg-error.so.0.8.0
7fdd32904000-7fdd32905000 r--p 00002000 08:01 917562 /lib/x86_64-linux-gnu/libgpg-error.so.0.8.0
7fdd32905000-7fdd32906000 rw-p 00003000 08:01 917562 /lib/x86_64-linux-gnu/libgpg-error.so.0.8.0
7fdd32906000-7fdd32980000 r-xp 00000000 08:01 922143 /lib/x86_64-linux-gnu/libgcrypt.so.11.7.0
7fdd32980000-7fdd32b80000 ---p 0007a000 08:01 922143 /lib/x86_64-linux-gnu/libgcrypt.so.11.7.0
7fdd32b80000-7fdd32b81000 r--p 0007a000 08:01 922143 /lib/x86_64-linux-gnu/libgcrypt.so.11.7.0
7fdd32b81000-7fdd32b84000 rw-p 0007b000 08:01 922143 /lib/x86_64-linux-gnu/libgcrypt.so.11.7.0
7fdd32b84000-7fdd32b92000 r-xp 00000000 00:15 9046248 /drizzle_mm_test/plugin/.libs/libmd5_plugin.so
7fdd32b92000-7fdd32d91000 ---p 0000e000 00:15 9046248 /drizzle_mm_test/plugin/.libs/libmd5_plugin.so
7fdd32d91000-7fdd32d92000 r--p 0000d000 00:15 9046248 /drizzle_mm_test/plugin/.libs/libmd5_plugin.so
7fdd32d92000-7fdd32d93000 rw-p 0000e000 00:15 9046248 /drizzle_mm_test/plugin/.libs/libmd5_plugin.so
7fdd32d93000-7fdd32df0000 r-xp 00000000 00:15 9046246 /drizzle_mm_test/plugin/.libs/liblogging_stats_plugin.so
7fdd32df0000-7fdd32ff0000 ---p 0005d000 00:15 9046246 /drizzle_mm_test/plugin/.libs/liblogging_stats_plugin.so
7fdd32ff0000-7fdd32ff3000 r--p 0005d000 00:15 9046246 /drizzle_mm_test/plugin/.libs/liblogging_stats_plugin.so
7fdd32ff3000-7fdd32ff5000 rw-p 00060000 00:15 9046246 /drizzle_mm_test/plugin/.libs/liblogging_stats_plugin.so
7fdd32ff5000-7fdd32ff6000 rw-p 00000000 00:00 0
7fdd32ff6000-7fdd33003000 r-xp 00000000 00:15 9046243 /drizzle_mm_test/plugin/.libs/liblength_plugin.so
7fdd33003000-7fdd33203000 ---p 0000d000 00:15 9046243 /drizzle_mm_test/plugin/.libs/liblength_plugin.so
7fdd33203000-7fdd33204000 r--p 0000d000 00:15 9046243 /drizzle_mm_test/plugin/.libs/liblength_plugin.so
7fdd33204000-7fdd33205000 rw-p 0000e000 00:15 9046243 /drizzle_mm_test/plugin/.libs/liblength_plugin.so
7fdd33205000-7fdd33215000 r-xp 00000000 00:15 9046230 /drizzle_mm_test/plugin/.libs/libhex_functions_plugin.so
7fdd33215000-7fdd33415000 ---p 00010000 00:15 9046230 /drizzle_mm_test/plugin/.libs/libhex_functions_plugin.so
7fdd33415000-7fdd33416000 r--p 00010000 00:15 9046230 /drizzle_mm_test/plugin/.libs/libhex_functions_plugin.so
7fdd33416000-7fdd33417000 rw-p 00011000 00:15 9046230 /drizzle_mm_test/plugin/.libs/libhex_functions_plugin.so20111014-163855
--
You received this bug notification because you are a member of UBUNTU -
AL - BR, which is subscribed to Drizzle.
https://bugs.launchpad.net/bugs/867866
Title:
multi-master replication test failing - duplicate trx message error/
first master rpl failure.
Status in A Lightweight SQL Database for Cloud Infrastructure and Web Applications:
Confirmed
Status in Drizzle fremont series:
Confirmed
Bug description:
So, I have created the infrastructure to setup multi-master topologies for testing.
The gist of the test is we spin 3 servers = 2 masters, one slave replicating from both.
master1 = we create test.t1 and some records
master2 = we create test.t2 and some records
When we try:
./dbqp --suite=slave --record multi_master_basic, the test fails as follows (slave crashes)
$ cat workdir/bot0/s2/var/log/s2.err
InnoDB: Doublewrite buffer not found: creating new
InnoDB: Doublewrite buffer created
InnoDB: 127 rollback segment(s) active.
InnoDB: Creating foreign key constraint system tables
InnoDB: Foreign key constraint system tables created
(SQLSTATE 00000) Duplicate entry '772-1' for key 'PRIMARY'
Failure while executing:
INSERT INTO `sys_replication`.`queue` (`master_id`, `trx_id`, `seg_id`, `commit_order`, `originating_server_uuid`, `originating_commit_id`, `msg`) VALUES (2, 772, 1, 1, 'ac9c8ac0-8f10-474b-9bbd-b61d2cdb2b93' , 1, 'transaction_context {
server_id: 1
transaction_id: 772
start_timestamp: 1317760732106016
end_timestamp: 1317760732106017
}
event {
type: STARTUP
}
segment_id: 1
end_segment: true
')
Replication slave: Unable to insert into queue.
Replication slave: drizzle_state_read:lost connection to server (EOF)
Lost connection to master. Reconnecting.
Replication slave: drizzle_state_connect:could not connect
111004 16:39:05 InnoDB: Starting shutdown...
Testing the setup with --start-and-exit shows that we only seem to be replicating from master 2, not master 1.
the config file is as follows:
ignore-errors
[master1]
master-host=127.0.0.1
master-port=9306
master-user=root
master-pass=''
[master2]
master-host=127.0.0.1
master-port=9312
master-user=root
master-pass=''
To manage notifications about this bug go to:
https://bugs.launchpad.net/drizzle/+bug/867866/+subscriptions
References
