← Back to team overview

maria-discuss team mailing list archive

Re: MariaDB Cluster wont start

 

You were right

This is the complete logs when i tray to start mariadb in the second node.


Sep 29 17:03:32 baruwAdb3 mysqld_safe: Starting mysqld daemon with
databases from /var/lib/mysql
Sep 29 17:03:32 baruwAdb3 mysqld_safe: WSREP: Running position recovery
with --log_error='/var/lib/mysql/wsrep_recovery.eRh9HO'
--pid-file='/var/lib/mysql/baruwAdb3-recover.pid'
Sep 29 17:03:32 baruwAdb3 mysqld: 140929 17:03:32 [Warning] option
'table_cache': unsigned value 2097152 adjusted to 524288
Sep 29 17:03:36 baruwAdb3 mysqld_safe: WSREP: Recovered position
0c63fe44-47fe-11e4-8d91-d3636a34c2fc:1
Sep 29 17:03:36 baruwAdb3 mysqld: 140929 17:03:36 [Warning] option
'table_cache': unsigned value 2097152 adjusted to 524288
Sep 29 17:03:36 baruwAdb3 mysqld: 140929 17:03:36 [Note] WSREP:
wsrep_start_position var submitted: '0c63fe44-47fe-11e4-8d91-d3636a34c2fc:1'
Sep 29 17:03:36 baruwAdb3 mysqld: 140929 17:03:36 [Note] WSREP: Read nil
XID from storage engines, skipping position init
Sep 29 17:03:36 baruwAdb3 mysqld: 140929 17:03:36 [Note] WSREP:
wsrep_load(): loading provider library '/usr/lib/galera/libgalera_smm.so'
Sep 29 17:03:36 baruwAdb3 mysqld: 140929 17:03:36 [Note] WSREP:
wsrep_load(): Galera 25.3.5-wheezy(rXXXX) by Codership Oy
<info@codership.comSep 29 17:03:36 baruwAdb3 mysqld: 140929 17:03:36 [Note]
WSREP: Setting initial position to 00000000-0000-0000-0000-000000000000:-1
Sep 29 17:03:36 baruwAdb3 mysqld: 140929 17:03:36 [Note] WSREP: protonet
asio version 0
Sep 29 17:03:36 baruwAdb3 mysqld: 140929 17:03:36 [Note] WSREP: Using
CRC-32C (optimized) for message checksums.
Sep 29 17:03:36 baruwAdb3 mysqld: 140929 17:03:36 [Note] WSREP: backend:
asio
Sep 29 17:03:36 baruwAdb3 mysqld: 140929 17:03:36 [Note] WSREP: GMCast
version 0
Sep 29 17:03:36 baruwAdb3 mysqld: 140929 17:03:36 [Note] WSREP:
(1475a37f-481c-11e4-b9cc-ee8fe601500f, 'tcp://0.0.0.0:4567') listening at
tcp://0.0.0.0:4567
Sep 29 17:03:36 baruwAdb3 mysqld: 140929 17:03:36 [Note] WSREP:
(1475a37f-481c-11e4-b9cc-ee8fe601500f, 'tcp://0.0.0.0:4567') multicast: ,
ttl: 1
Sep 29 17:03:36 baruwAdb3 mysqld: 140929 17:03:36 [Note] WSREP: EVS version
0
Sep 29 17:03:36 baruwAdb3 mysqld: 140929 17:03:36 [Note] WSREP: PC version 0
Sep 29 17:03:36 baruwAdb3 mysqld: 140929 17:03:36 [Note] WSREP: gcomm:
connecting to group 'baruwa_cluster', peer
'baruwadb1:,baruwadb2:,baruwadb3:'
Sep 29 17:03:36 baruwAdb3 mysqld: 140929 17:03:36 [Warning] WSREP:
(1475a37f-481c-11e4-b9cc-ee8fe601500f, 'tcp://0.0.0.0:4567') address 'tcp://
172.17.1.12:4567' points to own listening address, blacklisting
Sep 29 17:03:36 baruwAdb3 mysqld: 140929 17:03:36 [Note] WSREP:
(1475a37f-481c-11e4-b9cc-ee8fe601500f, 'tcp://0.0.0.0:4567') address 'tcp://
172.17.1.12:4567' pointing to uuid 1475a37f-481c-11e4-b9cc-ee8fe601500f is
blacklisted, skipping
Sep 29 17:03:37 baruwAdb3 mysqld: 140929 17:03:37 [Note] WSREP: declaring
55a2a1b4-481a-11e4-912d-93d52c4f97cc stable
Sep 29 17:03:37 baruwAdb3 mysqld: 140929 17:03:37 [Note] WSREP: Node
55a2a1b4-481a-11e4-912d-93d52c4f97cc state prim
Sep 29 17:03:37 baruwAdb3 mysqld: 140929 17:03:37 [Note] WSREP:
view(view_id(PRIM,1475a37f-481c-11e4-b9cc-ee8fe601500f,2) memb {
Sep 29 17:03:37 baruwAdb3 mysqld: #0111475a37f-481c-11e4-b9cc-ee8fe601500f,0
Sep 29 17:03:37 baruwAdb3 mysqld: #01155a2a1b4-481a-11e4-912d-93d52c4f97cc,0
Sep 29 17:03:37 baruwAdb3 mysqld: } joined {
Sep 29 17:03:37 baruwAdb3 mysqld: } left {
Sep 29 17:03:37 baruwAdb3 mysqld: } partitioned {
Sep 29 17:03:37 baruwAdb3 mysqld: })
Sep 29 17:03:37 baruwAdb3 mysqld: 140929 17:03:37 [Note] WSREP: discarding
pending addr without UUID: tcp://172.17.1.4:4567
Sep 29 17:03:37 baruwAdb3 mysqld: 140929 17:03:37 [Note] WSREP: discarding
pending addr proto entry 0x7f4043ae18c0
Sep 29 17:03:37 baruwAdb3 mysqld: 140929 17:03:37 [Note] WSREP: gcomm:
connected
Sep 29 17:03:37 baruwAdb3 mysqld: 140929 17:03:37 [Note] WSREP: Changing
maximum packet size to 64500, resulting msg size: 32636
Sep 29 17:03:37 baruwAdb3 mysqld: 140929 17:03:37 [Note] WSREP: Shifting
CLOSED -> OPEN (TO: 0)
Sep 29 17:03:37 baruwAdb3 mysqld: 140929 17:03:37 [Note] WSREP: Opened
channel 'baruwa_cluster'
Sep 29 17:03:37 baruwAdb3 mysqld: 140929 17:03:37 [Note] WSREP: Waiting for
SST to complete.
Sep 29 17:03:37 baruwAdb3 mysqld: 140929 17:03:37 [Note] WSREP: New
COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 2
Sep 29 17:03:37 baruwAdb3 mysqld: 140929 17:03:37 [Note] WSREP:
STATE_EXCHANGE: sent state UUID: 14c24d3f-481c-11e4-a0af-cae2115c6e2e
Sep 29 17:03:37 baruwAdb3 mysqld: 140929 17:03:37 [Note] WSREP: STATE
EXCHANGE: sent state msg: 14c24d3f-481c-11e4-a0af-cae2115c6e2e
Sep 29 17:03:37 baruwAdb3 mysqld: 140929 17:03:37 [Note] WSREP: STATE
EXCHANGE: got state msg: 14c24d3f-481c-11e4-a0af-cae2115c6e2e from 0
(baruwadb3)
Sep 29 17:03:37 baruwAdb3 mysqld: 140929 17:03:37 [Note] WSREP: STATE
EXCHANGE: got state msg: 14c24d3f-481c-11e4-a0af-cae2115c6e2e from 1
(baruwadb1)
Sep 29 17:03:37 baruwAdb3 mysqld: 140929 17:03:37 [Note] WSREP: Quorum
results:
Sep 29 17:03:37 baruwAdb3 mysqld: #011version    = 3,
Sep 29 17:03:37 baruwAdb3 mysqld: #011component  = PRIMARY,
Sep 29 17:03:37 baruwAdb3 mysqld: #011conf_id    = 1,
Sep 29 17:03:37 baruwAdb3 mysqld: #011members    = 1/2 (joined/total),
Sep 29 17:03:37 baruwAdb3 mysqld: #011act_id     = 1,
Sep 29 17:03:37 baruwAdb3 mysqld: #011last_appl. = -1,
Sep 29 17:03:37 baruwAdb3 mysqld: #011protocols  = 0/5/3 (gcs/repl/appl),
Sep 29 17:03:37 baruwAdb3 mysqld: #011group UUID =
0c63fe44-47fe-11e4-8d91-d3636a34c2fc
Sep 29 17:03:37 baruwAdb3 mysqld: 140929 17:03:37 [Note] WSREP:
Flow-control interval: [23, 23]
Sep 29 17:03:37 baruwAdb3 mysqld: 140929 17:03:37 [Note] WSREP: Shifting
OPEN -> PRIMARY (TO: 1)
Sep 29 17:03:37 baruwAdb3 mysqld: 140929 17:03:37 [Warning] WSREP: Gap in
state sequence. Need state transfer.
Sep 29 17:03:39 baruwAdb3 mysqld: 140929 17:03:39 [Note] WSREP: Running:
'wsrep_sst_rsync --role 'joiner' --address '172.17.1.12' --auth ''
--datadir '/var/lib/mysql/' --defaults-file '/etc/mysql/my.cnf' --parent
'26706'  '' '
Sep 29 17:03:39 baruwAdb3 rsyncd[26804]: rsyncd version 3.1.0 starting,
listening on port 4444
Sep 29 17:03:39 baruwAdb3 mysqld: 140929 17:03:39 [Note] WSREP: Prepared
SST request: rsync|172.17.1.12:4444/rsync_sst
Sep 29 17:03:39 baruwAdb3 mysqld: 140929 17:03:39 [Note] WSREP:
wsrep_notify_cmd is not defined, skipping notification.
Sep 29 17:03:39 baruwAdb3 mysqld: 140929 17:03:39 [Note] WSREP: REPL
Protocols: 5 (3, 1)
Sep 29 17:03:39 baruwAdb3 mysqld: 140929 17:03:39 [Note] WSREP: Service
thread queue flushed.
Sep 29 17:03:39 baruwAdb3 mysqld: 140929 17:03:39 [Note] WSREP: Assign
initial position for certification: 1, protocol version: 3
Sep 29 17:03:39 baruwAdb3 mysqld: 140929 17:03:39 [Note] WSREP: Service
thread queue flushed.
Sep 29 17:03:39 baruwAdb3 mysqld: 140929 17:03:39 [Warning] WSREP: Failed
to prepare for incremental state transfer: Local state UUID
(00000000-0000-0000-0000-000000000000) does not match group state UUID
(0c63fe44-47fe-11e4-8d91-d3636a34c2fc): 1 (Operation not permitted)
Sep 29 17:03:39 baruwAdb3 mysqld: #011 at
galera/src/replicator_str.cpp:prepare_for_IST():447. IST will be
unavailable.
Sep 29 17:03:39 baruwAdb3 mysqld: 140929 17:03:39 [Note] WSREP: Member 0.0
(baruwadb3) requested state transfer from '*any*'. Selected 1.0
(baruwadb1)(SYNCED) as donor.
Sep 29 17:03:39 baruwAdb3 mysqld: 140929 17:03:39 [Note] WSREP: Shifting
PRIMARY -> JOINER (TO: 1)
Sep 29 17:03:39 baruwAdb3 mysqld: 140929 17:03:39 [Note] WSREP: Requesting
state transfer: success, donor: 1
Sep 29 17:03:39 baruwAdb3 rsyncd[26819]: connect from baruwadb1.jovenclub.cu
(172.17.1.2)
Sep 29 17:03:39 baruwAdb3 rsyncd[26819]: rsync to rsync_sst/ from
baruwadb1.jovenclub.cu (172.17.1.2)
Sep 29 17:03:39 baruwAdb3 rsyncd[26819]: receiving file list
Sep 29 17:03:43 baruwAdb3 rsyncd[26929]: connect from baruwadb1.jovenclub.cu
(172.17.1.2)
Sep 29 17:03:43 baruwAdb3 rsyncd[26819]: sent 44 bytes  received
209,766,748 bytes  total size 209,715,200
Sep 29 17:03:43 baruwAdb3 rsyncd[26929]: rsync to rsync_sst-log_dir/ from
baruwadb1.jovenclub.cu (172.17.1.2)
Sep 29 17:03:43 baruwAdb3 rsyncd[26929]: receiving file list
Sep 29 17:04:05 baruwAdb3 /etc/init.d/mysql[27564]: 0 processes alive and
'/usr/bin/mysqladmin --defaults-file=/etc/mysql/debian.cnf ping' resulted in
Sep 29 17:04:05 baruwAdb3 /etc/init.d/mysql[27564]:
#007/usr/bin/mysqladmin: connect to server at 'localhost' failed
Sep 29 17:04:05 baruwAdb3 /etc/init.d/mysql[27564]: error: 'Can't connect
to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (111
"Connection refused")'
Sep 29 17:04:05 baruwAdb3 /etc/init.d/mysql[27564]: Check that mysqld is
running and that the socket: '/var/run/mysqld/mysqld.sock' exists!
Sep 29 17:04:05 baruwAdb3 /etc/init.d/mysql[27564]:
Sep 29 17:04:14 baruwAdb3 rsyncd[27730]: connect from baruwadb1.jovenclub.cu
(172.17.1.2)
Sep 29 17:04:14 baruwAdb3 rsyncd[27731]: connect from baruwadb1.jovenclub.cu
(172.17.1.2)
Sep 29 17:04:14 baruwAdb3 rsyncd[26929]: sent 63 bytes  received
1,074,004,182 bytes  total size 1,073,741,824
Sep 29 17:04:14 baruwAdb3 rsyncd[27730]: rsync to
rsync_sst/./performance_schema from baruwadb1.jovenclub.cu (172.17.1.2)
Sep 29 17:04:14 baruwAdb3 rsyncd[27731]: rsync to rsync_sst/./sa_bayes from
baruwadb1.jovenclub.cu (172.17.1.2)
Sep 29 17:04:14 baruwAdb3 rsyncd[27731]: receiving file list
Sep 29 17:04:14 baruwAdb3 rsyncd[27730]: receiving file list
Sep 29 17:04:15 baruwAdb3 rsyncd[27758]: connect from baruwadb1.jovenclub.cu
(172.17.1.2)
Sep 29 17:04:15 baruwAdb3 rsyncd[27730]: sent 1,036 bytes  received 100,517
bytes  total size 96,357
Sep 29 17:04:15 baruwAdb3 rsyncd[27758]: rsync to rsync_sst/./mysql from
baruwadb1.jovenclub.cu (172.17.1.2)
Sep 29 17:04:15 baruwAdb3 rsyncd[27758]: receiving file list
Sep 29 17:04:16 baruwAdb3 rsyncd[27772]: connect from baruwadb1.jovenclub.cu
(172.17.1.2)
Sep 29 17:04:16 baruwAdb3 rsyncd[27758]: sent 1,682 bytes  received
1,139,500 bytes  total size 1,133,625
Sep 29 17:04:16 baruwAdb3 rsyncd[27772]: rsync to rsync_sst/./baruwa from
baruwadb1.jovenclub.cu (172.17.1.2)
Sep 29 17:04:16 baruwAdb3 rsyncd[27772]: receiving file list
Sep 29 17:04:18 baruwAdb3 rsyncd[27818]: connect from baruwadb1.jovenclub.cu
(172.17.1.2)
Sep 29 17:04:18 baruwAdb3 rsyncd[27731]: sent 238 bytes  received
93,682,655 bytes  total size 93,659,026
Sep 29 17:04:18 baruwAdb3 rsyncd[27818]: rsync to rsync_sst/./test from
baruwadb1.jovenclub.cu (172.17.1.2)
Sep 29 17:04:18 baruwAdb3 rsyncd[27818]: receiving file list
Sep 29 17:04:18 baruwAdb3 rsyncd[27818]: sent 48 bytes  received 205 bytes
total size 61

Thanks for your time. Regards

Follow ups

References