maria-discuss team mailing list archive

Thread
Date
2 node cluster crash

To: maria-discuss@xxxxxxxxxxxxxxxxxxx
From: "absolutely_free@xxxxxxxxx" <absolutely_free@xxxxxxxxx>
Date: Wed, 10 Jun 2015 09:50:30 +0200 (CEST)
Reply-to: "absolutely_free@xxxxxxxxx" <absolutely_free@xxxxxxxxx>
Hi, 
I have two node cluster with Mariadb (5.5.43)
Both servers are equal (RAM, CPU, disk) and they are in same lan segment.
I use official MariaDB package from this repository:

deb http://mariadb.kisiek.net//repo/5.5/debian wheezy main

I use haproxy with this config:

# cat haproxy.cfg
global
        log /dev/log    local0
        log /dev/log    local1 notice
        chroot /var/lib/haproxy
        user haproxy
        group haproxy
        daemon
        maxconn 8192

defaults
        retries 2
        timeout connect 30s
        timeout server 600s
        timeout client 600s
        log     global
    
listen mariadb-cluster
        bind 0.0.0.0:3306
        mode tcp
        option mysql-check user haproxy
        option tcpka
        balance leastconn
        server node1 192.168.100.1:3307 check maxconn 4096
        server node2 192.168.100.2:3307 check maxconn 4096

listen webinterface
        bind 0.0.0.0:8080
        mode http
        stats enable
        stats uri /
        stats realm Strictly\ Private
        stats auth Username:Password

Often (about 1 time per week) several applications that rely on databases on this cluster encounter problems.
When this happens, I check on both machines and I see that mysql is running.
When I stop mysql on one node (let's say, node2), all returns working fine.
I am trying to find the root cause of this behavior.

This is "show innodb status" (during crash):

*************************** 1. row ***************************
  Type: InnoDB
  Name: 
Status: 
=====================================
150423 16:22:13 INNODB MONITOR OUTPUT
=====================================
Per second averages calculated from the last 35 seconds
-----------------
BACKGROUND THREAD
-----------------
srv_master_thread loops: 477706 1_second, 477700 sleeps, 47769 10_second, 15 background, 15 flush
srv_master_thread log flush and writes: 507815
----------
SEMAPHORES
----------
OS WAIT ARRAY INFO: reservation count 526170, signal count 1091151
Mutex spin waits 2133026, rounds 7742983, OS waits 186497
RW-shared spins 383830, rounds 2284949, OS waits 64704
RW-excl spins 20327, rounds 8918087, OS waits 271756
Spin rounds per wait: 3.63 mutex, 5.95 RW-shared, 438.73 RW-excl
--------
FILE I/O
--------
I/O thread 0 state: waiting for completed aio requests (insert buffer thread)
I/O thread 1 state: waiting for completed aio requests (log thread)
I/O thread 2 state: waiting for completed aio requests (read thread)
I/O thread 3 state: waiting for completed aio requests (read thread)
I/O thread 4 state: waiting for completed aio requests (read thread)
I/O thread 5 state: waiting for completed aio requests (read thread)
I/O thread 6 state: waiting for completed aio requests (write thread)
I/O thread 7 state: waiting for completed aio requests (write thread)
I/O thread 8 state: waiting for completed aio requests (write thread)
I/O thread 9 state: waiting for completed aio requests (write thread)
Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] ,
 ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0
Pending flushes (fsync) log: 0; buffer pool: 0
497313 OS file reads, 42388195 OS file writes, 22834746 OS fsyncs
0.17 reads/s, 16384 avg bytes/read, 1.57 writes/s, 1.00 fsyncs/s
-------------------------------------
INSERT BUFFER AND ADAPTIVE HASH INDEX
-------------------------------------
Ibuf: size 1, free list len 213, seg size 215, 46416 merges
merged operations:
 insert 121237, delete mark 2178, delete 701
discarded operations:
 insert 0, delete mark 0, delete 0
Hash table size 17700827, node heap has 4633 buffer(s)
65.94 hash searches/s, 155.85 non-hash searches/s
---
LOG
---
Log sequence number 513508907441
Log flushed up to   513508907225
Last checkpoint at  513508907225
Max checkpoint age    848635454
Checkpoint age target 822115597
Modified age          216
Checkpoint age        216
0 pending log writes, 0 pending chkp writes
21724628 log i/o's done, 0.63 log i/o's/second
----------------------
BUFFER POOL AND MEMORY
----------------------
Total memory allocated 8816427008; in additional pool allocated 0
Total memory allocated by read views 690656
Internal hash tables (constant factor + variable factor)
    Adaptive hash index 217517904     (141606616 + 75911288)
    Page hash           8851208 (buffer pool 0 only)
    Dictionary cache    38436882     (35403184 + 3033698)
    File system         328480     (82672 + 245808)
    Lock system         21567352     (21250568 + 316784)
    Recovery system     0     (0 + 0)
Dictionary memory allocated 3033698
Buffer pool size        524287
Buffer pool size, bytes 8589918208
Free buffers            0
Database pages          519654
Old database pages      191805
Modified db pages       2
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 379103, not young 0
0.17 youngs/s, 0.00 non-youngs/s
Pages read 496737, created 768744, written 20279635
0.17 reads/s, 0.00 creates/s, 0.83 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 519654, unzip_LRU len: 0
I/O sum[40]:cur[1], unzip sum[0]:cur[0]
--------------
ROW OPERATIONS
--------------
0 queries inside InnoDB, 0 queries in queue
1 read views open inside InnoDB
611 transactions active inside InnoDB
611 out of 1000 descriptors used
---OLDEST VIEW---
Normal read view
Read view low limit trx n:o 2E3F7975
Read view up limit trx id 2E3D4295
Read view low limit trx id 2E3F7975
Read view individually stored trx ids:
Read view trx id 2E3F7974
Read view trx id 2E3F7973
Read view trx id 2E3F796B
Read view trx id 2E3F7968
Read view trx id 2E3F7936
Read view trx id 2E3F791E
Read view trx id 2E3F7860
Read view trx id 2E3F7859
Read view trx id 2E3F77CE
Read view trx id 2E3F76D0
Read view trx id 2E3F75DB
Read view trx id 2E3F7578
Read view trx id 2E3F74E3
Read view trx id 2E3F74E0
Read view trx id 2E3F74C2
Read view trx id 2E3F74B1
Read view trx id 2E3F7473
Read view trx id 2E3F7472
Read view trx id 2E3F7441
Read view trx id 2E3F73D2
Read view trx id 2E3F72D1
Read view trx id 2E3F72D0
Read view trx id 2E3F7211
Read view trx id 2E3F71F1
Read view trx id 2E3F71EE
Read view trx id 2E3F71ED
Read view trx id 2E3F71E1
Read view trx id 2E3F71E0
Read view trx id 2E3F71DF
Read view trx id 2E3F713C
Read view trx id 2E3F707C
Read view trx id 2E3F6F91
Read view trx id 2E3F6C5D
Read view trx id 2E3F6C5C
Read view trx id 2E3F6C51
Read view trx id 2E3F6C50
Read view trx id 2E3F6C0B
Read view trx id 2E3F6B2D
Read view trx id 2E3F6AFE
Read view trx id 2E3F6AA5
Read view trx id 2E3F69B3
Read view trx id 2E3F695F
Read view trx id 2E3F6907
Read view trx id 2E3F68E0
Read view trx id 2E3F6898
Read view trx id 2E3F6878
Read view trx id 2E3F6854
Read view trx id 2E3F682D
Read view trx id 2E3F682A
Read view trx id 2E3F6813
Read view trx id 2E3F6531
Read view trx id 2E3F61E1
Read view trx id 2E3F6190
Read view trx id 2E3F6147
Read view trx id 2E3F60CE
Read view trx id 2E3F5FC3
Read view trx id 2E3F5FC1
Read view trx id 2E3F5F82
Read view trx id 2E3F5F79
Read view trx id 2E3F5F53
Read view trx id 2E3F5F52
Read view trx id 2E3F5EE0
Read view trx id 2E3F5CF1
Read view trx id 2E3F5B28
Read view trx id 2E3F5AFA
Read view trx id 2E3F5AE3
Read view trx id 2E3F5995
Read view trx id 2E3F58C8
Read view trx id 2E3F55DF
Read view trx id 2E3F5586
Read view trx id 2E3F5393
Read view trx id 2E3F5267
Read view trx id 2E3F524B
Read view trx id 2E3F5208
Read view trx id 2E3F4B19
Read view trx id 2E3F49C7
Read view trx id 2E3F4569
Read view trx id 2E3F40D8
Read view trx id 2E3F40D6
Read view trx id 2E3F4095
Read view trx id 2E3F4009
Read view trx id 2E3F3DF6
Read view trx id 2E3F3DE5
Read view trx id 2E3F39D1
Read view trx id 2E3F36E9
Read view trx id 2E3F3666
Read view trx id 2E3F35AF
Read view trx id 2E3F340D
Read view trx id 2E3F3389
Read view trx id 2E3F3302
Read view trx id 2E3F3298
Read view trx id 2E3F3279
Read view trx id 2E3F3278
Read view trx id 2E3F3199
Read view trx id 2E3F30F3
Read view trx id 2E3F2F41
Read view trx id 2E3F2ECE
Read view trx id 2E3F2E12
Read view trx id 2E3F2E07
Read view trx id 2E3F2D2B
Read view trx id 2E3F2D22
Read view trx id 2E3F2D21
Read view trx id 2E3F2CC1
Read view trx id 2E3F2A96
Read view trx id 2E3F2A6A
Read view trx id 2E3F2A08
Read view trx id 2E3F24C5
Read view trx id 2E3F236A
Read view trx id 2E3F2149
Read view trx id 2E3F20D8
Read view trx id 2E3F1FA4
Read view trx id 2E3F1F88
Read view trx id 2E3F1EBF
Read view trx id 2E3F1E02
Read view trx id 2E3F1C7C
Read view trx id 2E3F1BF9
Read view trx id 2E3F1BE1
Read view trx id 2E3F1BD2
Read view trx id 2E3F1BB5
Read view trx id 2E3F1BA6
Read view trx id 2E3F1BA5
Read view trx id 2E3F1A75
Read view trx id 2E3F19B2
Read view trx id 2E3F191B
Read view trx id 2E3F1848
Read view trx id 2E3F17A8
Read view trx id 2E3F1761
Read view trx id 2E3F1501
Read view trx id 2E3F14C6
Read view trx id 2E3F14C5
Read view trx id 2E3F13BA
Read view trx id 2E3F13B4
Read view trx id 2E3F11DE
Read view trx id 2E3F1102
Read view trx id 2E3F0E4D
Read view trx id 2E3F09EA
Read view trx id 2E3F0994
Read view trx id 2E3F0993
Read view trx id 2E3F08E3
Read view trx id 2E3F06FE
Read view trx id 2E3F06A3
Read view trx id 2E3F050C
Read view trx id 2E3F043F
Read view trx id 2E3F031D
Read view trx id 2E3F0314
Read view trx id 2E3F0194
Read view trx id 2E3F003D
Read view trx id 2E3EF007
Read view trx id 2E3EEEBD
Read view trx id 2E3EEE8C
Read view trx id 2E3EEE83
Read view trx id 2E3EEE70
Read view trx id 2E3EEE06
Read view trx id 2E3EE5B0
Read view trx id 2E3EE51A
Read view trx id 2E3EE3E3
Read view trx id 2E3EE3C6
Read view trx id 2E3EE3A4
Read view trx id 2E3EE38A
Read view trx id 2E3EE2E3
Read view trx id 2E3EE2CD
Read view trx id 2E3EE197
Read view trx id 2E3EE05C
Read view trx id 2E3EE05B
Read view trx id 2E3EDC81
Read view trx id 2E3EDC4F
Read view trx id 2E3EDC06
Read view trx id 2E3EDBC7
Read view trx id 2E3ED910
Read view trx id 2E3ED900
Read view trx id 2E3ED8E4
Read view trx id 2E3ED8B5
Read view trx id 2E3ED77F
Read view trx id 2E3ED765
Read view trx id 2E3ED656
Read view trx id 2E3ED3D3
Read view trx id 2E3ED330
Read view trx id 2E3ED214
Read view trx id 2E3ED1C1
Read view trx id 2E3ED1C0
Read view trx id 2E3ED1A0
Read view trx id 2E3ED0F5
Read view trx id 2E3ECEB3
Read view trx id 2E3ECE05
Read view trx id 2E3ECC27
Read view trx id 2E3ECBB1
Read view trx id 2E3ECB95
Read view trx id 2E3ECB4F
Read view trx id 2E3ECADE
Read view trx id 2E3ECAD5
Read view trx id 2E3EC83E
Read view trx id 2E3EC5A4
Read view trx id 2E3EC50E
Read view trx id 2E3EC4B1
Read view trx id 2E3EC287
Read view trx id 2E3EC080
Read view trx id 2E3EBFB7
Read view trx id 2E3EBE9E
Read view trx id 2E3EBD8A
Read view trx id 2E3EBCB8
Read view trx id 2E3EBC33
Read view trx id 2E3EBB70
Read view trx id 2E3EBB3B
Read view trx id 2E3EBA2C
Read view trx id 2E3EB631
Read view trx id 2E3EB435
Read view trx id 2E3EA1B8
Read view trx id 2E3E9FAA
Read view trx id 2E3E9F18
Read view trx id 2E3E9EE4
Read view trx id 2E3E9DAB
Read view trx id 2E3E9B23
Read view trx id 2E3E9902
Read view trx id 2E3E9785
Read view trx id 2E3E9781
Read view trx id 2E3E960F
Read view trx id 2E3E95FE
Read view trx id 2E3E95BA
Read view trx id 2E3E927B
Read view trx id 2E3E919B
Read view trx id 2E3E914B
Read view trx id 2E3E908D
Read view trx id 2E3E905A
Read view trx id 2E3E9052
Read view trx id 2E3E8FD0
Read view trx id 2E3E8FC9
Read view trx id 2E3E8FA0
Read view trx id 2E3E8F9F
Read view trx id 2E3E8F8A
Read view trx id 2E3E8F87
Read view trx id 2E3E8E9A
Read view trx id 2E3E8D88
Read view trx id 2E3E8C14
Read view trx id 2E3E8B71
Read view trx id 2E3E8B44
Read view trx id 2E3E8B1E
Read view trx id 2E3E8AD8
Read view trx id 2E3E8A48
Read view trx id 2E3E8980
Read view trx id 2E3E8893
Read view trx id 2E3E87EE
Read view trx id 2E3E87ED
Read view trx id 2E3E87D6
Read view trx id 2E3E87B5
Read view trx id 2E3E85A1
Read view trx id 2E3E8555
Read view trx id 2E3E8553
Read view trx id 2E3E853E
Read view trx id 2E3E84E7
Read view trx id 2E3E84E5
Read view trx id 2E3E84E4
Read view trx id 2E3E84E3
Read view trx id 2E3E849D
Read view trx id 2E3E849C
Read view trx id 2E3E841F
Read view trx id 2E3E83A0
Read view trx id 2E3E8202
Read view trx id 2E3E80AC
Read view trx id 2E3E80A3
Read view trx id 2E3E7EA5
Read view trx id 2E3E7CF5
Read view trx id 2E3E7B7B
Read view trx id 2E3E7B3A
Read view trx id 2E3E7A25
Read view trx id 2E3E7A16
Read view trx id 2E3E79F7
Read view trx id 2E3E7999
Read view trx id 2E3E7998
Read view trx id 2E3E790E
Read view trx id 2E3E7693
Read view trx id 2E3E7559
Read view trx id 2E3E658F
Read view trx id 2E3E64A2
Read view trx id 2E3E6467
Read view trx id 2E3E6236
Read view trx id 2E3E6155
Read view trx id 2E3E611B
Read view trx id 2E3E60B3
Read view trx id 2E3E609D
Read view trx id 2E3E5F9B
Read view trx id 2E3E5F95
Read view trx id 2E3E5E55
Read view trx id 2E3E5DE1
Read view trx id 2E3E5DB6
Read view trx id 2E3E5B49
Read view trx id 2E3E5AC6
Read view trx id 2E3E59CA
Read view trx id 2E3E57E5
Read view trx id 2E3E57E4
Read view trx id 2E3E57E3
Read view trx id 2E3E57A2
Read view trx id 2E3E576D
Read view trx id 2E3E5508
Read view trx id 2E3E5424
Read view trx id 2E3E53C6
Read view trx id 2E3E52D2
Read view trx id 2E3E52B8
Read view trx id 2E3E52B7
Read view trx id 2E3E5255
Read view trx id 2E3E5227
Read view trx id 2E3E5121
Read view trx id 2E3E5009
Read view trx id 2E3E4F09
Read view trx id 2E3E4EC7
Read view trx id 2E3E4EAA
Read view trx id 2E3E4EA9
Read view trx id 2E3E4DEA
Read view trx id 2E3E4DD9
Read view trx id 2E3E4C81
Read view trx id 2E3E4C63
Read view trx id 2E3E4AC2
Read view trx id 2E3E49F8
Read view trx id 2E3E4973
Read view trx id 2E3E4840
Read view trx id 2E3E42C6
Read view trx id 2E3E4228
Read view trx id 2E3E41C6
Read view trx id 2E3E416C
Read view trx id 2E3E4145
Read view trx id 2E3E3D89
Read view trx id 2E3E3CFF
Read view trx id 2E3E3B2B
Read view trx id 2E3E3ABD
Read view trx id 2E3E3A33
Read view trx id 2E3E3922
Read view trx id 2E3E36BF
Read view trx id 2E3E359B
Read view trx id 2E3E3585
Read view trx id 2E3E31A1
Read view trx id 2E3E309B
Read view trx id 2E3E3055
Read view trx id 2E3E2F6E
Read view trx id 2E3E2F2D
Read view trx id 2E3E2E8A
Read view trx id 2E3E2E89
Read view trx id 2E3E2DE5
Read view trx id 2E3E29F3
Read view trx id 2E3E2907
Read view trx id 2E3E286E
Read view trx id 2E3E282E
Read view trx id 2E3E27E8
Read view trx id 2E3E25FD
Read view trx id 2E3E215B
Read view trx id 2E3E1F9D
Read view trx id 2E3E1EDA
Read view trx id 2E3E1EBA
Read view trx id 2E3E1EAB
Read view trx id 2E3E1E22
Read view trx id 2E3E1E0A
Read view trx id 2E3E1D76
Read view trx id 2E3E1D35
Read view trx id 2E3E1D17
Read view trx id 2E3E1C6D
Read view trx id 2E3E19EF
Read view trx id 2E3E181F
Read view trx id 2E3E1652
Read view trx id 2E3E1198
Read view trx id 2E3E115B
Read view trx id 2E3E0D88
Read view trx id 2E3E0D87
Read view trx id 2E3E0D24
Read view trx id 2E3DF415
Read view trx id 2E3DF31E
Read view trx id 2E3DF31C
Read view trx id 2E3DF2D2
Read view trx id 2E3DF134
Read view trx id 2E3DF119
Read view trx id 2E3DF032
Read view trx id 2E3DF004
Read view trx id 2E3DEFA5
Read view trx id 2E3DEF63
Read view trx id 2E3DEE30
Read view trx id 2E3DEDD5
Read view trx id 2E3DEC83
Read view trx id 2E3DEC30
Read view trx id 2E3DEBF3
Read view trx id 2E3DEBEC
Read view trx id 2E3DEBE0
Read view trx id 2E3DEA93
Read view trx id 2E3DE9AF
Read view trx id 2E3DE975
Read view trx id 2E3DE8D5
Read view trx id 2E3DE8D4
Read view trx id 2E3DE879
Read view trx id 2E3DE868
Read view trx id 2E3DE801
Read view trx id 2E3DE7D6
Read view trx id 2E3DE7A0
Read view trx id 2E3DE79F
Read view trx id 2E3DE77A
Read view trx id 2E3DE779
Read view trx id 2E3DE75F
Read view trx id 2E3DE72B
Read view trx id 2E3DE6C6
Read view trx id 2E3DE65B
Read view trx id 2E3DE64A
Read view trx id 2E3DE616
Read view trx id 2E3DE541
Read view trx id 2E3DE52B
Read view trx id 2E3DE520
Read view trx id 2E3DE51F
Read view trx id 2E3DE4AD
Read view trx id 2E3DE443
Read view trx id 2E3DE43A
Read view trx id 2E3DE3E1
Read view trx id 2E3DE3B2
Read view trx id 2E3DE371
Read view trx id 2E3DE313
Read view trx id 2E3DE277
Read view trx id 2E3DE265
Read view trx id 2E3DE14F
Read view trx id 2E3DE111
Read view trx id 2E3DDFE6
Read view trx id 2E3DDEB5
Read view trx id 2E3DDE74
Read view trx id 2E3DDE6E
Read view trx id 2E3DDE6D
Read view trx id 2E3DDE69
Read view trx id 2E3DDE66
Read view trx id 2E3DDD86
Read view trx id 2E3DDD69
Read view trx id 2E3DDBF3
Read view trx id 2E3DDBC1
Read view trx id 2E3DDB2F
Read view trx id 2E3DD971
Read view trx id 2E3DD96D
Read view trx id 2E3DD91F
Read view trx id 2E3DD67A
Read view trx id 2E3DD663
Read view trx id 2E3DD60C
Read view trx id 2E3DD4C3
Read view trx id 2E3DD4BF
Read view trx id 2E3DD3F3
Read view trx id 2E3DD36B
Read view trx id 2E3DD2DE
Read view trx id 2E3DD2DD
Read view trx id 2E3DD2DC
Read view trx id 2E3DD2DB
Read view trx id 2E3DD2DA
Read view trx id 2E3DD2D4
Read view trx id 2E3DD2A9
Read view trx id 2E3DD2A8
Read view trx id 2E3DD28D
Read view trx id 2E3DD27B
Read view trx id 2E3DD18E
Read view trx id 2E3DD114
Read view trx id 2E3DCDF6
Read view trx id 2E3DCCB0
Read view trx id 2E3DCAB7
Read view trx id 2E3DCA02
Read view trx id 2E3DC8D3
Read view trx id 2E3DC8CB
Read view trx id 2E3DC8C5
Read view trx id 2E3DC852
Read view trx id 2E3DC7FA
Read view trx id 2E3DC60C
Read view trx id 2E3DBD6D
Read view trx id 2E3DB0C4
Read view trx id 2E3DAB06
Read view trx id 2E3DAA1A
Read view trx id 2E3DA9EB
Read view trx id 2E3DA5C3
Read view trx id 2E3DA5AB
Read view trx id 2E3DA58B
Read view trx id 2E3DA4CF
Read view trx id 2E3DA3E9
Read view trx id 2E3DA046
Read view trx id 2E3D9EB8
Read view trx id 2E3D9DC7
Read view trx id 2E3D9AAD
Read view trx id 2E3D9A82
Read view trx id 2E3D9A7A
Read view trx id 2E3D9A79
Read view trx id 2E3D99FA
Read view trx id 2E3D93AA
Read view trx id 2E3D9289
Read view trx id 2E3D9202
Read view trx id 2E3D9183
Read view trx id 2E3D9151
Read view trx id 2E3D90FB
Read view trx id 2E3D90DD
Read view trx id 2E3D8EB3
Read view trx id 2E3D8E51
Read view trx id 2E3D8D52
Read view trx id 2E3D8C30
Read view trx id 2E3D8C1A
Read view trx id 2E3D8C19
Read view trx id 2E3D8BA5
Read view trx id 2E3D89AD
Read view trx id 2E3D87AE
Read view trx id 2E3D8751
Read view trx id 2E3D8733
Read view trx id 2E3D867F
Read view trx id 2E3D8366
Read view trx id 2E3D81CE
Read view trx id 2E3D7F01
Read view trx id 2E3D7EFF
Read view trx id 2E3D7B73
Read view trx id 2E3D7B2D
Read view trx id 2E3D7A9A
Read view trx id 2E3D7907
Read view trx id 2E3D78A3
Read view trx id 2E3D785C
Read view trx id 2E3D77FF
Read view trx id 2E3D7639
Read view trx id 2E3D75BD
Read view trx id 2E3D73D5
Read view trx id 2E3D731E
Read view trx id 2E3D7134
Read view trx id 2E3D7083
Read view trx id 2E3D7082
Read view trx id 2E3D6FFD
Read view trx id 2E3D6FD4
Read view trx id 2E3D6FBC
Read view trx id 2E3D6F31
Read view trx id 2E3D6F30
Read view trx id 2E3D6F05
Read view trx id 2E3D6EFD
Read view trx id 2E3D6EF5
Read view trx id 2E3D6EA9
Read view trx id 2E3D6E0A
Read view trx id 2E3D6DA8
Read view trx id 2E3D6D19
Read view trx id 2E3D6B2E
Read view trx id 2E3D6967
Read view trx id 2E3D68FA
Read view trx id 2E3D61A2
Read view trx id 2E3D60F7
Read view trx id 2E3D60E5
Read view trx id 2E3D5FC7
Read view trx id 2E3D5F2D
Read view trx id 2E3D5E93
Read view trx id 2E3D5E4D
Read view trx id 2E3D5CA1
Read view trx id 2E3D5B90
Read view trx id 2E3D59B6
Read view trx id 2E3D5996
Read view trx id 2E3D5953
Read view trx id 2E3D58FB
Read view trx id 2E3D58E7
Read view trx id 2E3D58CF
Read view trx id 2E3D57DA
Read view trx id 2E3D57D2
Read view trx id 2E3D57BC
Read view trx id 2E3D5771
Read view trx id 2E3D5738
Read view trx id 2E3D5729
Read view trx id 2E3D5728
Read view trx id 2E3D56F6
Read view trx id 2E3D56F5
Read view trx id 2E3D56DE
Read view trx id 2E3D56DD
Read view trx id 2E3D56B3
Read view trx id 2E3D5696
Read view trx id 2E3D566B
Read view trx id 2E3D55E1
Read view trx id 2E3D55B1
Read view trx id 2E3D5599
Read view trx id 2E3D54F7
Read view trx id 2E3D54C8
Read view trx id 2E3D5496
Read view trx id 2E3D5463
Read view trx id 2E3D5422
Read view trx id 2E3D5382
Read view trx id 2E3D52EC
Read view trx id 2E3D5270
Read view trx id 2E3D521F
Read view trx id 2E3D50C3
Read view trx id 2E3D501C
Read view trx id 2E3D4F8A
Read view trx id 2E3D4EC8
Read view trx id 2E3D4E71
Read view trx id 2E3D4DFC
Read view trx id 2E3D4D7E
Read view trx id 2E3D4D3A
Read view trx id 2E3D4D06
Read view trx id 2E3D4D05
Read view trx id 2E3D4D04
Read view trx id 2E3D4D02
Read view trx id 2E3D4D01
Read view trx id 2E3D4CFF
Read view trx id 2E3D4CCD
Read view trx id 2E3D4C57
Read view trx id 2E3D4C4B
Read view trx id 2E3D4BC2
Read view trx id 2E3D4B78
Read view trx id 2E3D4B4C
Read view trx id 2E3D4B31
Read view trx id 2E3D48EE
Read view trx id 2E3D4860
Read view trx id 2E3D47DA
Read view trx id 2E3D4768
Read view trx id 2E3D4604
Read view trx id 2E3D45F8
Read view trx id 2E3D44F1
Read view trx id 2E3D44D9
Read view trx id 2E3D449E
Read view trx id 2E3D441E
Read view trx id 2E3D437C
Read view trx id 2E3D437B
Read view trx id 2E3D436E
Read view trx id 2E3D4302
Read view trx id 2E3D42BD
Read view trx id 2E3D42AD
Read view trx id 2E3D42A9
Read view trx id 2E3D42A8
Read view trx id 2E3D42A7
Read view trx id 2E3D42A6
Read view trx id 2E3D429E
Read view trx id 2E3D4295
-----------------
Main thread process no. 42459, id 140580935939840, state: sleeping
Number of rows inserted 1374217, updated 259069615, deleted 360531, read 5191170325
0.20 inserts/s, 0.54 updates/s, 0.00 deletes/s, 89.03 reads/s
------------

TABLE LOCK table `roundcube`.`session` trx id 2E3F7859 lock mode IX
RECORD LOCKS space id 227 page no 350 n bits 80 index `PRIMARY` of table `roundcube`.`session` trx id 2E3F7859 lock mode S locks rec but not gap waiting
---TRANSACTION 2E3F77CE, ACTIVE 6 sec inserting
mysql tables in use 1, locked 1
LOCK WAIT 2 lock struct(s), heap size 376, 1 row lock(s)
MySQL thread id 4288287, OS thread handle 0x7fdadfc8d700, query id 44864987 node1 192.168.100.1 roundcube update
INSERT INTO `session` (`sess_id`, `vars`, `ip`, `created`, `changed`) VALUES ('44f5q5mncbdqh1kskcsr7018s3', 'bGFuZ3VhZ2V8czo1OiJpdF9JVCI7dGVtcHxiOjE7c2tpbnxzOjU6Im1pY3NvIjs=', '213.xxxxxxxx', now(), now())
------- TRX HAS BEEN WAITING 6 SEC FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 227 page no 378 n bits 80 index `PRIMARY` of table `roundcube`.`session` trx id 2E3F77CE lock mode S locks rec but not gap waiting
------------------
---TRANSACTION 2E3D4302, ACTIVE 2079 sec
mysql tables in use 1, locked 1
94 lock struct(s), heap size 14776, 279 row lock(s), undo log entries 279
MySQL thread id 4283828, OS thread handle 0x7fdde7bfe700, query id 44714554 node1 192.168.100.1 db_dspam_admin query end
UPDATE dspam_token_data SET last_hit=CURRENT_DATE(),innocent_hits=innocent_hits+1 WHERE uid=0 AND token IN ('11208966279506063560','14355575044071913407','8469385930432278733','7966374547679573945','3426741056644935886','7964984027734112185','13945758747987769275','11207568337737381816','7966374566296844217','11208965523571177400','7964979679860519113','13945759985008537803','3426745367375869886','9262020720305336524','8302736407489378233','15747196743497709503','11208965542138376392','3618985354802917582','11208959957244014792','182759523735400642','9262016372543223740','7964972414031067065',
TABLE LOCK table `db_dspam`.`dspam_token_data` trx id 2E3D4302 lock mode IX
RECORD LOCKS space id 17 page no 28 n bits 384 index `id_token_data_01` of table `db_dspam`.`dspam_token_data` trx id 2E3D4302 lock_mode X locks rec but not gap
RECORD LOCKS space id 17 page no 1757 n bits 320 index `id_token_data_01` of table `db_dspam`.`dspam_token_data` trx id 2E3D4302 lock_mode X locks rec but not gap
RECORD LOCKS space id 17 page no 1655 n bits 384 index `id_token_data_01` of table `db_dspam`.`dspam_token_data` trx id 2E3D4302 lock_mode X locks rec but not gap
RECORD LOCKS space id 17 page no 9372 n bits 376 index `id_token_data_01` of table `db_dspam`.`dspam_token_data` trx id 2E3D4302 lock_mode X locks rec but not gap
RECORD LOCKS space id 17 page no 2630 n bits 248 index `id_token_data_01` of table `db_dspam`.`dspam_token_data` trx id 2E3D4302 lock_mode X locks rec but not gap
RECORD LOCKS space id 17 page no 986 n bits 376 index `id_token_data_01` of table `db_dspam`.`dspam_token_data` trx id 2E3D4302 lock_mode X locks rec but not gap
RECORD LOCKS space id 17 page no 52091 n bits 384 index `id_token_data_01` of table `db_dspam`.`dspam_token_data` trx id 2E3D4302 lock_mode X locks rec but not gap
RECORD LOCKS space id 17 page no 1959 n bits 384 index `id_token_data_01` of table `db_dspam`.`dspam_token_data` trx id 2E3D4302 lock_mode X locks rec but not gap
RECORD LOCKS space id 17 page no 51755 n bits 264 index `id_token_data_01` of table `db_dspam`.`dspam_token_data` trx id 2E3D4302 lock_mode X locks rec but not gap
TOO MANY LOCKS PRINTED FOR THIS TRX: SUPPRESSING FURTHER PRINTS

..
..

Thank you