← Back to team overview

maria-discuss team mailing list archive

Re: MariaDB server horribly slow on start

 

There is no supported downgrade path other than logical dump+restore.
There are also no packages built for distros where the major version is
older than what ships with the distro.

Since your queries seem to end up stuck in commit stage, it could be
related to redo log flushing, which behaves very erratically on 10.5+. If
it leaves the log to fill up to 90% and the state transfer hits, it could
be that with the checkpoint age already high, there just isn't enough
headroom to avoid a massive stall. Purely guessing here without any
telemetry.

So it seems you are, most unfortunately, in for an awful lot of pain.

You seem to have multiple instances of some settings in your config, that
is never a good sign.

Do you have PMM set up on this cluster and fully instrumented?


On Wed, 27 Jul 2022, 14:04 Cédric Counotte, <cedric.counotte@xxxxxxxxxx>
wrote:

> I started with 10.4.x back in January, exact same issue.
>
>
>
> Now on Ubuntu 22.04 it comes with MariaDB 10.6 and I was forced to upgrade!
>
>
>
> I’ll test downgrading somehow, but on mariadb downloads I can’t find 22.04
> version?!
>
> Download MariaDB Server - MariaDB.org
> <https://mariadb.org/download/?t=repo-config&d=21.10+%22impish%22&v=10.5&r_m=mva>
>
>
>
>
>
> *De :* Gordan Bobic <gordan.bobic@xxxxxxxxx>
> *Envoyé :* mercredi 27 juillet 2022 13:02
> *À :* Cédric Counotte <cedric.counotte@xxxxxxxxxx>
> *Cc :* William Edwards <wedwards@xxxxxxxxxxxxxx>;
> maria-discuss@xxxxxxxxxxxxxxxxxxx
> *Objet :* Re: [Maria-discuss] MariaDB server horribly slow on start
>
>
>
> IMO your biggest cause of problems is chasing the bleeding edge.
>
> You are on 10.6.x. I generally don't even deploy 10.5.x in production yet
> except in very exceptional circumstances (simple async replication, no
> expectation of extreme performance).
>
> Migrate to 10.4.x and stick with it until it goes EOL. Then maybe think
> about upgrading to 10.5.x in the 6 months following 10.4.x EOL.
>
>
>
> On Wed, Jul 27, 2022 at 1:50 PM Cédric Counotte <
> cedric.counotte@xxxxxxxxxx> wrote:
>
> I must add that the slowness occurs when the SST or IST are completed, and
> the server becomes available. Donor is also entirely blocked (it seems)
> when the queue is being processed.
>
>
>
> While the SST is performed, there is no issue.
>
>
>
> *De :* William Edwards <wedwards@xxxxxxxxxxxxxx>
> *Envoyé :* mercredi 27 juillet 2022 11:58
> *À :* Cédric Counotte <cedric.counotte@xxxxxxxxxx>
> *Cc :* maria-discuss@xxxxxxxxxxxxxxxxxxx
> *Objet :* Re: [Maria-discuss] MariaDB server horribly slow on start
>
>
>
>
>
> Op 27 jul. 2022 om 11:46 heeft Cédric Counotte <cedric.counotte@xxxxxxxxxx>
> het volgende geschreven:
>
> 
>
> Hello all. I hope I’m at the right place to ask this question.
>
> I opened a bug here: https://jira.mariadb.org/browse/MDEV-28969, however
> I was told to use this mailing list.
>
>
>
> We have 4 MariaDB servers in a Galera Cluster and it happens that a server
> has to be restarted (be it for a crash which I have to open a bug for) or
> maintenance.
>
>
>
> When that happens, the restarted server is causing huge slow down on the
> whole cluster, and it lasts for 10 to 30 minutes at the very least!
>
>
>
> And by huge, I mean huge, we end up with 500 to 800 pending queries on all
> servers as you can see on attached screenshots
>
> I’ve attached the configuration of any server for reference in case this
> is the source of the issue.
>
>
>
> Any way to solve this would be greatly appreciated.
>
>
>
> You seem to be focusing on effect. What is the cause? SST?
>
>
>
>
> Regards,
>
> 3C.
>
> _______________________________________________
> Mailing list: https://launchpad.net/~maria-discuss
> Post to     : maria-discuss@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~maria-discuss
> More help   : https://help.launchpad.net/ListHelp
>
> _______________________________________________
> Mailing list: https://launchpad.net/~maria-discuss
> Post to     : maria-discuss@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~maria-discuss
> More help   : https://help.launchpad.net/ListHelp
>
>

PNG image

PNG image


Follow ups

References