maria-discuss team mailing list archive

[Gordan Bobic <gordan.bobic@xxxxxxxxx>]

On Thu, Dec 8, 2022 at 10:59 PM Reindl Harald <h.reindl@xxxxxxxxxxxxx> wrote:
>
> the next reply-all clown.... boy i am subsribed so you can just respond
> to the list

You sure whine a lot about inconsequential things.

> Am 08.12.22 um 21:13 schrieb Gordan Bobic:
> > On Thu, Dec 8, 2022 at 9:42 PM Reindl Harald <h.reindl@xxxxxxxxxxxxx> wrote:
> >>
> >> Am 08.12.22 um 18:59 schrieb Gordan Bobic:
> >>> On Thu, Dec 8, 2022 at 7:28 PM Reindl Harald <h.reindl@xxxxxxxxxxxxx> wrote:
> >>>> MariaDB does the same as the filesystem
> >>>> InnoDB in fact is more ore less a FS on top of a FS
> >>>
> >>> So why do it at both levels?
> >>
> >> because the FS layer can't detect MariaDB errors?
> >
> > What is the net benefit of detecting said error? The way I see it, the
> > options are:
> > 1) MariaDB detects and error, crashes out
> > 2) MariaDB doesn't detect an error, ingests garbage, crashes out
>
> silent data corruption

At what layer? If at anything below, up to and including the file
system level, ZFS will detect and correct it for you.
MariaDB can at best only detect it for you. So catching it at a lower
level wins.

> > The only way an error will creep in without the error checking FS
> > spotting it is:
> > 1) manually corrupting the block by writing garbage over it
> > 2) MariaDB generated a duff block and wrote it out
> > 3) Some other hardware failure corrupted the block in MariaDB memory,
> > just before writing it to the file system
> >
> > If any of the latter set happens, the data is toast anyway.
> > If the former set happens, the data is toast anyway.
>
> silent data corruption

At what layer? If at anything below, up to and including the file
system level, ZFS will detect and correct it for you.
MariaDB can at best only detect it for you. So catching it at a lower
level wins.

> > Sure it's nice to get an error that confirms that your data is
> > corrupted, but that won't bring it back
> silent data corruption
>
> > Shift it down to a layer where at least a subset of the problemspace
> > can be fixed and we have a net gain in at least some cases.
>
> different world

A far better world that you seem have a hard time accepting exists.

> >>> And what makes doing it at MariaDB level
> >>> in any way better than doing it somewhere else?
> >>
> >> which magic should do it somewehre else?
> >
> > If a file system is in control of data mirroring and checksumming
> > every 16KB block, then if the data is corrupted on disk, file system
> > will detect it on a read and fetch a good copy from an uncorrupted
> > mirror.
>
> moron the files are fine from the viewpoint of the filessystem

At what layer? If at anything below, up to and including the file
system level, ZFS will detect and correct it for you.
MariaDB can at best only detect it for you. So catching it at a lower
level wins.
I am talking about total cumulative data integrity up to and including
what MariaDB reads from disks.
What layer in the stack, exactly, are you trying to fight "silent data
corruption" you mentioned above?

> > Application never knows something went wrong, file system replaces the
> > corrupted block on the other disk and everything carries on
> > uninterrupted.
>
> moron the files are fine from the viewpoint of the filessystem
>
> >> AGAIN: THE FILESYSTEM CAN'T DO ANYTHING BECAUSE IT'S NOT AFFECTED
> >
> > So your argument is that the page checksum is there to tell you that
> > your data is corrupted because of either:
> > 1) data was corrupted by the database itself, or
> > 2) a superuser overwriting the block on the disk
>
> and it's impressive how long it takes until you mororn understand such
> basics

And in that case what exactly will the ckecksum do for you?
What do you gain from it? The error message that the block is
corrupted just before the database crashes?

> > In either case, you are not getting the data back either way the the
> > database will stop working.
>
> yes, but it's a difference if you have silent data corruption detectet
> months later when clean backups are gone because retention times
>
> > So what is the point? I'd rather have the error fixable
>
> moron your filesystem can't fix that type of error
>
> that's why "Some of us run MariaDB on file systems that do their own
> block checksumming, and thus run innodb_checksum_algorithm=none" was an
> idiots response
>
> a different response would have been "i don't think that i need innodb
> checksums and want to disable it" but you moron pretended your
> filesystem does the same

Let's look at the scenarios here, assuming a block gets corrupted:
1) InnoDB has checksums.
1.1) Database doesn't step on the duff block, error doesn't get
detected. Until long after your backup retention expired.
1.2) Data corruption happens in memory before the checksum is
calculated. Checksum is based on corrupted data and doesn't help you.


2) InnoDB has no checksums. Database steps on the duff block.
2.1) Database crashes during a query. You get the query and tables
accessed during a stack trace.
You run a check table and it finds a corrupted table.
You restore it from backup.
2.2) Database doesn't crash because the damage merely corrupts a
single value but the record structure remains sound.

So it is that 2.2) point where the InnoDB checksum gives you anything.
It is a vanishingly narrow case. Sufficiently narrow that if you never
encountered a relatively common problem with upgrades due to redo log
format changes (there have been several over the years), you are
orders of magnitude less likely to have encountered this.