maria-developers team mailing list archive
Mailing list archive
Re: MDEV-14425: Is appending never slower than writing to a preallocated file?
> Le 2 févr. 2018 à 12:27, Marko Mäkelä <marko.makela@xxxxxxxxxxx> a écrit :
> I would like to redesign the InnoDB redo log format for better
> performance in MariaDB 10.4. Part of this would involve minimizing
> write amplification and optimizing for journaled file systems.
> Jun Su from Microsoft suggested to me that regular writes (as opposed
> to appends) could cause some write amplification inside journaled file
> InnoDB traditionally pre-allocates both data and log files. Maybe it
> was a good idea in 1994 when the code was initially conceived. But we
> have had journaled or copy-on-write file systems and also SSDs for
> quite some time now.
> I wrote two test programs that write a 2GiB file in 2KiB blocks,
> either pre-allocating the file upfront, or appending to the file. On
> the two SSDs that I tested (with ext4fs), appending was always faster.
> The programs are attached to
> I would appreciate it if someone can provide a counterexample where
> writing to a preallocated file would be faster than appending, on a
> modern file system. Also, I would like to see how HDDs would perform.
I've just tested your program on an XFS filesystem, the preallocate performance is almost the same than append:
time ./append foo1&time ./append foo2&time ./append foo3&time ./append foo4 <12:57:26
./append foo3 0.03s user 1.86s system 30% cpu 6.297 total
./append foo1 0.03s user 1.81s system 28% cpu 6.554 total
./append foo2 0.02s user 1.99s system 30% cpu 6.610 total
./append foo4 0.02s user 1.84s system 27% cpu 6.711 total
time ./preallocate foo1&time ./preallocate foo2&time ./preallocate foo3&time ./preallocate foo4 <12:59:09
./preallocate foo4 0.02s user 1.98s system 32% cpu 6.119 total
./preallocate foo1 0.03s user 2.01s system 33% cpu 6.120 total
./preallocate foo2 0.02s user 2.16s system 34% cpu 6.396 total <12:59:16
./preallocate foo3 0.04s user 1.88s system 28% cpu 6.687 total <12:59:17
Disk are 4 Intel SSD 520 (480GB) in Raid5 configuration on controller DELL PERC H710P
(tested several times)
I've also tested it with 4GB files, and on a slower system (still Intel SSD disks in Raid5 with XFS), append is also performing quite the same than preallocate.
M : +33 6 51 21 54 10
Softizy - At your side to Optimize your PHP / MySQL applications