maria-developers team mailing list archive

[jocelyn fournier <jocelyn.fournier@xxxxxxxxxxx>]

Hi Marko,

> Le 2 févr. 2018 à 12:27, Marko Mäkelä <marko.makela@xxxxxxxxxxx> a écrit :
> 
> I would like to redesign the InnoDB redo log format for better
> performance in MariaDB 10.4. Part of this would involve minimizing
> write amplification and optimizing for journaled file systems.
> 
> Jun Su from Microsoft suggested to me that regular writes (as opposed
> to appends) could cause some write amplification inside journaled file
> systems.
> 
> InnoDB traditionally pre-allocates both data and log files. Maybe it
> was a good idea in 1994 when the code was initially conceived. But we
> have had journaled or copy-on-write file systems and also SSDs for
> quite some time now.
> 
> I wrote two test programs that write a 2GiB file in 2KiB blocks,
> either pre-allocating the file upfront, or appending to the file. On
> the two SSDs that I tested (with ext4fs), appending was always faster.
> The programs are attached to
> https://jira.mariadb.org/browse/MDEV-14425
> 
> I would appreciate it if someone can provide a counterexample where
> writing to a preallocated file would be faster than appending, on a
> modern file system. Also, I would like to see how HDDs would perform.


I've just tested your program on an XFS filesystem, the preallocate performance is almost the same than append:

time ./append foo1&time ./append foo2&time ./append foo3&time ./append foo4                                                                                                                                                          <12:57:26
[1] 38171
[2] 38172
[3] 38174
./append foo3  0.03s user 1.86s system 30% cpu 6.297 total
./append foo1  0.03s user 1.81s system 28% cpu 6.554 total
./append foo2  0.02s user 1.99s system 30% cpu 6.610 total
./append foo4  0.02s user 1.84s system 27% cpu 6.711 total

time ./preallocate foo1&time ./preallocate foo2&time ./preallocate foo3&time ./preallocate foo4                                                                                                                                      <12:59:09
[1] 39461
[2] 39462
[3] 39464
./preallocate foo4  0.02s user 1.98s system 32% cpu 6.119 total
./preallocate foo1  0.03s user 2.01s system 33% cpu 6.120 total
./preallocate foo2  0.02s user 2.16s system 34% cpu 6.396 total                                                                                                                                                                      <12:59:16
./preallocate foo3  0.04s user 1.88s system 28% cpu 6.687 total                                                                                                                                                                      <12:59:17

Disk are 4 Intel SSD 520 (480GB) in Raid5 configuration on controller DELL PERC H710P
(tested several times)

I've also tested it with 4GB files, and on a slower system (still Intel SSD disks in Raid5 with XFS), append is also performing quite the same than preallocate.


HTH,

--
Jocelyn Fournier
Founder
M : +33 6 51 21 54 10
https://www.softizy.com
Softizy - At your side to Optimize your PHP / MySQL applications