graphite-dev team mailing list archive

[Amos Shapira <question191807@xxxxxxxxxxxxxxxxxxxxx>]

Question #191807 on Graphite changed:
https://answers.launchpad.net/graphite/+question/191807

Amos Shapira posted a new comment:
Yes I'm aware that all Whisper files need to be touched all the time,
that's where mmap actually shines and I tried to explain this in my
previous comments.

The crux of the matter is that all of this is VIRTUAL memory, mmap'ing
TB's of files into virtual memory doesn't mean that the files are
actually copied into RAM but only that each byte in the file has an
address in the user space memory to refer to it, and the kernel makes
sure that they are kept in sync.

When you DON'T use mmap, what the kernel does is to effectively mmap the files into its own memory (that's the "buffer cache") then copy parts of these memory pages to/from your provided read/write buffers. All you get from NOT mmap'ing files is the additional cost of:
1. seek system call
2. read system call, which:
  2.1. copies data from kernel memory to user space
3. write system call, which:
  3.1 copies data from user space back into kernel memory

All this is done for every read and write.

When you mmap() the file on FIRST ACCESS you:
1. open file (system call).
2. mmap system call, which just tells the kernel to manipulate some pointers.
3. close file (system call)

(system calls are implemented as CPU-level hardware interrupts,
requiring cache flashes, register saving and more work which can add up
to a lot of CPU time).

After that you just access the pages as you need, causing page faults to
read the file directly into a buffer which is accessible by the user
code, no system calls, no buffer allocation and no data copying.

The kernel is smart enough to page out Least Recently Used pages (and
flush them straight to the file on the disk if they are "dirty", no need
for Swap in/out), which is what it does when you use read/write anyway.

Does this answer your concerns?

-- 
You received this question notification because you are a member of
graphite-dev, which is an answer contact for Graphite.