pbxt-discuss team mailing list archive

[Michael Han <mh6@xxxxxxxxxxxx>]

On 1 Jul 2009, at 16:53, Paul McCullagh wrote:

> Hi All,
>
> My tests show that memory mapping has one major advantage over  
> normal file I/O:
>
> During a fsync(fh), pread(fh) and write(fh) can hang (actually  
> sometimes pread() hangs for no reason?!)
>
> With memory mapped files, a copy from or to the memory map never  
> hangs, even when an msync() is being done.
>
> The problem is during checkpoint. Here, a hanging read can bring  
> all activity to a halt for a couple of seconds. This happens due to  
> a cache miss.
>
> The major disadvantage of memory mapped files at the moment is the  
> "remap" function. This is when the size of the file is expanded,  
> and we upmap and remap the file. This could be fixed by mapping an  
> additional block of memory (but then we need to keep a list of the  
> memory maps in RAM for each file).
>
> I have only tested mac so far, but i believe this is the case with  
> most file systems (using XFS with direct I/O is the only exception  
> I have heard of).
>
> Any ideas welcome...
>
>
> --
> Paul McCullagh
> PrimeBase Technologies
> www.primebase.org
> www.blobstreaming.org
> pbxt.blogspot.com

Hi Paul,

I actually had some issues with the remapping in cases where the  
files grow larger than the main memory of the servers (don't ask).

But I can see the advantage of using mmap.

Michael


--
The Wellcome Trust Sanger Institute is operated by Genome Research 
Limited, a charity registered in England with number 1021457 and a 
company registered in England with number 2742969, whose registered 
office is 215 Euston Road, London, NW1 2BE.