openstack team mailing list archive

[Jonathan Lu <jojokururu@xxxxxxxxx>]

On 2013/6/17 18:59, Robert van Leeuwen wrote:
> > I'm facing the issue about the performance degradation, and once I glanced that changing the value in /proc/sys
> > /vm/vfs_cache_pressure will do a favour.
> > Can anyone explain to me whether and why it is useful?
> Hi,
>
> When this is set to a lower value the kernel will try to keep the inode/dentry cache longer in memory.
> Since the swift replicator is scanning the filesystem continuously it will eat up a lot of iops if those are not in memory.
>
> To see if a lot of cache misses are happening, for xfs, you can look at xs_dir_lookup and xs_ig_missed.
> ( look at http://xfs.org/index.php/Runtime_Stats )
>
> We greatly benefited from setting this to a low value but we have quite a lot of files on a node ( 30 million)
> Note that setting this to zero will result in the OOM killer killing the machine sooner or later.
> (especially if files are moved around due to a cluster change ;)
>
> Cheers,
> Robert van Leeuwen

Hi,
    We set this to a low value(20) and the performance is better than 
before. It seems quite useful.

    According to your description, this issue is related with the 
object quantity in the storage. We delete all the objects in the storage 
but it doesn't help anything. The only method to recover is to format 
and re-mount the storage node. We try to install swift on different 
environment but this degradation problem seems to be an inevitable one.

Cheers,
Jonathan Lu