openstack team mailing list archive

[Caitlin Bestler <Caitlin.Bestler@xxxxxxxxxxx>]

The LocalFS proposal was first posted a month ago, which is quoted below.

To quickly recap the purpose is to enable enhanced Object Servers that provide
features such as self-healing mirrors, which can achieve the same availability
with fewer network replications being required. Read the blueprint for details.

Specifically, Nexenta is interested in enabling use of ZFS self-healing mirroring.
A replica produced by the ZFS file system does not consume network bandwidth,
But unlike conventional simple disk mirroring, ZFS self-healing mirroring will detect
and heal latent disk errors.

Relying on self-healing mirroring essentially makes a replica with local IO bandwidth
rather than with network bandwidth. Physically the replicas are just as independent
as a network replica.

A deployment that has two network replicas, that each have two local copies, should
provide superior availability than three network replicas. The superior availability also
comes with reduced consumption of network bandwidth. Further, there would be no
degradation in transaction latency.

The first patch has been posted for review:

https://github.com/vineethrp/swift/commit/c062d4bad5fd8aa4a7bc5ad481b23c07216f281a

This patch creates the LocalFS class, and includes a default when no enhanced LocalFS
has been installed. Nexenta is developing the ZFS version for the LocalFS,  and the first
version is in the above patch. Others could implement the same capabilities over other
file systems.


There is one major issue that the concept of self-healing mirrors introduces. Basically
a replica of a partition can be in a degraded state when the full set of mirrors are not up.
Obviously such a replica will probably need to be replicated, but unlike an unmirrored
replica that has had a drive failure it is not useless. We suspect that a lower priority
replication is the correct response, but this is something that should be discussed
by the team as a whole.


On 09/15/2011 10:18 AM, Caitlin Bestler wrote:
> Greetings,
>
> A blueprint has been submitted for an extension to enable Local File Systems to take responsibility for
> certain operations, allowing generic Swift code to offload some burdens when these optional capabilities
> are available.
>
> The goal of this proposal is to allow an Object Server to take advantage of the capabilities of the ZFS
> file system, but it could be applied for other enhanced file systems as well.
>
> The blueprint is: https://blueprints.launchpad.net/swift/+spec/localfs
> The etherpad description is: http://etherpad.openstack.org/YMTqYzPmZQ
>
> This is the first of what will probably be a handful of proposals from Nexenta Systems, all with the goal
> of enabling value added Object Servers.
>
> So we should introduce ourselves.
>
> Nexenta brings open source solutions built on ZFS to provide software-based NAS/SAN appliances. The core value of the ZFS file system is delivered in an enterprise class storage solution. We intend to bring  the value of ZFS as a local file system to Cloud Storage as well.
>
> >From http://en.wikipedia.org/wiki/ZFS
>
> In computing, ZFS is a combined file system and logical volume manager designed by Sun Microsystems. The features of ZFS include data integrity verification against data corruption modes (like bit rot), support for high storage capacities, integration of the concepts of filesystem and volume management,snapshots and copy-on-write clones, continuous integrity checking and automatic repair, RAID-Z and native NFSv4 ACLs. ZFS is implemented as open-source software, licensed under the Common Development and Distribution License (CDDL). The ZFS name is a trademark of Oracle.[3]
>
>
>
> To take advantage of ZFS capabilities we will need to work with the Swift project to define how the core Swift code discovers and exploits optional capabilities.  This is a role similar to that of a graphics chip or network interface vendor working with an open source OS project. The goal is to enable enhanced functionality with interfaces that make  the enhanced functionality optional and largely vendor neutral. Other file system providers should be able to plug-in in their own solutions.
>
> Nexenta appliances are based on open source operating system that utilizes OpenSolaris, in a near future - Illumos, kernel. This means we will end up testing that the python code is truly OS independent, and we anticipate submitting a steady but hopefully small stream of patches to fix code that was inadvertently Linux dependent. The goal will be to supply patches that make the code truly generic, and hopefully avoid just accumulating any "if linux elif illumos elif bsd ..." sequences in Swift code.
>
> >From http://en.wikipedia.org/wiki/Illumos:
>
> Illumos is a derivative of OS/Net (aka ON), which basically is a Solaris/OpenSolaris kernel with the bulk of the drivers, core libraries, and basic utilities. It is dependent on OS/Net, which Illumos will follow very closely while allowing to retain changes to code which might be unacceptable to upstream OpenSolaris. Illumos is aiming at 100% ABI (Application Binary Interface) compatibility with Solaris ON, focusing just on the core foundation blocks.
>