openstack team mailing list archive

[John Dickinson <me@xxxxxx>]

Swift has many exciting features coming in the OpenStack Folsom
Release this fall, but where do we go from here? What's next for Swift
in grizzly?

I've got some ideas. I'd like to mention them and see where you the
community will take them. I've written up most of them into quick one-
line blueprints in Launchpad. If you'd like to contribute, grab the
blueprint and jump in.

- Optimize the "many small writes" workload. Swift actually handles
  many small concurrent writes very well. However, "many small writes"
  generally also implies that the cardinality of a single container
  gets very large. There are two ways this use case can be improved:

    - Implement transparent container sharding
      https://blueprints.launchpad.net/swift/+spec/container-sharding

    - Provide better listing traversal abstractions. Listing a few
      billion objects ten thousand at a time is somewhat impractical.

- Solve globally distributed clusters. How can I have servers in
  London and servers in San Jose in the same logical swift cluster
  with three replicas total, but guaranteed to have at least one
  replica in each cluster?
  https://blueprints.launchpad.net/swift/+spec/multi-region

- Support a single logical swift cluster with tiers of storage (eg
  cheap spinning disks and expensive high IOPS SSD arrays). Can, for
  example, a user choose to have a container and its objects be served
  from a particular tier of storage?
  https://blueprints.launchpad.net/swift/+spec/storage-tiers

- Some deployers have implemented metadata searching by intercepting
  write requests and sending the metadata to another system. Can
  metadata searching be implemented in swift itself? One possible
  implementation would be to dynamically generate indexes on the
  container DB.
  https://blueprints.launchpad.net/swift/+spec/searchable-metadata

- Support PUTs with unlimited size. Implement server-side large object
  splitting.
  https://blueprints.launchpad.net/swift/+spec/large-single-uploads

- Support the full HTTP spec for range requests
  https://blueprints.launchpad.net/swift/+spec/multi-range-support

- There are a few things that could be done to simplify installation

    - Create or refactor existing code into a single swift binary or
      startup script. Would it be possible, for example, to install
      swift and run one command with the data drives listed and swift
      "just works"?

    - Build a ring server that automatically discovers devices
      https://blueprints.launchpad.net/swift/+spec/ring-builder-server

    - Provide a simple, intuitive way to test a deployment after install
      https://blueprints.launchpad.net/swift/+spec/post-deploy-test

- Support concurrent reads to objects to support a read-heavy workload
  https://blueprints.launchpad.net/swift/+spec/concurrent-reads

If you are in the San Francisco area, we will have a swift meetup on
August 30 at Citizen Space SF at 6:30 pm.
http://www.meetup.com/openstack/events/77706042/

We will have a swift team meeting on Monday October 1 in
#openstack-meeting at 8pm UTC to discuss the plans for swift over the
next six months and the sessions for the design summit. If you are
interested in participating in swift development, please attend.

If you are a new contributor to swift, please read
http://wiki.openstack.org/HowToContribute.

--John

Attachment: smime.p7s
Description: S/MIME cryptographic signature