openstack team mailing list archive

[Dmitry Ukov <dukov@xxxxxxxxxxxx>]

Hi all,
*I want to introduce some ideas about Swift.*
*Let’s assume we have huge amount of data stored in Swift (e.g. 10Pb). This
data are dynamically changed by users. So we need to reduce network load
caused by replication and intensive data uploading/downloading.
My proposal is to create so called “Ring of rings”. For example we have 2
data centers with deployed Swift. We can distinguish some nodes for serving
“Ring of rings” (actually we need only Proxy Servers).

“Ring of rings” structure.
Each data center corresponds to zone (load balancer’s ip address and port
). Replica count is 1.

Load balancers for each data center have to rewrite
/<device>/<partition>/<account>/<container>/<object>
to
/<version>/<account>/<container>/<object>
Example for Nginx:
     rewrite ^/.+/[0-9]+/(.+)$ /v1/$1 break;      rewrite
^/.+/[0-9]+/(.+)/(.+)$ /v1/$1/$2 break;
     rewrite ^/.+/[0-9]+/(.+)/(.+)/(.+)$ /v1/$1/$2/$3 break;

Concept architecture
https://docs.google.com/drawings/pub?id=1RRqsTxfd0jtfUofFaGuwjyxCBUQ2keWzDzjle1kbqB4&w=1161&h=732

So we can use Ring to determine data center to send http request to.

What do you think about this scheme?
Feedback from the OpenStack/Swift community would be very appreciated.*

-- 
Kind regards
Dmitry Ukov
IT Engineer
Mirantis, Inc.