openstack team mailing list archive
Mailing list archive
Re: [Design] Why/how object storage(Swift) better than scale-out NAS?
While there are some similarities between HDFS and Openstack Object Storage
(Swift), the overall design of the systems are very different.
1. HDFS uses a central system to maintain file metadata (Namenode), where
as in Swift the metadata is distributed and replicated across the cluster.
Having a central meta-data system is a single point of failure for HDFS,
and makes it more difficult to scale to very large sizes.
2. Swift is designed with multi-tenancy in mind, where HDFS has no notion
3. HDFS is optimized for larger files (as is typical for processing data),
where Swift is designed to store any sized files.
4. Files in HDFS are write once, and can only have one writer at a time,
in Swift files can be written many times, and under concurrency, the last
5. HDFS is written in Java, where Swift is written in Python
TLDR: HDFS is designed to store a medium number of larges files to support
data processing, where Swift is designed as a more generic storage solution
to reliably store very large numbers of varying sized files.
(HDFS Architecture information attained from
Hope it help
2013/7/3 Li, Leon <Leon.Li2@xxxxxxx>
> ** **
> I have googled and found some answers but they are not to the point I
> think. Why foundation choice object storage other than scale-out NAS?****
> I see some points about benefit of object storage(swift)****
> **· **Storing billions of files.****
> **· **Storing Petabytes (millions of Gigabytes) of data.****
> **· **Use cheap servers****
> **· **Can have several copy for each file****
> However a scale-out NAS could also have these benefits, if you build the
> scale-out NAS with open source cluster FS(for example HDFS), just like many
> Internet company did.****
> ** **
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp