openstack team mailing list archive

[Kuo Hugo <tonytkdk@xxxxxxxxx>]

*REF :*
http://www.quora.com/What-features-differentiate-HDFS-and-OpenStack-Object-Storage
by ChunkThier

While there are some similarities between HDFS and Openstack Object Storage
(Swift), the overall design of the systems are very different.

1.  HDFS uses a central system to maintain file metadata (Namenode), where
as in Swift the metadata is distributed and replicated across the cluster.
Having a central meta-data system is a single point of failure for HDFS,
and makes it more difficult to scale to very large sizes.

2.  Swift is designed with multi-tenancy in mind, where HDFS has no notion
of multi-tenancy

3.  HDFS is optimized for larger files (as is typical for processing data),
where Swift is designed to store any sized files.

4.  Files in HDFS are write once, and can only have one writer at a time,
in Swift  files can be written many times, and under concurrency, the last
write wins.

5.  HDFS is written in Java, where Swift is written in Python

TLDR: HDFS is designed to store a medium number of larges files to support
data processing, where Swift is designed as a more generic storage solution
to reliably store very large numbers of varying sized files.

(HDFS Architecture information attained from
http://hadoop.apache.org/hdfs/do...<http://hadoop.apache.org/hdfs/docs/current/hdfs_design.html>
)

Hope it help

+Hugo Kuo+
hugo@xxxxxxxxxxxxxx
tonytkdk@xxxxxxxxx
+886 935004793


2013/7/3 Li, Leon <Leon.Li2@xxxxxxx>

> Hi,****
>
> ** **
>
> I have googled  and found some answers but they are not to the point I
> think. Why foundation choice object storage other than scale-out NAS?****
>
> I see some points about benefit of object storage(swift)****
>
> **·        **Storing billions of files.****
>
> **·        **Storing Petabytes (millions of Gigabytes) of data.****
>
> **·        **Use cheap servers****
>
> **·        **Can have several copy for each file****
>
> However a scale-out NAS could also have these benefits, if you build the
> scale-out NAS with open source cluster FS(for example HDFS), just like many
> Internet company did.****
>
> ** **
>
> Leon****
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>