openstack team mailing list archive

[Leander Bessa Beernaert <leanderbb@xxxxxxxxx>]

I'm getting around 5-6.5 GB a day of bytes written on Swift. I calculated
this by calling "swift stat && sleep 60s && swift stat". I did some
calculation based on those values to get to the end result.

Currently I'm resetting swift with a node size of 64, since 90% of the
files are less than 70KB in size. I think that might help.


On Mon, Jan 14, 2013 at 4:34 PM, Chuck Thier <cthier@xxxxxxxxx> wrote:

> Hey Leander,
>
> Can you post what performance you are getting?  If they are all
> sharing the same GigE network, you might also check that the links
> aren't being saturated, as it is pretty easy to saturate pushing 200k
> files around.
>
> --
> Chuck
>
> On Mon, Jan 14, 2013 at 10:15 AM, Leander Bessa Beernaert
> <leanderbb@xxxxxxxxx> wrote:
> > Well, I've fixed the node size and disabled the all the replicator and
> > auditor processes. However, it is even slower now than it was before :/.
> Any
> > suggestions?
> >
> >
> > On Mon, Jan 14, 2013 at 3:23 PM, Leander Bessa Beernaert
> > <leanderbb@xxxxxxxxx> wrote:
> >>
> >> Ok, thanks for all the tips/help.
> >>
> >> Regards,
> >>
> >> Leander
> >>
> >>
> >> On Mon, Jan 14, 2013 at 3:21 PM, Robert van Leeuwen
> >> <Robert.vanLeeuwen@xxxxxxxxxxxxx> wrote:
> >>>
> >>> > Allow me to rephrase.
> >>> > I've read somewhere (can't remember where) that it would be faster to
> >>> > upload files if they would be uploaded to separate containeres.
> >>> > This was suggested for a standard swift installation with a certain
> >>> > replication factor.
> >>> > Since I'll be uploading the files with the replicators turned off,
> does
> >>> > it really matter if I insert a group of them in separate containeres?
> >>>
> >>> My guess is this concerns the SQLite database load distribution.
> >>> So yes, it still matters.
> >>>
> >>> Just to be clear: turning replicators off does not matter at ALL when
> >>> putting files in a healthy cluster.
> >>> Files will be "replicated" / put on all required nodes at the moment
> the
> >>> put request is done.
> >>> The put request will only give an OK when there is quorum writing the
> >>> file (the file is stored on more than half of the required object
> nodes)
> >>> The replicator daemons do not have anything to do with this.
> >>>
> >>> Cheers,
> >>> Robert
> >>>
> >>> _______________________________________________
> >>> Mailing list: https://launchpad.net/~openstack
> >>> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> >>> Unsubscribe : https://launchpad.net/~openstack
> >>> More help   : https://help.launchpad.net/ListHelp
> >>>
> >>
> >
> >
> > _______________________________________________
> > Mailing list: https://launchpad.net/~openstack
> > Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> > Unsubscribe : https://launchpad.net/~openstack
> > More help   : https://help.launchpad.net/ListHelp
> >
>