openstack team mailing list archive

[Mark Brown <ntdeveloper2002@xxxxxxxxx>]

Has anyone used container sync in their implementations? It would be great to know your experiences, because real world use case studies are scarce:) Anything to be aware of, whether its performance, setup, problems seen, recommendations..?

--Mark

________________________________
 From: Mark Brown <ntdeveloper2002@xxxxxxxxx>
To: David Hadas <DAVIDH@xxxxxxxxxx> 
Cc: "openstack@xxxxxxxxxxxxxxxxxxx" <openstack@xxxxxxxxxxxxxxxxxxx> 
Sent: Sunday, May 12, 2013 11:25 PM
Subject: Re: [Openstack] Swift questions.
 


Thanks again, David. Definitely helps.

Is the alternative node you refer to here the "handoff" node? Is the handoff node something that is in the ring database? I am trying to piece together where in the stack this would happen. If it is transparent, it would probably happen in the object server somehow, but it would need to know where the handoff node is.

-- Mark.


________________________________
 From: David Hadas <DAVIDH@xxxxxxxxxx>
To: Mark Brown <ntdeveloper2002@xxxxxxxxx> 
Cc: "openstack@xxxxxxxxxxxxxxxxxxx" <openstack@xxxxxxxxxxxxxxxxxxx> 
Sent: Sunday, May 12, 2013 11:13 PM
Subject: Re: [Openstack] Swift questions.
 


Mark, 

It would mark the server as having insufficient storage (to avoid retrying the same server for a while) and try to place the object in an alternative node -  same as it would do with any other error. Once the object is placed in an alternative node, the alternative node would try to send the object back to its originated place every once in a while such that if the problem is reolved, it will move to the right place. 

This being said, there are N replicas - and this would repeat for each of the N replicas. So if one server is full you can still access the data using the other N-1 replicas and you still maintain N replicas in your system of the object (although one of them is misplaced until the administrator would
 resolve the space issue). 

Swift doe snot use df, it would try to allocate the space on the disk and if it fails it fails.
Failure is just a natural part of live as far as swift is concerned ;)

Hope this helps. 

DH



Regards, 
David Hadas
Research Staff Member, Master Inventor

IBM Research Labs, Haifa
Tel:    Int+972-4-829-6104
Fax:   Int+972-4-829-6112

-----Mark Brown <ntdeveloper2002@xxxxxxxxx> wrote: -----
To: David Hadas/Haifa/IBM@IBMIL
From: Mark Brown <ntdeveloper2002@xxxxxxxxx>
Date: 05/12/2013 08:27PM
Cc: "openstack@xxxxxxxxxxxxxxxxxxx" <openstack@xxxxxxxxxxxxxxxxxxx>
Subject: Re: [Openstack] Swift questions.


Thanks for the response David.

I do understand Swift, by its design, tries to keep things in balance among various nodes. I was curious what it does when it encounters a full partition(say hard disk is full)? Lets just say it is balanced and the all nodes are nearing capacity. If I dont add any nodes, what happens when it tries to write on a specific node (which it was directed to based on the hashing ring) and there is not enough space to write the object?

Also, what does it use to determine a full partition? Does it use a df? 

Mark


________________________________
 From: David Hadas <DAVIDH@xxxxxxxxxx>
To: Mark Brown <ntdeveloper2002@xxxxxxxxx> 
Cc: "openstack@xxxxxxxxxxxxxxxxxxx" <openstack@xxxxxxxxxxxxxxxxxxx> 
Sent: Sunday, May 12, 2013 10:36 PM
Subject: Re: [Openstack] Swift questions.
 


Mark, 

Regarding your first Q:

Swift evenly balance the hard-drives such that in a correctly configured system, you should expect one hard-drive being more full than the other. 
There is manual a mechanism in swift to balance hard-drives by moving partitions
 to/from hard-drive but you should need to use it under normal conditions, it is likely that if your had-drives get full the right thing to do would be to add more hard-drives.

In any case you should care not about individual partitions 'getting full' as partitions are not allocated any specific space and can grow and shrink as needed as long as the hard-drive they are in have space.


DH



Regards, 
David Hadas
Research Staff Member, Master Inventor

IBM Research Labs, Haifa
Tel:    Int+972-4-829-6104
Fax:   Int+972-4-829-6112

-----"Openstack" <openstack-bounces+davidh=il.ibm.com@xxxxxxxxxxxxxxxxxxx> wrote: -----
To: "openstack@xxxxxxxxxxxxxxxxxxx" <openstack@xxxxxxxxxxxxxxxxxxx>
From: Mark Brown 
Sent by: "Openstack" 
Date:
 05/12/2013 07:50PM
Subject: [Openstack] Swift questions.


Hello guys,
Been looking at Swift for some projects, and had some very basic questions.

1. How does Swift determine a certain partition is full? And when it does detect that, what does it do? Does it return an error to the client?

2. Regarding container sync, has anyone used container sync in their implementations? It would be great to know your experiences, because real world use case studies are scarce:)


-- Mark

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to     : openstack@xxxxxxxxxxxxxxxxxxx
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp






_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to     : openstack@xxxxxxxxxxxxxxxxxxx
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp