openstack team mailing list archive

[Vishvananda Ishaya <vishvananda@xxxxxxxxx>]

This topic might be better posted on openstack-dev

Vish

On Feb 26, 2013, at 11:24 AM, Kun Huang <aragongareth@xxxxxxxxx> wrote:

> Hi swift developer,
> 
> I'm confused about implementation of ring structure.
> 
> in the RingBuilder, line 671 ~ 681
> 
> 
>         for part, replace_replicas in reassign_parts:
> 
> 
>             # Gather up what other tiers (zones, ip_ports, and devices) the
> 
>             # replicas not-to-be-moved are in for this part.
> 
>             other_replicas = defaultdict(int)
> 
> 
>             unique_tiers_by_tier_len = defaultdict(set)
> 
> 
>             for replica in xrange(self.replicas):
> 
> 
>                 if replica not in replace_replicas:
> 
> 
>                     dev = self.devs[self._replica2part2dev[replica][part]]
> 
> 
>                     for tier in tiers_for_dev(dev):
> 
> 
>                         other_replicas[tier] += 1
> 
> 
>                         unique_tiers_by_tier_len[len(tier)].add(tier)
> 
> 
> 
> 
> this while loop results in other_replicas and unique_tiers_by_tier_len, but I don't have a confirmed understanding about these two data.
> 
> 
> and in line 684 to 725: (I removed comments)
> 
> 
>                 tier = ()
> 
> 
>                 depth = 1
> 
> 
>                 while depth <= max_tier_depth:
> 
> 
>                     candidate_tiers = tier2children[tier]
> 
> 
>                     candidates_with_replicas = \
> 
>                         unique_tiers_by_tier_len[len(tier) + 1]
> 
> 
>                     if len(candidate_tiers) > len(candidates_with_replicas):
> 
> 
>                         for t in reversed(candidate_tiers):
> 
> 
>                             if other_replicas[t] == 0:
> 
> 
>                                 tier = t
> 
> 
>                                 break
> 
>                     else:
> 
>                         min_count = min(other_replicas[t]
> 
> 
>                                         for t in candidate_tiers)
> 
> 
>                         tier = (t for t in reversed(candidate_tiers)
> 
> 
>                                 if other_replicas[t] == min_count).next()
> 
> 
>                     depth += 1
> 
> 
> 
> this loop search the tier tree, from () to (zone, ip, device), for finding a tier, and choosing a dev by that tier, and storing dev id in part key(line 762, that's a kind of final ring structure?). And the propose of tree structure is:
> 
> """
> This is already supported in Swift with the concept of availability zones. Swift will place each replica in different availability zones, if possible. If you only have one zone, Swift will place the replicas on different machines. If you only have one machine, Swift will place the replicas on different drives.
> 
> -- John
> """, right?
> 
> 
> And the last loop line 732 to line 760 only works on sorting for next big loop? (line 683)
> 
> 
> The propose of rebalance is to put every replica on correct partition, and put every partition on correct device? 
> 
> 
> 
> Gareth
> 
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp