openstack team mailing list archive

[Kun Huang <aragongareth@xxxxxxxxx>]

Hi swift developer,

I'm confused about implementation of ring structure.

in the RingBuilder, line 671 ~ 681

        for part, replace_replicas in reassign_parts:
            # Gather up what other tiers (zones, ip_ports, and devices) the
            # replicas not-to-be-moved are in for this part.
            other_replicas = defaultdict(int)
            unique_tiers_by_tier_len = defaultdict(set)
            for replica in xrange(self.replicas):
                if replica not in replace_replicas:
                    dev = self.devs[self._replica2part2dev[replica][part]]
                    for tier in tiers_for_dev(dev):
                        other_replicas[tier] += 1
                        unique_tiers_by_tier_len[len(tier)].add(tier)


this while loop results in other_replicas and
unique_tiers_by_tier_len, but I don't have a confirmed understanding
about these two data.

and in line 684 to 725: (I removed comments)


                tier = ()
                depth = 1
                while depth <= max_tier_depth:
                    candidate_tiers = tier2children[tier]
                    candidates_with_replicas = \
                        unique_tiers_by_tier_len[len(tier) + 1]
                    if len(candidate_tiers) > len(candidates_with_replicas):
                        for t in reversed(candidate_tiers):
                            if other_replicas[t] == 0:
                                tier = t
                                break
                    else:
                        min_count = min(other_replicas[t]
                                        for t in candidate_tiers)
                        tier = (t for t in reversed(candidate_tiers)
                                if other_replicas[t] == min_count).next()
                    depth += 1

this loop search the tier tree, from () to (zone, ip, device), for
finding a tier, and choosing a dev by that tier, and storing dev id in
part key(line 762, that's a kind of final ring structure?). And the
propose of tree structure is:

"""

This is already supported in Swift with the concept of availability
zones. Swift will place each replica in different availability zones,
if possible. If you only have one zone, Swift will place the replicas
on different machines. If you only have one machine, Swift will place
the replicas on different drives.

-- John

""", right?


And the last loop line 732 to line 760 only works on sorting for next
big loop? (line 683)


The propose of rebalance is to put every replica on correct partition,
and put every partition on correct device?



Gareth