Consistent hashing implemented simply in Python

I have implemented consistent hashing in Python. The module is called hash_ring and you can get it right away. This post will explain the motivation behind the project and details. I think other languages such as Ruby can reuse my code since it's fairly simple :-)

To install the project, simply do:

sudo easy_install hash_ring

Example of usage when mapping keys to memcached servers:

memcache_servers = ['192.168.0.246:11212',
                    '192.168.0.247:11212',
                    '192.168.0.249:11212']

ring = HashRing(memcache_servers)
server = ring.get_node('my_key')

The motivation behind hash_ring

Consistent hashing is really neat and can be used anywhere where you have a list of servers and you need to map some keys (objects) to these servers. An example is memcached or a distributed system.

A problem when you use memcached clients is that you map keys to servers in following way:

server = serverlist[ hash(key) % len(serverlist) ]

The major problem with this approach is that you'll invalidate all your caches when you add or remove memcache servers to the list - and this invalidation can be very expensive if you rely on caching.

This problem was solved 10 years ago by David Karger et al and they have published following articles that explain the idea of consistent caching in greater details:

Another motivation is that I am currently looking into building a distributed hash map - and consistent hashing is essential in such a system. Here are a few widely used systems that use consistent hashing:

How consistent hashing works

Consistent hashing is fairly simple (and genius way of distributing keys). It can be best explained by the idea that you have a ring that goes from 0 to some big number. Given a node A, you find a placement for A on the ring by running hash_function(A), the hash_function should generally mix the values well - good candidates for the hash function are MD5 or SHA1. Given a (key, value) pair you find the key's placement on the ring by running hash_function(key). A node holds all the keys that have a value lower than itself, but greater than the preceding node.

Tom White has written a great blog post about consistent hashing, take a look at it, it explains the idea in much greater detail.

Python implementation

I think my Python implementation is beatiful so I will share the full implementation. The code speaks for itself or something:

import md5

class HashRing(object):

    def __init__(self, nodes=None, replicas=3):
        """Manages a hash ring.

        `nodes` is a list of objects that have a proper __str__ representation.
        `replicas` indicates how many virtual points should be used pr. node,
        replicas are required to improve the distribution.
        """
        self.replicas = replicas

        self.ring = dict()
        self._sorted_keys = []

        if nodes:
            for node in nodes:
                self.add_node(node)

    def add_node(self, node):
        """Adds a `node` to the hash ring (including a number of replicas).
        """
        for i in xrange(0, self.replicas):
            key = self.gen_key('%s:%s' % (node, i))
            self.ring[key] = node
            self._sorted_keys.append(key)

        self._sorted_keys.sort()

    def remove_node(self, node):
        """Removes `node` from the hash ring and its replicas.
        """
        for i in xrange(0, self.replicas):
            key = self.gen_key('%s:%s' % (node, i))
            del self.ring[key]
            self._sorted_keys.remove(key)

    def get_node(self, string_key):
        """Given a string key a corresponding node in the hash ring is returned.

        If the hash ring is empty, `None` is returned.
        """
        return self.get_node_pos(string_key)[0]

    def get_node_pos(self, string_key):
        """Given a string key a corresponding node in the hash ring is returned
        along with it's position in the ring.

        If the hash ring is empty, (`None`, `None`) is returned.
        """
        if not self.ring:
            return None, None

        key = self.gen_key(string_key)

        nodes = self._sorted_keys
        for i in xrange(0, len(nodes)):
            node = nodes[i]
            if key <= node:
                return self.ring[node], i

        return self.ring[nodes[0]], 0

    def get_nodes(self, string_key):
        """Given a string key it returns the nodes as a generator that can hold the key.

        The generator is never ending and iterates through the ring
        starting at the correct position.
        """
        if not self.ring:
            yield None, None

        node, pos = self.get_node_pos(string_key)
        for key in self._sorted_keys[pos:]:
            yield self.ring[key]

        while True:
            for key in self._sorted_keys:
                yield self.ring[key]

    def gen_key(self, key):
        """Given a string key it returns a long value,
        this long value represents a place on the hash ring.

        md5 is currently used because it mixes well.
        """
        m = md5.new()
        m.update(key)
        return long(m.hexdigest(), 16)
Announcements · Code · Python 19. Nov 2008
14 comments so far

Say you have a ring S1 to S3, and an object with key K like this with S1 having the smallest key:
S1 - S2 - K - S3 - S1

If you now add S4 and have a ring like this:
S1 - S2 - K - S4 - S3 - S1

Afaik then any lookups for K will now be redirected to S4 instead of S3, but as S4 was just added it will report "No value with that key". Is that not a problem or is it assumed that all clients will be able to recreate any value?

Assuming all clients are able to do just that, then you are now able to (unlike the normal memcached solution):

  • Add new servers with most keys still valid.
  • Remove at most 1 server without having to invalidate, but not any sibling till all values are replicated again.

Am I correct ? :)

Spand:
You are correct in your assumption. When adding a server to the ring one will only invalidate some of the keys - instead of all, like the current Python implementation.

An approach thought to solve this issue is to replicate data to the successors of N. E.g. if N replicates its data to N+1 and N+2, then if N crashes the keys will be found on the successors of N - i.e. no keys will be invalidated. In memcached thought a missing node will just result in a cache miss - which isn't that big a deal.

If you remove an item from a sorted list, the list remains sorted. You can skip the sort() step in remove_node.

Also, I'd sort only once at the end of add_node, rather than on every loop iteration.

Marius:
You are absolutely right. Fixed and released ;-)

Wow, you have managed it to appear on reddit. Nice to see you blog there :)

Nice, this should be used in memcache client. :-)

Thanks for nice code.

I would just suggest to use enumerate()
for a more beautiful loop over the _sorted_keys:


for i, node in enumerate(self._sorted_keys):
...

Christoph:
I have actually been there a few times now 8)

Alexander:
I will in the upcoming days write a guide on how to use hash_ring with python-memcache.

Ivo:
Thanks for the tip, I have forgot about enumerate as I don't use indexes that often.

If you invalidate all your caches and rely on cahing it can be time consuming.

The Python code implementation example you provide Amix is a work of art! Nice! :)

Nice work... I wish I could write python like you, it would make my life a lot easier.

I like it

Hi Amir, thanks for the write-up. You may want to consider using bisect.insort() to maintain the sorted list on insertion, instead of having to re-sort it every time.

http://docs.python.org/library...

عقارات السعودية - عقارات الرياض - عقارات الخرج - عقارات مكة المكرمة - عقارات جدة - عقارات الطائف - عقارات المدينة المنورة - عقارات ينبع - عقارات الدمام - عقارات الخبر - عقارات الأحساء - عقارات القصيم - عقارات عسير - عقارات حائل - عقارات تبوك - عقارات الباحة - عقارات الحدود الشمالية - عقارات الجوف - عقارات جازان - عقارات نجران - عقارات مصر - عقارات القاهرة - عقارات الجيزة - عقارات حلوان - عقارات 6 اكتوبر - عقارات الاسكندرية - عقارات الساحل الشمالي - عقارات البحيرة - عقارات السويس - عقارات الاسماعلية - عقارات بورسعيد - عقارات البحر الاحمر - عقارات مطروح - عقارات جنوب سيناء - عقارات شمال سيناء - عقارات دمياط - عقارات الدقهلية - عقارات كفر الشيخ - عقارات الشرقية - عقارات الغربية - عقارات القليوبية - عقارات المنوفبة - عقارات الفيوم - عقارات قنا - عقارات المنيا - عقارات اسيوط - عقارات بني سويف - عقارات سوهاج - عقارات الوادي الجديد - عقارات الأقصر - عقارات اسوان - عقارات الامارات - عقارات ابوظبي - عقارات العين - عقارات دبي - عقارات جبل علي - عقارات الشارقة - عقارات رأس الخيمة - عقارات عجمان - عقارات أم القيوين - عقارات الفجيرة - الوطن العربي - عقارات الكويت - عقارات عمان - عقارات قطر - عقارات البحرين - عقارات الاردن - عقارات لبنان - عقارات المغرب - عقارات السودان - عقارات سوريا - وظائف - وظائف في السعودية - وظائف في الامارات - وظائف في مصر - وظائف في الكويت - وظائف في عمان - وظائف في قطر - وظائف في البحرين - وظائف في الاردن - وظائف في لبنان - وظائف في المغرب - وظائف في السودان - وظائف في سوريا - خدمات - العروض الجديدة - الرعاية و الإعلان - الدعم الفني - العقارات العام - شراء شقق - شراء شقة - شقق بالتقسيط - شقة بالتقسيط - شقق تمليك - شقة تمليك - شقق سكنية - شقة سكنية - شقق فندقية - شقة فندقية - شقق للايجار - شقة للايجار - شقق للبيع - شقة للبيع - شقق مفروشة - شقة مفروشة - غير مصنف - مطلوب شقق - مطلوب شقة
اهداف الدوري الأنجليزي
اهداف الدوري الأسباني
اهداف دوري ابطال اوروبا
مهارات ولقطات منوعة
اهداف الدوري السعودي
اهداف تصفيات كأس العالم
اهداف الدوري المصري
اهداف الدوري الالماني
أهداف كأس الخليج
هدف تيوب

Post a comment
Commenting on this post has expired.
© 2000-2009 amix. Powered by Skeletonz.