hacking and gonzo since 2000

MemcacheRing: Consistent hashing for python-memcache

I have added MemcacheRing to hash_ring 1.1. This class encapsulates python-memcache's Client class so keys are distributed using consistent hashing.

You can read more about consistent hashing in two other posts:

Usage of MemcacheRing:

from hash_ring import MemcacheRing
 
mc = MemcacheRing(['192.168.0.23:11212', '192.168.0.66:11212'])
mc.set('hello', 'world')
print mc.get('hello')

Implementation:

import memcache
import types

from hash_ring import HashRing

class MemcacheRing(memcache.Client):
    """Extends python-memcache so it uses consistent hashing to
    distribute the keys.
    """
    def __init__(self, servers, *k, **kw):
        self.hash_ring = HashRing(servers)

        memcache.Client.__init__(self, servers, *k, **kw)

        self.server_mapping = {}
        for server_uri, server_obj in zip(servers, self.servers):
            self.server_mapping[server_uri] = server_obj

    def _get_server(self, key):
        if type(key) == types.TupleType:
            return memcache.Client._get_server(key)

        for i in range(self._SERVER_RETRIES):
            iterator = self.hash_ring.iterate_nodes(key)
            for server_uri in iterator:
                server_obj = self.server_mapping[server_uri]
                if server_obj.connect():
                    return server_obj, key

        return None, None

Other than this, I am also using the idea of consistent hashing to implement a distributed hash map. This may also become open-source some time in the future.

Announcements · Code · Python • 25. Nov 2008