bitmapist: Powerful realtime analytics with Redis 2.6's bitmaps and Python
I just released bitmapist (GitHub) - a powerful realtime analytics library that can help you answer following questions:
This library is very easy to use and enables you to create your own reports easily.
Using Redis bitmaps you can store events for millions of users in a very little amount of memory (megabytes). You should be careful about using huge ids (e.g. 2^32 or bigger) as this could require larger amounts of memory.
If you want to read more about bitmaps please read following:
Requires Redis 2.6+ and newest version of redis-py.
sudo pip install bitmapist
Setting things up:
from datetime import datetime, timedelta from bitmapist import setup_redis, delete_all_events, mark_event,\ MonthEvents, WeekEvents, DayEvents, HourEvents,\ BitOpAnd, BitOpOr now = datetime.utcnow() last_month = datetime.utcnow() - timedelta(days=30)
Mark user 123 as active and has played a song:
mark_event('active', 123) mark_event('song:played', 123)
Answer if user 123 has been active this month:
assert 123 in MonthEvents('active', now.year, now.month) assert 123 in MonthEvents('song:played', now.year, now.month)
How many users have been active this week?
print len(WeekEvents('active', now.year, now.isocalendar()))
Perform bit operations. How many users that have been active last month are still active this month?
active_2_months = BitOpAnd( MonthEvents('active', last_month.year, last_month.month), MonthEvents('active', now.year, now.month) ) print len(active_2_months) # Is 123 active for 2 months? assert 123 in active_2_months
Work with nested bit operations (imagine what you can do with this ;-))!
active_2_months = BitOpAnd( BitOpAnd( MonthEvents('active', last_month.year, last_month.month), MonthEvents('active', now.year, now.month) ), MonthEvents('active', now.year, now.month) ) print len(active_2_months) assert 123 in active_2_months