Plurk Comet: Handling of 100.000+ open connections

Comet is a buzz word, just like Ajax, but a bit cooler. The bottom line in comet is that the server can push data to clients when new data arrives.

I think that comet is the next big thing and this trend can be seen in an upcoming product like Google Wave that use comet heavily for creating real time updates.

Plurk is growing fast and we are becoming one of the largest Python sites on the Internet. We serve many thousands of concurrent users pr. day (100.000+), so implementing comet for Plurk is a big challenge and I have spent around a week on fiddling with different solutions.

Here are some of the technologies I have tired:

• Python Twisted: Non-blocking server in Python. Unfortunately it ate a lot of CPU and could not scale
• Jetty: They claim to have good support for comet (and they do if you don't serve 100.000 clients at once). The Jetty installation we ran ate around 2GB of ram on 10.000 active users, which is unacceptable for our needs
• Apache Tomcat: Same with Jetty, it eats tons of memory, even thought they do support comet connections
• Apache Mina: A NIO (non-blocking IO) framework which I used to build a HTTP server. Unfortunately Mina is very badly documented and it did not scale up in production

After trying these out and found out that they could not handle a massive load I was about to give up. But then, I stumbled upon the savior:

• JBoss Netty: A NIO framework, done by one of Apache Mina's founders

Netty is not that documented, but it's really well designed and after some hacking around the performance is pretty amazing.

The bottom line

Using Netty we have comet running on 100.000+ open connections - this uses some GB of memory and 20% of CPU on a quad core server. I.e. we have solved the C10k * 10 problem using non-blocking technology and some pretty impressive libraries (namely Java NIO and Netty).

A big kudos goes to Trustin Lee for his amazing work on Netty!