hacking and gonzo since 2000

Plurk: Instant conversations using Comet

The web is becoming more and more realtime and we at Plurk are following the trend. Yesterday we released instant notifications so users can have conversations with sub-second delays - - it's like group based instant messaging.

Implementing this has not been easy given the complexities and size of Plurk, some problems include:

huge amount of active users - hundred thousands of online users at once
huge amount of usage - to this day our users have posted over 1 billion unique messages
complexity of Plurk's features - such as mute, responses seen, private plurks etc.

A demo can be seen here - if that's too boring sign up, invite some friends and try it out live:

Our implementation uses node.js and a big kudos goes to them for their wonderful work.

We have rewritten to node.js from Java+JBoss Netty, because the old solution had scalability problems. We have also been able to build a very sophisticated messaging system given JavaScript expressiveness and the simplicity of node.js. Generally, we have found node.js and V8 very impressive - and it uses about 10 times less memory than Java+Netty did. The bottom line is if you implement anything comet related - - try to use node.js!

We may release our comet implementation in the future, but currently it's very tied up to Plurk's architecture.

I have already shared a lot of my knowledge on comet - including code and a talk about node.js for Taipei Open Source Group. Check out these posts for more info:

Code · Plurk · Tips • 30. Jan

10 comments so far

george 30. Jan

Hey, amix -- could you please give some more details about what scalability problems you ran into with Netty?

I'm implementing a network service now where I'm using Netty s o I would really like to hear what issues you faced at large scale.

amix 30. Jan

george:
Generally, node.js uses a little more CPU than Java+Netty, but much less memory (like noted, about 10x for our usage). And memory consumption was a concern for us. The other problem we had was lost connections/lost responses and this was a major issue as it made our comet communication unreliable.

Generally thought Netty is a lot more low level than node.js is. I would recommend looking at both, but personally I am amazed by node.js and V8. It should be said that I was also amazed by Netty and I still think it's a great product.

Tim 30. Jan: Hi Amix,
I love to hear more about configuration you setup for Java+Netty before hitting that scalability issue. If my system requires transactional support, I would think it's better for me to go with Netty. What do you think?

Peter Svensson 30. Jan

Hi Amix, Would you like to talk about the plurk/node hack at the Scandinavian Web Developer Conference 2010?

http://www.swdc-central.com

We're kind of packed, but I'd really like you to come.

Cheers,
Peter Svensson, Organizer (and some)

Imo 30. Jan: For someone who has only messed around with javascript to get things working on a site because they had to, where should I go to get up to speed on state of the art javascript and node.js? Any books for either topic that you force your devs to read or that you all agree on? Thanks!

Carl Byström 30. Jan

As george, I'm also a bit curious about what made you leave Netty for node.js.

Was it the memory usage with that made you consider node.js?
How many machines is your Netty vs node.js deployed on? What kind of server specs?

amix 1. Feb

Tim:
I don't think Netty offers better transaction support and if you need channel locks you can implement it pretty easily using memcached, check this post out for more info:

http://amix.dk/blog/post/19386

Other than this I'll write two follow ups on this why one should use node.js and why it's a good solution for doing comet.

Peter Svensson:
I'll follow up with an email.

Sergey Mikhanov 2. Feb

Hi,

Have you, by any chance, ever compared your solution to the Twisted-based one? It's easy to see that Node.js can beat Netty in terms of memory consumption, but what about Twisted?

BTW, Twisted fulfills your rationale for using Node.js (from your comment on Hacker News): "closures are a natural part of the language, and are well suited for event processing".

Toby M 7. Feb: Sergey: from Amir's other post, "Twisted: Non-blocking server in Python. Unfortunately it ate a lot of CPU and could not scale"

Tim 9. Feb: Amix,
Thanks for the response. You're awesome on tech side :) but I think you can get much further with a redesign site for Plurk. A clean interface will get you further.