The art of scaling

The wise men say it's a good thing to have scalability issues - and generally it is, because it means you have created something that people actually want to use. But scaling something up is a real pain - - especially the database layer where each optimization can take days to complete.

I will share one tip that can make a big difference, I call it "The art of scaling":

The art of scaling is really the art of knowing - - the art of knowing your problem and having sufficient experience to solve it the right way. The art of scaling is not to do premature optimizations, but to apply optimizations where optimizations are needed. One will find this information in doing profiling and monitoring of the whole system.

I plurked earlier today how happy I was that Plurk was very fast again. The reason for the speed improvement is a new custom timeline database that improves our performance at least 8 times. This is the load one on of our MySQL servers after the release:

To implement this timeline database was pretty easy, it took around 2 days. To get the idea and to get enough information about the problem domain has taken over 1 year.

You have probably heard that premature optimizations are the root of all evil - - and they are! Another root of evil is not keeping a track of how healthy your system is and what your application spends time on.

In other words, The art of scaling can be summed up in following statements:

• it's unlikely that you will have sufficient experience to implement the right solution the first time
• don't optimize unless you have to
• don't trust your code - ensure that you know where most of the time is spent
• be proactive, instead of reactive: solve issues before they arise, but only solve those problems that your monitoring data tells you will arise or are arising

Some related blog posts about this subject:

• How Plurk does systems monitoring
• Reactive vs. proactive development