Python unladen-swallow at least 33% slower than Python 2.5.1

unladen-swallow is an optimization of Python done by some Google engineers, it's a very exciting project since it promises a JIT version of Python that's highly optimized for web applications. I evaluated if it would be wise to upgrade Plurk to the 2009Q1 version. Unfortunately the results are not that impressive :-(

I did a simple benchmark on my MacBook (2GHz Intel Core Duo 2 with 4GB of RAM):

This request does a lot of requests to memcached and does a lot unpicking - something that unladed swallow should optimize on. I benchmarked this against Python 2.5.1.

unladed swallow benchmark

Version: 

Python 2.6.1 (r261:811, Aug 26 2009, 22:30:27) 
[GCC 4.0.1 (Apple Inc. build 5465)] on darwin
[Unladen Swallow 2009Q1]

ab benchmark: 

Concurrency Level:      10
Time taken for tests:   47.242 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      6036000 bytes
HTML transferred:       5899000 bytes
Requests per second:    21.17 [#/sec] (mean)
Time per request:       472.418 [ms] (mean)
Time per request:       47.242 [ms] (mean, across all concurrent requests)
Transfer rate:          124.77 [Kbytes/sec] received

Resource usage: 

  PID COMMAND      %CPU   TIME   #TH #PRTS #MREGS RPRVT  RSHRD  RSIZE  VSIZE
  560 python2.6    0.0%  1:17.72   9    63    325   66M   188K    69M    89M

Python 2.5.1 benchmark

Version: 

Python 2.5.1 (r251:54863, Feb  6 2009, 19:02:12) 
[GCC 4.0.1 (Apple Inc. build 5465)] on darwin

ab benchmark: 

Concurrency Level:      10
Time taken for tests:   30.138 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      6036000 bytes
HTML transferred:       5899000 bytes
Requests per second:    33.18 [#/sec] (mean)
Time per request:       301.381 [ms] (mean)
Time per request:       30.138 [ms] (mean, across all concurrent requests)
Transfer rate:          195.58 [Kbytes/sec] received

Resource usage: 

  PID COMMAND      %CPU   TIME   #TH #PRTS #MREGS RPRVT  RSHRD  RSIZE  VSIZE
  554 Python       0.0%  0:38.49   9    71    349   44M   700K    47M    69M

Conclusion

From my benchmarks it's clearly to see that:

  • unladen-swallow 2009Q1 is slower than Python 2.5.1 (33% slower)
  • unladen-swallow 2009Q1 uses much more memory than Python 2.5.1 (33% more memory)
  • unladen-swallow 2009Q1 uses much more CPU than Python 2.5.1 (50% more CPU)

Like all benchmarks this benchmark should be taken with a grain of salt. But generally, I am not that eager to upgrade to unladen-swallow or Python2.6 just yet ;-)

This could be related that 2009Q1 is based on Python 2.6,x which is slower than Python 2.5.x. But still, it's not good when future versions are much slower, use much more CPU and much more memory... it should go the other way around.

27. Aug 2009 • Benchmarks · Code · Plurk · Python
© Amir Salihefendic