Wedoist features a realtime system that's powered by comet where the state is synced across all open sessions. Today we have rewritten our client side implementation to use Cross-Origin Resource Sharing, which is a relatively new technology to do cross domain communication. This technology is supported by following browsers:

• Firefox 3.5+
• Safari 4+
• Google Chrome 3+
• Internet Explore 8+

Implementing comet is a challenge, both on the backend and the client side. In this blog post I will only feature the client side implementation.

How is our comet setup?

comet0X.wedoist.com points to a nginx server that proxies to JBoss Netty servers. This makes it possible to support SSL and load balance.

This solution scales really well, but we need to do cross-domain requests to comet0X.wedoist.com. For this we used ScriptCommunicator, which is one of my more popular open-source projects. It basically uses script tags for communication and it resolves to hacks in order to detect errors.

Using Cross-Origin Resource Sharing

The main points of Cross-Origin Resource Sharing are following:

• Your backend server specifies which origins it accepts requests from via a special Access-Control-Allow-Origin header
• On Chrome, Safari of Firefox you use XMLHttpRequest object and check for withCredentials property
• On Internet Explorer 8+ you use XDomainRequest object
• For everything else you resolve back to script tags

Our implementation of this strategy is shared below.

Specifying Access-Control-Allow-Origin

You need to specify what origins you are accepting requests from and you do this by adding a special Access-Control-Allow-Origin header.

In nginx you can do following to only accept from https://wedoist.com;

location / {
    add_header Access-Control-Allow-Origin https://wedoist.com;
    proxy_pass http://0.0.0.0:8001/;
}

To accept from everywhere you would do this (it isn't recommended to due CSRF exploits!):

location / {
    add_header Access-Control-Allow-Origin *;
    proxy_pass http://0.0.0.0:8001/;
}

Implementation of the new script communicator

You can download and fork the code here:
https://github.com/amix/Script...

Using the new script communicator

To use the new script communicator simply do following:

var url = 'http://some_domain.com/give_me_js_back.php?i=42&callback=foocall'
ScriptCommunicator.sourceJavascript(url, on_success, on_error)

on_success and on_error are functions. on_error is only supported by browser that support Cross-Origin Resource Sharing, i.e.:

• Firefox 3.5+
• Safari 4+
• Google Chrome 3+
• Internet Explore 8+

Like the old ScriptCommunicator implementation your server should return JavaScript that can be eval'ed.

References

• Wikipedia: Cross-Origin Resource Sharing
• Enable cross-origin resource sharing an info site about CORS
• W3's CORS specification
• Mozilla's HTTP access control
• Cross-domain Ajax with Cross-Origin Resource Sharing, a good tutorial on how to get started

Last saturday we switched Wedoist from Rackspace's Cloud to Amazon AWS. In this post I will present why we have done this and why we picked Amazon AWS instead of Rackspace's Cloud or dedicated hosting.

Why not Rackspace's Cloud?

We initially used Rackspace because we thought they had lots of experience and expertise in hosting. Now we are not so sure and we don't have a huge trust in their systems or their network.

Our problem was that a lot of users had reported performance issues. Wedoist is used in about 50 countries and each time we debugged these issues they were from users that came from "remote countries" such as Brazil, Hong Kong or Taiwan. Having developed web-applications for a long time we were sure it was a network issue and even I could reproduce this issue in Chile. This was our main reason for moving to Amazon AWS: Rackspace's network does not seem to work that well in "remote areas" of the world.

The second reason is that Amazon AWS seems to be a superior product that offers more solutions such as:

• Amazon RDS: Managed SQL databases
• Amazon SES: Simple Email Service
• Amazon DynamoDB: Fully managed NoSQL database

Amazon AWS also seems to be in rapid development, for example, Amazon DynamoDB was released last week (and it seems to be an awesome database solution!)

Why not dedicated hosting?

You will get a lot more hardware by going for dedicated hosting (either buying or leasing servers). For example, if you read pinboard's article on dedicated hosting, A Short Rant About Hosting, you will find prices that are much cheaper than Amazon's.

Here are our reasons why we didn't go the dedicated hosting route:

Cost isn't everything:
For us cost isn't the only thing we care about. Our focus is on building software and serving our users in best ways possible. Setting things up ourselfs would save money, but would require a lot more work, be more limited and possible produce more errors.

We don't have resources to use multiple data centers:
Some things would not be possible at all, because setting it up would be very expensive - - an example is minute snapshots of our databases that are stored in 3 different data centers and can be restored with a few clicks. Implementing backup strategies is complex, especially if you have a lot of data, and Amazon AWS saves us a lot of headaches in this area.

We can easily scale:
With Amazon AWS scaling up or down is easy: we can add servers, we can upgrade to bigger machines, we can downgrade, we can script everything so extra machines are added in the morning and shutdown in the afternoon. All this with a few clicks or a few scripts and in matter of minutes.

Our focus isn't on managing servers:
The bottom line is that we can focus on building great software. Using Amazon AWS is more costly than dedicated hosting, but our products aren't free, so paying for premium hosting isn't a huge deal.

Amazon AWS seems to be a marketleader
Amazon AWS has an impressive customer base that includes Dropbox, Netflix or Yelp (just to name a few). We feel in good company by being there as well.

The context

In 2006, when I started Todoist, MochiKit was one of the better JavaScript libraries. Today MochiKit is largely forgotten and the state of the art JavaScript libraries are backbone.js, spine.js or ember.js. All of these newer libraries use the Model-View-Controller (MVC) pattern, a pattern I have written about in the past:

• Model View Controller: History, theory and usage [2011]
• MVCxN pattern [2006]

Model View Controller rewrite in JavaScript/CoffeeScript

I will now show a series of screenshots of what the problem is and how I find a solution that makes the code simpler and more maintainable.

Each project in Todoist has a counter that shows how many tasks the project holds. In the old codebase there's one creator and 7 (yes 7!) update functions:

I could rewrite this to use fewer update functions or even just one, but I could also go an extra mile to ensure that such blunders do not happen in the future. The better way to do this is to use the MVC pattern: the basic idea is that we create a counter that autoupdates itself each time the model changes.

I looked at how modern libraries handled this (I was largely inspired by backbone.js ) and I began scribbling down how an API could look like:

I did not want to use backbone.js directly as I don't like the way routes work or that it's dependent on underscore (adding backbone.js and underscore would result in 2000 lines of extra code). I also think I can learn more by implementing stuff myself and I can do a customized solution that works well with my existing code.

After this step I began to implement a test to see if my API made sense:

I tried to use the new code for the counter (I had two solutions, either binding it to a project object or using ProjectsModel). I tried to use them both throughout some code to see how they would behave in different contexts:

Then I set these solutions against each other - - reflecting on pros and cons of each solution:

The solution I ended up with was following:

Would love to hear about how you attack a rewrite. Happy hacking!

Some of our stats:

• Over 300.000 users
• Rapid growth
• Our business is profitable
• We started fulltime with the company just 6 months ago. Imagine the future :-)

Join us either freelance or full-time and work on something that makes the world more productive.

Send your resume to amix@amix.dk, be sure to include some code you are proud of (or a link to your GitHub/BitBucket profile).

Want to become more productive in 2012? Upgrade for $14/year at todoist.com/premiumPromotion.

Todoist is espeically powerful with browser plugins (that integrate nicely wtih Gmail). Check out:

• Todoist for Google Chrome
• Todoist for Firefox

To find Users in just .js files recursively you would do:

:vimgrep /Users/ **/*.js

To find Users in .js and .coffee files recursively you would do:

:vimgrep /Users/ **/*.js **/*.coffee

This is super useful when you are refactoring code.

I am unsure if Ack.vim can do something similar. If you know, please leave it as a comment!

You can check out some of my other Vim tips here.

Over the years something has changed in me and I am currently obsessed with design. I also think design applies to everything - - from clothing, to code, to food. And design isn't only about how it looks like, but also how it works.

I think my change has been fueled by Apple's amazing comeback. One of Apple's biggest differentiating factors is their amazing designs. They built the biggest technology company based on great designs - outperforming companies that solely focused on functionality.

In most people's vocabularies, design means veneer. It's interior decorating. It's the fabric of the curtains of the sofa. But to me, nothing could be further from the meaning of design. Design is the fundamental soul of a human-made creation that ends up expressing itself in successive outer layers of the product or service.
— Steve Jobs

Here is a redesign of Todoist's and Wedoist's fronpages. It's not live yet, but will be live during the next week:

I was in same group as Herval and he is a good guy. This said, I have a huge problem with this general mentality and I think a lot of today's entrepreneurs are spoiled brats that don't have their perspectives right.

I have had an awesome experience in Chile and it has been one of my best decisions I have made. I have met my girlfriend here, I have scaled my business a lot, I have learned to surf, I have met amazing people from all around the world etc. On top of this, my company got $40.000 free money. The reimbursement process could be improved, but generally I did not have that much problems with it.

The problem I see is that people expect everything on a silver plate. They expect everything to be perfect. And of course, if you expect this you will be disappointed by Chile and Start-Up Chile - - because it isn't perfect and they have a long way to go, but they are trying hard to create a good platform for entrepreneurs and so far I think they are doing a great job (and a better job than most other countries in the world).

Maybe this lack of perspectives is culturally/experience bound. For me, my dad's story as an entrepreneur gives me perspective that I have it good and that I have much better opportunities than he ever had. My dad was an entrepreneur that quit his factory job to start his own business in his early 20's. He worked 12 hours pr. day at least. He worked and scaled his business for over 20 years - - to provide education for my siblings and good life for my family. Then the Bosnian war came and we lost it _all_. We had to relocate to Denmark, a country where we did not speak the language (and I can tell you that Danish isn't an easy language to learn). What did my dad do? He learned Danish and started another successful business in his late 40's.

So when I see people complain about the process of getting free money or that the chairs in the offices are bad then I laugh, because I think they have no perspective of what it takes to build a business, how hard building a business is for majority of people or what struggles other people have.

I have before covered CoffeeScript and you should read my post if you don't know what CoffeeScript is:

• CoffeeScript: The beautiful way to write JavaScript

The Pythonic extension of js2coffee

js2coffee is great, but I don't like some of the code it produces as it's more similar to Ruby than Python. I have forked js2coffee and provided a --pythonic option which does following things:

• Uses 4 spaces for each step of indent
• Uses parenthesized function calls - - I really dislike Ruby's optional parenthesizes on function calls, it produces unceccary bugs and makes code harder to understand (at least, when having a Python background)
• Does not use implicit returns
• Does not use unless - - for me, code that uses unless is much harder to understand

Here's a little showcase of what --pythonic does different:

$ ~/> cat test.'s

The JavaScript code we are compiling to CoffeeScript:

function helloWorld(some_value) {
    if(some_value != 'hello')
        return;
    aFunctionCall('hello');
    return "value";
}

$ js2coffee test.js

Compiling it via the standard js2coffee produces following code:

helloWorld = (some_value) ->
  return  unless some_value is "hello"
  aFunctionCall "hello"
  "value"

$ js2coffee --pythonic test.js

Compiling it with the --pythonic option produced following code - - which for me is easier to understand given my Python background:

helloWorld = (some_value) ->
    return if some_value isnt "hello"
    aFunctionCall("hello")
    return "value"

Google Closure + CoffeeScript?

Another project I am currently working on is to improve CoffeeScript's compiler so it can work with Google Closure. What this enables is to write high-level code in CoffeeScript that gets minimized and optimized by Google Closure's compiler (which does non-trivial code transformations and optimizations).

There's already a project that does something similar to this:

• Closure Mode for CoffeeScript

Currently reading: Autobiography of a Yogi by Paramahansa Yogananda

In 1946, Paramahansa Yogananda (January 5, 1893–March 7, 1952), published his life story, Autobiography of a Yogi, which introduced many westerners to meditation and yoga. It has since been translated into 25 languages, and the various editions published since its inception have sold over a million copies worldwide.

The book describes Yogananda's search for a guru, and his encounters with leading spiritual figures such as Therese Neumann, the Hindu saint Sri Anandamoyi Ma, Mohandas Gandhi, Rabindranath Tagore, Nobel Prize-winning physicist Sir C. V. Raman, and noted American plant scientist Luther Burbank, to whom it is dedicated.

Following Ask HN: Best book you read in 2011 I have added following to my reading list:

The Undiscovered Self by Carl G. Jung

In his classic, provocative work, Dr. Carl Jung-one of psychiatry's greatest minds-argues that the future depends on our ability to resist society's mass movements. Only by understanding our unconscious inner nature-"the undiscovered self"-can we gain the self-knowledge that is antithetical to ideological fanaticism. But this requires facing the duality of the human psyche-the existence of good and evil in us all. In this seminal book, Jung compellingly argues that only then can we cope and resist the dangers posed by those in power.

The Checklist Manifesto: How to Get Things Right by Atul Gawande

We live in a world of great and increasing complexity, where even the most expert professionals struggle to master the tasks they face. Longer training, ever more advanced technologies—neither seems to prevent grievous errors. But in a hopeful turn, acclaimed surgeon and writer Atul Gawande finds a remedy in the humblest and simplest of techniques: the checklist. First introduced decades ago by the U.S. Air Force, checklists have enabled pilots to fly aircraft of mind-boggling sophistication. Now innovative checklists are being adopted in hospitals around the world, helping doctors and nurses respond to everything from flu epidemics to avalanches. Even in the immensely complex world of surgery, a simple ninety-second variant has cut the rate of fatalities by more than a third.

Image source

I wanted to rename todoist.users to todoist.apps.sessions, since it was a more appropriate name. Here's how I made sure that all the imports got updated:

1. Search for all occurrences of users in Python files, :vimgrep "users" **/*.py":

2. Show all results in a quick list window by typing :cope in command mode:

3. Copy the results of the quick list window into a new buffer. By for example typing gg => V => G => y => :ene => p

4. Delete all lines that have models in them (as we want to ignore todoist.models.users). You do this by typing :g/models/d.

5. Delete all lines that don't have import in them (we want only to focus on import lines). You do this by typing :g!/import/d

After this step I only see results that are relevant for my refactoring.

A bonus for you vimrc

Add this to your .vimrc to quickly transform your quicklist into a new buffer that's syntax highlighted:

map <leader>cc :botright cope<cr>
map <leader>co ggVGy:tabnew<cr>:set syntax=qf<cr>pgg

Anytime you press leader + co in a quicklist window it will isolate the quicklist in a new tab and syntax highlight it.

py_static_check is based on pyflakes and adds following features:

• Ability to specify what star imports resolve to (-s argument). pyflakes does not work with star imports
• Ability to ignore unused import warnings (-i argument). Checking a big code base will pollute your warnings with unsued import warnings
• Better sorting of warnings/errors

To install it do following:

$ sudo easy_install py_static_check

To fork it (and improve it) go to github:

• https://github.com/amix/py_static_check

Here are some of the things py_static_check can do.

Catch undefined names, even for star imports

Example code:

from os import *

def function_with_error():
    print path
    print paths

star_imports.py:

import os
STAR_IMPORTS = {
    'os': os.__all__,
}

Running it with py_static_check (with -s argument):

$ py_static_check -s tests/star_import.py tests/undefined_name_star.py
tests/undefined_name.py:5: undefined name 'paths

Running it with pyflakes:

$ pyflakes tests/undefined_name_star.py 
tests/undefined_name_star.py:6: 'from os import *' used; unable to detect undefined names

Ignore not used warnings

Example code:

from os import path

Running it with py_static_check (with -i argument):

$ py_static_check -i tests/ignore_not_used.py

$ py_static_check tests/ignore_not_used.py
tests/ignore_not_used.py:10: 'path' imported but unused

Assigned but never used

Like pyflakes it can catch a lot of errors, such as defining a variable without using it.

Example code:

def some_function():
    def inner_fn():
        local_var = ""

Running it with py_static_check:

$ py_static_check tests/assigned_but_never_used.py
tests/assigned_but_never_used.py:8: local variable 'local_var' is assigned to but never used

Hokusai

My task was to ensure that the correct language is set when sending out emails.

My first iteration used following code:

#Remember the current language
current_lang = get_current_lang()

#Set language by uid if possible
user_lang = get_lang_by_uid(uid)
if user_lang:
    set_current_lang(user_lang)

#… MAIL CODE … 

#Revert back to old language
set_current_lang(current_lang)

Now this is fine and well, but not that clean, especially when you have to use it in multiple places. I knew there must be a better way to do this. After some iterations I came up with this marvel:

with set_lang_by_uid(uid):
    #… MAIL CODE …

A one liner that I can use everywhere!

The implementation of set_lang_by_uid

Implementing set_lang_by_uid is trivial - that's the great thing about using the with statement!

Here's how it's implemented:

from contextlib import contextmanager

@contextmanager
def set_lang_by_uid(uid):
    user_lang = get_lang_by_uid(uid)
    if user_lang:
        current_lang = get_current_lang()
        set_current_lang(user_lang)
        yield
        set_current_lang(current_lang)
    else:
        yield

More references

Check out my older writing on the with statement:

• Implementing file locks using Python's with statement
• Python's with statement