Creating email applications using Lamson

Email is an old and proven technology and integrating with email is very important, especially if you target businesses. Unfortunately building email applications has been a major pain until recently. In this post I will explore my venture into building email support for Wedoist using a modern framework like Lamson.

Outdated tools and the new hope

The problem with building email applications is that the email tools are as old as email itself (from the 1970's). Fortunately for us, in the recent times there has been a major contribution in this field by Zed Shaw. He has created a modern framework for building email applications called Lamson.

What Wedoist should support

Wedoist is a project management tool and I think it would be super useful to add status updates or tasks via email. For example, if you get an error report via email you can delegate it by forwarding it to the proper project.

What I wanted to implement are following emails:

• inbox.[project_id]@wedoist.net: Send an email to this address to add a task to [project_id]
• status.[project_id]@wedoist.net: Send an email to this address to add a status update to [project_id]
• comment.[item_id]@wedoist.net: Reply-to email that can be used to leave a comment on tasks/updates directly from an user's email client

All of these have been implemented, so let's take a look how they were done, how it can be done and why Lamson is a great way to do it.

Is it hard to build your own email server?

No, it's not hard, but building your own SMTP server is just the first step, just like coding a HTTP server is just a tiny part of implementing a web application.

To show how easy it is here is a very simple non-blocking SMTP server built in Python:

import smtpd
import asyncore

class MailDummy(smtpd.SMTPServer):

    def process_message(self, peer, mailfrom, rcpttos, data):
        print mailfrom

if __name__ == '__main__':
    server = MailDummy(('0.0.0.0', 25), None)
    asyncore.loop()

The Lamson way

Lamson does a lot more than just being a SMTP server. It's a framework for building kickass email applications.

Lamson provides following tools:

• Routing is based on regular expressions and Finite State Machines. Might sound complicated, but it's very easy to use and understand and much more elegant than the alias based hell
• With Lamson you can use a modern template system such as Jinja2 or Mako
• Modern storage system, basically you can use any storage system that you like (such as SQLAlchemy, lightcloud or whatever Python supports)
• Model-View-Controller paradigm, similar to Django, Rails or any other modern web-framework
• nose testing environment

So basically Lamson provides the same tools for building email-applications as the tools we use to build web-applications. It's pretty neat. Let's take a look at my Wedoist code to see how Lamson code feels like.

Implementing status.[project_id]@wedoist.net

The first step is to implement a handler. For status updates this is done from app/handlers/status_updates.py and the code looks like this:

@route("status.(project_id)@(host)", project_id="[0-9]+")
@stateless
def STATUS(message, project_id=None, host=None):
    project_id = long(project_id)

    user = _resolve_user(project_id, message, host)
    if not user:
        return

    try:
        project = ProjectsApi().get(project_id)
        subject, body = parse_subject_body(message)

        if subject:
            item = StatusesApi().add(project.id, subject)

            if body:
                CommentsApi().add(project.id,
                                  'status',
                                  item.id,
                                  body)

            signals.send_signal('email_status', item)
    except PermissionError as p_err:
        _send_permission_error(project_id, message, p_err)
    except ModelError as m_err:
        _send_model_error(project_id, message, m_err)

Decorators are used to tell that status[project_id]@wedoist.net should be routed to this function. It's quite elegant and much more readable than your average email code.

Unit testing your email application

As I mature as a developer I find testing more and more important, especially in a language like Python. Lamson supports unit-testing and makes it trivial. Here is how one of the tests for status[project_id]@wedoist.net looks like:

def test_unknown_user():
    assert len(ItemsApi().getActive(PROJECT.inbox_list)) == 0

    helpers.say('attacker@test.com',
                INBOX_EMAIL,
                'Some random rambling',
                'Body body',
                except_reply='noreply')

    assert len(ItemsApi().getActive(PROJECT.inbox_list)) == 0

It's neat and a very nice way to develop email-applications.

Start building email support for your application

I highly recommend using Lamson. I basically built Wedoist's email support in one day - it is in production now and you can check it out.

Happy hacking :-)