WSGI Production Setup: uWSGI, supervisor and nginx

In this post I present a uWSGI, supervisor and nginx setup, which is probably the currently best way to run WSGI applications (including Django).

For Todoist and Wedoist we ran CherryPy's WSGI server for a long time. CherryPy has served us well, but after upgrading to a recent version we ran into deadlock issues. This forced us to look for alternatives, since we could not figure out what the problem was (and it's unsustainable to downgrade to a very old version).

Why we picked uWSGI?

• It's implemented in C and a lot of benchmarks point that it's one of the fastest WSGI servers. benchmark 1 benchmark 2
• nginx ships with uWSGI support since version 0.8.40
• The support between nginx and uWSGI is great since they speak over an optimized protocol instead of HTTP
• It supports LOTS of features, one of the best is killing and restarting dead processes
• It supports graceful restart of servers (without losing any requests!). This is great since we do multiple deployments pr. day
• It seems to be well supported and under active development
• A lot of others are choosing it as their production setup

How to run it behind supervisor

supervisor lets you easily control and monitor other processes (and restart them if they crash).

Our /etc/supervisor.d/uwsgi looks something like this:

[unix_http_server]
file=/var/run/supervisor_uwsgi.sock 

[supervisord]
pidfile=/var/run/supervisord_uwsgi.pid 
logfile_backups=1

[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

[supervisorctl]
serverurl=unix:///var/run/supervisor_uwsgi.sock

[program:todoist1]
command = /usr/local/bin/uwsgi -s 127.0.0.1:14001 
          --file /home/ubuntu/todoist/uwsgi_todoist.py --callable app 
          --processes 2 -t 60 --disable-logging -M --need-app -b 32768
user=ubuntu
stopsignal=INT

The interesting part of the uWSGI configuration are following:

• -t 60: If a process is not responsive for 60 seconds it is killed and restarted
• --disable-logging: We disable logging, since we use a central log
• -M: indicates master mode
• --need-app: If the app crashes on start then uWSGI crashes as well
• -b 32768: Sets a bigger buffer size. We needed this because we ran into a invalid request block size error

nginx setup

Our nginx setup looks something like this. Do note that we are using nginx'es upstream to distribute the load (for CherryPy we used haproxy):

http {

    upstream todoist_uwsgi { 
        server localhost:14001; 
        server localhost:14002; 
        ...
    }

...

    server {

        ...

        location / {
            error_page 502 /error_502.html;

            include uwsgi_params;

            uwsgi_param X-Real-IP $remote_addr;
            uwsgi_param Host $http_host;

            uwsgi_pass todoist_uwsgi;
        }
    }

...

}

Special case for slow requests

In Wedoist it's possible to upload large files.

To bypass this you must handle uploading requests specially and set a much larger timeout.

For Weodist we are running servers that only handle uploads:

[program:wedoist8]
command = /usr/bin/uwsgi -s 127.0.0.1:14008 
          --file /home/ec2-user/wedoist/uwsgi_wedoist.py --callable app 
          --processes 4 -t 6000 --disable-logging -M --need-app -b 32768
user=ec2-user
stopsignal=INT

As you can see -t is set to 6000 seconds, which means that the process/request will be considered dead after not responding for 6000 seconds.

In nginx configuration we also set uwsgi_read_timeout and uwsgi_send_timeout to 6000 and we redirect upload requests to a special upstream servers:

location /Uploader/attachFile {
    error_page 502 /error_502.html; 

    uwsgi_read_timeout 6000;
    uwsgi_send_timeout 6000;

    include uwsgi_params;

    uwsgi_param X-Forwarded-Proto https;
    uwsgi_param X-Real-IP $remote_addr;
    uwsgi_param Host $http_host;
    
    uwsgi_pass upload_cores;
}

That's about it. Hope somebody will find this useful :-)

Happy hacking!