Parallelism

The NIPAP backend is written using Twisted which doesn't support forking or running things in threads, instead it relies on asynchronous programming and deferred tasks to kind of simulate the effect of running multiple parallel request handlers (either threaded or forked).

To avoid specifically talking about forking or threads, I will refer to the whole business of dealing with multiple requests in parallel simply as 'parallelism' for the rest of this document.

Twisted appealed to us (the original developers of NIPAP) because it seemed like a sweet framework that would be able to leverage some form of parallelism in the future - we just needed to modify some code. Besides, a lot of cool things, like parts of Spotify's backend (according to rumour), was written in Twisted which added to its appeal.

Usually in a project you just want to get things going and trying out all the pesky little details of your framework require so much time that doesn't produce anything usable - so, we never looked much deeper at the asynchronous coding required for parallelism in Twisted until much much later. Over time, our view of Twisted changed somewhat - it's still a nice framework just not really for us. It doesn't feel very Pythonesque and the code changes required for asynchronous query handling to work just seemed to imposing.

Instead we came to look at other things, like Flask, and with the Flask-XML-RPC plugin, it seemed like it could be our drop-in replacement for Twisted. Luckily (we did something right!), most backend code is split into a backend.py file which isn't specific to Twisted or even XML-RPC at all and xmlrpc.py which kind of wraps around the functions in backend.py and does XML-RPC stuff (like exception/error translation) as well as providing some Twisted glue. The idea here was that it should be easy to add a REST/JSON API. I never thought we'd be ripping out Twisted, but with our split it was actually pretty easy.

There's now a proof of concept for a Flask-XMLRPC backend which replaces xmlrpc.py with flask_xmlrpc.py and modifies part of nipapd to start up the Flask API with the Tornado framework.

Tornado is essentially a small web server that can serve Flask apps. It provides multiple means of parallelism and the one chosen for NIPAP is pre-forking as we don't need to share any state across multiple handlers and forking is dead simple for this use case. Most time spent handling requests is not spent in NIPAP at all but rather in the database and since PostgreSQL is pre-forked, with one backend per client, we fork as many request handlers as there are CPU cores (default behaviour of Tornado).