Mail archive

[acf] ACF2: http server and transaction manager

From: Natanael Copa <>
Date: Thu, 6 Dec 2012 17:13:14 +0100

Some questions for ACF2 implementation.

Do we go for CGI (like current ACF?)
Do we go for FCGI (with like lighttpd?)
Do we pick a more dedicated app server, like mongrel2, xavante, luvit?

In any case, the idea is that you can create a "transaction" in the
transaction manager. The properties of the transaction can be changed
over multiple http requests, and when the "commit" action happens the
transaction is passed on to the storage, or "persistance manager".

How does the http request communicate with the state?

I have tried to collect some of the options we have and I wonder if
there are other (good) options?

Transaction state options:
 * save the current transaction state in (tmpfs) filesystem
   - dump luatables in /tmp (as serialized lua, json or similar?)
   - in a database, like sqlite/gdbm/tdb/redit
   - works with forking (if cgit and for privilege separation)

 * keep state in memory let the transaction manager run as a separate:
   - process:
     - independent daemon. http requests (web app) talks to it via some
       communication channel listed below.
     - each transaction forks its own process. subsequent requests (web
       app) talks to it via some IPC channel listed below.
   - thread:
     - separate worker thread created on first requests that lives
       "forever". requests talks via some IPC channel.
     - each transaction create a thread that livs as long as the
       transaction. subsequent requests talks to it via some
       communication channel.

   Possible communication channels for the above:
   - zeromq
   - lua lanes' lindas (for lua-lanes). Threads only.
   - dbus
   - roll our own, using unix sockets or pipes.
   - using filesystem (some db or tmp file) as listed above

HTTP server options:
 * non-threading/non-forking app servers:
   - xavante - uses lua coroutines to avoid blocking
   - luvit - uses an event library (libuv) and callbacks to avoid
   - lighttpd + mod_magnet (blocks so fairly useless)

 * threading app servers:
   - nginx-lua (
   - mongrel2
   - apache + mod_lua
   - fcgi (wsapi.fcgi) with one of:
     - lighttpd
     - nginx
     - cherokee?

 * cgi webservers (with haserl?)
   - mini_httpd
   - busybox httpd + stunnel (for https)
   - lighttpd
   (lots more)

 * unknown webservers from luci
   - zhttpd:
   - lucihttpd:
   I don't know what those does or if they are useful for us.

I suppose that even if we pick a non-threading http server like xavante
or luvit, we could still let the transaction manager run in its own
thread/process as listed above, using lua-lanes or something. It might
require some clever use of coroutines or callbacks for signals (eg
catch sigchild from a transaction subprocess) I havent checked if libuv
(used in luvit) can do that but I assume it does, like libev does.

There is also a question of privilege separation, which basically means
a separate process for the privileged stuff. Does the transaction
manager need to run as root? Probably not. But persistance manager will
need it, so maybe we want/need persistance manager run in separate
process. Does it need run all the time, as a daemon? probably not.
fork/exec of a suid root executable could work too.

In general, I think it would be nice to not be tied to any specific
webserver (use plain CGI or FCGI). CGI is nice because the web servers
available are so small, but since each http reques require fork, it
will not scale to a 16000 request per second app. It might be fast
enough for our purpose though. FCGI sounds tempting but then webserver
is a bit bigger (lighttpd).

If we have *many* http requests and each is very fast to handle, then
we probably want luvit or xavante. but those also have a scalability
problem (i think) due to they are non-threading.

Other options we have?

Other ideas?

Some useful hits from Google:

What web server to use for Lua web development:

Lua Webserver:

Subject: http serving options:

Received on Thu Dec 06 2012 - 17:13:14 GMT