I spent the majority of my weekend writing a generic caching server for Erlang to be used with my authentication/authorization lib. Originally I had created an auth lib (it still exists, but will be dead) at Google Code but decided to translate it from Python to Erlang, and hopefully make it better. One of the requirements was a limited in memory caching server. I could use memcached, but I couldn't think of a better first project to write in erlang to get my feet wet before starting the big authentication library.

It took a while to get into Erlang, honestly. The language is simple and elegant, but can become very advanced very quickly for an imperative programmer. For those of you who may not know what Erlang is, it is a functional programming language developed at Ericsson that lends itself towards massive concurrency and extreme fault tolerance. It has recently become much more popular, as words like "cloud computing" and "distributed hashtable" are becoming more common place.

There are two books I recommend for Erlang, to be read in this order...there is the backbone of Erlang, "Programming Erlang: Software for a Concurrent World" that will give you a very nice introduction into Erlang and take you into a world of concurrency. The second book was recently released, "Erlang Programming - A Concurrent Approach to Software Development". Notice a theme? The second book takes you much more out of your comfort zone if you are not used to functional programming, and gives you some very advanced coding examples to follow. I actually started reading this book first but by the 6th or 7th chapter was very lost, at which time I picked up Joe Armstrong's book, which prepared me for the latter.

But back to the topic at hand, I probably spent about ten hours altogether on my caching server, nearly half of that writing a testing library to test concurrent connections and the speed of my library. I will probably end up using or writing a memcached client anyway, but it was very good experience. Erlang is indeed elegant after you begin writing it, and the OTP is very powerful at creating a stable and fast app. I used the wonderful ets library to use as an in memory key value store, and primarily just wrote my own get and set methods that ensured that the cache stayed near the memory boundary.

I decided to follow memcache's python API, with set, get, del, and add functions. The set function is actually what checks expiration and memory constraints after storing a new value. It took a while to research and understand ets, but after I did I was somewhat shocked at its speed and versatility.

To mark items as expirable, I have a timeout key, as well as storing the created timestamp and the number of access times. I prefer to keep highly accessed values in the cache, so I add two minutes to the create time per access to extend its life. If there is a timeout time, the "get" catches it and doesn't return the value, otherwise the "set" function will expire "first in", least accessed items first. It's a very simple scheme that is probably entirely incorrect for most settings, but it works for me for now.

I created both the memory free-er and the client api as servers, each held by a supervisor as to not interfere with each other, held by another supervisor. I probably overdid it with supervisors, but it was the easiest way at the time to write it as I wanted to maintain my ets database until a last resort crash. It turned out to work very well, with bugs in the program crashing the bottom tier supervisors at times, but not the top. Of course with a supervisor behaviour, the servers are started back immediately. When a request comes in, a new process is created to handle it then returns as a gen_server reply (somewhat like an rpc call).

Timing it, I used the wonderful plists library to create 10-3000 connections at a time. I actually timed over 300,000 connections, each over 1024 bytes in size on a 16MB cache, but the server could not handle 300,000 connections at a time. It could however handle 3,000 connections and responded to each single get request in less than 35 microseconds...that's right, micro, not milli. The set functions took a bit longer depending on if the cache was full. Originally I had implemented the polling memory free-er using the ets select statement checking for expiration times, but this proved too slow, taking over 50-100 milliseconds. I re-wrote it using ets:foldl which takes a function and runs it over every item in the table, and the set time on a full table collapsed to 5-7 milliseconds regardless of the number of items in the table. An empty table had comparable results with get, being 30-50 microseconds. It amazed me erlang could iterate over that many items that quickly...perhaps that is not fast for C, but being very used to Python, iterating 16,000-100,000 items can take more than 5 milliseconds per function call, much less with 3,000 threads of execution. If I had written it like memcached using the idea of memory blocks, the performance would probably be even better.

I definitely feel that I learned a lot from the project...I am excited to continue with my authentication library that I hope will eventually turn into an entire profile, authentication, and authorization store for use with single sign on for multiple backends, including Django.

For now, you can check out erl_tempcache at github.