Open-sourcing at Inaka (lasse, egithub, tirerl & more)
A little over a year ago, when I started working at Inaka, I knew very little about Erlang and its ecosystem. Over that time I had the opportunity to create and contribute to a number of open-source Erlang projects, and in the process I learnt not only Erlang but a whole bunch of things.
All of the open-source projects I worked on were needed for a specific client's project or an internal tool we were working on. At Inaka we think it is a good idea to share what we do, specially because we use a lot of open-source libraries created by others. And giving back to the open-source community not only generates a warm feeling inside, but also allows the company to be known.
So let's roll back a year from now.
On my second day after I had settled in, had my morning coffee and configured my development environment, Brujo (our CTO) came to me and said:
I have a project you can practice your Erlang-fu with... and it's going to be open-source.
The project in question was lasse, a Server-Sent Event (SSE) handler for Cowboy. SSE is a simple way to keep a communication open between the client and the server, and Cowboy is currently the most powerful Erlang web server. Cowboy handlers are implemented as a group of callback functions, which are included in the request-response flow through Cowboy's internal logic.
At Inaka we used to reimplement the logic to handle SSEs with Cowboy for every project that needed it, which were a lot. So it made sense to avoid writing the same code that handled SSEs for each one of them.
Since a handler is just a couple of functions here and there, the task was simple enough for an Erlang newbie. At the same time there were a couple of things I had to go over before I could even start typing code, such as:
Having a clear objective helped me to organize work and sink in the Erlang concepts I had been reading about. So a couple of days later I had a working version of the code, which was later used in client and open-source projects (e.g. fiar).
On to the next one
The second Erlang project I got to work on was Elvis, a style checker for Erlang source code. This was a challenging project to say the least, but it was a lot of fun.
Part of the fun stuff involved integrating Elvis with GitHub as a webhook. In order to accomplish this we needed to interact with GitHub's API, so a library to do just that was in order...
The GitHub API is quite big, but for our purposes we only needed a specific portion of it at first. Over time though, we added an almost full set of the API calls, included a way of handling the limits set by GitHub on requests per time interval, and other goodies that we either needed or thought were a good idea to include.
One of these goodies, for example, is the possibility of choosing between different JSON libraries. The reason behind this decision is the impossibility of creating escripts that include NIFs (i.e. jiffy).
Two Inaka projects, Elvis and
Gadget, currently use
erlang-github extensively, so you
can be sure this library is under on-going development. We are regularly fixing bugs and
introducing improvements to make the experience better.
Last but, not least
The last open-source library we are going to talk about in this blog post is called tirerl. But first let me tell you a little bit about SumoDB.
SumoDB is an abstraction layer to persist information in different data stores. The most used backend we have is MySql, but we recently added new backends like Riak, PostgreSQL, Mnesia and ElasticSearch, with various degrees of completion.
The backend for ElasticSearch uses our own open-source library called tirerl.
This library is simply an Erlang interface to ElasticSearch's HTTP API. It exposes almost all calls to the mentioned API in a clear and accessible way. It was born as a rewrite of the existing erlasticsearch, after trying out the existing ElasticSearch libraries for Erlang and finding it wasn't simple to use them as SumoDB backends. Since the ElasticSearch data model doesn't exactly fit into the SumoDB abstractions, it was quite challenging to come up with an implementation that made sense.
Although this library hasn't seen much action in the past 3 months, it will probably get a little bit of love in the near future, in terms of documentation, tests and examples. So stay tuned!
All the libraries we have mentioned here are the product of hours dedicated to build open-source projects. They are far from perfect, with varying degrees of completion and documentation. It might be fair to say this is what happens in the open-source world, projects are started to fill a need and put on hold (sometimes indefinitely) when that need is sufficiently fulfilled.
The good thing about the projects we have mentioned here is that they are required to be able to perform in production environments. So most of them have been not only tested through integration, functional and unit testing, but through real-world usage, which is ultimately the decisive factor when judging the quality and reliability of any piece of software.
Finally I want to thank Inaka for giving me the chance to contribute to the open-source community and being able to learn so much along the way.