Latest blog entries
The Art of Writing a Blogpost
Mar 09 2017 : Matias Vera
Feb 14 2017 : Felipe Ripoll
Do you need a blazing fast reverse geocoder? Enter offline-geocoder!
Jan 18 2017 : Roberto Romero
MongooseIM has RESTful services!! Here I show how you can use them in an iOS application.
Dec 13 2016 : Sergio Abraham
20 Questions, or Maybe a Few More
Nov 16 2016 : Stephanie Goldner
Because conferences and meetups are not just about the technical stuff.
Nov 01 2016 : Pablo Villar
Sharing some light on how it is to partner with us.
Oct 27 2016 : Inaka
How to easily play a sound in Android
Oct 25 2016 : Giaquinta Emiliano
We're publishing our work guidelines for the world to see.
Oct 13 2016 : Brujo Benavides
Using niffy to simplify working with NIFs on Erlang
Oct 05 2016 : Hernan Rivas Acosta
How to write clear function signatures, yet expressive, while following Swift 3 API design guidelines.
Sep 16 2016 : Pablo Villar
How to automatically trigger rails tests with a Jenkins job
Sep 14 2016 : Demian Sciessere
A description of our usual stack for building REST servers in Erlang
Sep 06 2016 : Brujo Benavides
Using Erlang's External Term Format
Aug 17 2016 : Hernan Rivas Acosta
Integrating our Android linter with Github's pull requests
Aug 04 2016 : Fernando Ramirez and Euen Lopez
Introducing how to implement passwordless login with phoenix framework
Jul 27 2016 : Thiago Borges
Our newest game to test your Beam Skills
Jul 14 2016 : Brujo Benavides
Three Open Source Projects, one App
Jun 28 2016 : Andrés Gerace
Running credo checks for elixir code on your github pull requests
Jun 16 2016 : Alejandro Mataloni
Thoughts on rebar3
Jun 08 2016 : Hernán Rivas Acosta
Thoughts on rebar3
Moving to rebar3
Recently we were tasked with moving a project from erlang.mk to rebar3 as it's quickly becoming the default way of managing OTP applications. The process was not particularly complicated; but since this project has its fair share of unusual stuff, it ended up revealing some of the strengths, weaknesses, and quirks of rebar 3, and also testing how flexible it actually is.
Now, the project in question consists of 5 different 'subprojects' (2 servers, 2 clients and one common library) that need to work in tandem. They were all hosted in the same git repository and had a lot of things in common; for example, Makefiles and configurations.
So, what did it take to move to rebar3? To be fair, not much, but that was thanks to some of rebar3's features. For example, one thing we were using was the ability to include a generic Makefile in the project's specific Makefiles (in the same way erlang.mk was included in the generic Makefile). That made everything easier to maintain, even though we panicked a bit when we first take a look at it and found no 'include' equivalent on rebar3. However, one of the most interesting features of rebar is that you can replace some configuration files with some
.script ones that will get executed and whose output will be used instead of the orginal file.
What this means in practice is that you can replace
rebar.config with a
rebar.config.script file that will get executed (using erlang's
file:script/2) to obtain the actual
rebar.config. And this is an awesome feature that not only made it possible to merge multiple config files, but also opened the door to some preprocessing. For example, we have all the dependencies in a separate file and during the preprocessing stage we just replace the single atom we have on the deps entry with those versions (basically a map with lists:keyfind).
Another obstacle was the fact that we needed local dependencies. Again, it's not obvious how to use them with rebar3, but after some digging we found out about the _checkouts directory and added a symlink to the local libraries that were common between projects. In case you are wondering, rebar will always use libraries in the _checkouts directory instead of the ones it downloads from the internet, which means that using a simlink works perfectly fine.
And since we are talking about dependencies, quite possibly the best feature in rebar3 is the
rebar.lock file, which contains the actual versions (the git sha) of every single library that was downloaded. And if such file exists, rebar will use that version instead of the one in the deps entry. This fixes the problem with git tags being unreliable (I remember Joe Armstrong complaining about that) and ensures that every commited version actually works. Useful feature, though I just wish they would've called it
Unfortunately, the tests (or at least some of them) refused to run. The reason being that
ct_slave:start was not using the correct cookies and we had no way of fixing that (really, I could write an essay on all the things that got in the way of fixing that issue). On the bright side, however,
ct_slave was able to start the node, but it was unable to connect to it afterwards. So to go around the issue, we first had to call
net_kernel:start/1 and then
ct_slave:start/2, ignoring the error it returned. And since it didn't matter, we just used a small timeout as it was going to fail anyway.
The problem with this approach is that you are also unable to stop the node using the
ct_slave module, so we decided to kill it manually
_ = spawn(Node, erlang, halt, ),
But it was not all just smooth sailing with the occasional
ct_slave hiccup. Rebar3 is not exactly keen on supporting Makefiles directly (unlike erlang.mk which supports rebar projects seamlessly), so at first we had trouble compiling jiffy (and others). Yes, this can be solved with overrides, but the solution is less than elegant and not particularly clean, so I'm glad we only had to do it once (on our common
rebar.config file). Despite all that, we can still say that this feature is a powerful one and can solve most (maybe all) of the issues you can find when some libraries refuse to cooperate with rebar3.
The other big issue we encountered was that our common tests required some flags to be passed, like the one to set the '.config' file. The problem is that rebar3 refuses to support this as a concious choice (though previous rebar versions did). One possible solution is calling common test with the flags
ERL_FLAGS="-config config/test.config" rebar3 ct, but that is not particularly nice and it is bound to be a problem when adding new people to the project. Besides, after a while, if your tests fail because you simply forget and call 'ct' directly, what was first a minor annoyance will turn into full on flow breaking, trust me. Our solution for that was (funny enough) adding Makefiles, so that we could just call
make ct instead and let the Makefile hide away the fact that we are force to pass the ERL_FLAGS manually every time.
And that's not all. Having a Makefile makes it really tempting to start adding other features that were removed. For example, in rebar2 you could call
rebar co instead of
rebar compile, a minor thing that may only matter because I was used to it. But since I was used to it, we just needed to add another target on the Makefile and voilà,
make co works "again". Similarly,
rebar3 clean does not really clean everything (it leaves the rebar.lock file and maybe more importantly, the _build directory) so adding a phony 'clean' target to the Makefile is a must.
Back to square one (kinda)
So we ended up settling with having to use GNU Make to wrap all rebar3 calls as it was easier, more consistent and actually allowed us to run our tests properly. Kind of ironic, as our motivation for migrating to rebar3 was to get away from those very Makefiles we ended up using.