inaka

Latest blog entries

/
The Art of Writing a Blogpost

The Art of Writing a Blogpost

Apr 11 2017 : Matias Vera

/
SpellingCI: No more spelling mistakes in your markdown flies!

Feb 14 2017 : Felipe Ripoll

/
Fast reverse geocoding with offline-geocoder

Do you need a blazing fast reverse geocoder? Enter offline-geocoder!

Jan 18 2017 : Roberto Romero

/
Using Jayme to connect to the new MongooseIM REST services

MongooseIM has RESTful services!! Here I show how you can use them in an iOS application.

Dec 13 2016 : Sergio Abraham

/
20 Questions, or Maybe a Few More

20 Questions, or Maybe a Few More

Nov 16 2016 : Stephanie Goldner

/
The Power of Meeting People

Because conferences and meetups are not just about the technical stuff.

Nov 01 2016 : Pablo Villar

/
Finding the right partner for your app build

Sharing some light on how it is to partner with us.

Oct 27 2016 : Inaka

/
Just Play my Sound

How to easily play a sound in Android

Oct 25 2016 : Giaquinta Emiliano

/
Opening our Guidelines to the World

We're publishing our work guidelines for the world to see.

Oct 13 2016 : Brujo Benavides

/
Using NIFs: the easy way

Using niffy to simplify working with NIFs on Erlang

Oct 05 2016 : Hernan Rivas Acosta

/
Function Naming In Swift 3

How to write clear function signatures, yet expressive, while following Swift 3 API design guidelines.

Sep 16 2016 : Pablo Villar

/
Jenkins automated tests for Rails

How to automatically trigger rails tests with a Jenkins job

Sep 14 2016 : Demian Sciessere

/
Erlang REST Server Stack

A description of our usual stack for building REST servers in Erlang

Sep 06 2016 : Brujo Benavides

/
Replacing JSON when talking to Erlang

Using Erlang's External Term Format

Aug 17 2016 : Hernan Rivas Acosta

/
Gadget + Lewis = Android Lint CI

Integrating our Android linter with Github's pull requests

Aug 04 2016 : Fernando Ramirez and Euen Lopez

/
Passwordless login with phoenix

Introducing how to implement passwordless login with phoenix framework

Jul 27 2016 : Thiago Borges

/
Beam Olympics

Our newest game to test your Beam Skills

Jul 14 2016 : Brujo Benavides

/
Otec

Three Open Source Projects, one App

Jun 28 2016 : Andrés Gerace

/
CredoCI

Running credo checks for elixir code on your github pull requests

Jun 16 2016 : Alejandro Mataloni

/
Thoughts on rebar3

Thoughts on rebar3

Jun 08 2016 : Hernán Rivas Acosta

/
See all Inaka's blog posts >>

/
Erlang Operational Configuration Simplicity

Iñaki Garay wrote this on July 14, 2015 under configuration, erlang, management, ops .

Strap yourselves in, we're off to DevOps land. Hic sunt dracones.

Abstract

We currently use erlang.mk and relx to generate our Erlang application releases. In this article we detail how we modified our build and deploy processes to allow splitting the application configuration by using the include mechanism in OTP application configuration files.

Motivation

Sometimes I feel like the phrase that best describes my life is "Once you know what you are doing, it's easy." Getting my Erlang-based systems up on the server in a secure, timely, flexible and robust manner certainly qualifies for that.

Just one aspect of this is configuration management.

Once the basics of building the application and getting it on the server are taken care of, we would like to be able to:

  • tune the configuration for different environments such as staging or production,
  • hide sensitive information such as secret tokens and database access passwords,
  • dynamically set variables such as ports, urls, etc.

Erlang applications are configured via the application environment variables.

When packaged into a release and running in embedded mode, this file is the sys.config file.

Usually, our configuration variables can be divided into two sets: * those which do not vary across environments * those which do

How we did it before

Our usual approach to these problems was to have a sys.config.template file which included template variables. These variables would be instantiated with mustache during the build or deploy process.

Some preliminary setup

To establish some common ground, let us suppose we have a minimal OTP application called example, which does nothing except start a supervisor. Since we are using erlang.mk, the directory structure is:

$ find .
./example/
./example/Makefile
./example/erlang.mk
./example/relx.config
./example/ebin/
./example/src/
./example/src/example.app.src
./example/src/example.erl
./example/src/example_sup.erl

./example/Makefile:

PROJECT=example

SHELL_OPTS = -s ${PROJECT}

include erlang.mk

./example/relx.config:

{ release
, {example, "0.0.1"}
, [ kernel
  , stdlib
  , example
  ]
}.
{extended_start_script, true}.

Our application source files are:

./example/src/example.app.src:

{ application
, example
, [ {description, "An example application."}
  , {vsn, "0.0.1"}
  , {applications,
    [ kernel
    , stdlib
    ]}
  , {modules, []}
  , {mod, {example, []}}
  , {registered, []}
  ]
}.

./example/src/example.erl:

-module(example).

-behaviour(application).

-export(
  [ start/0
  , stop/0
  , start/2
  , stop/1
  ]).

start() ->
  application:ensure_all_started(example).

stop() ->
  application:stop(example).

start(_StartType, _Args) ->
  example_sup:start_link().

stop(_State) ->
  ok.

./example/src/example_sup.erl:

-module(example_sup).

-behaviour(supervisor).

-export([init/1, start_link/0]).

start_link() ->
  supervisor:start_link({local, ?MODULE}, ?MODULE, []).

init([]) ->
  io:format("Starting example_sup~n"),
  {ok, { {one_for_one, 10, 60}, []} }.

Now that we have this in place we can compile the application and generate a release by executing make, and start a shell running the application with:

% make shell
 GEN    shell
Erlang/OTP 17 [erts-6.4] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]

Eshell V6.4  (abort with ^G)
Starting example_sup
1> q().
ok

Configuring our application

Let's add some configuration:

  1. create config/sys.config,
  2. tell our Makefile that's the config file and
  3. tell relx.config to use that for the release configuration
  4. tell our supervisor to print out its application environment variables

./example/config/sys.config:

[
  { example
  , [ {env_var_1, value_1}
    , {env_var_2, value_2}
    ]
  }
].

Changes to ./example/Makefile:

% git diff Makefile
diff --git a/Makefile b/Makefile
index a548a52..bb03489 100644
--- a/Makefile
+++ b/Makefile
@@ -1,3 +1,7 @@
 PROJECT = example

+CONFIG ?= config/sys.config
+
+SHELL_OPTS = -s ${PROJECT} -config ${CONFIG}
+
 include erlang.mk

Changes to ./example/relx.config:

% git diff relx.config
diff --git a/relx.config b/relx.config
index aeeee8c..d387fab 100644
--- a/relx.config
+++ b/relx.config
@@ -5,5 +5,6 @@
   , example
   ]
 }.
+{sys_config, "config/sys.config"}.
 {extended_start_script, true}.

We'll tell our supervisor to print out the application environment when starting, so we can see it:

% git diff src/example_sup.erl
diff --git a/src/example_sup.erl b/src/example_sup.erl
index 16079ca..ae428b4 100644
--- a/src/example_sup.erl
+++ b/src/example_sup.erl
@@ -9,5 +9,7 @@ start_link() ->

 init([]) ->
   io:format("Starting example_sup~n"),
+  AppEnv = application:get_all_env(example),
+  io:format("AppEnv: ~p~n", [AppEnv]),
   {ok, { {one_for_one, 10, 60}, []} }.

Now when we start the application we see:

% make shell
 GEN    shell
Erlang/OTP 17 [erts-6.4] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]

Starting example_sup
AppEnv: [{env_var_1,value_1},{env_var_2,value_2},{included_applications,[]}]
Eshell V6.4  (abort with ^G)
1> q().
ok

Great, so now we have a minimal configurable application.

The Trick

The other day while browsing the official Erlang documentation, we came across this interesting tidbit of information:

Therefore, in Erlang 5.4/OTP R10B, the syntax of sys.config was extended to allow pointing out other .config files:

[{Application, [{Par, Val}]} | File].

File = string() is the name of another .config file. The extension .config may be omitted. It is recommended to use absolute paths. A relative path is relative the current working directory of the emulator.

When traversing the contents of sys.config and a filename is encountered, its contents are read and merged with the result so far. When an application configuration tuple {Application, Env} is found, it is merged with the result so far. Merging means that new parameters are added and existing parameter values overwritten.

Let's use the configuration include trick to create a mechanism through which our deploy process can inject settings into a release.

So we'll create ./example/config/extra.config, it looks pretty similar to config/sys.config:

% cat config/extra.config
[
  { example
  , [ {env_var_2, new_value_2}
    , {env_var_3, value_3}
    ]
  }
].

As you can see, we added a new setting, env_var_3, which might be some secret setting we don't want people outside of ops to be aware of, and we shadowed env_var_2 with a new value, which could be the case when we have some setting that differs across development, staging or production environments, e.g. some IP address or a port.

Now we need to tell config/sys.config to include this file:

% git diff config/sys.config
diff --git a/config/sys.config b/config/sys.config
index 011471c..7fc1087 100644
--- a/config/sys.config
+++ b/config/sys.config
@@ -4,4 +4,5 @@
     , {env_var_2, value_2}
     ]
   }
+, "config/extra"
 ].

We also need to tell relx to copy this extra file over to the generated release, so that devs working on their local can test said release before committing with values appropriate to their local environment. We achieve this by means of an overlay:

% git diff relx.config
diff --git a/relx.config b/relx.config
index d387fab..b63b249 100644
--- a/relx.config
+++ b/relx.config
@@ -6,5 +6,11 @@
   ]
 }.
 {sys_config, "config/sys.config"}.
+{ overlay
+, [ { copy
+    , "config/extra.config"
+    , "config/extra.config"
+    }
+  ]
+}.
 {extended_start_script, true}.

Two final details, before we move on.

1) We don't want the config/extra.config file to be included in the git repo, because it shouldn't include secret or dynamic values and because it will be modified by local devs and the build/deploy process. So we'll abstain from staging it and add it to the .gitignore.

% git diff .gitignore
diff --git a/.gitignore b/.gitignore
index 8e46d5a..4758a57 100644
--- a/.gitignore
+++ b/.gitignore
@@ -8,3 +8,4 @@ ebin
 rel/example_project
 .concrete/DEV_MODE
 .rebar
+config/extra.config

2) But now someone cloning our repo for the first time and running make won't end up with a runnable application!

So we'll have 'make' check to see whether the config/extra.config file exists and, if not, create and initialize it (did you know that wasn't a word until the 1950s?).

% git diff Makefile
diff --git a/Makefile b/Makefile
index bb03489..0e8a034 100644
--- a/Makefile
+++ b/Makefile
@@ -4,4 +4,7 @@ CONFIG ?= config/sys.config

 SHELL_OPTS = -s ${PROJECT} -config ${CONFIG}

+EXTRA_CONFIG ?= config/extra.config
+all::
+       @if ! [ -a ${EXTRA_CONFIG} ]; then echo "[]." > ${EXTRA_CONFIG}; fi
+
 include erlang.mk

Now let's see our end result:

% make shell
 GEN    shell
Erlang/OTP 17 [erts-6.4] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]

Starting example_sup
AppEnv: [{env_var_3,value_3},
         {env_var_1,value_1},
         {env_var_2,new_value_2},
         {included_applications,[]}]
Eshell V6.4  (abort with ^G)
1> q().
ok

Awesome!

But does it work with the release as well?

So far for convenience we've been using erlang.mk's shell target to test, as developers would on their local machine. What about the release?

% _rel/example/bin/example console
Exec: /home/igaray/Projects/personal/example/_rel/example/erts-6.4/bin/erlexec -boot /home/igaray/Projects/personal/example/_rel/example/releases/0.0.1/example -env ERL_LIBS /home/igaray/Projects/personal/example/_rel/example/releases/0.0.1/lib -config /home/igaray/Projects/personal/example/_rel/example/releases/0.0.1/sys.config -args_file /home/igaray/Projects/personal/example/_rel/example/releases/0.0.1/vm.args -- console
Root: /home/igaray/Projects/personal/example/_rel/example
/home/igaray/Projects/personal/example/_rel/example
Erlang/OTP 17 [erts-6.4] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]

Starting example_sup
AppEnv: [{env_var_3,value_3},
         {env_var_1,value_1},
         {env_var_2,new_value_2},
         {included_applications,[]}]
Eshell V6.4  (abort with ^G)
(example@127.0.0.1)1> q().
ok

Yes it does :)

Tests

So all this is nice, but besides local and staging/production, there is another common environment we haven't mentioned, which is the test environment.

erlang.mk provides a test configuration mechanism via the CT_OPTS variable, which contains the flags passed to ct_run. Another erlang.mk variable relevant to testing is the TEST_ERLC_OPTS variable, which contains flags passed to erlc when compiling for test runs.

An example CT_OPTS and TEST_ERLC_OPTS might be:

CONFIG=config/sys.config
TEST_CONFIG= config/test.config

CT_OPTS = -cover test/example.spec -erl_args -config ${CONFIG} -config ${TEST_CONFIG}
TEST_ERLC_OPTS += +'{parse_transform, lager_transform}' +debug_info

Notice that in -erl_args (which is a ct_run argument which specifies arguments to be passed to erl when running the common test tests) we include TWO -config arguments. Erlang will read the two config files and merge them in a similar manner to the include mechanism.

We first tell the BEAM to load the regular config file, and then have another, separate, config file just to override the environment variables specific to the testing environment. This ${TEST_CONFIG} should also be added to the .gitignorefile, excluded from the repo and created if necessary by the Makefile`.

Caveat Emptor

Bear in mind that the include mechanism will only work if the configuration file is named sys.config and only sys.config.

We placed the include at the very end, but if you have several includes, make sure to test and see what environment you end up with, lest some value be inadvertently shadowed.