One of the things that I know of certain companies that require high security do, is to store the configuration settings in an encrypted file (stored in a logical place, away from the code somewhere, i.e. using your approach number 3.). The program will then prompt for the proper passphrase to decrypt this file when it is started.

This means that the only way for an attacker to read the secret configuration information, is to either be able to read your keystrokes while you restart the server, or read (and basically disassemble) part of the RAM that your running program is using.

I’ve also heard about servers deployed this way that ran fine for years, and then when they finally had to restart, they asked for a password that nobody remembered anymore…

Docker Online Meetup: What’s new in Docker 1.13

While this isn’t Elixir specific, we have a two prong approach for handling our secret config for our highly available and global Asp.Net systems, utilizing both configuration in Environment Variables and in a configuration environment.

Originally we were doing all configuration via environment variables. This became tedious and difficult to properly maintain as we scaled the number of configuration settings we had to maintain as well as scaling our systems to different geographic data centers, (not to mention staging systems that can be swapped between prod and staging). A lot of configuration stayed the same between each region but some things didn’t and keeping track of that was difficult.

So how we handled it was by loading configuration twice. First past is reading the deployed configuration file (which really only has defaults) then overriding them with environment variables. We then pull the connection string out and then repeat the process with the database (so config file overwritten by database overwritten by environment variables).

Each server can specify in the environment variables a configuration profile (for example it’s region), which adds an extra layer of optional overrides. This allows us to do things like keep our east coast servers queuing up data in our east coast queues and keep latencies as a whole down.

This allows us to keep the environment variable maintenance down (only requiring database connection string and prod/staging specific settings) while making it very easy to query to see what settings will be applicable to a specific server. Not only that, since our databases are backed up in a way that allow us to restore to any point in time if something gets fubar we have a way to restore them back.

As part of another project I’ve been looking at ways to securely store secrets, and one of the things that stood out when trying Vault was the ways that apps could be used with it.

So for the case of getting credentials for an app, using the Cubbyhole secret backend seems like a good way to mitigate unwanted access to tokens.

The basic idea is;

• Enable Cubbyhole secret backend. (Vault enables this by default)
• Create temporary token with use count of 2. (The use count specifies number of times the specific token can be used)
• Create permanent token.
• Store permanent token in temporary token’s cubbyhole.
• Pass temporary token to container application using environment variable.
• Application reads permanent token from cubbyhole using temporary token.

Even if temporary token is read by malicious user later, there is no use for it since the use count for temporary token would have expired. Out of the specified initial use count of 2 for temporary token, first count is used when writing the permanent token and the second count is used when reading the permanent token.

I’m not sure how easy it would be to integrate, I’m thinking some sort of upstart script that gets the temporary token then passes it to the elixir app on startup, then internally the app does the rest - doing something like Preparing for Production - #2 by sasajuric

Some more info on response wrapping using Cubbyholes:

This post has an example of that, as well as the DB secret backend for authenticating DB users:

At the end of Paul Schoenfelder’s (@bitwalker) excellent deployment talk on Distillery, someone asks this very question about managing secrets. His response is brief and I won’t paraphrase!