Environment Variables or YAML config files

Question

The background:
Step 1 -> We have a box that runs unit and functional tests of an application by running it in test mode with a specific configuration.
Step 2 -> Upon success of Step 1, we run integration tests of an application by running it in test mode with different configuration set, in another box.
Step 3 -> Upon success of step 2, we run the performance tests of an application by running it in production mode, in the performance test box.
Step 4 -> Upon success of step 3, the build is considered stable and the UAT box is updated with that code base and the application is run in production mode, for the customer review and feedback. Step 5 -> With GO from the customer, the production box is updated with the code base.

Now, from above steps we observe that in steps 1 and 2, while the application runs in test mode it has different configuration. Similar is the case with steps 3,4 and 5.

In such situations, what is the recommended practice? We were having YAML configuration files, but personally I felt that maintaining numerous configuration files doesn't make sense. And so, I changed from the practise of
"Config file per environment"
to
"Config file per rails mode, externalizing the variables to linux environment".

Am I on right track? Doesn't my action, simplify things?

What are the pros and cons of these two approaches?

Answer 1

In my experience, environment variables are the configuration option of last-resort. They absolutely have their place, but applications should generally check some other more reliable and explicitly intentional configuration option first. I would highly recommend loading configuration from a YAML file and only use an environment variable as a fallback. Always assume that your environment variable is going to apply to everything system-wide and assume that it's going to accidentally get unset or set to the wrong value at some point. i.e., your app shouldn't commit seppuku because some configuration directory got set to / and it doesn't have permissions to it (or worse you wipe your drive because the app ran as root). Or more likely, something like your RAILS_ENV was set to test when it should've been production and no one realized and now users are writing data to the wrong place or to /dev/null on account of all the 500s.

Personally, I like to drop logger.warn messages anytime I fallback to an environment variable for a configuration value.

With your precise issue, honestly, I'd probably pass in the configuration value for which environment to start on the command line.