How to config Meteor on AWS/EBS using METEOR_SETTINGS environment variable

Question

Trying to set up a Meteor on an AWS/EBS (Amazon Web Services, Elastic Beanstalk) environment.

A Meteor dev-run can be passed a command line flag: --settings settings.json where settings.json is a file containing server/client key/value configs (as properly-formatted JSON).

Instead of passing the config file in command line, Meteor's deployment uses a METEOR_SETTINGS environment variable. If provided it is expected to contain a json document such as contents of settings.json, for example:

$ METEOR_SETTINGS=$(cat settings.json)
$ echo $METEOR_SETTINGS
{ "public": { "s3path": "https://d2v4p3rms9rvi3.cloudfront.net" } }

The problem is that when I set the value of METEOR_SETTINGS to this value in the EBS console:

AWS/EBS discards the quotes, escapes the slashes (as seen in screenshot), and sends Meteor:

{public:{s3path:https:\/\/d2v4p3rms9rvi3.cloudfront.net}}

As indicated by the node start up error:

-------------------------------------
/var/log/nodejs/nodejs.log
-------------------------------------
npm WARN deprecated backwards-incompatible changes made to `npm run-script` and
npm WARN deprecated semver behavior.

> [email protected] start /var/app/current
> node main.js


/var/app/current/programs/server/boot.js:283
}).run();
   ^
Error: METEOR_SETTINGS are not valid JSON: {public:{s3path:https:\/\/d2v4p3rms9rvi3.cloudfront.net}}
    at packages/meteor/packages/meteor.js:21:1
    at Package (packages/meteor/packages/meteor.js:42:1)
    at /var/app/current/programs/server/packages/meteor.js:1277:4
    at /var/app/current/programs/server/packages/meteor.js:1286:3
    at /var/app/current/programs/server/boot.js:242:10
    at Array.forEach (native)
    at Function._.each._.forEach (/var/app/current/node_modules/underscore/underscore.js:79:11)
    at /var/app/current/programs/server/boot.js:137:5

Bumping against this problem I tried all sorts of variations for the JSON object in the value field: escaping the quotes, enclosing the entire json part with single quotes, replacing double-quotes with single-quotes, and other attempts - neither solved it.

Question is:

How can METEOR_SETTINGS be set so that Meteor rcv & parse it correctly?

Note: one of the requirements is that the same build deploys to dev, staging and production environments. Configs need to be set separately for each environment thus if there's another way to inject the settings into the EBS environment w/o modifying the build that will also solve it.

Answer 1

After discussing this issue with AWS support I realized that AWS/EBS does not support storing JSON in environment variables. This is because the environment variables are stored as key/value strings in unencoded JSON (apparently, in CloudFormation). The bottom line here a bit disappointing:

METEOR_SETTINGS cannot be used in the AWS/EBS console

This is indeed unfortunate, however there are a couple of workarounds.

1st Workaround

Move the json configs into an s3 bucket and place the following content in a .ebextensions/app.config file:

container_commands: 
  01_setvariable: 
    command: "aws s3 cp s3://<bucket-name>/nodejs.conf /tmp/deployment/config/#etc#init#nodejs.conf 

This will entirely override /etc/init/nodejs.conf with content retrieved from your s3 bucket. Naturally there's an opportunity to set/override individual settings using fine-tuned/fancy bash scripting.

I ended up not choosing this method, because it involves another entity (an S3 bucket) and the dev iteration requires a new version deploy, which isn't terribly fast.

2nd Workaround

Note: this is a simple code-hack I came up with. It seems to put all this mess behind while not requiring much effort.

My original need was to propagate AWS/EBS env vars to the client, so I decided to bypass the METEOR_SETTINGS variable and populate Meteor.settings.public directly with env vars from node's process.env space. The whitelisting is managed by a simple list. Add a server/lib/config.js file with:

Meteor.startup(function () {
    // public settings that need to be exposed to the client can be added here
    var publicEnvs = {
        S3_PATH: 's3path'
    };
    var modified;
    _.each(publicEnvs, (value, key) => {
        let envValue = process.env[key];
        if (envValue) {
            Meteor.settings.public[value] = envValue;
            modified = true;
        }
    });
    if (modified) {
        __meteor_runtime_config__.PUBLIC_SETTINGS = Meteor.settings.public;
    }
});

Hurray, your client can access the env vars of your choice!

For example with this change, an S3_PATH environment variable defined in the EBS console can be accessed as Meteor.settings.public.s3path on the client. Quite simple, and without many moving parts :)

Answer 2

Tested another workaround.

after meteor build --directory edit main.js as following

process.argv.splice(2, 0, 'program.json');
var settingfilename = './settings.json';
if (process.env.METEOR_SETTING_FILE)
  settingfilename = process.env.METEOR_SETTING_FILE;
var settings = require(settingfilename);
if (settings) {
  try {
    process.env.METEOR_SETTINGS = JSON.stringify(settings);
  } catch (e) {
    console.error(e);
  }
}
process.chdir(require('path').join(__dirname, 'programs', 'server'));
require('./programs/server/boot.js');

and copy settings.json into bundle/ and eb init and eb deploy.

you can set the other settings file with adding METEOR_SETTING_FILE at Environment Properties in Configuration tab from EB Console.

editing file is needed after every build.

---

added the patch file to use in the build script like ed - ../build/bundle/main.js < main.js.patch

main.js.patch

8a
var settingfilename = './settings.json';
if (process.env.METEOR_SETTING_FILE)
  settingfilename = process.env.METEOR_SETTING_FILE;
var settings = require(settingfilename);
if (settings) {
  try {
    process.env.METEOR_SETTINGS = JSON.stringify(settings);
  } catch (e) {
    console.error(e);
  }
}
// console.log (JSON.stringify(process.env));
.
w