How to Handle Runtime Configuration of Symfony2 Using Consul Service Discovery

Question

Our team is presently exploring the idea of service discovery for a Symfony2 application using Consul. Being in the relative frontier, there's very little out there in the way of discussion. So far we've discovered:

• Runtime configuration has previously been shot down.
• A bundle exists to handle such a use case, but has also hasn't seen a lot of activity as of late.
• One of the contributors of said bundle suggested that External Parameters might be the answer to the problem.
• Sensio has created its own Consul SDK. However, there seems to be little in the way of documentation/official blog articles re: Symfony2 integration
• Consol provides watches which can be triggered on various changes

Current thoughts are to explore utilizing the Consul watchers to re-trigger a cache build along with external parameters. That said, there is some concern on the overhead of such an operation if services change semi-frequently.

Based on the above, and knowledge of Consul/Symfony internals, would that be an advisable approach? If not, why, and what alternatives are available?

Answer 1

In the company I work, we took a different route.

Instead of fighting against Symfony to accept runtime configuration (something it should, like Spring Data Consul, for example), we decided to make Consul update Symfony configuration, in a similar in concept, different in implementation than Frank did.

We installed Consul and Consul Template. We create a K/V entry pair that contains the entire parameters.yml file. Example:

Key: eblock/config/parameters.yml

parameters:
    router.request_context.host: dev.eblock.ca
    router.request_context.scheme: http
    router.request_context.base_url: /

Then a consul template configuration file was added at location /opt/consul-template/config/eblock.cfg:

template {
    source = "/opt/consul-template/templates/eblock-parameters.yml.ctmpl"
    destination = "/var/www/eblock/app/config/parameters.yml"
    command = "/opt/eblock/scripts/parameters_updated.sh"
}

The contents of ctmpl file are:

{{key "eblock/config/parameters.yml"}}

Finally, our parameters_updated.sh script does:

#!/bin/bash

readonly PROGNAME=$(basename "$0")
readonly LOCKFILE_DIR=/tmp
readonly LOCK_FD=201

lock() {
    local prefix=$1
    local fd=${2:-$LOCK_FD}
    local lock_file=$LOCKFILE_DIR/$prefix.lock

    # create lock file
    eval "exec $fd>$lock_file"

    # acquire the lock
    flock -n $fd \
        && return 0 \
            || return 1
}

lock $PROGNAME || exit 0

export HOME=/root
logger "Starting composer install" && \
/usr/local/bin/composer install -d=/var/www/eblock/ --no-interaction && \
logger "Running composer dump-autoload" && \
/usr/local/bin/composer dump-autoload -d=/var/www/eblock/--optimize && \
logger "Running app/console c:c/c:w" && \
/usr/bin/php /var/www/eblock/app/console c:c -e=prod --no-warmup && \
/usr/bin/php /var/www/eblock/app/console c:w -e=prod && \
logger "Running doctrine commands" && \
/usr/bin/php /var/www/eblock/app/console doctrine:database:create --env=prod --if-not-exists && \
/usr/bin/php /var/www/eblock/app/console doctrine:migrations:migrate -n --env=prod && \
logger "Restarting php-fpm" && \
/bin/systemctl restart php-fpm &

Knowing that both consul and consul-template services are up, as soon as your value changes in the specified key for consul template, it'll dump the file into configured destination and run the command for parameters updated.

It works like a charm. =)

Answer 2

A simple KV watcher that puts the value into parameters.yml, triggers a cache:clear is the simplest option in my opinion and also provides the benefit of compilation so that it doesn't have to go to Consul each time to check if values are updated. Like you said, some overhead but seems to be ok if you don't change your parameters every 5 minutes.

We're exploring that option now but if you made any progress on this, an update would be appreciated.

[Update 2016-02-23] We've implemented the idea I mentioned above and it works as expected: well. Mind you, we change our parameters only on deploy of a new version (because we also use service discovery by Consul so no need to update service lists in parameters). We mostly did it because it saves us the boring job of changing parameters on several servers. As usual: this might not work for you but I think you would be safe if, like I said before, you don't change your parameters every 5 mins :)