Using Consul with Azure App Services

Question

I'd like to have all of my configuration settings all in one place for all of my azure web app services, as well as resources outside of Azure. Consul's key value store seems like it could be a good fit (I'm happy to hear other suggestions if something else fits better). From my admittedly limited understanding of Consul, each node requires an agent to be running in order to access to key value store.

Is this correct? If so, how can I do this, would it be via a continuous webjob in Azure? If not, how can I access the KV store without an agent?

Answer 1

it looks like we will not be able to use consul with Azure App Service (aka Web Apps) at all.

Here is what I've tried.

1. Naive approach - consul as WebJob

Due to networking restrictions attempt to connect to ANY localhost ports what did not spawn with processes that belong to App Service (Web App) itself will end up with the following exception.

An attempt was made to access a socket in a way forbidden by its access permissions 127.0.0.1:8500.

Reference from documentation:

https://github.com/projectkudu/kudu/wiki/Azure-Web-App-sandbox#networking-restrictionsconsiderations

The only way an application can be accessed via the internet is through the already-exposed HTTP (80) and HTTPS (443) TCP ports; applications may not listen on other ports for packets arriving from the internet. However, applications may create a socket which can listen for connections from within the sandbox. For example, two processes within the same app may communicate with one another via TCP sockets; connection attempts incoming from outside the sandbox, albeit they be on the same machine, will fail. See the next topic for additional detail.

Here is an interesting piece:

Connection attempts to local addresses (e.g. localhost, 127.0.0.1) and the machine's own IP will fail, except if another process in the same sandbox has created a listening socket on the destination port.

2. Consul spawned from App Service itself

I've copied consul to Web App (as build output) and added the following lines to application startup code:

var consul = Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "bin/consul/consul.exe");

Process.Start(consul, "agent --data-dir=../../data");
Process.Start(consul, "join my-cluster-dns.name");

... and it joined the cluster and I even was able to connect to consul via 127.0.0.1:8500 from the App Service (Web App) itself.

However, it is still useless setup as Consul agent MUST be reachable from server, so all I was able to see from cluster standpoint is a dead node with failing "serf" health-check. Again, according to the documentation there is no work around this: "The only way an application can be accessed via the internet is through the already-exposed HTTP (80) and HTTPS (443) TCP ports".

https://www.consul.io/docs/agent/basics.html

Not all Consul agents in a cluster have to use the same port, but this address MUST be reachable by all other nodes.

Summary

All-in-all, probably there is no way to properly host/use Consul with Azure App Services.

Answer 2

You don't need Consul Agent to retrieve configuration for your application.

You can use library Winton.Extensions.Configuration.Consul. It introduces Configuration Provider (docs) which can be integrated within your application.

Here sample configuration (full sample project available here)

internal sealed class Program
{
    public static IHostBuilder CreateHostBuilder(string[] args)
    {
        return Host
            .CreateDefaultBuilder(args)
            .ConfigureWebHostDefaults(builder => builder.UseStartup<Startup>())
            .ConfigureAppConfiguration(
                builder =>
                {
                    builder
                        .AddConsul(
                            "appsettings.json",
                            options =>
                            {
                                options.ConsulConfigurationOptions =
                                    cco => { cco.Address = new Uri("http://consul:8500"); };
                                options.Optional = true;
                                options.PollWaitTime = TimeSpan.FromSeconds(5);
                                options.ReloadOnChange = true;
                            })
                        .AddEnvironmentVariables();
                });
    }

    public static void Main(string[] args)
    {
        CreateHostBuilder(args).Build().Run();
    }
}

Your app configuration will be updated periodically.