.Net Core appsettings.json best practices - override dev settings (or vice versa)?

Question

Looking for a sensible approach in regards to structuring appsettings.json files in .Net Core.

Should the base 'appsettings.json' file be configured for running in a dev environment, and then environment based overrides such as appsettings.production.json overwrite specific keys for production?

Or should appsettings.json only contain config that is shared across ALL environments, and then a specific appsettings.development/staging.json file used to explicitly set keys for those environments?

My concern is - say an app is deployed to a live server, but a key stored in an environment variable (e.g. to override connection string) is missing or spelt wrong etc. In this case the app would fall back to the base appsettings.json connection string which would be the incorrect DB for the live environment. A scenario like this sounds pretty disasterous, particularly as this could easily go unnoticed?

So really the question boils down to - should the contents of the base appsettings.json file be 'dev' values as default (e.g. dev DBs, sandbox APIs) that are overridden with production data, or vice versa?

Answer 1

I think this has the boring answer; it depends. But my favorite approach is this:

appsetting.json (base settings)
appsettings.development.json (dev with no secrets)
appsettings.production.json (production with no secrets)

Appsettings where the values that are secret only exist in the base setting while the other are written in the respective appsettings.[env].json. So example database connection key only exists in base setting with the local database. It´s the environments job to replace it

Example for database connection and logging

appsettings.json

{
"ConnectionStrings": {
  “dbConnection: “data source=localhost” <—— only here
},
“environment”: “local”,
"Logging": {
  "LogLevel": {
    "Default": “Verbose”
  }
},
}

appsettings.development.json

{
“environment”: “development”,
"Logging": {
  "LogLevel": {
    "Default": “Warning”
  }
},
}

appsettings.production.json

{
“environment”: “production”,
"Logging": {
  "LogLevel": {
    "Default": “Information”
  }
},
}

My concern is - say an app is deployed to a live server, but a key stored in an environment variable (e.g. to override connection string) is missing or spelt wrong etc. In this case the app would fall back to the base appsettings.json connection string which would be the incorrect DB for the live environment. A scenario like this sounds pretty disasterous, particularly as this could easily go unnoticed?

You can always do this. But some sanity tests should do it. Do a simple health check where you ping the database if your infrastructure / deploy pipeline allows it.

Answer 2

UPDATE: I wanted to append to this that I no longer use Azure App Configuration, only the KeyVault. I was disappointed that the pricing allows for 1 free AppConfig per subscription, and then appears to charge more than 1.20$CDN a day for each beyond the first. Which for me meant an extra 36$ a month for something that will only be used once a day (assuming I let the web app wind down at night). The price point just doesn't make any sense whatsoever. I'd just as soon use multiple KeyVault, as their cost looks to be purely based on usage, and at a pittance per thousands of transactions. I don't understand what the intended use case for AppConfig is to justify its price point.

I've gotten to the habit storing my configuration in Azure under an AzureAppConfig and/or an AzureKeyVault. It gives me a central location to manage my dev, staging/test, production settings to and doesn't require me to complicate my deployment with manipulating appsettings files, or storing in them in some sort of deployment repo. It's really only ever read from azure when the application starts (I didn't need to be able to refresh them while my app was running). That being said, it made it a little interesting for the local dev story because I personally wanted the order of operations to be appsettings.json, appsettings.{environment}.json, AzureAppConfig, KeyVault, then finally secrets.json. That way, no matter what, I could override a setting from azure with my local secrets file (even if the setting I was overriding wasn't technically a secret).

I basically ended up writing some custom code in program.cs to handle loading the config sources from Azure, then finish with looking for the JsonConfigurationSource that had a Path of "secrets.json", then bump that to be the last item in my IConfigurationBuilder.Sources.

For me, my files get used as follows

• appsettings.json - Common settings that would need to be set for any environment, and will likely never change depending on the environment. appsettings.{environment}.json - Mostly just en empty JSON files that basically just name the AzureAppConfig & AzuerKeyVault resource names to connect to
• AzureAppConfig - Basically for anything that would be different between Production, Staging/Testing, or local Development, AND is not a sensitive piece of information. API endpoints addresses, IP addresses, various URLs, error logging information, that sort of thing.
• AzureKeyVault - Anything sensitive. Usernames, passwords, keys for external APIs (auth, license keys, connection strings, etc).

The thing is, even if you put a setting in appsettings.json, that doesn't mean you can't override it with appsettings.{enviroment}.json or somewhere else. I've frequently put a settings in the root setting file with a value of NULL, just to remind me that it's a setting used in the app. So a better question might be, do you want to be able to run your app (as in no errors) with nothing but the base appsettings.json and secrets.json? Or would the contents from appsettings.{enviroment}.json always be needed to successfully spin up?

The other thing to look at based off of your question is validation for your configuration. Later versions of Microsoft.Extensions.Options offer various ways to validate your options so that you can try and catch instances where something was left empty/undefined. I typically decorate my POCO Options classes with data annotation attributes and then use ValidateDataAnnotations() to verify they get setup correctly.

For example

services.AddOptions<MailOptions>().Bind(configuration.GetSection("MailSettings")).ValidateDataAnnotations();

It's worth noting this validation runs only when you try to request something like the MailOptions I use as an example above, from DI (so not at startup) For this reason, I also created your my own IStartupFilter to preemptively request one or more of my Options classes from the service provider when the app starts up, in order to force that same Validation to run before the app even starts accepting requests.

public class EagerOptionsValidationStartupFilter : IStartupFilter
{
    public readonly ICollection<Type> EagerValidateTypes = new List<Type>();
    private readonly IServiceProvider serviceProvider;

    public EagerOptionsValidationStartupFilter(IServiceProvider serviceProvider)
    {
        this.serviceProvider = serviceProvider;
    }

    public Action<IApplicationBuilder> Configure(Action<IApplicationBuilder> next)
    {
        foreach (var eagerType in EagerValidateTypes)
        {
            dynamic test = serviceProvider.GetService(typeof(IOptions<>).MakeGenericType(eagerType));
            _ = test.Value;
        }

        return next;
    }
}

startup.cs

public void ConfigureServices(IServiceCollection services)
{

    services.AddTransient<IStartupFilter>(x =>
        new EagerOptionsValidationStartupFilter(x)
        {
            EagerValidateTypes = {
                typeof(MailOptions),
                typeof(OtherOptions),
                typeof(MoreImportantOptions)
            }
        });
}

Answer 3

There are several ways how to shape your settings (thats the beauty of .NET Core). The way how I normally do it, is the following:

appsetting.json (template)
appsettings.development.json (dev with no secrets)

I actually do not put any settings in appsettings.json. I use it as a template map of the settings that must (can) be set during deployment.

// appsettings.json

{
  "ConnectionStrings": {
    "dbConnection": "************************"
  },
  "environment": "************************",
  "Logging": {
    "LogLevel": {
      "Default": "************************"
    }
  },
}

That way if I miss any setting, it will become apparent later on that it was forgotten. I dont have to worry about accidentally using settings that "slipped" through the hierarchy. Therefore if you look at your other jsons, they are complete and there are no hidden settings.

// appsettings.Development.json

{
  "ConnectionStrings": {
    "dbConnection": "data source=localhost"
  },
  "environment": "local",
  "Logging": {
     "LogLevel": {
      "Default": "Verbose"
    }
  }
}

Sharing settings seems to be a good idea for small applications. It actually gives more problems if your application gets more complex.

Answer 4

A few principals come into play here:

First, any broken/missing items should error out vs. silently work in some subset of cases. This is valuable because it uncovers issues early on in development. Only put values in the base file that are constant across environments or will reveal missing values when not overridden e.g. under test. This enables you to write negative test cases to a known value, which can help uncover errors in more complex configurations.

Second, any extra deployed content is added risk so deploy nothing extra. Put the appropriate values for each environment into the environment-specific file and nothing else. These values should override the base file, enabling you to deploy and run without manual intervention. Use the out-of-box configuration loader to load (only) the correct file for the current environment.

Third, it can be helpful to have a way to override values in the environment without re-deploying any files. The value here depends on your environment and situation e.g. security event. As a result, environment variables should override the preceding two sources.

If you're using a centralized configuration source, can you allow a deployed file to override it? This is a dev-sec-ops/policy question. Your answer will determine where centralized config should fall in the list. The farther down you put it the more likely your developers will need to run an instance locally.

There may be other considerations or additional layers that make sense in your project. The important thing is to have a "why" for the choices you make, and to be able to explain and justify them logically in your context.