Startup fails after deployment when using connection strings configSource attribute along with Azure App Service environment connection strings

Question

I'm trying to follow Best practices for private config data and connection strings in configuration in ASP.NET and Azure and Best practices for deploying passwords and other sensitive data to ASP.NET and Azure App Service.

Steps I took:

I have an ASP.NET 4.6 Web App with a regular web.config file. I created two files for my secrets: Web.Secrets.AppSettings.config and Web.Secrets.ConnectionString.config, put corresponding secrets into them according to the tutorial and modified the root web.config so it looks like that:

<configuration>
  <appSettings file="Web.Secrets.AppSettings.config">
  </appSettings>
  <connectionStrings configSource="Web.Secrets.ConnectionStrings.config">
  </connectionStrings>
  <!--...-->
</configuration>

Then I created a new Azure App Service in Azure Portal, opened it's Application Settings and added the secrets into the corresponding sections (App Settings and Connection Strings).

After that I deployed my Web App to this Azure App Service and right after that at the startup got yellow screen of death with the following message:

Server Error in '/' Application.

Configuration Error

Description: An error occurred during the processing of a configuration file required to service this request. Please review the specific error details below and modify your configuration file appropriately.

Parser Error Message: Unable to open configSource file 'Web.Secrets.ConnectionStrings.config'.

Source Error:

An application error occurred on the server. The current custom error settings for this application prevent the details of the application error from being viewed remotely (for security reasons). It could, however, be viewed by browsers running on the local server machine.

Source File: D:\home\site\wwwroot\web.config Line: 8

Version Information: Microsoft .NET Framework Version:4.0.30319; ASP.NET Version:4.6.1590.0

Of course Web.Secrets.ConnectionStrings.config and Web.Secrets.AppSettings.config are not copied and it's exactly what I need. The corresponding secrets should be taken from the environment variables.

There is a stack trace in HTML Source of the error page:

[ConfigurationErrorsException]: Unable to open configSource file 'Web.Secrets.ConnectionStrings.config'. (D:\home\site\wwwroot\web.config line 8)
   at System.Configuration.BaseConfigurationRecord.EvaluateOne(String[] keys, SectionInput input, Boolean isTrusted, FactoryRecord factoryRecord, SectionRecord sectionRecord, Object parentResult)
   at System.Configuration.BaseConfigurationRecord.Evaluate(FactoryRecord factoryRecord, SectionRecord sectionRecord, Object parentResult, Boolean getLkg, Boolean getRuntimeObject, Object& result, Object& resultRuntimeObject)
   at System.Configuration.BaseConfigurationRecord.GetSectionRecursive(String configKey, Boolean getLkg, Boolean checkPermission, Boolean getRuntimeObject, Boolean requestIsHere, Object& result, Object& resultRuntimeObject)
   at System.Configuration.BaseConfigurationRecord.GetSection(String configKey)
   at System.Web.Configuration.HttpConfigurationSystem.GetApplicationSection(String sectionName)
   at System.Web.Configuration.HttpConfigurationSystem.GetSection(String sectionName)
   at System.Web.Configuration.HttpConfigurationSystem.System.Configuration.Internal.IInternalConfigSystem.GetSection(String configKey)
   at System.Configuration.ConfigurationManager.GetSection(String sectionName)
   at System.Configuration.ConfigurationManager.get_ConnectionStrings()
   at EnvSettings.SettingsProcessor.SetConnectionString(String name, String connString, String providerName)
   at EnvSettings.SettingsProcessor.Start()
[InvalidOperationException]: The pre-application start initialization method Start on type EnvSettings.SettingsProcessor threw an exception with the following error message: Unable to open configSource file 'Web.Secrets.ConnectionStrings.config'. (D:\home\site\wwwroot\web.config line 8).
   at System.Web.Compilation.BuildManager.InvokePreStartInitMethodsCore(ICollection`1 methods, Func`1 setHostingEnvironmentCultures)
   at System.Web.Compilation.BuildManager.InvokePreStartInitMethods(ICollection`1 methods)
   at System.Web.Compilation.BuildManager.CallPreStartInitMethods(String preStartInitListPath, Boolean& isRefAssemblyLoaded)
   at System.Web.Compilation.BuildManager.ExecutePreAppStart()
   at System.Web.Hosting.HostingEnvironment.Initialize(ApplicationManager appManager, IApplicationHost appHost, IConfigMapPathFactory configMapPathFactory, HostingEnvironmentParameters hostingParameters, PolicyLevel policyLevel, Exception appDomainCreationException)
[HttpException]: The pre-application start initialization method Start on type EnvSettings.SettingsProc

What am I doing wrong? Or it's just a bug in Azure?

Important notes

• The app fails right on the startup. I don't even touch anything related to config files anywhere.
• If I delete the connection string from connection strings section in Application Settings of Azure App Service, the app starts fine. If I get it back, the app starts failing at the startup again. This is very odd! Just think about it! There IS connection string - fail, there is NO connection string - fine.
• When I run the app locally and there is no Web.Secrets.ConnectionStrings.config file, the app runs just fine. The app fails only when being deployed at Azure App Service. Hence, the issue is Azure App Service specific.
• It is being reproduced even with a plain empty ASP.NET Web Application project.
• The part about appSettings works just fine, only the part with connectionStrings fails.

Default Azure App Service Environment Variables:

WEBSITE_NODE_DEFAULT_VERSION: 4.4.7

Possible workaround:

Config transformation could be used for removing the appropriate attribute in Web.config. For example, that's how Web.Release.config might look like:

<?xml version="1.0" encoding="utf-8"?>
<configuration xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
  <connectionStrings xdt:Transform="RemoveAttributes(configSource)"/>
  <system.web>
    <compilation xdt:Transform="RemoveAttributes(debug)" />
  </system.web>
</configuration>

Answer 1

I looked into this, and my conclusion is that this is not an Azure issue, but is the way the config system behaves with a connection string configSource. Specifically, the behavior is that when you specify such directive, that file must be present, or any attempt to access connection strings blows up. e.g. outside of Azure, set up your web.config pointing to a missing configSource and run:

ConnectionStringSettings settings = ConfigurationManager.ConnectionStrings["foo"];

And it will blow up in the same way (Unable to open configSource file 'Web.Secrets.ConnectionStrings.config'.).

It's interesting, because with App Settings, it's able to simply ignore a file directive when the file is missing.

But none of that is Azure specific. It's just .NET framework config system behavior. I create a trivial Console app which demonstrates that: https://github.com/davidebbo-test/ConsoleAppWithMissingConfigSourceFile

You'll need to either yank the attribute (as you're showing), or deploy a dummy file to keep it happy.