Internal .Net Framework Data Provider error 6 in SQL Azure

Question

I regularly experience the above error when creating connections to Azure SQL databases. I've implemented ReliableSqlConnection with retry logic in attempt to avoid this issue but it has been to no avail. Following is an example error stack trace:

System.InvalidOperationException
Internal .Net Framework Data Provider error 6. 
System.InvalidOperationException: Internal .Net Framework Data Provider error 6. 
at System.Data.ProviderBase.DbConnectionPool.CreateObject(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal oldConnection)
at System.Data.ProviderBase.DbConnectionPool.UserCreateRequest(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal oldConnection)
at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, UInt32 waitForMultipleObjectsTimeout, Boolean allowCreate, Boolean onlyOneCheckConnection, DbConnectionOptions userOptions, DbConnectionInternal& connection)
at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal& connection)
at System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection)
at System.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions)
at System.Data.ProviderBase.DbConnectionClosed.TryOpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions)
at System.Data.SqlClient.SqlConnection.TryOpenInner(TaskCompletionSource`1 retry)
at System.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1 retry)
at System.Data.SqlClient.SqlConnection.Open()
at Microsoft.Practices.EnterpriseLibrary.TransientFaultHandling.ReliableSqlConnection.<Open>b__1()
at Microsoft.Practices.EnterpriseLibrary.TransientFaultHandling.RetryPolicy.<>c__DisplayClass1.<ExecuteAction>b__0()
at Microsoft.Practices.EnterpriseLibrary.TransientFaultHandling.RetryPolicy.ExecuteAction[TResult](Func`1 func)
at Microsoft.Practices.EnterpriseLibrary.TransientFaultHandling.RetryPolicy.ExecuteAction(Action action)
at Microsoft.Practices.EnterpriseLibrary.TransientFaultHandling.ReliableSqlConnection.<Open>b__0()
at Microsoft.Practices.EnterpriseLibrary.TransientFaultHandling.RetryPolicy.<>c__DisplayClass1.<ExecuteAction>b__0()
at Microsoft.Practices.EnterpriseLibrary.TransientFaultHandling.RetryPolicy.ExecuteAction[TResult](Func`1 func)
at Microsoft.Practices.EnterpriseLibrary.TransientFaultHandling.RetryPolicy.ExecuteAction(Action action)
at Microsoft.Practices.EnterpriseLibrary.TransientFaultHandling.ReliableSqlConnection.Open(RetryPolicy retryPolicy)

This issue happens when creating a new database in an elastic pool. The SQL command text is execute in the following method:

public void ExecuteCommandText(string commandText)
    {
        if (IsNullOrEmpty(commandText))
            throw new ArgumentNullException(nameof(commandText));

        List<string> commandSteps = SplitCommandText(commandText);

        using (var sqlConnection = CreateConnection())
        {
            foreach (string commandStep in commandSteps)
            {
                using (SqlCommand command = sqlConnection.CreateCommand())
                {
                    command.CommandText = commandStep;
                    command.CommandTimeout = _commandTimeout;
                    command.ExecuteNonQuery();
                    command.Dispose();
                }
            }
        }
    }

Where:

private ReliableSqlConnection CreateConnection()
    {
        if (IsNullOrEmpty(ConnectionString))
            throw new InvalidOperationException("Connection string is not defined.");

        ReliableSqlConnection sqlConnection = new ReliableSqlConnection(ConnectionString, _retryPolicy, _retryPolicy);
        sqlConnection.Open();
        return sqlConnection;
    }

And:

var retryStrategy = new ExponentialBackoff(5, TimeSpan.FromSeconds(1), TimeSpan.FromSeconds(60), TimeSpan.FromSeconds(2));
_retryPolicy = new RetryPolicy<SqlDatabaseTransientErrorDetectionStrategy>(retryStrategy);

The connection string is of the format:

$"Server=tcp:{serverName},1433;Data Source={serverName};Persist Security Info=False;User ID='{user}';Password='{password}';MultipleActiveResultSets=False;Encrypt=True;TrustServerCertificate=False;Connection Timeout=90;"

Also, I've checked the eDTU usage on my elastic pool:

Elastic pool eDTU usage

The first spike is from creating 1 database in the pool, the second is for 2, the third is for 3 and the fourth is for creating 4 databases concurrently. The elastic pool service tier is Standard 100.

Answer 1

This is one of Azure SQL transient error. A lot of info is here https://msdn.microsoft.com/en-us/library/dn440719(v=pandp.60).aspx. Seem the same error when trying to do backups via code. In those cases I drop and re-initiate.