Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Executing an Elastic Scale multi shard query on the database context

I'm busy implementing the new Elastic Scale technnology in a project I'm working on at the moment. This technology appears to solve some complex issues we were having while designing the new application foundation.

So far, the examples look great and I'm busy implementing this in our newly created DAL.

For our application we can't rely solely on Elastic Scale in Azure. The application has to be able to run on a single instance machine, on-premise, also. Therefore I've created the following code to query the database which works quite well, also with Elastic Scale.

public IEnumerable<AnEntity> All()
{
    var dbConnection = GetConnection();
    using (var context = new OurDatabaseContext(dbConnection))
    {
        var theEntities = context.EntityTable;
        return theEntities.ToArray();
    }
}
private IDbConnection GetConnection()
{
    var connectionInstance = connection[ConnectionStringNames.TheDatabase];
    var dbConnection = connectionInstance.Create();
    return dbConnection;
}

The connectionInstance is configured via IoC which will create an IDbConnection which we can use in the OurDatabaseContext. All pretty much straightforward.

The main issue I'm facing is doing a MultiShardConnection, provided by Elastic Scale and implemented in the examples.

So my question is, is it possible to use a MultiShardConnection with a database context (like the ones of LINQ2SQL (which we are using) or EF).

If not, is the only solution to use the MultiShardConnection in combination with the MultiShardCommand? Or when will such a feature become available?

like image 542
Jan_V Avatar asked Dec 17 '14 15:12

Jan_V


People also ask

How do I shard an Azure SQL Database?

To easily scale out databases on Azure SQL Database, use a shard map manager. The shard map manager is a special database that maintains global mapping information about all shards (databases) in a shard set. The metadata allows an application to connect to the correct database based upon the value of the sharding key.

How do you redistribute the data to the new databases without disrupting the data integrity?

How do you redistribute the data to the new databases without disrupting the data integrity? Use the split-merge tool to move data from constrained databases to the new databases. The split-merge tool runs as an Azure web service.

What is a shard in Azure?

A shard is an individual partition that exists on separate database server instance to spread load. Auto sharding or data sharding is needed when a dataset is too big to be stored in a single database. As both the database size and number of transactions increase, so does the response time for querying the database.


1 Answers

As i currently know there is no trivial way to make a multishard connection with a dbContext.

Using MultiShardConnection + MultiShardCommand is pretty straitforward from the examples. However there are not all methods available ( for example ReadAsync).

BUT, i think that most multishard connections can be bypassed by using the right Sharding Key ( mapping key ). I solved my problem by adding one more sharding Key to my ShardMapManager. If you want describe your specific reason you need MultiShard Connection and ill edit my post. There is always the option to create multiple dbContexts.

like image 162
Nikatlas Avatar answered Nov 02 '22 05:11

Nikatlas