Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Parallel EF Core queries with DbContext injection in ASP.NET Core

I have writing an ASP.NET Core web application that needs all the data from some tables of my database to later organize it into readable format for some analysis.

My problem is that this data is potentially massive, and so in order to increase performance i decided to get this data in parallel and not one table at a time.

My issue is that i dont quite understand how to achieve this with the inherit dependency injection as in order to be able to do parallel work, i need to instantiate the DbContext for each of these parallel work.

The below code produces this exception:

---> (Inner Exception #6) System.ObjectDisposedException: Cannot access a disposed object. A common cause of this error is disposing a context that was resolved from dependency injection and then later trying to use the same context instance elsewhere in your application. This may occur if you are calling Dispose() on the context, or wrapping the context in a using statement. If you are using dependency injection, you should let the dependency injection container take care of disposing context instances.
Object name: 'MyDbContext'.
   at Microsoft.EntityFrameworkCore.DbContext.CheckDisposed()
   at Microsoft.EntityFrameworkCore.DbContext.get_InternalServiceProvider()
   at Microsoft.EntityFrameworkCore.DbContext.get_ChangeTracker()

ASP.NET Core project:

Startup.cs:

public void ConfigureServices(IServiceCollection services)
{
    services.AddMvc().SetCompatibilityVersion(CompatibilityVersion.Version_2_1);
    services.AddDistributedMemoryCache();

    services.AddDbContext<AmsdbaContext>(options =>
        options.UseSqlServer(Configuration.GetConnectionString("ConnectionString"))
            .UseQueryTrackingBehavior(QueryTrackingBehavior.NoTracking));

    services.AddSession(options =>
    {
        options.Cookie.HttpOnly = true;
    });
}

public void Configure(IApplicationBuilder app, ILoggerFactory loggerFactory)
{
    if (HostingEnvironment.IsDevelopment())
    {
        app.UseDeveloperExceptionPage();
    }
    else
    {
        app.UseExceptionHandler("/Error");
        app.UseHsts();
    }

    loggerFactory.AddLog4Net();
    app.UseStaticFiles();
    app.UseCookiePolicy();
    app.UseSession();
    app.UseMvc();
}

Controller's action method:

[HttpPost("[controller]/[action]")]
public ActionResult GenerateAllData()
{
    List<CardData> cardsData;

    using (var scope = _serviceScopeFactory.CreateScope())
    using (var dataFetcher = new DataFetcher(scope))
    {
        cardsData = dataFetcher.GetAllData(); // Calling the method that invokes the method 'InitializeData' from below code
    }

    return something...;
}

.NET Core Library project:

DataFetcher's InitializeData - to get all table records according to some irrelevant parameters:

private void InitializeData()
{
    var tbl1task = GetTbl1FromDatabaseTask();
    var tbl2task = GetTbl2FromDatabaseTask();
    var tbl3task = GetTbl3FromDatabaseTask();

    var tasks = new List<Task>
    {
        tbl1task,
        tbl2task,
        tbl3task,
    };

    Task.WaitAll(tasks.ToArray());

    Tbl1 = tbl1task.Result;
    Tbl2 = tbl2task.Result;
    Tbl3 = tbl3task.Result;
}

DataFetcher's sample task:

private async Task<List<SomeData>> GetTbl1FromDatabaseTask()
{
    using (var amsdbaContext = _serviceScope.ServiceProvider.GetRequiredService<AmsdbaContext>())
    {
        amsdbaContext.ChangeTracker.QueryTrackingBehavior = QueryTrackingBehavior.NoTracking;
        return await amsdbaContext.StagingRule.Where(x => x.SectionId == _sectionId).ToListAsync();
    }
}
like image 615
Tomer Something Avatar asked Oct 17 '18 13:10

Tomer Something


People also ask

How do you inject DbContext in dependency injection for ASP.NET Core?

DbContext in dependency injection for ASP.NET CoreThe context is configured to use the SQL Server database provider and will read the connection string from ASP.NET Core configuration. It typically does not matter where in ConfigureServices the call to AddDbContext is made.

Should DbContext be Singleton?

1 Answer. First, DbContext is a lightweight object; it is designed to be used once per business transaction. Making your DbContext a Singleton and reusing it throughout the application can cause other problems, like concurrency and memory leak issues. And the DbContext class is not thread safe.

How do I dispose of DbContext in EF core?

When the controller is being disposed, call dispose on your repository and that should dispose the context. If you are using a service layer and not talking to the repository directly from the controller, then call dispose on the service which will call dispose on repo which will dispose the context.

What is DbContext in EF core?

A DbContext instance represents a session with the database and can be used to query and save instances of your entities. DbContext is a combination of the Unit Of Work and Repository patterns. Entity Framework Core does not support multiple parallel operations being run on the same DbContext instance.


2 Answers

I'm not sure you do actually need multiple contexts here. You have have noticed that in the EF Core docs, there's this conspicuous warning:

Warning

EF Core does not support multiple parallel operations being run on the same context instance. You should always wait for an operation to complete before beginning the next operation. This is typically done by using the await keyword on each asynchronous operation.

This is not entirely accurate, or rather, it's simply worded somewhat confusingly. You can actually run parallel queries on a single context instance. The issue comes in with EF's change tracking and object fixup. These types of things don't support multiple operations happening at the same time, as they need to have a stable state to work from when doing their work. However, that really just limits your ability to do certain things. For example, if you were to run parallel saves/select queries, the results could be garbled. You might not get back things that are actually there now or change tracking could get messed up while it's attempt to create the necessary insert/update statements, etc. However, if you're doing non-atomic queries, such as selects on independent tables as you wish to do here, there's no real issue, especially, if you're not planning on doing further operations like edits on the entities you're selecting out, and just planning on returning them to a view or something.

If you truly determine you need separate contexts, your best bet is new up your context with a using. I haven't actually tried this before, but you should be able to inject DbContextOptions<AmsdbaContext> into your class where these operations are happening. It should already be registered in the service collection since it's injected into your context when the service collection instantiates that. If not, you can always just build a new one:

var options = new DbContextOptionsBuilder()
    .UseSqlServer(connectionString)
    .Build()
    .Options;

In either case, then:

List<Tbl1> tbl1data;
List<Tbl2> tbl2data;
List<Tbl3> tbl3data;

using (var tbl1Context = new AmsdbaContext(options))
using (var tbl2Context = new AmsdbaContext(options))
using (var tbl3Context = new AmsdbaContext(options))
{
    var tbl1task = tbl1Context.Tbl1.ToListAsync();
    var tbl2task = tbl2Context.Tbl2.ToListAsync();
    var tbl3task = tbl3Context.Tbl3.ToListAsync();

    tbl1data = await tbl1task;
    tbl2data = await tbl2task;
    tbl3data = await tbl3task;
}

It's better to use await to get the actual result. This way, you don't even need WaitAll/WhenAll/etc. and you're not blocking on the call to Result. Since tasks return hot, or already started, simply postponing calling await until each has been created is enough to buy you parallel processing.

Just be careful with this that you select everything you need within the usings. Now that EF Core supports lazy-loading, if you're using that, an attempt to access a reference or collection property that hasn't been loaded will trigger an ObjectDisposedException, since the context will be gone.

like image 145
Chris Pratt Avatar answered Sep 30 '22 06:09

Chris Pratt


Simple answer is - you do not. You need an alternative way to generate dbcontext instances. The standard approach is to get the same instance on all requests for a DbContext in the same HttpRequest. You can possibly override ServiceLifetime, but that then changes the behavior of ALL requests.

  • You can register a second DbContext (subclass, interface) with a different service lifetime. Even then you need to handle the creation manually as you need to call it once for every thread.

  • You manaully create them.

Standard DI simply comes to an end here. It is QUITE lacking, even compared to older MS DI frameworks where you possibly could put up a separate processing class with an attribute to override creation.

like image 45
TomTom Avatar answered Sep 30 '22 04:09

TomTom