Error in EF Core: A second operation started on this context before a previous operation completed

Question

I got strange error in my web api app: A second operation started on this context before a previous operation completed. Any instance members are not guaranteed to be thread safe.

I have a process to calculate User's payday and update its User Leave balances. so I need to iterate users and get its leave balance and then do update on each balances. I have no idea how to solve this error yet. The error triggered when I call this.SaveAll() which is containb

    public async Task<bool> SaveAll()
    {
        return await _context.SaveChangesAsync() > 0;
    }

also datacontext is injected with this code:

private readonly DataContext _context;
    private readonly IAdminSettingsRepository _settingRepo;
    private readonly IAppRepository _appRepository;
    public PayrollRepository(DataContext context, IAdminSettingsRepository settingRepo, IAppRepository appRepository)
    {
        _context = context;
        _settingRepo = settingRepo;
        _appRepository = appRepository;
    }

the process as follow:

public async Task<bool> ProcessPayCalendar(PayCalendar payCalendar)
        {
            List<User> users = payCalendar.Users.ToList();
            SickLeaveEntitlement sickLeaveEntitlement = await _settingRepo.GetSickLeaveEntitlement();
            AnnualLeaveEntitlement annualLeaveEntitlement = await _settingRepo.GetAnnualLeaveEntitlement();
            LongServiceLeaveEntitlement longServiceLeaveEntitlement = await _settingRepo.GetLongServiceLeaveEntitlement();
            DateTime tenYearsAgo = DateTime.Today.AddYears(-10);
            DateTime currentMonth = new DateTime(payCalendar.NextPaymentDate.Year, payCalendar.NextPaymentDate.Month, 1);
            DateTime previousMonth = currentMonth.AddMonths(-1);


            /// begin calculation for leave calendar
            users.ForEach(async user =>
            {
                decimal hoursWorked = this.CalculateWorkingHours(user);
                decimal hourlyRate = this.CalculateHourlyRate(user);
                // create pay day for user
                Payday payday = new Payday();
                payday.UserId = user.Id;
                payday.PayPeriodStart = payCalendar.PayPeriodStartDate;
                payday.PayPeriodEnd = payCalendar.PayPeriodEndDate;
                payday.PaymentDate = payCalendar.NextPaymentDate;
                payday.HoursWorked = hoursWorked;
                payday.SickLeaveAccrual = this.CalculateSickLeaveEntitlement(hoursWorked, sickLeaveEntitlement);
                payday.AnnualLeaveAccrual = this.CalculateAnnualLeaveEntitlement(hoursWorked, annualLeaveEntitlement);
                payday.LongServiceLeaveAccrual = (user.StartDateCurrentAnnualSalary > tenYearsAgo) ? 0m : this.CalculateLongServiceEntitlement(hoursWorked, longServiceLeaveEntitlement);
                payday.SickLeaveAccrualValue = payday.SickLeaveAccrual * hourlyRate;
                payday.AnnualLeaveAccrualValue = payday.AnnualLeaveAccrual * hourlyRate;
                payday.LongServiceLeaveAccrualValue = payday.LongServiceLeaveAccrual * hourlyRate;

                this.Add(payday);

                // do calculation on leave balance 
                // jika leave balance di awal bulan maka lakukan replikasi dr bulan sebelumnya.
                // closebalance menjadi opening balance bulan selanjutnya.
                LeaveBalance sickLeaveBalance = await this.GetUserLeaveBalance(user.Id, sickLeaveEntitlement, "sickLeave");
                if (sickLeaveBalance == null)
                {
                    // sickLeaveBalance =  await this.CreateLeaveBalance(user.Id,sickLeaveEntitlement,"sickLeave",payday.PaymentDate);
                    // sickLeaveBalance.CurrentBalance =+ payday.SickLeaveAccrual ;
                    // sickLeaveBalance.CurrentBalanceValue += payday.SickLeaveAccrualValue;
                    throw new Exception($"Sick Leave balance for user: {user.Username} is not found. Please report this as bug");
                }
                else
                {
                    sickLeaveBalance.CurrentBalance = +payday.SickLeaveAccrual;
                    sickLeaveBalance.CurrentBalanceValue += payday.SickLeaveAccrualValue;
                    sickLeaveBalance.LastUpdate = payday.PaymentDate;
                }

                LeaveBalance annualLeaveBalance = await this.GetUserLeaveBalance(user.Id, annualLeaveEntitlement, "annualLeave");
                if (annualLeaveBalance == null)
                {
                    throw new Exception($"Annual Leave balance for user: {user.Username} is not found. Please report this as bug");
                }
                else
                {
                    annualLeaveBalance.CurrentBalance = +payday.AnnualLeaveAccrual;
                    annualLeaveBalance.CurrentBalanceValue += payday.AnnualLeaveAccrualValue;   
                    annualLeaveBalance.LastUpdate = payday.PaymentDate;

                }
                LeaveBalance longServiceLeaveBalance = await this.GetUserLeaveBalance(user.Id, longServiceLeaveEntitlement, "longServiceLeave");
                if (longServiceLeaveBalance == null)
                {
                    throw new Exception($"Long Service Leave balance for user: {user.Username} is not found. Please report this as bug");
                }
                else
                {
                    longServiceLeaveBalance.CurrentBalance = +payday.LongServiceLeaveAccrual;
                    longServiceLeaveBalance.CurrentBalanceValue += payday.LongServiceLeaveAccrualValue;
                    longServiceLeaveBalance.LastUpdate = payday.PaymentDate;

                }
            });
            return await this.SaveAll();
        }

the complete error:

fail: Microsoft.EntityFrameworkCore.Update[10000] An exception occurred in the database while saving changes for context type 'CRSApp.API.Data.DataContext'. System.InvalidOperationException: A second operation started on this context before a previous operation completed. Any instance members are not guaranteed to be thread safe. at Microsoft.EntityFrameworkCore.Internal.ConcurrencyDetector.EnterCriticalSection() at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChangesAsync(IReadOnlyList1 entriesToSave, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.DbContext.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken) System.InvalidOperationException: A second operation started on this context before a previous operation completed. Any instance members are not guaranteed to be thread safe. at Microsoft.EntityFrameworkCore.Internal.ConcurrencyDetector.EnterCriticalSection() at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChangesAsync(IReadOnlyList1 entriesToSave, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.DbContext.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken) fail: Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware[1] An unhandled exception has occurred while executing the request. System.InvalidOperationException: A second operation started on this context before a previous operation completed. Any instance members are not guaranteed to be thread safe. at Microsoft.EntityFrameworkCore.Internal.ConcurrencyDetector.EnterCriticalSection() at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChangesAsync(IReadOnlyList1 entriesToSave, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.DbContext.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken) at CRSApp.API.Data.PayrollRepository.SaveAll() in E:\CRSApp\crsapp.api\Data\PayrollRepository.cs:line 35 at CRSApp.API.Data.PayrollRepository.ProcessPayCalendar(PayCalendar payCalendar) in E:\CRSApp\crsapp.api\Data\PayrollRepository.cs:line 229 at CRSApp.API.Controllers.Admin.PayrollController.ProcessPayCalendar(PayCalendarParam param) in E:\CRSApp\crsapp.api\Controllers\Admin\PayrollController.cs:line 68 at Microsoft.AspNetCore.Mvc.Internal.ActionMethodExecutor.TaskOfIActionResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments) at System.Threading.Tasks.ValueTask1.get_Result() at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.InvokeActionMethodAsync() at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.InvokeNextActionFilterAsync() at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.Rethrow(ActionExecutedContext context) at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted) at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.InvokeInnerFilterAsync() at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeNextResourceFilter() at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Rethrow(ResourceExecutedContext context) at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted) at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeFilterPipelineAsync() at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeAsync() at Microsoft.AspNetCore.Builder.RouterMiddleware.Invoke(HttpContext httpContext) at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context) at Microsoft.AspNetCore.Cors.Infrastructure.CorsMiddleware.Invoke(HttpContext context) at Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context) at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.Invoke(HttpContext context)

Fyi: I use Dotnet Core 2.1.1 as stated bellow.

<ItemGroup>
    <PackageReference Include="Microsoft.AspNetCore.App" Version="2.1.1"/>
    <PackageReference Include="Microsoft.VisualStudio.Web.CodeGeneration.Design" Version="2.1.0" PrivateAssets="All"/>
    <PackageReference Include="AutoMapper.Extensions.Microsoft.DependencyInjection" Version="4.0.1"/>
    <PackageReference Include="Pomelo.EntityFrameworkCore.MySql" Version="2.1.1"/>
    <PackageReference Include="MailKit" Version="2.0.6"/>
  </ItemGroup>

Answer 1

DbContext is not thread safe

I think this ForEach is your problem.

 users.ForEach(async user =>
        {
           //....

            LeaveBalance sickLeaveBalance = await this.GetUserLeaveBalance(user.Id,});
           // .....
        });

as you used async keyword so for each user an action will be invoke asynchronously so your code execution will be like the execution of code below:

foreach(var user in users){
     //without await
     //it's an async method
    DoSomeThingAsync(user); //you called GetUserLeaveBalance in DoSomeThingAsync
}

and as i guess, GetUserLeaveBalance is using DbContext so you are using DbContext asynchronously and you will face with such error like

A second operation started on this context before a previous operation completed ...

You need change your ForEach to something like this

foreach(var user in users){
    await DoSomeThingAsync(user); //you called GetUserLeaveBalance in DoSomeThingAsync
}