Performance problems with EF core QueryFilter

Question

In our system we ran into performance problems while using QueryFilters in EF core. The problem is that EF core filters inside a LEFT JOIN instead of doing the filtering outside of it.

The generated SQL looks something like this:

SELECT [pom].[Id],
        [pom].[DeleteDate],
        [pom].[UpdateDate],
        [pom].[Version],
        [t].[Id],
        [t].[MandatorId],
        [t].[NetPriceAmount],
        [t].[NetPriceCurrencyIso4217Code]
FROM [externaldata].[PurchaseOfferMetadata] AS [pom]
    LEFT JOIN (
    SELECT [po].[Id],
            [po].[MandatorId],
            [po].[NetPriceAmount],
            [po].[NetPriceCurrencyIso4217Code]
    FROM [externaldata].[PurchaseOffer] AS [po]
    WHERE [po].[MandatorId] = 1
) AS [t] ON [pom].[Id] = [t].[Id]
WHERE [pom].[Id] IN 
    (CAST(3094411 AS bigint), 
    CAST(4757070 AS bigint), 
    CAST(4757112 AS bigint), 
    CAST(5571232 AS bigint))

The problematic part is WHERE [po].[MandatorId] = 1. If this would be in the second WHERE statement, the query runs much faster.

The database model is configured like this:

modelBuilder.Entity<PurchaseOffer>()
      .HasQueryFilter(po => po.MandatorId == 1)
      .ToTable(nameof(PurchaseOffer), schema: ExternalDataSchemaName);

modelBuilder.Entity<PurchaseOfferMetadata>()
      .HasOne(pom => pom.PurchaseOffer)
      .WithOne(po => po.Metadata)
      .HasForeignKey<PurchaseOffer>(po => po.Id);

On the database we have set the foreign key like this:

IF OBJECT_ID('[externaldata].[FK_PurchaseOffer_PurchaseOfferMetadata]', 'F') IS NULL
BEGIN
    ALTER TABLE [externaldata].[PurchaseOffer] ADD CONSTRAINT [FK_PurchaseOffer_PurchaseOfferMetadata] FOREIGN KEY
    (
        [Id]
    )
    REFERENCES [externaldata].[PurchaseOfferMetadata] ([Id])
END;

The EF core query looks like this:

var existingPurchaseOfferMetadatasById = await db.PurchaseOfferMetadatas
   .Where(pom => purchaseOfferIds.Contains(pom.Id))
   .Include(pom => pom.PurchaseOffer)
   .ToDictionaryAsync(pom => pom.Id, cancellationToken);

Currently we have the following amount of records in each table:

• PurchaseOfferMetadata: 12'654'639
• PurchaseOffer: 1'689'634

Has anyone also encountered this problem and might found a solution to this?

Answer 1

EF has historically not done a great job of creating the most optimized SQL queries when joins are complex. I can't answer how you get Linq to understand your database optimization design; however, if you know the SQL that should be generated, then I am proposing that for better complex READ performance, use a NuGet package like Dapper to write the exact SQL you want and get the joins your database deserves.

using Dapper;

[...]

public class SalsaZima
{
  public int ID { get; set; }
  public string MySalsaColumn { get; set; }
  public string MyZimaColumn { get; set; }
}

[...]

// get data from database
using (IDbConnection dp = Dapper)
{
  string query = @"SELECT 
                     s.ID
                    ,s.MySalsaColumn
                    ,z.MyZimaColumn
                   FROM dbo.Salsa s 
                   LEFT JOIN dbo.Zima z
                     ON s.ZimaID = z.ID";

  return dp.Query<SalsaZima>(query).ToList();
}