Can a String based Include alternative be created in Entity Framework Core?

Question

On an API I need dynamic include, but EF Core does not support string-based include.

Because of this, I created a mapper which maps strings to lambda expressions added to a list as:

List<List<Expression>> expressions = new List<List<Expression>>();

Consider the following specific types:

public class EFContext 
{
    public DbSet<P1> P1s { get; set; }
    public DbSet<P1> P2s { get; set; }
    public DbSet<P1> P3s { get; set; }
}

public class P1 
{
    public P2 P2 { get; set; }
    public P3 P3 { get; set; }
}

public class P2 
{
    public P3 P3 { get; set; }
}

public class P3 { }

Include and ThenInclude are normally used as follows:

EFContext efcontext = new EFContext();
IQueryable<P1> result = efcontext.P1s
        .Include(p1 => p1.P2)
        .ThenInclude(p2 => p2.P3)
        .Include(p1 => p1.P3);

They can also be used the following way:

Expression<Func<P1, P2>> p1p2 = p1 => p1.P2;
Expression<Func<P1, P3>> p1p3 = p1 => p1.P3;
Expression<Func<P2, P3>> p2p3 = p2 => p2.P3;

List<List<Expression>> expressions = new List<List<Expression>> 
    {
        new List<Expression> { p1p2, p1p3 },
        new List<Expression> { p2p3 }
    };

EFContext efcontext = new EFContext();

IIncludableQueryable<P1, P2> q1 = EntityFrameworkQueryableExtensions
            .Include(efcontext.P1s, p1p2);
    
IIncludableQueryable<P1, P3> q2 = EntityFrameworkQueryableExtensions
            .ThenInclude(q1, p2p3);

IIncludableQueryable<P1, P3> q3 = EntityFrameworkQueryableExtensions
            .Include(q2, p1p3);

result = q3.AsQueryable();

The problem is that my method receives a list of Expressions and I only have the base type in T:

public static class IncludeExtensions<T> 
{
    public static IQueryable<T> IncludeAll(this IQueryable<T> collection, List<List<Expression>> expressions) 
    {
        MethodInfo include = typeof(EntityFrameworkQueryableExtensions)
            .GetTypeInfo()
            .GetDeclaredMethods(nameof(EntityFrameworkQueryableExtensions.Include))
            .Single(mi => mi.GetParameters()
                .Any(pi => pi.Name == "navigationPropertyPath"));

        MethodInfo includeAfterCollection = typeof(EntityFrameworkQueryableExtensions)
            .GetTypeInfo()
            .GetDeclaredMethods(nameof(EntityFrameworkQueryableExtensions.ThenInclude))
            .Single(mi => 
                !mi.GetParameters()[0].ParameterType.GenericTypeArguments[1].IsGenericParameter);

        MethodInfo includeAfterReference = typeof(EntityFrameworkQueryableExtensions)
            .GetTypeInfo()
            .GetDeclaredMethods(nameof(EntityFrameworkQueryableExtensions.ThenInclude))
            .Single(mi => mi.GetParameters()[0].ParameterType.GenericTypeArguments[1].IsGenericParameter);

        foreach (List<Expression> path in expressions) 
        {
            bool start = true;

            foreach (Expression expression in path) 
            {
                if (start) 
                {
                    MethodInfo method = include.MakeGenericMethod(typeof(T), ((LambdaExpression)expression).ReturnType);

                    IIncludableQueryable<T,?> result = method.Invoke(null, new Object[] { collection, expression });

                    start = false;
                } 
                else 
                {
                    MethodInfo method = includeAfterReference.MakeGenericMethod(typeof(T), typeof(?), ((LambdaExpression)expression).ReturnType);

                    IIncludableQueryable <T,?> result = method.Invoke(null, new Object[] { collection, expression });
                }           
            }
        }
        
        return collection; // (to be replaced by final as Queryable)
    }
}

The main problem has been resolving the correct types for each Include and ThenInclude step and also which ThenInclude to use.

Is this even possible with the current EF7 Core? Did someone find a solution for dynamic Include?

The Include and ThenIncludeAfterReference and ThenIncludeAfterCollection methods are part of EntityFrameworkQueryableExtensions class in EntityFramework Github's repository.

Answer 1

Update:

Starting with v1.1.0, the string based include is now part of EF Core, so the issue and the below solution are obsolete.

Original answer:

Interesting exercise for the weekend.

Solution:

I've ended up with the following extension method:

public static class IncludeExtensions
{
    private static readonly MethodInfo IncludeMethodInfo = typeof(EntityFrameworkQueryableExtensions).GetTypeInfo()
        .GetDeclaredMethods(nameof(EntityFrameworkQueryableExtensions.Include)).Single(mi => mi.GetParameters().Any(pi => pi.Name == "navigationPropertyPath"));

    private static readonly MethodInfo IncludeAfterCollectionMethodInfo = typeof(EntityFrameworkQueryableExtensions).GetTypeInfo()
        .GetDeclaredMethods(nameof(EntityFrameworkQueryableExtensions.ThenInclude)).Single(mi => !mi.GetParameters()[0].ParameterType.GenericTypeArguments[1].IsGenericParameter);

    private static readonly MethodInfo IncludeAfterReferenceMethodInfo = typeof(EntityFrameworkQueryableExtensions).GetTypeInfo()
        .GetDeclaredMethods(nameof(EntityFrameworkQueryableExtensions.ThenInclude)).Single(mi => mi.GetParameters()[0].ParameterType.GenericTypeArguments[1].IsGenericParameter);

    public static IQueryable<TEntity> Include<TEntity>(this IQueryable<TEntity> source, params string[] propertyPaths)
        where TEntity : class
    {
        var entityType = typeof(TEntity);
        object query = source;
        foreach (var propertyPath in propertyPaths)
        {
            Type prevPropertyType = null;
            foreach (var propertyName in propertyPath.Split('.'))
            {
                Type parameterType;
                MethodInfo method;
                if (prevPropertyType == null)
                {
                    parameterType = entityType;
                    method = IncludeMethodInfo;
                }
                else
                {
                    parameterType = prevPropertyType;
                    method = IncludeAfterReferenceMethodInfo;
                    if (parameterType.IsConstructedGenericType && parameterType.GenericTypeArguments.Length == 1)
                    {
                        var elementType = parameterType.GenericTypeArguments[0];
                        var collectionType = typeof(ICollection<>).MakeGenericType(elementType);
                        if (collectionType.IsAssignableFrom(parameterType))
                        {
                            parameterType = elementType;
                            method = IncludeAfterCollectionMethodInfo;
                        }
                    }
                }
                var parameter = Expression.Parameter(parameterType, "e");
                var property = Expression.PropertyOrField(parameter, propertyName);
                if (prevPropertyType == null)
                    method = method.MakeGenericMethod(entityType, property.Type);
                else
                    method = method.MakeGenericMethod(entityType, parameter.Type, property.Type);
                query = method.Invoke(null, new object[] { query, Expression.Lambda(property, parameter) });
                prevPropertyType = property.Type;
            }
        }
        return (IQueryable<TEntity>)query;
    }
}

Test:

Model:

public class P
{
    public int Id { get; set; }
    public string Info { get; set; }
}

public class P1 : P
{
    public P2 P2 { get; set; }
    public P3 P3 { get; set; }
}

public class P2 : P
{
    public P4 P4 { get; set; }
    public ICollection<P1> P1s { get; set; }
}

public class P3 : P
{
    public ICollection<P1> P1s { get; set; }
}

public class P4 : P
{
    public ICollection<P2> P2s { get; set; }
}

public class MyDbContext : DbContext
{
    public DbSet<P1> P1s { get; set; }
    public DbSet<P2> P2s { get; set; }
    public DbSet<P3> P3s { get; set; }
    public DbSet<P4> P4s { get; set; }

    // ...

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        modelBuilder.Entity<P1>().HasOne(e => e.P2).WithMany(e => e.P1s).HasForeignKey("P2Id").IsRequired();
        modelBuilder.Entity<P1>().HasOne(e => e.P3).WithMany(e => e.P1s).HasForeignKey("P3Id").IsRequired();
        modelBuilder.Entity<P2>().HasOne(e => e.P4).WithMany(e => e.P2s).HasForeignKey("P4Id").IsRequired();
        base.OnModelCreating(modelBuilder);
    }
}

Usage:

var db = new MyDbContext();

// Sample query using Include/ThenInclude
var queryA = db.P3s
    .Include(e => e.P1s)
        .ThenInclude(e => e.P2)
            .ThenInclude(e => e.P4)
    .Include(e => e.P1s)
        .ThenInclude(e => e.P3);

// The same query using string Includes
var queryB = db.P3s
    .Include("P1s.P2.P4", "P1s.P3");

How it works:

Given a type TEntity and a string property path of the form Prop1.Prop2...PropN, we split the path and do the following:

For the first property we just call via reflection the EntityFrameworkQueryableExtensions.Include method:

public static IIncludableQueryable<TEntity, TProperty>
Include<TEntity, TProperty>
(
    this IQueryable<TEntity> source,
    Expression<Func<TEntity, TProperty>> navigationPropertyPath
)

and store the result. We know TEntity and TProperty is the type of the property.

For the next properties it's a bit more complex. We need to call one of the following ThenInclude overloads:

public static IIncludableQueryable<TEntity, TProperty>
ThenInclude<TEntity, TPreviousProperty, TProperty>
(
    this IIncludableQueryable<TEntity, ICollection<TPreviousProperty>> source,
    Expression<Func<TPreviousProperty, TProperty>> navigationPropertyPath
)

and

public static IIncludableQueryable<TEntity, TProperty>
ThenInclude<TEntity, TPreviousProperty, TProperty>
(
    this IIncludableQueryable<TEntity, TPreviousProperty> source,
    Expression<Func<TPreviousProperty, TProperty>> navigationPropertyPath
)

source is the current result. TEntity is one and the same for all calls. But what is TPreviousProperty and how we decide which method to call.

Well, first we use a variable to remember what was the TProperty in the previous call. Then we check if it is a collection property type, and if yes, we call the first overload with TPreviousProperty type extracted from the generic arguments of the collection type, otherwise simply call the second overload with that type.

And that's all. Nothing fancy, just emulating an explicit Include / ThenInclude call chains via reflection.

Answer 2

String-based Include() shipped in EF Core 1.1. I would suggest you try upgrading and removing any workarounds you had to add to your code to address this limitation.