Reusable Calculations For LINQ Projections In Entity Framework (Code First)

Question

My domain model has a lot of complex financial data that is the result of fairly complex calculations on multiple properties of various entities. I generally include these as [NotMapped] properties on the appropriate domain model (I know, I know - there's plenty of debate around putting business logic in your entities - being pragmatic, it just works well with AutoMapper and lets me define reusable DataAnnotations - a discussion of whether this is good or not is not my question).

This works fine as long as I want to materialize the entire entity (and any other dependent entities, either via .Include() LINQ calls or via additional queries after materialization) and then map these properties to the view model after the query. The problem comes in when trying to optimize problematic queries by projecting to a view model instead of materializing the entire entity.

Consider the following domain models (obviously simplified):

public class Customer
{
 public virtual ICollection<Holding> Holdings { get; private set; }

 [NotMapped]
 public decimal AccountValue
 {
  get { return Holdings.Sum(x => x.Value); }
 }
}

public class Holding
{
 public virtual Stock Stock { get; set; }
 public int Quantity { get; set; }

 [NotMapped]
 public decimal Value
 {
  get { return Quantity * Stock.Price; }
 }
}

public class Stock
{
 public string Symbol { get; set; }
 public decimal Price { get; set; }
}

And the following view model:

public class CustomerViewModel
{
 public decimal AccountValue { get; set; }
}

If I attempt to project directly like this:

List<CustomerViewModel> customers = MyContext.Customers
 .Select(x => new CustomerViewModel()
 {
  AccountValue = x.AccountValue
 })
 .ToList();

I end up with the following NotSupportedException: Additional information: The specified type member 'AccountValue' is not supported in LINQ to Entities. Only initializers, entity members, and entity navigation properties are supported.

Which is expected. I get it - Entity Framework can't convert the property getters into a valid LINQ expression. However, if I project using the exact same code but within the projection, it works fine:

List<CustomerViewModel> customers = MyContext.Customers
 .Select(x => new CustomerViewModel()
 {
  AccountValue = x.Holdings.Sum(y => y.Quantity * y.Stock.Price)
 })
 .ToList();

So we can conclude that the actual logic is convertible to a SQL query (I.e., there's nothing exotic like reading from disk, accessing external variables, etc.).

So here's the question: is there any way at all to make logic that should be convertible to SQL reusable within LINQ to entity projections?

Consider that this calculation may be used within many different view models. Copying it to the projection in each action is cumbersome and error prone. What if the calculation changes to include a multiplier? We'd have to manually locate and change it everywhere it's used.

One thing I have tried is encapsulating the logic within an IQueryable extension:

public static IQueryable<CustomerViewModel> WithAccountValue(
 this IQueryable<Customer> query)
{
 return query.Select(x => new CustomerViewModel()
 {
  AccountValue = x.Holdings.Sum(y => y.Quantity * y.Stock.Price)
 });
}

Which can be used like this:

List<CustomerViewModel> customers = MyContext.Customers
 .WithAccountValue()
 .ToList();

That works well enough in a simple contrived case like this, but it's not composable. Because the result of the extension is an IQueryable<CustomerViewModel> and not a IQueryable<Customer> you can't chain them together. If I had two such properties in one view model, one of them in another view model, and then the other in a third view model, I would have no way of using the same extension for all three view models - which would defeat the whole purpose. With this approach, it's all or nothing. Every view model has to have the exact same set of calculated properties (which is rarely the case).

Sorry for the long-winded question. I prefer to provide as much detail as possible to make sure folks understand the question and potentially help others down the road. I just feel like I'm missing something here that would make all of this snap into focus.

Answer 1

I did a lot of research on this the last several days because it's been a bit of a pain point in constructing efficient Entity Framework queries. I've found several different approaches that all essentially boil down to the same underlying concept. The key is to take the calculated property (or method), convert it into an Expression that the query provider knows how to translate into SQL, and then feed that into the EF query provider.

I found the following libraries/code that attempted to solve this problem:

LINQ Expression Projection

http://www.codeproject.com/Articles/402594/Black-Art-LINQ-expressions-reuse and http://linqexprprojection.codeplex.com/

This library allows you to write your reusable logic directly as an Expression and then provides the conversion to get that Expression into your LINQ query (since the query can't directly use an Expression). The funny thing is that it'll be translated back to an Expression by the query provider. The declaration of your reusable logic looks like this:

private static Expression<Func<Project, double>> projectAverageEffectiveAreaSelector =
 proj => proj.Subprojects.Where(sp => sp.Area < 1000).Average(sp => sp.Area);

And you use it like this:

var proj1AndAea =
 ctx.Projects
  .AsExpressionProjectable()
  .Where(p => p.ID == 1)
  .Select(p => new 
  {  
   AEA = Utilities.projectAverageEffectiveAreaSelector.Project<double>() 
  });

Notice the .AsExpressionProjectable() extension to set up projection support. Then you use the .Project<T>() extension on one of your Expression definitions to get the Expression into the query.

LINQ Translations

http://damieng.com/blog/2009/06/24/client-side-properties-and-any-remote-linq-provider and https://github.com/damieng/Linq.Translations

This approach is pretty similar to the LINQ Expression Projection concept except it's a little more flexible and has several points for extension. The trade off is that it's also a little more complex to use. Essentially you still define your reusable logic as an Expression and then rely on the library to convert that into something the query can use. See the blog post for more details.

DelegateDecompiler

http://lostechies.com/jimmybogard/2014/05/07/projecting-computed-properties-with-linq-and-automapper/ and https://github.com/hazzik/DelegateDecompiler

I found DelegateDecompiler via the blog post on Jimmy Bogard's blog. It has been a lifesaver. It works well, is well architected, and requires a lot less ceremony. It does not require you to define your reusable calculations as an Expression. Instead, it constructs the necessary Expression by using Mono.Reflection to decompile your code on the fly. It knows which properties, methods, etc. need to be decompiled by having you decorate them with ComputedAttribute or by using the .Computed() extension within the query:

class Employee
{
 [Computed]
 public string FullName
 {
  get { return FirstName + " " + LastName; }
 }
 public string LastName { get; set; }
 public string FirstName { get; set; }
}

This can also be easily extended, which is a nice touch. For example, I set it up to look for the NotMapped data annotation instead of having to explicitly use the ComputedAttribute.

Once you've set up your entity, you just trigger decompilation by using the .Decompile() extension:

var employees = ctx.Employees
 .Select(x => new
 {
  FullName = x.FullName
 })
 .Decompile()
 .ToList();

Answer 2

You can encapsulate logic by creating a class that contains the original Entity and the additional calculated property. You then create helper methods that project to the class.

For example, if we were trying to calculate the tax for an Employee and a Contractor entity, we could do this:

---

//This is our container for our original entity and the calculated field
public class PersonAndTax<T> 
{
    public T Entity { get; set; }
    public double Tax { get; set; }
}

---

public class PersonAndTaxHelper
{
    // This is our middle translation class
    // Each Entity will use a different way to calculate income
    private class PersonAndIncome<T>
    {
        public T Entity { get; set; }
        public int Income { get; set; }
    }

---

Income calculating methods

    public static IQueryable<PersonAndTax<Employee>> GetEmployeeAndTax(IQueryable<Employee> employees)
    {
        var query = from x in employees
                    select new PersonAndIncome<Employee>
                    {
                        Entity = x,
                        Income = x.YearlySalary
                    };
        return CalcualateTax(query);
    }

    public static IQueryable<PersonAndTax<Contractor>> GetContratorAndTax(IQueryable<Contractor> contractors)
    {
        var query = from x in contractors
                    select new PersonAndIncome<Contractor>
                    {
                        Entity = x,
                        Income = x.Contracts.Sum(y => y.Total) 
                    };

        return CalcualateTax(query);
    }

---

Tax calculation is defined in one place

    private static IQueryable<PersonAndTax<T>> CalcualateTax<T>(IQueryable<PersonAndIncome<T>> personAndIncomeQuery)
    {
        var query = from x in personAndIncomeQuery
                    select new PersonAndTax<T>
                    {
                        Entity = x.Entity,
                        Tax = x.Income * 0.3
                    };
        return query;
    }
}

---

Our view model projections using the Tax property

    var contractorViewModel = from x in PersonAndTaxHelper.GetContratorAndTax(context.Contractors)
                            select new
                            {
                                x.Entity.Name,
                                x.Entity.BusinessName
                                x.Tax,
                            };

    var employeeViewModel = from x in PersonAndTaxHelper.GetEmployeeAndTax(context.Employees)
                            select new
                            {
                                x.Entity.Name,
                                x.Entity.YearsOfService
                                x.Tax,
                            };