Tracing Linq expression evaluation

Question

I'm wondering if it's possible to write a "passthrough" extension method for IQueryable which would write a debugstring whenever the queryable is evaluated, in other words, the debug print should be a side-effect of evaluation.

Something like:

var qr = SomeSource.Where(...).OrderBy(...).Trace("SomeSource evaluated at {0}", DateTime.Now)
var qr2 = qr.Where(...);

When I construct a linq query and pass it as a data source to some object, I'd like to know when and how often does the object evaluate my query. I suppose it can be achieved in other ways, like for example wrapping IEnumerable.GetEnumerator, but I'd like to do it generically for any linq query.

Answer 1

I've done something similar, but more complex (because it also manipulates the expressions as it processes them). In order to accomplish it, I created a wrapper class that implemented IQueryable and contained a reference to the thing I actually wanted to query. I made it pass all interface members through to the referenced object except for the Provider property, which returned a reference to another class I created that inherited from IQueryProvider. IQueryProvider has the methods that get called whenever a query is constructed or executed. So you could do something like this if you don't mind being forced to always query your wrapper object(s) instead of the original object(s).

You should also be aware, if you're using LINQ-to-SQL, there's a Log property on the DataContext that you can use to route lots of debug information wherever you want.

Sample code:

Make your own IQueryable to control the QueryProvider that gets returned.

Public Class MyQueryable(Of TableType)
   Implements IQueryable(Of TableType)

   Private innerQueryable As IQueryable(Of TableType)
   Private myProvider As MyQueryProvider = Nothing

   Public Sub New(ByVal innerQueryable As IQueryable(Of TableType))
      Me.innerQueryable = innerQueryable
   End Sub

   Public Function GetEnumerator() As System.Collections.Generic.IEnumerator(Of TableType) Implements System.Collections.Generic.IEnumerable(Of TableType).GetEnumerator
      Return innerQueryable.GetEnumerator()
   End Function

   Public Function GetEnumerator1() As System.Collections.IEnumerator Implements System.Collections.IEnumerable.GetEnumerator
      Return innerQueryable.GetEnumerator()
   End Function

   Public ReadOnly Property ElementType() As System.Type Implements System.Linq.IQueryable.ElementType
      Get
         Return innerQueryable.ElementType
      End Get
   End Property

   Public ReadOnly Property Expression() As System.Linq.Expressions.Expression Implements System.Linq.IQueryable.Expression
      Get
         Return innerQueryable.Expression
      End Get
   End Property

   Public ReadOnly Property Provider() As System.Linq.IQueryProvider Implements System.Linq.IQueryable.Provider
      Get
         If myProvider Is Nothing Then myProvider = New MyQueryProvider(innerQueryable.Provider)
         Return myProvider
      End Get
   End Property

   Friend ReadOnly Property innerTable() As System.Data.Linq.ITable
      Get
         If TypeOf innerQueryable Is System.Data.Linq.ITable Then
            Return DirectCast(innerQueryable, System.Data.Linq.ITable)
         End If
         Throw New InvalidOperationException("Attempted to treat a MyQueryable as a table that is not a table")
      End Get
   End Property
End Class

Make a custom query provider to control the expression tree that gets generated.

Public Class MyQueryProvider
   Implements IQueryProvider

   Private innerProvider As IQueryProvider

   Public Sub New(ByVal innerProvider As IQueryProvider)
      Me.innerProvider = innerProvider
   End Sub
   Public Function CreateQuery(ByVal expression As System.Linq.Expressions.Expression) As System.Linq.IQueryable Implements System.Linq.IQueryProvider.CreateQuery
      Return innerProvider.CreateQuery(expression)
   End Function

   Public Function CreateQuery1(Of TElement)(ByVal expression As System.Linq.Expressions.Expression) As System.Linq.IQueryable(Of TElement) Implements System.Linq.IQueryProvider.CreateQuery
      Dim newQuery = innerProvider.CreateQuery(Of TElement)(ConvertExpression(expression))
      If TypeOf newQuery Is IOrderedQueryable(Of TElement) Then
         Return New MyOrderedQueryable(Of TElement)(DirectCast(newQuery, IOrderedQueryable(Of TElement)))
      Else
         Return New MyQueryable(Of TElement)(newQuery)
      End If
   End Function

   Public Shared Function ConvertExpression(ByVal expression As Expression) As Expression
      If TypeOf expression Is MethodCallExpression Then
         Dim mexp = DirectCast(expression, MethodCallExpression)
         Return Expressions.MethodCallExpression.Call(ConvertExpression(mexp.Object), _
            mexp.Method, (From row In mexp.Arguments Select ConvertExpression(row)).ToArray())
      ElseIf TypeOf expression Is BinaryExpression Then
         Dim bexp As BinaryExpression = DirectCast(expression, BinaryExpression)
         Dim memberInfo As NestedMember = Nothing
         Dim constExp As Expression = Nothing
         Dim memberOnLeft As Boolean
         Dim doConvert = True
         '' [etc... lots of code to generate a manipulated expression tree
      ElseIf TypeOf expression Is LambdaExpression Then
         Dim lexp = DirectCast(expression, LambdaExpression)
         Return LambdaExpression.Lambda( _
            ConvertExpression(lexp.Body), (From row In lexp.Parameters Select _
            DirectCast(ConvertExpression(row), ParameterExpression)).ToArray())
      ElseIf TypeOf expression Is ConditionalExpression Then
         Dim cexp = DirectCast(expression, ConditionalExpression)
         Return ConditionalExpression.Condition(ConvertExpression(cexp.Test), _
                                                ConvertExpression(cexp.IfTrue), _
                                                ConvertExpression(cexp.IfFalse))
      ElseIf TypeOf expression Is InvocationExpression Then
         Dim iexp = DirectCast(expression, InvocationExpression)
         Return InvocationExpression.Invoke( _
            ConvertExpression(iexp.Expression), (From row In iexp.Arguments _
            Select ConvertExpression(row)).ToArray())
      ElseIf TypeOf expression Is MemberExpression Then
         '' [etc... lots of code to generate a manipulated expression tree
      ElseIf TypeOf expression Is UnaryExpression Then
         '' [etc... lots of code to generate a manipulated expression tree
      Else
         Return expression
      End If
   End Function

   Public Function Execute(ByVal expression As System.Linq.Expressions.Expression) As Object Implements System.Linq.IQueryProvider.Execute
      Return innerProvider.Execute(expression)
   End Function

   Public Function Execute1(Of TResult)(ByVal expression As System.Linq.Expressions.Expression) As TResult Implements System.Linq.IQueryProvider.Execute
      Return innerProvider.Execute(Of TResult)(ConvertExpression(expression))
   End Function
End Class

Then extend your derived DataContext by providing wrapped queryables:

Partial Public Class MyDataContext
  Private myQueries As Dictionary(Of System.Type, Object) = New Dictionary(Of System.Type, Object)
  Public ReadOnly Property My_AccountCategories() As MyQueryable(Of AccountCategory)
     Get
        Dim result As Object = Nothing
        If (Me.myQueries.TryGetValue(GetType(AccountCategory), result) = false) Then
           result = New MyQueryable(Of AccountCategory)(Me.AccountCategories)
           Me.myQueries(GetType(AccountCategory)) = result
        End If
        Return CType(result,MyQueryable(Of AccountCategory))
     End Get
  End Property
  Public ReadOnly Property My_AccountSegmentations() As MyQueryable(Of AccountSegmentation)
     Get
        Dim result As Object = Nothing
        If (Me.myQueries.TryGetValue(GetType(AccountSegmentation), result) = false) Then
           result = New MyQueryable(Of AccountSegmentation)(Me.AccountSegmentations)
           Me.myQueries(GetType(AccountSegmentation)) = result
        End If
        Return CType(result,MyQueryable(Of AccountSegmentation))
     End Get
  End Property
End Class

Answer 2

Define a new extension method:

    public static IEnumerable<T> Trace<T>(this IEnumerable<T> input, 
                                            string format, 
                                            params object[] data) 
    {
        if (input == null)
            throw new ArgumentNullException("input");

        return TraceImpl(input, format, data);
    }

    private static IEnumerable<T> TraceImpl<T>(IEnumerable<T> input, 
                                                string format, 
                                                params object[] data) 
    {
        System.Diagnostics.Trace.WriteLine(string.Format(format, data));

        foreach (T element in input)
            yield return element;
    }

This should print a trace everytime you iterate over it. Thanks to Jon Skeet for the inspiration.

Personally, I would replace format and data with an Action delegate, so you could perform any task (not relevant to the collection) instead of simply tracing.

Edit: I have the feeling that this might only work for linq-to-objects. For IQueryable<T>, you'd have to adjust the expression tree parsers, which you don't have access to. Sorry :-/