Returning IQueryable but need Raven stats to insert TotalResults header

Question

I have a WebApi method which returns an IQueryable of RavenDB documents. The caller needs to know the number of possible results (because the actual results are limited/paged).

So, I have something like this at the end of my WebApi method:

HttpContext.Current.Response.AddHeader("Total-Result-Count", 
    resultsStats.TotalResults.ToString())

Unfortunately, this won't work, because the IQueryable hasnt actually executed yet - so the stats will be empty.

How do I go about deferring the population of the stats response-header until AFTER the query has executed?

[UPDATE]

I attempted to apply an ActionFilter to capture the result after the controller action had executed... but it seems the ActionFilter is invoked BEFORE the IQueryable is actually enumerated...

public class CountQueryableResultsActionFilter : ActionFilterAttribute
{
    public override void OnActionExecuted(HttpActionExecutedContext filterContext)
    {
        var controllerStats = filterContext.ActionContext.ControllerContext.Controller as IControllerStatistics;
        System.Web.HttpContext.Current.Response.AddHeader("Total-Result-Count", controllerStats.TotalResults.ToString());
    }
}

IF, I called "IQueryable.ToArray()" at the end of the WebApi method, then the Linq query gets executed immediately, it generates statistics, and everything works - but that will prevent the user from being able to apply their own OData filters etc...

Answer 1

Ok - I figured it out.

The following will result in only a single Raven query being issued, which returns both the result, and the result-count.

Thanks to David Ruttka for his experiments in this area. I have adapted his code to work with with RavenDb. This code will return the results, and the result-count through one database query, as RavenDB intended.

I have appended my code below - to use this, you must return IRavenQueryable<T> from your WebApi method (not IQueryable<T>). Then, appending $inlinecount=allpages to your Uri will invoke the handler. This code will not break the other OData query extensions ($take, $skip etc)

Note: This code uses the 'inline' technique, in that the statistics are returned in the message body - you could change the code to inject the stats in the header if you liked - I just chose to go with the standard way that OData works.

You could adapt this code to include any and all of the statistics that Raven generates.

Use the following code to register the handler with ASP.NET (in your Global.asax.cs)

RegistrationCode:

GlobalConfiguration.Configuration.MessageHandlers.Add(new WebApi.Extensions.InlineRavenCountHandler());

Handler code:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Net.Http;
using System.Threading;
using System.Threading.Tasks;
using System.Reflection;
using System.Net.Http.Headers;
using System.Net;

namespace WebApi.Extensions
{
    public class InlineRavenCountHandler : DelegatingHandler
    {
        protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
        {
            if (!ShouldInlineCount(request))
                return base.SendAsync(request, cancellationToken);

            // Otherwise, we have a continuation to work our magic...
            return base.SendAsync(request, cancellationToken).ContinueWith(
                t =>
                {
                    var response = t.Result;

                    // Is this a response we can work with?
                    if (!ResponseIsValid(response)) return response;

                    var pagedResultsValue = this.GetValueFromObjectContent(response.Content);
                    Type queriedType;

                    // Can we find the underlying type of the results?
                    if (pagedResultsValue is IQueryable)
                    {
                        queriedType = ((IQueryable)pagedResultsValue).ElementType;

                        // we need to work with an instance of IRavenQueryable to support statistics
                        var genericQueryableType = typeof(Raven.Client.Linq.IRavenQueryable<>).MakeGenericType(queriedType);

                        if (genericQueryableType.IsInstanceOfType(pagedResultsValue))
                        {
                            Raven.Client.Linq.RavenQueryStatistics stats = null;

                            // register our statistics object with the Raven query provider.
                            // After the query executes, this object will contain the appropriate stats data
                            dynamic dynamicResults = pagedResultsValue;
                            dynamicResults.Statistics(out stats);


                            // Create the return object.
                            var resultsValueMethod =
                                this.GetType().GetMethod(
                                    "CreateResultValue", BindingFlags.Instance | BindingFlags.NonPublic).MakeGenericMethod(
                                        new[] { queriedType });

                            // Create the result value with dynamic type
                            var resultValue = resultsValueMethod.Invoke(
                                this, new[] { stats, pagedResultsValue });

                            // Push the new content and return the response
                            response.Content = CreateObjectContent(
                                resultValue, response.Content.Headers.ContentType);
                            return response;

                        }
                        else
                            return response;
                    }
                    else
                        return response;
               });
        }

        private bool ResponseIsValid(HttpResponseMessage response)
        {
            // Only do work if the response is OK
            if (response == null || response.StatusCode != HttpStatusCode.OK) return false;

            // Only do work if we are an ObjectContent
            return response.Content is ObjectContent;
        }

        private bool ShouldInlineCount(HttpRequestMessage request)
        {
            var queryParams = request.RequestUri.ParseQueryString();
            var inlinecount = queryParams["$inlinecount"];
            return string.Compare(inlinecount, "allpages", true) == 0;
        }

    // Dynamically invoked for the T returned by the resulting ApiController
    private ResultValue<T> CreateResultValue<T>(Raven.Client.Linq.RavenQueryStatistics stats, IQueryable<T> pagedResults)
    {
        var genericType = typeof(ResultValue<>);
        var constructedType = genericType.MakeGenericType(new[] { typeof(T) });

        var ctor = constructedType
            .GetConstructors().First();

        var instance = ctor.Invoke(null);

        var resultsProperty = constructedType.GetProperty("Results");
        resultsProperty.SetValue(instance, pagedResults.ToArray(), null);

        var countProperty = constructedType.GetProperty("Count");
        countProperty.SetValue(instance, stats.TotalResults, null);

        return instance as ResultValue<T>;
    }

        // We need this because ObjectContent's Value property is internal
        private object GetValueFromObjectContent(HttpContent content)
        {
            if (!(content is ObjectContent)) return null;

            var valueProperty = typeof(ObjectContent).GetProperty("Value", BindingFlags.Instance | BindingFlags.NonPublic);
            if (valueProperty == null) return null;

            return valueProperty.GetValue(content, null);
        }

        // We need this because ObjectContent's constructors are internal
        private ObjectContent CreateObjectContent(object value, MediaTypeHeaderValue mthv)
        {
            if (value == null) return null;

            var ctor = typeof(ObjectContent).GetConstructors(BindingFlags.NonPublic | BindingFlags.Instance).FirstOrDefault(
                ci =>
                {
                    var parameters = ci.GetParameters();
                    if (parameters.Length != 3) return false;
                    if (parameters[0].ParameterType != typeof(Type)) return false;
                    if (parameters[1].ParameterType != typeof(object)) return false;
                    if (parameters[2].ParameterType != typeof(MediaTypeHeaderValue)) return false;
                    return true;
                });

            if (ctor == null) return null;

            return ctor.Invoke(new[] { value.GetType(), value, mthv }) as ObjectContent;
        }
    }

    public class ResultValue<T>
    {
        public int Count { get; set; }
        public T[] Results { get; set; }
    }
}

Answer 2

You can wrap the IQueryable and intercept the GetEnumerator. A sample of this is for example here: http://blogs.msdn.com/b/alexj/archive/2010/03/01/tip-55-how-to-extend-an-iqueryable-by-wrapping-it.aspx. It does something a bit different but it should give you the idea.

Also - the caller can use $inlinecount=allpages in the URL to do this using the OData protocol. Although I'm not sure if WebAPI supports this query option yet.