Bundling API requests

Question

I am creating a REST API and I have been playing with the idea of allowing bundling of requests from clients. By bundling I mean they can send one request, containing multiple "real" requests, and they get delivered to the client together. Typically javascript ajax requests. Something like this:

POST /bundlerequest

["/person/3243", "/person/3243/friends", "/comments/3243?pagesize=10&page=1", "/products", "/product/categories" ] 

(The bundled request can only be GET requests, as of now at least) This is intended to return something like this

{
    "success" : ["/person/3243", "/person/3243/friends", "/comments/3243?pagesize=10&page=1", "/products", "/product/categories" ],
    "error" : [],
    "completiontime" : 94,
    other relevant metadata...
    "responses" : [
        {"key" : "/person/3243" , "data" : {"name" : "John", ...} },
        {"key" : "/person/3243/friends" , "data" : [{"name": "Peter", "commonfriends" : 5, ...}] },
        etc...
    ]
}

The benefits of this bundling is that it reduces the number of requests and that is especially important on mobile devices for instance.

So my first question is, is my approach to this a good one? Does anyone have experience with doing something like this?

AFAIK the common way of solving this is to write server side code to return combined data, that I believe is relevant for the client(s). (The twitter user stream for instance does this, combining person info, latest tweets, latest personal messages etc.) But this makes the API very opinionated and when the client needs changes the server might need to change to accomodate to optimize.

And the second question is how to implement this?

My backend is ASP.NET MVC 3 and IIS 7. Should I implement it in the application, having an bundlerequest action that internally calls the other actions specified in the request?

Could it be implemented in IIS 7 directly? Writing a module that transparently intercepts requests to /bundlerequest and then calls all the corresponding sub requests, making the application totally unaware of the bundling that happens? This would also allow me to implement this in an application-agnostic way.

Answer 1

You could use an asynchronous controller to aggregate those requests on the server. Let's first start by defining a view model that will be returned by the controller:

public class BundleRequest
{
    public string[] Urls { get; set; }
}

public class BundleResponse
{
    public IList<string> Success { get; set; }
    public IList<string> Error { get; set; }
    public IList<Response> Responses { get; set; }
}

public class Response
{
    public string Key { get; set; }
    public object Data { get; set; }
}

then the controller:

public class BundleController : AsyncController
{
    public void IndexAsync(BundleRequest request)
    {
        AsyncManager.OutstandingOperations.Increment();
        var tasks = request.Urls.Select(url =>
        {
            var r = WebRequest.Create(url);
            return Task.Factory.FromAsync<WebResponse>(r.BeginGetResponse, r.EndGetResponse, url);
        }).ToArray();

        Task.Factory.ContinueWhenAll(tasks, completedTasks =>
        {
            var bundleResponse = new BundleResponse
            {
                Success = new List<string>(),
                Error = new List<string>(),
                Responses = new List<Response>()
            };
            foreach (var task in completedTasks)
            {
                var url = task.AsyncState as string;
                if (task.Exception == null)
                {
                    using (var response = task.Result)
                    using (var stream = response.GetResponseStream())
                    using (var reader = new StreamReader(stream))
                    {
                        bundleResponse.Success.Add(url);
                        bundleResponse.Responses.Add(new Response
                        {
                            Key = url,
                            Data = new JavaScriptSerializer().DeserializeObject(reader.ReadToEnd())
                        });
                    }
                }
                else
                {
                    bundleResponse.Error.Add(url);
                }
            }
            AsyncManager.Parameters["response"] = bundleResponse;
            AsyncManager.OutstandingOperations.Decrement();
        });
    }

    public ActionResult IndexCompleted(BundleResponse response)
    {
        return Json(response, JsonRequestBehavior.AllowGet);
    }
}

and now we can invoke it:

var urls = [ 
    '@Url.Action("index", "person", new { id = 3243 }, Request.Url.Scheme, Request.Url.Host)', 
    '@Url.Action("friends", "person", new { id = 3243 }, Request.Url.Scheme, Request.Url.Host)', 
    '@Url.Action("index", "comments", new { id = 3243, pagesize = 10, page = 1 }, Request.Url.Scheme, Request.Url.Host)',
    '@Url.Action("index", "products", null, Request.Url.Scheme, Request.Url.Host)', 
    '@Url.Action("categories", "product", null, Request.Url.Scheme, Request.Url.Host)' 
];
$.ajax({
    url: '@Url.Action("Index", "Bundle")',
    type: 'POST',
    contentType: 'application/json; charset=utf-8',
    data: JSON.stringify(urls),
    success: function(bundleResponse) {
        // TODO: do something with the response
    }
});

Of course some tweaking might be necessary to adapt this to your specific needs. For example you mentioned sending AJAX requests with session expired which might redirect to the Logon page and thus not capturing the error. That's indeed a PITA in ASP.NET. Phil Haack blogged a possible way to circumvent this undesired behavior in a RESTful manner. You just need to add a custom header to the requests.