Designing a fluent Javascript interface to abstract away the asynchronous nature of AJAX

Question

How would I design an API to hide the asynchronous nature of AJAX and HTTP requests, or basically delay it to provide a fluent interface. To show an example from Twitter's new Anywhere API:

// get @ded's first 20 statuses, filter only the tweets that
// mention photography, and render each into an HTML element
T.User.find('ded').timeline().first(20).filter(filterer).each(function(status) {
    $('div#tweets').append('<p>' + status.text + '</p>');
});

function filterer(status) {
    return status.text.match(/photography/);
}

vs this (asynchronous nature of each call is clearly visible)

T.User.find('ded', function(user) {
    user.timeline(function(statuses) {
        statuses.first(20).filter(filterer).each(function(status) {
            $('div#tweets').append('<p>' + status.text + '</p>');
        });
    });
});

function filterer(status) {
    return status.text.match(/photography/);
}

It finds the user, gets their tweet timeline, filters only the first 20 tweets, applies a custom filter, and ultimately uses the callback function to process each tweet.

I am guessing that a well designed API like this should work like a query builder (think ORMs) where each function call builds the query (HTTP URL in this case), until it hits a looping function such as each/map/etc., the HTTP call is made and the passed in function becomes the callback.

An easy development route would be to make each AJAX call synchronous, but that's probably not the best solution. I am interested in figuring out a way to make it asynchronous, and still hide the asynchronous nature of AJAX.

Answer 1

Give a look to the following article published just a couple of days ago by Dustin Diaz, Twitter Engineer on @anywhere:

• Asynchronous method queue chaining in JavaScript

He talks about a really nice technique that allows you to implement a fluent interface on asynchronous methods, basically methods chained together independent of a callback, using a really simple Queue implementation.

Answer 2

I'm developing FutureJS which was originally based on Crockford's promises (original slides). The current goal is to be the Async Toolbox of JavaScript and eliminate chaining clutter.

Futures.chainify(providers, consumers, context, params)

Asynchronous method queueing allows you to chain actions on data which may or may not be readily available. This is how Twitter's @Anywhere api works.

You might want a model which remotely fetches data in this fashion:

Contacts.all(params).randomize().limit(10).display();
Contacts.one(id, params).display();

Which could be implemented like so:

var Contacts = Futures.chainify({
  // Providers must be promisables
  all: function(params) {
    var p = Futures.promise();
    $.ajaxSetup({ error: p.smash });
    $.getJSON('http://graph.facebook.com/me/friends', params, p.fulfill);
    $.ajaxSetup({ error: undefined });
    return p.passable();
  },
  one: function(id, params) {
    var p = Futures.promise();
    $.ajaxSetup({ error: p.smash });
    $.getJSON('http://graph.facebook.com/' + id, params, p.fulfill);
    $.ajaxSetup({ error: undefined });
    return p.passable();
  }
},{
  // Consumers will be called in synchronous order
  // with the `lastResult` of the previous provider or consumer.
  // They should return either lastResult or a promise
  randomize: function(data, params) {
    data.sort(function(){ return Math.round(Math.random())-0.5); // Underscore.js
    return Futures.promise(data); // Promise rename to `immediate`
  },
  limit: function(data, n, params) {
    data = data.first(n);
    return Futures.promise(data);
  },
  display: function(data, params) {
    $('#friend-area').render(directive, data); // jQuery+PURE
    // always return the data, even if you don't modify it!
    // otherwise your results could be unexpected
    return data;
  }
});

Things to know:

• providers - promisables which return data
• consumers - functions which use and or change data
  • the first argument must be data
  • when returning a promisable the next method in the chain will not execute until the promise is fulfilled
  • when returning a "literal object" the next method in the chain will use that object
  • when returning undefined (or not returning anything) the next method in the chain will use the defined object
• context - apply()d to each provider and consumer, thus becoming the this object
• params - reserved for future use

Alternatively you could use synchronous callback chaining - what you may have seen elsewhere as chain().next() or then():

Futures.sequence(function(callback) {

    $.getJSON("http://example.com", {}, callback);

}).then(function(callback, result, i, arr) {

    var data = transform_result(result);
    $.getJSON("http://example.com", data, callback);

}).then(...)

I named it sequence rather than chain since _.js already has a method named chain and I'd like to use _.methodName for my library as well.

Take a peek and let me know what you think.

FuturesJS will work alongside jQuery, Dojo, etc without issue. There are no dependencies. It will work with Node.js (and Rhino when using env.js).

=8^D

P.S. As to the ORM / MVC fix - you can check out JavaScriptMVC and SproutCore. I'm also working on my own solution called TriforceJS, but I don't have anything ready for release yet.

P.P.S Example of promisables

var doStuff = function (httpResult) {
    // do stuff
  },
  doMoreStuff = function (httpResult) {
    // do more stuff
  };

function fetchRemoteData(params) {
  var promise = Futures.promise();
  $.getJSON("www.example.com", params, promise.fulfill, 'jsonp');
  return promise;
}

p = fetchRemoteData(params);
p.when(doStuff);
p.when(doMoreStuff);