JavaScript generator-style async

Question

From what I understand, the future style to write async code in JS is to use generators instead of callbacks. At least, or esp. in the V8 / Nodejs community. Is that right? (But that might be debatable and is not my main question here.)

To write async code with generators, I have found a few libraries:

• gen-run (What I'm currently using.)
• co
• task.js
• Galaxy

They all look kind of similar and I'm not that sure which of them to use (or if that even matters). (However, that might again be debatable and is also not my main question here -- but I still would be very happy about any advice.)

(I'm anyway only using pure V8 - if that matters. I'm not using Nodejs but I use pure V8 in my custom C++ app. However, I already have a few node-style elements in my code, including my custom require().)

Now I have some function X written in callback-style, which itself calls other async functions with callback arguments, e.g.:

function X(v, callback) {
    return Y(onGotY);
    function onGotY(err, res) {
        if(err) return callback(err);
        return Z(onGotZ);
    }
    function onGotZ(err, res, resExtended) {
        if(err) return callback(err);
        return callback(null, v + res + resExtended);
    }
}

And I want to turn X into a generator, e.g. I guess function* X(v) { ... }. How would that look like?

Answer 1

I went with my very simple own lib which works quite well for my small V8 environment and also makes it easy to debug because the JS callstack stays intact. To make it work with Nodejs or on the web, it would need some modifications, though.

Rationales here:

For debugging, we don't really want async code - we want to have nice understandable call stacks in debuggers, esp. in the node-inspector debugger. Also note that so far, all async code is completely artificial and we execute everything completely in sync, somewhat emulated via V8 Microtasks. Callback-style async code is already hard to understand in call stacks. Generator-style async code looses the call stack information completely in conventional debuggers - that includes the current Chrome Beta Developer Tools V8 debugger used with node-inspector. That is very much the nature of generators (coroutines in general). Later versions of the debugger might handle that, but that is not the case today. We even need some special C++ code to get the info. Example code can be found here: https://github.com/bjouhier/galaxy-stack/blob/master/src/galaxy-stack.cc

So if we want to have useful debuggers, today, we cannot use generators -- at least not as it is usually done. We still want to use generator-style async code because it makes the code much more readable, compared to callback-style code. We introduce a new function async to overcome this. Imagine the following callback-style async code:

function doSthX(a, b, callback) { ... }

function doSthY(c, callback) {
    doSthX(c/2, 42, function(err, res) {
        if(err) return callback(err);
        callback(null, c + res);
    })
}

Now, the same code with the async function and generator-style code:

function* doSthX(a, b) { ... }

function* doSthY(c) {
    var res = yield async(doSthX(c/2, 42));
    return c + res;
}

We can provide two versions of async(iter):

1. Run the iterator iter on top of the stack. That will keep a sane call stack and everything is run serially.
2. Yield the iterator down to some lower handler and make it really async.

Note that there are a few notable libraries which can be used for the second approach:

https://github.com/visionmedia/co http://taskjs.org/ https://github.com/creationix/gen-run https://github.com/bjouhier/galaxy

For now, we just implement the first approach - to make debugging easier. If we once want to have both, we can introduce a debug flag to switch via both implementations.

global.async = async;
function async(iter) {
    // must be an iterator
    assert(iter.next);

    var gotValue;
    var sendValue;
    while(true) {
        var next = iter.next(sendValue);
        gotValue = next.value;

        if(!next.done) {
            // We expect gotValue as a value returned from this function `async`.
            assert(gotValue.getResult);
            var res = gotValue.getResult();
            sendValue = res;
        }

        if(next.done) break;
    }

    return {
        getResult: function() {
            return gotValue;
        }
    };
}


// Like `async`, but wraps a callback-style function.
global.async_call_cb = async_call_cb;
function async_call_cb(f, thisArg /* , ... */) {
    assert(f.apply && f.call);
    var args = Array.prototype.slice.call(arguments, 2);
    return async((function*() {
        var gotCalled = false;
        var res;

        // This expects that the callback is run on top of the stack.
        // We will get this if we always use the wrapped enqueueMicrotask().
        // If we have to force this somehow else at some point, we could
        // call runMicrotasks() here - or some other waiter function,
        // to wait for our callback.
        args.push(callback);
        f.apply(thisArg, args);

        function callback(err, _res) {
            assert(!gotCalled);
            if(err) throw err;
            gotCalled = true;
            res = _res;
        }

        assert(gotCalled);
        return res;
    })());
}



// get the result synchronously from async
global.sync_from_async = sync_from_async;
function sync_from_async(s) {
    assert(s.getResult); // see async()
    return s.getResult();
}


// creates a node.js callback-style function from async
global.callback_from_async = callback_from_async;
function callback_from_async(s) {
    return function(callback) {
        var res;
        try { res = sync_from_async(s); }
        catch(err) {
            return callback(err);
        }
        return callback(null, res); 
    };
}


global.sync_get = sync_get;
function sync_get(iter) {
    return sync_from_async(async(iter));
}


// this is like in gen-run.
// it's supposed to run the main-function which is expected to be a generator.
// f must be a generator
// returns the result.
global.run = run;
function run(f) {
    return sync_get(f());
}