How can I invoke a callback given to a yield in javascript v8 generator code?

Question

I'm attempting to understand javascript generators in node.js v8, without using any third party libraries.

I want to try having a generator that invokes an asynchronous callback (something like a web request) and returns the value that is called back.

My code (using Node v0.11.12) looks like:

var callbackWrapper = function (){
  return function(callback) {
    callback(null, 5);
  };
};

var result = null;
var generator = (function *(){
  result = yield callbackWrapper();
})();

var nextIteration = generator.next();

At the end of this code, nextIteration.value is a function, and result is still null.

It was my understanding that if yield is given a method with a callback, it invokes that callback.

My goal is to somehow get the value, 5, that is called back.

What is the way to do this? Do I have to pass some function to nextIteration.value and invoke it?

Answer 1

Let's clarify some things.

It was my understanding that if yield is given a method with a callback, it invokes that callback.

yield does not invoke anything. The purpose of yield is, well, to yield... When you define a generator you define an entity which yields values. It has nothing to do with async code. We can, however, leverage an important property of generators to handle async code.

Generators, by their definition, yield values. Here's the important part - between one yield to another the generator is suspended. In other words, it waits until it is asked to yield the next value. The internal state of the generator is kept in memory (on the stack) until the generator is exhausted (no more values to yield).

Another important property of generators is that we can send values back, thus changing their internal state.

So if we can suspend generators (make them wait) and we can send values back; we can essentially make them wait for some async operation to complete and then send the result back.

What is the way to do this? Do I have to pass some function to nodeIteration.value and invoke it?

Basically you need to wrap you code with a generator. Every time you start an async operation you make the generator wait by using yield. When the async operation completes you resume your generator by sending the result back with next(result).

I wrote an extensive post to explain this issue, which I'm sure will help you understand. Take a look: http://eyalarubas.com/javascript-generators-and-callbacks.html

Answer 2

Generators don't handle node style callbacks on their own. Instead it's returning the function that you wrapped inside of the callbackWrapper thunk. As yield only returns a value and then pauses execution at that point in time.

Generators weren't really designed for control flow but you can build on top of them to create control flow libraries like co, suspend, etc..

Basically what these libraries do (I'm oversimplifying here), is take your generator function and recursively call it until it tells them that it has finished.

Each of these libraries handles the internal yields in different ways, for example co turns everything it can handle into thunks internally. While suspend uses node-style callbacks for everything internally.

At each yield they check to see what was yielded to them a thunk, promise, generator, or whatever constructs that library handles, and abstracts the control out based on when they are completed.

You can build a structure around generators to handle asynchronous thunked functions but it's not for the feint of heart. Basically it would go something like this (Note: don't use this other than for playing around as its missing all the normal checks, error handling, etc..):

function controlFlow(genFunc){
  // check to make sure we have a generator function otherwise explode

  var generator; // reference for an initialized generator

  // return a funcion so that execution time can be postponed
  return function(){
    runGen(genFunc());
  }

  function runGen(generator){
    var ret = generator.next();

    // here the generator is already finished we we'll return the final value
    if(ret.done){
      return ret.value
    }

    // here we'll handle the yielded value no matter what type it is
    // I'm being naive here for examples sake don't do this
    if(typeof ret.value === 'function'){
      // oh look we have a regular function (very faulty logic)
      // we wouldn't do this either but yeah
      ret.value(function(err, result){
        console.log(result);
      });
    }

   // oh we got another item like an array or object that means parallelism or serialization depending on what we got back
   // turn array, object, whatever into callback mechanisms and then track their completion
   // we're just going to fake it here and just handle the next call
   runGen(generator);
  }
}

function thunked(callback){
  return function(){
    callback(null, 5);
  };
};

function regular(callback){
  console.log('regular function call');
  callback(null, 'value');
};

controlFlow(function *(){
  yield thunked(function(err, result){
    console.log(err);
    console.log(result);
  });
  yield regular;
  yield thunked(function(err, result){
    console.log('Another Thunked');
  });
  yield regular(function(err, result){
    console.log(err);
    console.log(result);
  });
})();