Chaining difference in ES6 Promises and PEP3148 Futures

Question

I am somewhat puzzled about reasoning in difference in implementation of ES6 Promises and PEP3148 Futures. In Javascript, when Promise is resolved with another Promise, "outer" promise inherits the value of "inner" promise once it's resolved or rejected. In Python, "outer" future is instead immediately resolved with "inner" future itself, not with it's eventual value, and that is the problem.

To illustrate this, I had provided two code snippets for both platforms. In Python, code looks like this:

import asyncio

async def foo():
    return asyncio.sleep(delay=2, result=42)

async def bar():
    return foo()

async def main():
    print(await bar())

asyncio.get_event_loop().run_until_complete(main())

In Javascript, fully equivalent code is this:

function sleep(delay, result) {
    return new Promise((resolve, reject) => {
        setTimeout(() => {
            resolve(result);
        }, delay * 1000);
    });
}

async function foo() {
    return sleep(2, 42);
}

async function bar() {
    return foo();
}

(async function main() {
    console.log(await bar());
})();

sleep function provided for the sake of completeness.

Javascript code prints 42, as expected. Python code prints <coroutine object foo at 0x102a05678> and complaints about "coroutine 'foo' was never awaited".

This way, JS allows you to choose the point when the control will be passed away of your current context of execution, by immediately awaiting promises, or letting caller to await them. Python literally leaves you no other option than always await the Future/coroutine, because otherwise you will have to unwind the Future chain in a loop yourself with an ugly wrapper function like this:

async def unwind(value):
    while hasattr(value, '__await__'):
        value = await value

    return value

So, the question: Is there any reasoning behind this decision? Why Python disallows chained Futures? Were there any discussions about it? Is there anything that can be done to make behavior match Promises closer?

Answer 1

Let me show a quick comparison of JavaScript's Promises, and Python's futures, where I can point out the main usecases and reveal the reasons behind decisions.

I will use the following dummy example to demonstrate the use of async functions:

async function concatNamesById(id1, id2) {
  return (await getNameById(id1)) + ', ' + (await getNameById(id2));
}

Asynchronous JavaScript

Back in the day, before the concept of Promises emerged, people wrote their code using callbacks. There are still various conventions about which argument should be the callback, how errors should be handled, etc... At the end, our function looks something like this:

// using callbacks
function concatNamesById(id1, id2, callback) {
    getNameById(id1, function(err, name1) {
        if (err) {
            callback(err);
        } else {
            getNameById(id2, function(err, name2) {
                if (err) {
                    callback(err);
                } else {
                    callback(null, name1 + ', ' + name2);
                }
            });
        }
    });
}

This does the same as the example, and yes, I used 4 indentation spaces intentionally to magnify the problem with the so called callback hell or pyramid of doom. People using JavaScript were writing code like this for many years!

Then Kris Kowal came with his flaming Q library and saved the disappointed JavaScript community by introducing the concept of Promises. The name is intentionally not "future" or "task". The main goal of the Promise concept is to get rid of the pyramid. To achieve this promises have a then method which not only allows you to subscribe to the event that is fired when the promised value is obtained, but will also return another promise, allowing chaining. That is what make Promises and futures a different concept. Promises are a bit more.

// using chained promises
function concatNamesById(id1, id2) {
  var name1;
  return getNameById(id1).then(function(temp) {
    name1 = temp;
    return getNameById(id2); // Here we return a promise from 'then'
  }) // this then returns a new promise, resolving to 'getNameById(id2)', allows chaining
  .then(function(name2) {    
    return name1 + ', ' + name2; // Here we return an immediate value from then
  }); // the final then also returns a promise, which is ultimately returned
}

See? It is essential to unwrap promises returned from then callbacks to build a clean, transparent chain. (I myself wrote this kind of asynchronouos code for over a year.) However, things get complicated when you need some control flow like conditional branching or loops. When the first compilers/transpilers (like 6to5) for ES6 appeared, people slowly started to use generators. ES6 generators are two directional, meaning the generator not only produces values, but can receive supplied values on each iteration. This allowed us to write the following code:

// using generators and promises
const concatNamesById = Q.async(function*(id1, id2) {
  return (yield getNameById(id1)) + ', ' + (yield getNameById(id2));
});

Still using promises, Q.async makes an async function from a generator. There is no black magic there, this wrapper function is implemented using nothing but promise.then (more or less). We are nearly there.

Today, since the ES7 specification for async-await is pretty mature, anyone can compile asynchronous ES7 code to ES5 using BabelJS.

// using real async-await
async function concatNamesById(id1, id2) {
  return (await getNameById(id1)) + ', ' + (await getNameById(id2));
}

Returning promise from an async function

So this works:

async foo() {
  return /* await */ sleep('bar', 1000);
  // No await is needed!
}

and so does this:

async foo() {
  return await await await 'bar';
  // You can write await pretty much everywhere you want!
}

This kind of weak/dynamic/duck typing fits well in JavaScript's view of the world.

You are right that you can return a promise from an async function without await, and it gets unwinded. It is not really a decision, but is a direct consequence of how promise.then works, as it unwinds the promises to make chaining comfortable. Still, I think it is a good practice to write await before every async call to make it clear you are aware of the call being asynchronous. We do have multiple bugs every day because of missing await keywords, as they will not cause instantaneous errors, just a bunch of random tasks running in parallel. I love debugging them. Seriously.

Asynchronous Python

Let's see what Python people did before async-await coroutines were introduced in python:

def concatNamesById(id1, id2):
  return getNameById(id1) + ', ' + getNameById(id2);

Wait what? Where are the futures? Where is the callback pyramid? The point is that Python people do not have any of the problems JavaScript people had. They just use blocking calls.

The big difference between JavaScript and Python

So why didn't JavaScript people use blocking calls? Because they couldn't! Well, they wanted to. Believe me. Before they introduced WebWorkers all JavaScript code ran on the gui thread, and any blocking call caused the ui to freeze! That is undesireable, so people writing the specifications did everything to prevent such things. As of today, the only ways to block the UI thread in a browser I am aware of:

• Using XMLHttpRequest with the deprecated async = false option
• Using a spin-wait
• (Or do actual heavy computation)

Currently you cannot implement spin-locks and similar things in JavaScript, there is just no way. (Until browser vendors start to implement things like Shared Array Buffers, which I am afraid of will bring a special kind of hell upon us as soon as enthusiast amateurs will start to use them)

On the other hand there is nothing wrong with blocking calls in Python as usually there is no such thing as a 'gui thread'. If you still need some parallelism, you can start a new thread, and work on that. This is useful when you want to run multiple SOAP requests at once, but not as useful when you want to utilize the computational power of all the cpu cores in your laptop, as the Global Interpreter Lock will prevent you to do so. (This was worked around by the multiprocessing module, but that's another story)

So, why do Python people need coroutines? The main answer is that reactive programming got really popular nowadays. There are of course other aspects, like not wanting to start a new thread for every restful query you make, (some Python libraries are known to leak thread ids until they eventually crash) or just wanting to get rid of all the unnecessary multithreading primitives like mutexes and semaphores. (I mean those primitives can be omitted if your code can be rewritten to coroutines. Sure they are needed when you do real multithreading.) And that is why futures were developed.

Python's futures do not allow chaining in any form. They are not intended to be used that way. Remember, JavaScript's promises were to change the pyramid scheme to a nice chain scheme, so unwinding was necessary. But automatic unwinding needs specific code to be written, and needs the future resolution to make distinction between supplied values by their types or properties. That is, it would be more complex (=== harder to debug), and would be a little step towards weak typing, which is against the main principles of python. Python's futures are lightweight, clean and easy to understand. They just don't need automatic unwinding.