What was the motivation for introducing a separate microtask queue which the event loop prioritises over the task queue?

Question

My understanding of how asynchronous tasks are scheduled in JS

Please do correct me if I'm wrong about anything:

The JS runtime engine agents are driven by an event loop, which collects any user and other events, enqueuing tasks to handle each callback.

The event loop runs continuously and has the following thought process:

• Is the execution context stack (commonly referred to as the call stack) empty?
• If it is, then insert any microtasks in the microtask queue (or job queue) into the call stack. Keep doing this until the microtask queue is empty.
• If microtask queue is empty, then insert the oldest task from the task queue (or callback queue) into the call stack

So there are two key differences b/w how tasks and microtasks are handled:

• Microtasks (e.g. promises use microtask queue to run their callbacks) are prioritised over tasks (e.g. callbacks from othe web APIs such as setTimeout)
• Additionally, all microtasks are completed before any other event handling or rendering or any other task takes place. Thus, the application environment is basically the same between microtasks.

Promises were introduced in ES6 2015. I assume the microtask queue was also introduced in ES6.

My question

What was the motivation for introducing the microtask queue? Why not just keep using the task queue for promises as well?

Update #1 - I'm looking for a definite historical reason(s) for this change to the spec - i.e. what was the problem it was designed to solve, rather than an opinionated answer about the benefits of the microtask queue.

References:

• In depth: Microtasks and the JavaScript runtime environment
• HTML spec event loop processing model
• Javascript-hard-parts-v2
• loupe - Visualisation tool to understand JavaScript's call stack/event loop/callback queue interaction
• Using microtasks in JavaScript with queueMicrotask()

Answer 1

Promises were introduced in ES6 2015. I assume the microtask queue was also introduced in ES6.

Actually the microtask task queue was not introduced by ECMAScript standards at all: the ES6 standard specified putting promise handling jobs for a settled promise in a queue named "PromiseJobs" under TriggerPromiseReactions, using the abstract process EnqueueJob to enter the job in a job queue implemented by the host environment, without prescribing how the host queue should be handled.

Prior to adoption by ECMAScript

Promise libraries were developed in user land. The bit of code that executed promise handlers, monitored if they threw or returned a value and had access to the resolve and reject functions of the next promise in a promise chain was called the "trampoline". While part of the Promise library, the trampoline was not considered part of user code and the claim of calling promise handlers with a clean stack excluded stack space occupied by the trampoline.

Settlement of a promises with a list of handlers to call for the settled status (fulfilled or rejected) required starting the trampoline to run promise jobs if it were not already running.

The means of starting trampoline execution with an empty stack was limited to existing Browser APIs including setTimeout, setImmediate and the Mutation Observer API. The Mutation Observer uses the microtask queue and may be the reason for its introduction (not sure of the exact browser history).

Of the event loop interfacing possibilities, setImmediate was never implemented by Mozilla at least, Mutation Observers were available in IE11 according to MDN, and setTimeout under some circumstances would be throttled so it would take at least some milliseconds to execute a callback even if the delay time were set to zero.

Developer Competition

To an outside observer promise library developers competed with each other to see who could come up with the fastest time to begin executing a promise handler after promise settlement.

This saw the introduction of setImmediate polyfills which picked the fastest strategy of starting a callback to the trampoline from the event loop depending on what was available in the browser. YuzuJS / setImmediate on GitHub is a prime example of such a polyfill and its readme well worth reading.

History After adoption in ECMAScript 2015

Promises were included in ES6 without specifying the priority host implementations should give to promise jobs.

The author of the YuzuJS/setImmediate polyfill above also made a submission to the TC39 committee to specify that promise jobs should be given high priority in ECMAScript. The submission was ultimately rejected as an implementation issue not belonging to the language standard. Arguments supporting the submission are unavailable on TC39's tracking site given it doesn't reference rejected proposals.

Subsequently the HTML5 specification introduced rules for Promise implementation in browsers. The section on how to implement ECMAScipt's EnqueueJob abstract operation in host browsers specifies that they go in the microtask queue.

---

Answer

What was the motivation for introducing the microtask queue? Why not just keep using the task queue for promises as well?

• The micro task queue was introduced to support Mutation Observer Events, as detailed by Jake Archibald at JSConf Asia 2018 ( 24:07)¹ during his "In the loop" presentation.

• Early developers of promise libraries found ways to enter jobs in the micro task queue and in doing so minimized the time between settling promises and running their promise reaction jobs. To some extent this created competition between developers but also facilitated continuing asynchronous program operation as soon as possible after handling completion of one step in a sequence of asynchronous operations.

• By design fulfillment and rejection handlers can be added to promises that have already been settled. If such cases there is no need to wait for something to happen before proceeding to the next step of a promise chain. Using the microtask queue here means the next promise handler is executed asynchronously, with a clean stack, more or less immediately.

Ultimately the decision to specify the microtask queue was made by prominent developers and corporations based on their expert opinion. While that may be excellent choice, the absolute necessity of doing so is moot.

---

See also Using microtasks in JavaScript with queueMicrotask() on MDN.

---

¹ Thanks to @Minh Nghĩa 's comment for the link to Jake Archibald's "In the Loop" (0:00) talk - ☆☆☆☆☆. Highlights include

• An event loop executes one task from the task queue at a time, all tasks in the animation queue except for tasks added while executing the queue, and all tasks in the microtask queue until it's empty.
• Dependence on tricky execution order of event handlers and promise callbacks can cause unit testing failures because events dispatched programatically execute event handlers synchronously, not via the event loop.

Answer 2

One advantage is fewer possible differences in observable behavior between implementations.

If these queues weren't categorized, then there would be undefined behavior when determining how to order a setTimeout(..., 0) callback vs. a promise.then(...) callback strictly according to the specification.

I would argue that the choice of categorizing these queues into microtasks and "macro" tasks decreases the kinds of bugs possible due to race conditions in asynchronicity.

This benefit appeals particularly to JavaScript library developers, whose goal is generally to produce highly optimized code while maintaining consistent observable behavior across engines.