Node.js - module caching, partially done objects, and cyclical dependencies?

Question

In the node.js documentation regarding module caching, the following statement is made:

Multiple calls to require('foo') may not cause the module code to be executed multiple times. This is an important feature. With it, "partially done" objects can be returned, thus allowing transitive dependencies to be loaded even when they would cause cycles.

I'm a bit confused about the last sentence. What is a "partially done" object? How does this relate to allowing (or avoiding) cyclical dependencies?

Answer 1

If you require a package from a file, and that causes a file in that package to require the file that caused the initial require then you have a cyclic dependency. By default, it would just go in circles. In order to prevent this, one can keep a marker where the original require started so that the next time that file is require'd it will start from that point rather than the beginning. It's not flawless, but in the case of loading a package you are generally only interested in the exports, and it works well in that case.

I pushed a diff for node-browserify a while back for a primitive method of "partially done" exports. Basically, each time something is require'd it will check the amount of exports. If there are more exports, it means the package was incomplete the last time, and could still be processing. If there are no new exports (the new and old count are equal), then it means the package is done, and can be cached so that the module code is not executed multiple times. Being that it is in the browser, there's no control over execution flow, and thus the module code would be repeated partially (in steps) until done. Whereas I'm sure Node.js has more elegant handling.

Answer 2

There is a nice and clear example provided in the node.js documentation.It sheds more light on "partially done" object and cyclic dependencies.

When there are circular require() calls, a module might not have finished executing when it is returned.

Consider this situation:

a.js:

console.log('a starting');
exports.done = false;
const b = require('./b.js');
console.log('in a, b.done = %j', b.done);
exports.done = true;
console.log('a done');

b.js:

console.log('b starting');
exports.done = false;
const a = require('./a.js');
console.log('in b, a.done = %j', a.done);
exports.done = true;
console.log('b done');

main.js:

console.log('main starting');
const a = require('./a.js');
const b = require('./b.js');
console.log('in main, a.done=%j, b.done=%j', a.done, b.done);

When main.js loads a.js, then a.js in turn loads b.js. At that point, b.js tries to load a.js. In order to prevent an infinite loop, an unfinished copy of the a.js exports object is returned to the b.js module. b.js then finishes loading, and its exports object is provided to the a.js module.

By the time main.js has loaded both modules, they're both finished. The output of this program would thus be:

node main.js

main starting
a starting
b starting
in b, a.done = false
b done
in a, b.done = true
a done
in main, a.done=true, b.done=true