Pipelined code generation in a webpack plugin

Question

Problem:

I'm trying to write a webpack plugin to integrate a source-code generator into my webpack build. My complete scenario is complex, so I've broken it down into a simpler progression of questions.

Part I:

I have a code generator that generates a %.js file from a %.proto file. For example, with source files foo.proto and bar.proto, I want my plugin to produce the following compilation steps:

                ┌─────────┐
    foo.proto ──┤ codegen ├──> foo.js
                └─────────┘
                ┌─────────┐
    bar.proto ──┤ codegen ├──> bar.js
                └─────────┘

Where am I meant to register this dependency on each %.proto file (for file watching) and declare the produced assets (%.js) on the compilation object?

This scenario could be achieved with a loader by using require('codegen!foo.proto'), but by Part III you'll see why loaders won't be appropriate.

My intention would be expressed in make as:

%.js: %.proto
    codegen $^ $@

Part II:

The generated %.js files emitted by my generator are now in ES6 syntax, so need to be transpiled to ES5. I already have babel-loader configured for transpilation of ES6 source, if that's helpful. Continuing the example, the steps would be:

                ┌─────────┐  ┌───────┐
    foo.proto ──┤ codegen ├──┤ babel ├──> foo.js
                └─────────┘  └───────┘
                ┌─────────┐  ┌───────┐
    bar.proto ──┤ codegen ├──┤ babel ├──> bar.js
                └─────────┘  └───────┘

i.e., I want:

%.js: %.proto
    codegen $^ | babel -o $@

Should I:

• be doing the transpilation within my plugin task, hiding it from the webpack compilation?
• be getting webpack to do the transpilation via creating additional tasks on the compilation object?
• be emitting the generated js in a manner that will allow webpack to transform it through the appropriate loader pipeline it's already using for other source?

Part III:

My generator now takes an additional input file of %.fragment.js. How can I express this dependency on the webpack compilation, such that file watching will rebuild the assets when either %.proto or %.fragment.js is changed? This multi-source dependency is why I don't think loaders are an appropriate direction to head in.

                  ┌─────────┐  ┌───────┐
      foo.proto ──┤ codegen ├──┤ babel ├──> foo.js
foo.fragment.js ──┤         │  │       │
                  └─────────┘  └───────┘
                  ┌─────────┐  ┌───────┐
      bar.proto ──┤ codegen ├──┤ babel ├──> bar.js
bar.fragment.js ──┤         │  │       │
                  └─────────┘  └───────┘

My intention is:

%.js: %.proto %.fragment.js
    codegen $^ | babel -o $@

In this post, I saw a mention of "child compilations". Is there any webpack documentation of what those are or how they're intended to be used?

Or, is this kind of scenario not what webpack is intended to support, even via custom plugins?

Answer 1

Your problem can be solved with loaders. I recommend to read the guidelines before work.

First by prioprity is [loader] do only a single task. So, your loader for proto files will just generate ES6 js file .

---

Q: Where am I meant to register this dependency on each %.proto file (for file watching) and declare the produced assets (%.js) on the compilation object?

A: You should require your proto files in common way (as you described):

require("foo.proto");

and producing additional assets with emitFile function:

emitFile(name: string, content: Buffer|String, sourceMap: {...})

---

Q: Should I emitting the generated js in a manner that will allow webpack to transform it through the appropriate loader pipeline it's already using for other source?

A: Yep, your loader must do only a single task: generate ES6 js file from proto file. And then resulting file will be passed to babel:

{test: /\.proto$/, loader: 'babel-loader!proto-loader'}

---

Q: My generator now takes an additional input file of %.fragment.js. How can I express this dependency on the webpack compilation, such that file watching will rebuild the assets when either %.proto or %.fragment.js is changed?

A: You must mark dependencies with addDependency function (example from docs):

// Loader adding a header
var path = require("path");
module.exports = function(source) {
    this.cacheable();
    var callback = this.async();
    var headerPath = path.resolve("header.js");
    this.addDependency(headerPath);
    fs.readFile(headerPath, "utf-8", function(err, header) {
        if(err) return callback(err);
        callback(null, header + "\n" + source);
    });
};