Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Dynamic loading of external modules in webpack fails

I am trying to set up the following architecture: a core React application that gets built with some basic functionality, and the ability to load additional React components at runtime. These additional React components can be loaded on-demand, and they are not available at build time for the core application (so they cannot be included in the bundles for the core application, and must be built separately). After researching for some time, I came across Webpack Externals, which seemed like a good fit. I am now building my modules separately using the following webpack.config.js:

const path = require('path');
const fs = require('fs');

process.env.BABEL_ENV = 'production';
process.env.NODE_ENV = 'production';

const appDirectory = fs.realpathSync(process.cwd());
const resolveApp = relativePath => path.resolve(appDirectory, relativePath);

module.exports = {
  entry: './src/MyModule.jsx',
  output: {
    path: path.resolve(__dirname, 'dist'),
    filename: 'MyModule.js',
    library: 'MyModule',
    libraryTarget: 'umd'
  },
   externals: {
    "react": "react",
    "semantic-ui-react": "semantic-ui-react"
   },
   module: {
    rules: [
        {
            test: /\.(js|jsx|mjs)$/,
            include: resolveApp('src'),
            loader: require.resolve('babel-loader'),
            options: {              
              compact: true,
            },
        }
    ]
  },
  resolve: {
    extensions: ['.wasm', '.mjs', '.js', '.json', '.jsx']
  }
};

Took a look at the generated MyModule.js file, and it looks correct to me.

Now, in my core app, I am importing the module as follows:

let myComponent = React.lazy(() => import(componentName + '.js'));

where componentName is the variable that matches the name of my module, in this case, "MyModule" The name is not known at build time, and the file is not present in the src folder at build time. To avoid errors from webpack when building this code with an unknown import, I have added the following to my webpack.config.js for the core project:

module.exports = {
    externals: function (context, request, callback/*(err, result)*/) {
        if (request === './MyModule.js') {
            callback(null, "umd " + request);
        } else {
            callback();
        }
    }
}

I have confirmed that the function in externals gets called during the build, and the if condition is matched for this module. The build succeeds, and I am able to run my core application.

Then, to test dynamic loading, I drop MyModule.js into the static/js folder where the bundles for my core app live, then I navigate to the page in my core app that requests MyModule via let myComponent = React.lazy(() => import(componentName + '.js'));

I see a runtime error in the console on the import line,

TypeError: undefined is not a function
    at Array.map (<anonymous>)
    at webpackAsyncContext 

My guess is it's failing to find the module. I don't understand where it is looking for the module, or how to get more information to debug.

like image 231
SlyCaptainFlint Avatar asked Jun 20 '19 17:06

SlyCaptainFlint


People also ask

How does webpack handle dynamic imports?

Whenever you import a file in your code, Webpack will look for it in the project directory and copy it to the build folder with a new name, then Webpack replaces the import code in your bundled JavaScript file with the path to the newly copied file.

How does webpack externals work?

Webpack externals tell Webpack to exclude a certain import from the bundle. Often externals are used to exclude imports that will be loaded via CDN. For example, suppose you are implementing server-side rendering with Vue and Express, but your client-side code imports Vue via a CDN.

Does webpack use CommonJS?

As you may know, webpack supports a couple of module types out of the box, including both CommonJS and ES modules. Webpack also works on both client- and server-side JavaScript, so with webpack, we can also easily handle assets and resources like images, fonts, stylesheets, and so on.

What is chunking in webpack?

Chunk: This webpack-specific term is used internally to manage the bundling process. Bundles are composed out of chunks, of which there are several types (e.g. entry and child).


1 Answers

Turns out that I was making a couple of incorrect assumptions about webpack and dynamic loading.

I was having issues with two things - the kind of module I was loading, and the way that I was loading it.

  1. Dynamic importing is not yet a standard ES feature - it is due to be standardized in ES 2020. This dynamic import will only return a module if the module object you are attempting to load is an ES6 module (aka something that contains an 'export ModuleName'). If you attempt to load something packed up as a CommonJS module, AMD, UMD, the import will succeed, but you will get an empty object. Webpack does not appear to support creating bundles in ES6 format - it can create a variety of module types, and in my config file above, I was actually creating UMD modules (configured via libraryTarget setting).

  2. I had issues with the import statement itself because I was using it within an app bundled by Webpack. Webpack reinterprets the standard ES import statement. Within a standard webpack config (including the one you get from CRA), webpack uses this statement as a split point for bundles, so even modules that are dynamically imported are expected to be there at webpack build time (and the build process will fail if they are not available). I had tried to use webpack externals to tell webpack to load the modules dynamically, which allowed the build to succeed without the modules being there. However, the app still used Webpack's import function instead of the standard JS import function at runtime. I confirmed this by attempting to run import('modulename') from the browser console and getting a different result than my app, which was bundled with webpack.

To solve problem #2, you can tell Webpack to not reinterpret the ES dynamic import by adding some annotation to the import statement.

import(/*webpackIgnore: true*/ 'path/to/module.js');

This will both prevent Webpack from attempting to find and bundle the dynamically imported module at build time, and attempting to import it at runtime. This will make behavior in the app match behavior in the browser console.

Problem #1 was a bit more difficult to solve. As I mentioned above, importing a non-ES6 module will return an empty object (if you await the promise or use .then()). However, as it turns out, the file itself does load and the code gets executed. You can export the module in the "window" format using Webpack, and then load it as follows.

await import(/*webpackIgnore: true*/`path/to/module.js`);
let myModule = window['module'].default;

Another potential solution that avoids using the window object is building the module using a system capable of producing ES6 modules (so, not Webpack). I ended up using Rollup to create an ES6 module that pulled all dependencies into a single file, and ran the output through Babel. This produced a module that loaded successfully via a dynamic ES import. The following was my rollup.config.js (note that I included all external node modules needed in my module - this bloated the module size but is a requirement for my specific application - yours will likely differ and you will need to configure rollup to exclude the modules)

// node-resolve will resolve all the node dependencies
import commonjs from 'rollup-plugin-commonjs';
import resolve from 'rollup-plugin-node-resolve';
import babel from 'rollup-plugin-babel';
import replace from 'rollup-plugin-replace';

export default {
  input: 'src/myModule.jsx',
  output: {
    file: 'dist/bundle.js',
    format: 'esm'
  },
  plugins: [
    resolve(),
    babel({
      exclude: 'node_modules/**'
    }),
    commonjs({
      include: 'node_modules/**',      
      namedExports: {
        'node_modules/react/index.js': ['Children', 'Component', 'PropTypes',   'PureComponent', 'React', 'createElement', 'createRef', 'isValidElement', 'cloneElement', 'Fragment'],
        'node_modules/react-dom/index.js': ['render', 'createElement', 'findDOMNode', 'createPortal'],
        'node_modules/react-is/index.js': ['isForwardRef']
      }
    }),
    replace({
      'process.env.NODE_ENV': JSON.stringify( 'production' )
    })
  ]
}
like image 53
SlyCaptainFlint Avatar answered Oct 02 '22 15:10

SlyCaptainFlint