Next.js bundle size is exploding as a result of dynamic component lookup, how to solve?

Question

tldr:

Check the repo - common.js includes all dependencies, even though, only one is used on the respective page.

• http://localhost:3000/components/ComponentOne
• http://localhost:3000/components/ComponentTwo

Livedemo: click here

More Details:

I have an app (find attached a grossly simplified version) where based on a user input a different component is rendered. Finding the component to be rendered happens via a component-map. It makes sense that the common.js includes all dependencies for the switcher page, where both components have to be accessible (and thus their dependencies). But it does not make sense for the individual pages to include the respectively other dependency.

To summarize:

• I want to be able to have a large group of components, that can are rendered based on a user's input. It is not feasible for my use-case to serialize and deserialize (as shown here) them as the components are wildly different and require different dependencies
• I also want to render every component out to its own statically generated page, where I retrieve additional SEO information from a database. In this case, however, I only want to load the required dependencies for the particular component at hand.

http://localhost:3000

Selecting ComponentOne results in:

Uses recharts.js

Selecting ComponentTwo results in:

Uses victory.js

Answer 1

PROBLEM

TLDR: Next's Webpack configuration is chunking dynamically loaded components as its own chunk, which may create duplicated or combined chunk dependencies.

With your example, I duplicated component 1 and 2 as component 3 and 4 respectively. However, with component 4 (which is a copy of component 2), I added an additional moment-timezone dependency. The result is a separated chunk with duplicated victory-pie dependencies (it also imported the entire library for both victory and moment-timezone packages):

EXPLANATION

Even though there is quite a lot of dependency sharing between the two 3rd party charting packages (mainly both share d3 dependencies), if the components are reusing 3rd party libraries that happen to have shared dependencies and be dynamically loaded across multiple routes, Webpack may attempt to combine these 3rd party chunks into one combined chunk: instead of the expected two or more chunks:

But, as you'll notice in the chunk screenshot right above, even if the 3rd party packages aren't being reused/reimported across multiple routes, you still have duplicated dependencies (for example, both large peach and lime-green chunks in the screenshot above contain duplicated d3-scale, d3-time, d3-path, and so on dependency chunks).

Unfortunately, this is a necessary and expected behavior of a component being imported via next/dynamic (also applies to using Webpack's dynamic import statements) because it must traverse the entire dependency graph for each dynamically imported component and (potentially) add them as their own chunk -- in other words, in the case of a dynamic loaded component, Webpack doesn't know what component is being loaded during runtime, so it must create an entire chunk for it to be able to be loaded upon request (even if other components may share the same dependencies, it won't know). Not only that, since it doesn't know what's being imported/used within the dynamic component, it can't tree-shake dependencies! This, as a result, creates incredibly large and duplicated chunks as you add more dynamically loaded components.

SOLUTION

Unfortunately, there's really no fix. Even when I tried to manually separate and group these dependencies as their own separate chunks (to reduce redundancy/build size), the component would no longer render. Which makes sense, when each component is chunked in a way to be its own separate "app" within the main app.

In this case, the simplest solution would to be render a static image in lieu of a dynamically loaded React component (like a thumbnail for a video).

OTHER THOUGHTS

I took a look into Next's Webpack configuration and was able to make some progress. You can create your own webpack splitChunks rules for Next to use, which will help reduce some chunk redundancy; but, even then, I was still getting duplicate chunks (derived mostly from d3 shared dependencies). You can try it out. Definitely not for the faint of heart as you'll be chasing a rabbit down a dark hole and you won’t achieve chunk distribution perfection. That said, it does help reduce the build size...

Here's some prelimary work to use as a foundation for your next.config.js file:

next.config.js

module.exports = {
  webpack(config, { isServer }) {
    /* adds client-side webpack optimization rules for splitting chunks during build-time */
    if (!isServer) {
      config.optimization.splitChunks.cacheGroups = {
        ...config.optimization.splitChunks.cacheGroups,
        victory: {
          test: /[\\/]node_modules[\\/](victory-pie|victory-core|victory-pie\/es)[\\/]/,
          name: "victory",
          priority: 50,
          reuseExistingChunk: true,
        },
        recharts: {
          test: /[\\/]node_modules[\\/](recharts|recharts-scale)[\\/]/,
          priority: 20,
          name: "recharts",
          reuseExistingChunk: true,
        },
        lodash: {
          test: /[\\/]node_modules[\\/](lodash)[\\/]/,
          name: "lodash",
          reuseExistingChunk: true,
          priority: 40,
        },
      };
    }
    /* return new config to next */
    return config;
  },
};