[Community thread] Compilation performance optimization tips & tricks (please share yours)

[kaelig]

While setting up Storybook on a pretty large TypeScript codebase (the Shopify admin), I found that some configuration tweaks led to significantly better development performance (both build times & incremental compilation times).

Finding these optimizations tweaks was a rather painful process, so I thought I'd share them back in case it could be useful to other people.

Please consider sharing your optimization tricks in the comments below, and help the community get faster Storybooks.

🙏❤

---

Our config (and build performance telemetry plugin)

Here's our Stack: React 16, Webpack 4, TypeScript (compiled using Babel), GraphQL, Apollo, and a few big-ish decorators including Polaris React's theme & localization providers.

At the time of posting this, loading our Storybook downloads ~30MB of assets, for the most part JS bundles.

To make it easy to benchmark the effect of the optimizations, I keep all of them behind the flag enableExperimentalPerformanceTweaks, and created a TelemetryPlugin that reports a few metrics into a dashboard, so I can track possible performance regressions/improvements when updating Storybook, or as the number of stories increases.

Here are all of the optimization tweaks in our config, annotated with the performance benefit they bring. Your mileage may vary, and some optimizations will result in lack of certain functionality (worth it for us). Feedback and questions are welcome!

Setup

// Experimental performance tweaks are enabled by default.
// If you run into any Storybook build issues, set this to `false`
// before opening an issue.
const enableExperimentalPerformanceTweaks = true;

// MASSIVE performance impact
// Choose which React docgen processor to run:
//
// - false (default): fastest, types aren't parsed
// - 'react-docgen-typescript': slow, but accurate
// - 'react-docgen': fast, but useless for our codebase
//
// Read more: https://github.com/storybookjs/storybook/blob/013f2658e30ea25d1e10b5823cead0eb8329f4c3/addons/docs/react/README.md#typescript-props-with-react-docgen
// Override using STORYBOOK_REACT_DOCGEN (useful for Chromatic builds):
// $ STORYBOOK_REACT_DOCGEN=react-docgen-typescript yarn run storybook
const reactDocgen = process.env.STORYBOOK_REACT_DOCGEN || false;

// You can help make Storybook faster. Here are some tips to
// profile and debug performance issues:
//
// To log the webpack config into a `webpackconfig.log` file:
//
// $ yarn run storybook --debug-webpack > webpackconfig.log
// $ ctrl + c (to exit)
//
// To analyze the bundle, run:
// $ BUNDLE_ANALYZER=true yarn run storybook

Bundle splitting

If you have a large amount of dependencies, you can easily end up with very large >10mb bundles. Large bundles are slower to load than multiple smaller bundles (especially when developing in GitHub Codespaces or equivalent), so this helps us parallelize downloads and save some load time.

We apply this setting both to the manager and preview (iframe) using config.optimization = { splitChunks };.

// Split bundles into smaller ones, to avoid having two >10MB bundles
const splitChunks = {
  chunks: "all",
  maxInitialRequests: Infinity,
  minSize: 0,
  cacheGroups: {
    default: false,
    storiesGroup: {
      test: /\.stories\./,
    },
    axeVendor: {
      test: /[\\/]node_modules[\\/](axe-core)[\\/]/,
    },
    coreVendor: {
      test: /[\\/]node_modules[\\/](core-js)[\\/]/,
    },
    storybookVendor: {
      test: /[\\/]node_modules[\\/](@storybook|@emotion|@mdx-js|overlayscrollbars|polished|@popperjs)[\\/]/,
    },
    babelVendor: {
      test: /[\\/]node_modules[\\/]@?babel/,
    },
    reactVendor: {
      test: /[\\/]node_modules[\\/](react|react-dom)[\\/]/,
    },
    lodashVendor: {
      test: /[\\/]node_modules[\\/](lodash)[\\/]/,
    },
    vendor: {
      test: /[\\/]node_modules[\\/](!react)(!react-dom)(!lodash)(!@babel)(!babel)(!axe-core)(!@storybook)(!@emotion)(!@mdx-js)(!overlayscrollbars)(!polished)(!@popperjs)(!core-js)/,
    },
  },
};

General settings

module.exports = {
  // When adding directories to this setting, be as specific as possible.
  // Wildcard globbing patterns negatively impact incremental build times,
  // as webpack’s watcher has to cover more ground.
  stories: [
    // Global stories, such as icons
    "../app/stories/**/*.stories.@(mdx|tsx)",
  ],
  addons: [
    // your addons here
  ],
  typescript: {
    reactDocgen: reactDocgen,
  },

Preview (iframe) webpack config

  webpackFinal: async (config) => {
    if (process.env.BUNDLE_ANALYZER) {
      const BundleAnalyzerPlugin = require("webpack-bundle-analyzer")
        .BundleAnalyzerPlugin;
      config.plugins.unshift(
        new BundleAnalyzerPlugin({ generateStatsFile: true })
      );
    }

    // These performance tweaks aren’t needed for Storybook to run in web.
    // They may present tradeoffs, but in aggregate, make developing in Storybook much faster and more pleasant.
    if (enableExperimentalPerformanceTweaks) {
      // MEDIUM performance impact
      // Disable ProgressPlugin
      config.plugins = config.plugins.filter(
        (plugin) => plugin.constructor.name !== "ProgressPlugin"
      );

      // MEDIUM performance impact
      // Disable sourcemaps
      config.devtool = false;

      // NEGLIGIBLE performance impact
      // CaseSensitivePathsPlugin can be hungry on large codebases
      // let's disable it, it's not so helpful when iterating on the UI.
      config.plugins = config.plugins.filter(
        (plugin) => plugin.constructor.name !== "CaseSensitivePathsPlugin"
      );

      // NEGLIGIBLE performance impact
      // Tracking missing node modules may not be that useful
      config.plugins = config.plugins.filter(
        (plugin) => plugin.constructor.name !== "WatchMissingNodeModulesPlugin"
      );

      // General best practices for large codebases
      config.optimization = {
        ...config.optimization,
        removeAvailableModules: true,
        removeEmptyChunks: true,
        splitChunks,
      };
      config.output.pathinfo = false;

      // This breaks `yarn link`
      config.resolve.symlinks = false;
    }

    // Here's how we track initial and incremental build times
    // to keep an eye on development speed in Storybook (see plugin source below)
    config.plugins.push(new TelemetryPlugin());

    return config;
  }

Manager webpack config

  managerWebpack: async (config) => {
    if (enableExperimentalPerformanceTweaks) {
      // Makes the vendor bundle smaller by ~30%
      // and somehow doesn't make the initial build slower
      config.mode = "production";

      // NEGLIGIBLE performance impact
      // CaseSensitivePathsPlugin can be hungry on large codebases
      // let's disable it, it's not so helpful when iterating on the UI.
      config.plugins = config.plugins.filter(
        (plugin) => plugin.constructor.name !== "CaseSensitivePathsPlugin"
      );

      // General best practices for large codebases
      config.optimization = {
        ...config.optimization,
        removeAvailableModules: true,
        removeEmptyChunks: true,
        splitChunks,
      };
      config.output.pathinfo = false;

      // This breaks `yarn link`
      config.resolve.symlinks = false;
    }

    return config;
  },
};

Performance telemetry plugin

We keep an eye on compilation time and bundle size over time, to see the impact of Storybook updates, config changes, and # of stories on our development experience.

/**
 * Log build metrics into Datadog
 */
class TelemetryPlugin {
  // Here you’d import your own statsd client:
  #statsdClient = require("./statsd-client");
  #buildNumber = 0;

  apply(compiler) {
    compiler.hooks.done.intercept({
      register: (tap) => {
        if (tap.name !== "webpack-hot-middleware") return tap;

        // intercept webpack-hot-middleware to collect telemetry data on build times
        const originalFn = tap.fn.bind(tap);
        tap.fn = (stats) => {
          originalFn(stats);
          const compilationTime = stats.endTime - stats.startTime;
          let eventName = "preview_initial_build_duration_ms";

          if (this.#buildNumber > 0) {
            eventName = "preview_incremental_build_duration_ms";
          }

          // Time spent on the previewer webpack build
          // when we perform a cold-cache build in CI
          if (process.env.WARM_UP_CACHE) {
            eventName = "preview_warm_up_cache_build_duration_ms";
          }

          this.#statsdClient.timing(eventName, compilationTime);

          this.#buildNumber++;
        };
        return tap;
      },
    });
  }
}

[cherninely]

Hi, @kaelig, thx for your tips & tricks.
My tips is not currently ready, but i would like to share my thoughts for now.

In our company we have rich codebase and bundles can be like 50mb plus and build time can be 5 mins plus.
Most tricks i tried speed up bundle size or build time for about 5-10% i think. Or even more, but with storybook consumptions tradeoffs.

So we decided to change our speeding up strategy from speeding up webpack & co to microfrontends.
What i mean:
Usually it's possible to split up your package from 1 large storybook to different smaller ones.
If its done, then its possible to add selectivity and build/deploy only what changed.

According to my calculations with this strategy we can speed up both build time and bundle size 3 times.
This is much more significant, then every trick i tried.
Actually, in my opinion, it doesn't mean, that developers shouldn't try to speed up storybook, but eventually its all about backlog priorities.
We are in progress now to try this strategy out. Will share results later if i won't forget.

[kaelig]

This article by @liorheber contains a few great performance tips as well: https://medium.com/kenshoos-engineering-blog/how-to-test-400-react-components-without-breaking-a-sweat-aa304a5cc72b ❤️