Feature request: Store results of performance tests somehow and somewhere for the future usage (e.g. performance regression testing on CI)

storybook-addon-performance 🚀

A storybook addon to help better understand and debug performance for React components

Highlights 🌟

Zero config (except for interactions): Generate performance information relating to server-side rendering and client-side mounting without any configuration
Pin results: You can run some tasks, pin the result, make some changes, rerun the tasks and see what changed
Save/Load results: You can run some tasks, save the results as a local artifact, and run them again later by loading the artifact back into the addon.
Interactions: Add your own custom user interactions to run as a parameter to your story. This lets you time how long interactions take. The API for this is super flexible and powerful!
Control: Run all tasks for an overview, or run individual tasks to drill down on specific problems
Marked: All tasks are marked with the User Timing API to allow for easy debugging of individual tasks in your browser's performance profiler

Installation

Install storybook-addon-performance

# pnpm
pnpm add storybook-addon-performance --dev

# yarn
yarn add storybook-addon-performance --dev

# npm
npm install storybook-addon-performance --save-dev

Register the addon in .storybook/main.js

module.exports = {
  addons: ['storybook-addon-performance'],
};

Add the decorator

You can either add the decorator globally to every story in .storybook/preview.js (recommended)

import { withPerformance } from 'storybook-addon-performance';

export const decorators = [withPerformance];

Or you can add it to individual stories:

Using Component Story Format (CSF)

import MyComponent from './MyComponent';
import { withPerformance } from 'storybook-addon-performance';

export default {
  title: 'MyComponent',
  component: MyComponent,
  decorators: [withPerformance],
};

Using StoriesOf API

import MyComponent from './MyComponent';
import { withPerformance } from 'storybook-addon-performance';

storiesOf('MyComponent', module)
  .addDecorator(withPerformance)
  .add('MyComponent', () => <MyComponent />);

Usage: Interactions

Interaction tasks are a task type that can be defined and run on a story-by-story basis. They are useful for timing the interactive performance of your components.

To define your interaction tasks, first create an array of objects, each containing the name and description (optional) of the task, and a run function that performs whatever tasks you'd like to measure:

import { InteractionTaskArgs, PublicInteractionTask } from 'storybook-addon-performance';
import { findByText, fireEvent } from '@testing-library/dom';

// ...

const interactionTasks: PublicInteractionTask[] = [
  {
    name: 'Display dropdown',
    description: 'Open the dropdown and wait for Option 5 to load',
    run: async ({ container }: InteractionTaskArgs): Promise<void> => {
      const element: HTMLElement | null = container.querySelector('.addon__dropdown-indicator');
      invariant(element);
      fireEvent.mouseDown(element);
      await findByText(container, 'Option 5', undefined, { timeout: 20000 });
    },
  },
];

The run function in each task object takes two arguments:

container: an HTMLElement container that contains a rendered instance of the story component
controls: contains an async timing function that can be optionally called to specify when to start and finish measurements; otherwise the time taken to complete the entire run function is measured. Useful when a task involves some set-up work.

To use, wrap the operations in question with controls.time as shown below:
```
run: async ({ container }: InteractionTaskArgs): Promise<void> => {
  // setup
  await controls.time(async () => {
    // interaction task you'd like to measure
  });
};
```

Note that you can use whatever libraries you'd like to perform these interaction tests – the example above uses @testing-library/dom to open the select in the example and wait for a specific item.

You can then include the array of interaction tasks inside the performance parameters of your story, with the key interactions:

// Using the Component Story Format (CSF)
// https://storybook.js.org/docs/formats/component-story-format/
import { findByText, fireEvent } from '@testing-library/dom';
import { PublicInteractionTask } from 'storybook-addon-performance';
import React from 'react';
import Select from 'react-select';
import invariant from 'tiny-invariant';

export default {
  title: 'React select example',
};

const interactionTasks: PublicInteractionTask[] = [
  {
    name: 'Display dropdown',
    description: 'Open the dropdown and wait for Option 5 to load',
    run: async ({ container }: InteractionTaskArgs): Promise<void> => {
      const element: HTMLElement | null = container.querySelector('.addon__dropdown-indicator');
      invariant(element);
      fireEvent.mouseDown(element);
      await findByText(container, 'Option 5', undefined, { timeout: 20000 });
    },
  },
];

select.storyName = 'React Select';
select.parameters = {
  performance: {
    interactions: interactionTasks,
  },
};

Supplied types

As seen above, the plugin exports two type definitions to assist with creating your own interaction tasks:

PublicInteractionTask: defines the object structure for an interaction task; pass an array of these tasks as a parameter to storybook, as shown above.
InteractionTaskArgs: the arguments for an interaction task's run function

Usage: Saving and loading results

You can save the result of a performance task as a local artifact by using the Save API. The Save API creates a story-specific artifact which can be then be loaded at a later time to be used as a benchmark. This can be useful for CI or testing a change in branch vs the trunk. You can use this API via the Save result / Load result buttons in the UI.

Some caveats with this API:

Storybook run performance results are variable, and can change depending on CPU utilisation / memory when the tests are run. If you intend to save an artifact, ensure you're re-running / comparing your results in an environment that is as similar as possible to the environment it was originally run.
For this API to work correctly the task artifact should be based on the same number of samples / copies as the original test.

For more consistent results we suggest recording artifacts using 10 copies / 10 samples.

Usage: Filtering task groups

Some components are not designed to work in server side rendering, or on the client. To support this we have created a allowlist that you can optionally pass in to only allow the groups to run that you want to. To configure this option, set the allowedGroups option as part of a story's parameters.

Default value: ['server', 'client'] (run everything)

// Using [Component Story Format (CSF)](https://storybook.js.org/docs/formats/component-story-format/)
export const onlyClient = () => <p>A story only measuring client-side performance 👩‍💻</p>;

onlyClient.parameters = {
  performance: {
    allowedGroups: ['client'],
  },
};

export const onlyServer = () => <p>A story only measuring server-side performance ‍☁️</p>;

onlyServer.parameters = {
  performance: {
    allowedGroups: ['server'],
  },
};

A Note on Performance Metrics 💡

In order to get the most accurate performance metrics possible, you should use a production build of Storybook. For more background, see the React optimizing performance documentation.

While this add-on does work with a dev build, you'll see more variability in results.

Local addon development

In the storybook-addon-performance folder (packages/storybook-addon-performance)

# Start the typescript watcher and a local storybook:
pnpm dev

# Start just the typescript watcher
# This is needed as storybook does not compile addons
pnpm typescript:watch

# Start the local storybook
pnpm storybook:dev