growthbook / growthbook-js

Powerful A/B testing for JavaScript - both browser and NodeJS

Open Source Feature Flagging and A/B Testing

Get up and running in 1 minute with:

git clone https://github.com/growthbook/growthbook.git
cd growthbook
docker-compose up -d

Then visit http://localhost:3000

Our Philosophy

The top 1% of companies spend thousands of hours building their own feature flagging and A/B testing platforms in-house. The other 99% are left paying for expensive 3rd party SaaS tools or hacking together unmaintained open source libraries.

We want to give all companies the flexibility and power of a fully-featured in-house platform without needing to build it themselves.

Major Features

• 🏁 Feature flags with advanced targeting, gradual rollouts, and experiments
• 💻 SDKs for React, Javascript, PHP, Ruby, Python, Go, Android, iOS, and more!
• 🆎 Powerful A/B test analysis with advanced statistics (CUPED, Sequential testing, Bayesian, SRM checks, and more)
• ❄️ Use your existing data stack - BigQuery, Mixpanel, Redshift, Google Analytics, and more
• ⬇️ Drill down into A/B test results by browser, country, or any other custom attribute
• 🪐 Export reports as a Jupyter Notebook!
• 📝 Document everything with screenshots and GitHub Flavored Markdown throughout
• 🔔 Webhooks and a REST API for building integrations

Try GrowthBook

Managed Cloud Hosting

Create a free GrowthBook Cloud account to get started.

Open Source

The included docker-compose.yml file contains the GrowthBook App and a MongoDB instance (for storing cached experiment results and metadata):

git clone https://github.com/growthbook/growthbook.git
cd growthbook
docker-compose up -d

Then visit http://localhost:3000 to view the app.

Check out the full Self-Hosting Instructions for more details.

Documentation and Support

View the GrowthBook Docs for info on how to configure and use the platform.

Join our Slack community if you get stuck, want to chat, or are thinking of a new feature.

Or email us at [email protected] if Slack isn't your thing.

We're here to help - and to make GrowthBook even better!

Contributors

We ❤️ all contributions, big and small!

Read CONTRIBUTING.md for how to setup your local development environment.

If you want to, you can reach out via Slack or email and we'll set up a pair programming session to get you started.

License

GrowthBook is an Open Core product. The bulk of the code is under the permissive MIT license, and the packages/enterprise directory has its own separate commercial license.

View the LICENSE file in this repository for the full text and details.

growthbook-js's Issues

Easier way to add new defaultProperties or experimentConfig

Currently:

growthbook.configure({
  defaultTrackingProps: {
    ...growthbook.getDefaultTrackingProps(),
    newProp: "hello"
  }
});

Change default behavior to merge instead of overwrite. Provide hook to reset back to empty array.

// Default behavior: merge
growthbook.configure({
  defaultTrackingProps: {
    newProp: "hello"
  }
});

// Hook to reset
growthbook.resetDefaultTrackingProps();

Same should be true for experimentConfig

Preserve variation when coverage changes

Currently, 100% coverage means:

• 0 to 0.5 gets baseline
• 0.5 to 1 gets variation

and 50% coverage means:

• 0 to 0.25 gets baseline
• 0.25 to 0.5 gets variation
• 0.5 to 1 is not in test

So if coverage is reduced while a test is running, everyone who was in the variation gets excluded from the test and the original baseline people get split.

If we change 50% coverage to the following, it will preserve the original mapping better:

• 0 to 0.25 gets baseline
• 0.25 to 0.5 is not in test
• 0.5 to 0.75 gets variation
• 0.75 to 1 is not in test

Or some version of the above (basically keep original mapping breakdown and exclude every xth person)

Querystring and/or localStorage experiment override for testing

Persist previously chosen variation

If weights or coverage change during a test, it should only affect new people who have never been bucketed before. May need to persist in local storage.

Open question: what to do about experimentByUser across devices? This will break the guarantee that the same userId will always get the same variation.

Custom track method

Configure a custom track method to override the built-in one.

// Use segment for event tracking
growthbook.configure({
  track: analytics.track
});

QA Mode

Make it easier to test experiments during automated testing. Example below uses Jest, but these new helper methods and qa flag should be applicable to any testing library.

describe('my app', () => {
  beforeAll(() => {
    // Flag to turn on QA mode
    // Force variation -1 for all experiments unless explicitly forced to something else
    // Disable auto experiments
    client.config.qa = true;
  })
  beforeEach(() => {
    // Undo any forces applied within test cases
    client.forcedVariations.clear();
  })

  it("works for control", () => {
    // test the control behavior
  })

  it("works for variation", () => {
    // force a non-control variation
    client.forceVariations.set("my-test", 1);

    // test the variation 1 behavior
  })
})

Dev Variation Switcher

Show a variation switcher UI overlaid on the page during development. For each experiment that is being used on a page, show the key and let the user switch between variations. Persist the last chosen variations in sessionStorage.

Usage:

If the extra code is small enough, it can be added to the main package and used like this:

client.enableDevMode();

Otherwise, it can be included as a separate dynamic import inside a node_env check:

if(process.env.NODE_ENV === "development") {
  import("@growthbook/growthbook/dev")
    .then(dev => {
      dev(client);
    })
}

This option is significantly more complicated, so hopefully the code stays small enough...

How it works:

1. For each user:
2. Subscribe to the user for any assigned variation or activeExperiment changes
3. When subscribe function triggers, get list of assigned variations and currently active experiments
4. If list.length > 0, render the UI for the variation switcher
5. When a new variation is chosen, call client.forcedVariations.set(key, variation) and user.experiment(exp)

Numbers 1, 2, and 4 rely on new features that will be added in #10