isabella232 / bugbot

Making life easier for people who report or triage Electron issues.

BugBot

Overview

1. People file Electron bug reports with tests.
2. BugBot can run the test on many platforms and Electron versions, then report back in-issue.
3. BugBot can bisect regressions down to a range of commits, then report back in-issue.
4. Users and maintainers can see where the bug is reproducible.

Usage

By bug reporters

BugBot only needs one thing: a test that exits with exitCode 0 if Electron works, or nonzero if Electron has a bug.

One easy way to make a test is with Electron Fiddle. Its "File > New Test" has a test template with some lightweight test helpers and its "Publish" button can upload the test to the gist.github.com pastebin.

If you want to do it manually, the only requirement is that 0/1 exitCode. You can write it from scratch or use electron-quick-start's test-template branch as a starting point, which is the same template used by Electron Fiddle. You can clone it from the command line with git clone --branch test-template https://github.com/electron/electron-quick-start, then use a web browser to upload it to gist.github.com.

After uploading your test to gist.github.com, file an Electron bug report. The bug report template will ask for a "Testcase Gist URL", which is where you'll provide the Gist URL.

By maintainers

Much like trop, you can start BugBot with issue comments. To begin bisection, add a comment that looks like this:

/bugbot test [gist] [platforms...] [versions...]

• If no gist is given, use the issue body's Testcase Gist URL.
• If no platforms are given, test on Linux, macOS, and Windows.
• If no versions are given, test the first version and latest version of each prerelease branch, of each supported branch, and of the two branches before that.

/bugbot bisect [gist] [goodVersion [badVersion]]

• If no gist is given, use the issue body's Testcase Gist URL.
• If no goodVersion is given, use the issue body's Last Known Working Electron Version or an old version.
• If no badVersion is given, use the issue body's Electron Version or the latest release.

Deployment

Environment variables

Name	Module	Description	Default
BUGBOT_BROKER_CERT or BUGBOT_BROKER_CERT_PATH	Required by Broker if BUGBOT_BROKER_URL is https	The data (or the path to it) to use as the cert option to https.createServer().	None
BUGBOT_BROKER_KEY or BUGBOT_BROKER_KEY_PATH	Required by Broker if BUGBOT_BROKER_URL is https	The data (or the path to it) to use as the key option to https.createServer().	None
BUGBOT_BROKER_URL	Required by all	The base URL for the broker, e.g. https://bugbot.electronjs.org:8443.	None
BUGBOT_CHILD_TIMEOUT_MS	Runner	When to cancel a hung child	5 minutes
BUGBOT_FIDDLE_EXEC	Runner	Used to invoke electron-fiddle. This can include other space-delimited command-line arguments, e.g. xvfb-run electron-fiddle	'which electron-fiddle'
BUGBOT_POLL_INTERVAL_MS	Issue Manager, Runner	How frequently to poll the Broker	issue-manager: 500 msec. runner: 20 sec
BUGBOT_AUTH_TOKEN	Required: Issue Manager, Runner; Optional: Broker	The auth token for communications with the Broker
BUGBOT_GITHUB_LOGIN	Issue Manager	The name of the GitHub app registered for the Probot client
BUGBOT_LOG_METRICS_URL	Broker	The remote Loki endpoint to send log metrics to
BUGBOT_LOG_METRICS_AUTH	Broker	The (Basic) auth credentials to authenticate log metrics requests with

Development

BugBot is split into multiple modules, connected in development via Yarn Workspaces and TypeScript Project References. Each module can be found within the modules/ top-level directory.

After cloning BugBot, run the following commands to set up your workspace:

$ yarn
$ yarn run build

Note: The latter command is required due to a caveat with TypeScript Project References and will hopefully be remedied automatically in the future. If you don't run yarn run build then you may see errors in your editor for modules that have not been built (or are not up-to-date and need to be rebuilt).