Go rules for Bazel
Travis | Jenkins |
---|---|
- November 27, 2017
- Release 0.7.1 is now available.
- November 3, 2017
- Release 0.7.0 is now available.
- October 16, 2017
- We have a new mailing list: bazel-go-discuss. All questions about building Go with Bazel and using Gazelle are welcome.
Contents
- Mailing list: bazel-go-discuss
- Core api
- Workspace rules
- Toolchains
- Protobuf rules
- Extra rules
- Deprecated rules
- Build modes
The rules are in the alpha stage of development. They support:
- libraries
- binaries
- tests
- vendoring
- cgo
- cross compilation
- auto generating BUILD files via gazelle
- protocol buffers
They currently do not support (in order of importance):
- bazel-style auto generating BUILD (where the library name is other than go_default_library)
- C/C++ interoperation except cgo (swig etc.)
- coverage
- test sharding
Note: | The latest version of these rules (0.7.1) requires Bazel โฅ 0.7.0 to work. |
---|
The master
branch is only guaranteed to work with the latest version of Bazel.
Create a file at the top of your repository named WORKSPACE and add one of the snippets below, verbatim. This will let Bazel fetch necessary dependencies from this repository and a few others.
If you want to use the latest stable release, add the following:
http_archive( name = "io_bazel_rules_go", url = "https://github.com/bazelbuild/rules_go/releases/download/0.7.1/rules_go-0.7.1.tar.gz", sha256 = "341d5eacef704415386974bc82a1783a8b7ffbff2ab6ba02375e1ca20d9b031c", ) load("@io_bazel_rules_go//go:def.bzl", "go_rules_dependencies", "go_register_toolchains") go_rules_dependencies() go_register_toolchains()
If you want to use a specific commit (for example, something close to
master
), add the following instead:git_repository( name = "io_bazel_rules_go", remote = "https://github.com/bazelbuild/rules_go.git", commit = "a390e7f7eac912f6e67dc54acf67aa974d05f9c3", ) load("@io_bazel_rules_go//go:def.bzl", "go_rules_dependencies", "go_register_toolchains") go_rules_dependencies() go_register_toolchains()
If you plan to use the proto rules (
go_proto_library
andgo_grpc_library
), add the following to WORKSPACE.load("@io_bazel_rules_go//proto:def.bzl", "proto_register_toolchains") proto_register_toolchains()
You can add more external dependencies to this file later (see go_repository).
Add a file named
BUILD.bazel
in the root directory of your project. In general, you need one of these files in every directory with Go code, but you need one in the root directory even if your project doesn't have any Go code there.If your project can be built with
go build
, you can generate your build files using Gazelle. If your project isn't compatible with go build or if you prefer not to use Gazelle, you can write build files by hand.
If your project can be built with go build
, you can generate and update your
build files automatically using gazelle, a tool included in this repository.
Add the code below to the
BUILD.bazel
file in your repository's root directory. Replace theprefix
string with the prefix you chose for your project earlier.load("@io_bazel_rules_go//go:def.bzl", "gazelle") gazelle( name = "gazelle", prefix = "github.com/example/project", )
If your project uses vendoring, add
external = "vendored",
below theprefix
line.After adding the
gazelle
rule, run the command below:bazel run //:gazelle
This will generate a
BUILD.bazel
file for each Go package in your repository. You can run the same command in the future to update existing build files with new source files, dependencies, and options.
If your project doesn't follow go build
conventions or you prefer not to use
gazelle, you can write build files by hand.
In each directory that contains Go code, create a file named
BUILD.bazel
Add a
load
statement at the top of the file for the rules you use.load("@io_bazel_rules_go//go:def.bzl", "go_binary", "go_library", "go_test")
For each library, add a go_library rule like the one below. Source files are listed in
srcs
. Other packages you import are listed indeps
using Bazel labels that refer to other go_library rules. The library's import path should be specified withimportpath
.go_library( name = "go_default_library", srcs = [ "foo.go", "bar.go", ], deps = [ "//tools:go_default_library", "@org_golang_x_utils//stuff:go_default_library", ], importpath = "github.com/example/project/foo", visibility = ["//visibility:public"], )
For each test, add a go_test rule like either of the ones below. You'll need separate go_test rules for internal and external tests.
# Internal test go_test( name = "go_default_test", srcs = ["foo_test.go"], importpath = "github.com/example/project/foo", library = ":go_default_library", ) # External test go_test( name = "go_default_xtest", srcs = ["bar_test.go"], deps = [":go_default_library"], importpath = "github.com/example/project/foo", )
For each binary, add a go_binary rule like the one below.
go_binary( name = "foo", srcs = ["main.go"], deps = [":go_default_library"], importpath = "github.com/example/project/foo", )
For instructions on how to depend on external libraries, see _vendoring
Yes, this setup was deliberately chosen to be compatible with go build
.
Make sure your project appears in GOPATH
, and it should work.
Note that go build
won't be aware of dependencies listed in WORKSPACE
, so
these will be downloaded into GOPATH
. You may also need to check in generated
files.
This was used to keep import paths consistent in libraries that can be built
with go build
before the importpath
attribute was available.
In order to compile and link correctly, the Go rules need to be able to
translate Bazel labels to Go import paths. Libraries that don't set the
importpath
attribute explicitly have an implicit dependency on //:go_prefix
,
a special rule that specifies an import path prefix. The import path is
the prefix concatenated with the Bazel package and target name. For example,
if your prefix was github.com/example/project
, and your library was
//foo/bar:bar
, the Go rules would decide the import path was
github.com/example/project/foo/bar/bar
. The stutter at the end is incompatible
with go build
, so if the label name is go_default_library
, the import path
is just the prefix concatenated with the package name. So if your library is
//foo/bar:go_default_library
, the import path is
github.com/example/project/foo/bar
.
We are working on deprecating go_prefix
and making importpath
mandatory (see
#721). When this work is complete, the go_default_library
name won't be needed.
We may decide to stop using this name in the future (see #265).
Bazel executes tests in a sandbox, which means tests don't automatically have
access to files. You must include test files using the data
attribute.
For example, if you want to include everything in the testdata
directory:
go_test(
name = "go_default_test",
srcs = ["foo_test.go"],
data = glob(["testdata/**"]),
importpath = "github.com/example/project/foo",
)
By default, tests are run in the directory of the build file that defined them.
Note that this follows the Go testing convention, not the Bazel convention
followed by other languages, which run in the repository root. This means
that you can access test files using relative paths. You can change the test
directory using the rundir
attribute. See go_test.
Gazelle will automatically add a data
attribute like the one above if you
have a testdata
directory unless it contains buildable .go files or
build files, in which case, testdata
is treated as a normal package.
References:
- Running Bazel Tests on Travis CI by Kevin Burke
- korfuri/bazel-travis Use Bazel with Travis CI
- Our own Travis configuration file
In order to run Bazel tests on Travis CI, you'll need to install Bazel in the
before_install
script. See our configuration file linked above.
You'll want to run Bazel with a number of flags to prevent it from consuming a huge amount of memory in the test environment.
--batch
: Don't start the Bazel server.--host_jvm_args=-Xmx500m --host_jvm_args=-Xms500m
: Set the maximum and initial JVM heap size. Keeping the same means the JVM won't spend time growing the heap. The choice of heap size is somewhat arbitrary; other configuration files recommend limits as high as 2500m. Higher values mean a faster build, but higher risk of OOM kill.--bazelrc=.test-bazelrc
: Use a Bazel configuration file specific to Travis CI. You can put most of the remaining options in here.build --spawn_strategy=standalone --genrule_strategy=standalone
: Disable sandboxing for the build. Sandboxing may fail inside of Travis's containers because themount
system call is not permitted.test --test_strategy=standalone
: Disable sandboxing for tests as well.--local_resources=1536,1.5,0.5
: Set Bazel limits on available RAM in MB, available cores for compute, and available cores for I/O. Higher values mean a faster build, but higher contention and risk of OOM kill.--noshow_progress
: Suppress progress messages in output for cleaner logs.--verbose_failures
: Get more detailed failure messages.--test_output=errors
: Show test stderr in the Travis log. Normally, test output is written log files which Travis does not save or report.
Downloads on Travis are relatively slow (the network is heavily
contended), so you'll want to minimize the amount of network I/O in
your build. Downloading Bazel and a Go SDK is a huge part of that. To
avoid downloading a Go SDK, you may request a container with a
preinstalled version of Go in your .travis.yml
file, then call
go_rules_dependencies(go_version = "host")
in a Travis-specific
WORKSPACE
file.
You may be tempted to put Bazel's cache in your Travis cache. Although this can speed up your build significantly, Travis stores its cache on Amazon, and it takes a very long time to transfer. Clean builds seem faster in practice.