Boring-registry is an open source module and provider registry compatible with Terraform and OpenTofu.
Table of Contents
- Overview
- Installation
- Configuration
- Internal Storage Layout
- Publishing Modules
- Publishing Providers
- Provider Network Mirror
With boring-registry, you can upload and distribute your own modules and providers, as an alternative to publishing them on HashiCorp's public Terraform Registry.
Support for the Module Registry Protocol, Provider Registry Protocol, and Provider Network Mirror Protocol allows it to work natively with Terraform and OpenTofu.
- Module Registry
- Provider Registry
- Network mirror for providers
- Pull-through mirror for providers
- Support for S3, GCS, Azure Blob Storage, and MinIO object storage
helm upgrade --install --wait --namespace default boring-registry oci://ghcr.io/boring-registry/charts/boring-registry
Images are published to ghcr.io/boring-registry/boring-registry
for every tagged release of the project.
Run make
to build the project and install the boring-registry
executable into $GOPATH/bin
.
Then start the server with $GOPATH/bin/boring-registry
, or if $GOPATH/bin
is already on your $PATH
, you can simply run boring-registry
.
The boring-registry does not rely on a configuration file. Instead, everything can be configured using flags or environment variables.
Important Note:
- Flags have higher priority than environment variables
- All environment variables are prefixed with
BORING_REGISTRY_
Example: To enable debug logging you can either pass the flag: --debug
or set the environment variable: BORING_REGISTRY_DEBUG=true
.
To run the server you need to specify which storage backend to use:
Minimal configuration using the S3 storage backend:
$ boring-registry server \
--storage-s3-bucket=terraform-registry-test
Make sure the server has AWS credentials set (e.g. AWS_ACCESS_KEY_ID
and AWS_SECRET_ACCESS_KEY
).
Minimal example using the GCS storage backend:
$ boring-registry server \
--storage-gcs-bucket=terraform-registry-test
Make sure the server has GCP credentials set (e.g. GOOGLE_CLOUD_PROJECT
).
Minimal example using the S3 storage backend with MinIO:
$ boring-registry server \
--storage-s3-bucket=terraform-registry-test \
--storage-s3-region=eu-east-1 \
--storage-s3-pathstyle=true \
--storage-s3-endpoint=https://minio.example.com
Minimal example using the Azure storage backend:
$ boring-registry server \
--storage-azure-account=registry \
--storage-azure-container=terraform-registry-test
Make sure the server has Azure credentials set. The Azure backend supports the following authentication methods:
- Environment Variables
- Service principal with client secret (
AZURE_TENANT_ID
,AZURE_CLIENT_ID
,AZURE_CLIENT_SECRET
) - Service principal with certificate (
AZURE_TENANT_ID
,AZURE_CLIENT_ID
,AZURE_CLIENT_CERTIFICATE_PATH
,AZURE_CLIENT_CERTIFICATE_PASSWORD
) - User with username and password (
AZURE_TENANT_ID
,AZURE_CLIENT_ID
,AZURE_USERNAME
,AZURE_PASSWORD
)
- Service principal with client secret (
- Managed Identity
- Azure CLI
Make sure the used identity has the role Storage Blob Data Contributor
on the Storage Account.
The storage backend has to be specified for the upload
command as well. Check the module upload section below.
The boring-registry can be configured with a set of API keys to match for by using the --auth-static-token="very-secure-token"
flag or by providing it as an environment variable BORING_REGISTRY_AUTH_STATIC_TOKEN="very-secure-token"
.
Multiple API keys can be configured by passing comma-separated tokens to the --auth-static-token="first-token,second-token"
flag or environment variable BORING_REGISTRY_AUTH_STATIC_TOKEN="first-token,second-token"
.
The token can be passed to Terraform inside the ~/.terraformrc
configuration file:
credentials "boring-registry.example.com" {
token = "very-secure-token"
}
The boring-registry is using the following storage layout inside the storage backend:
<bucket_prefix>
├── modules
│ └── <namespace>
│ └── <name>
│ └── <provider>
│ ├── <namespace>-<name>-<provider>-<version>.tar.gz
│ └── <namespace>-<name>-<provider>-<version>.tar.gz
├── providers
│ └── <namespace>
│ ├── signing-keys.json
│ └── <name>
│ ├── terraform-provider-<name>_<version>_SHA256SUMS
│ ├── terraform-provider-<name>_<version>_SHA256SUMS.sig
│ └── terraform-provider-<name>_<version>_<os>_<arch>.zip
└── mirror
└── providers
└── <hostname>
└── <namespace>
├── signing-keys.json
└── <name>
├── terraform-provider-<name>_<version>_SHA256SUMS
├── terraform-provider-<name>_<version>_SHA256SUMS.sig
└── terraform-provider-<name>_<version>_<os>_<arch>.zip
The <bucket_prefix>
is an optional prefix under which the boring-registry storage is organized and can be set with the --storage-s3-prefix
or --storage-gcs-prefix
flags.
An example without any placeholders could be the following:
<bucket_prefix>
├── modules
│ └── acme
│ └── tls-private-key
│ └── aws
│ ├── acme-tls-private-key-aws-0.1.0.tar.gz
│ └── acme-tls-private-key-aws-0.2.0.tar.gz
├── providers
│ └── acme
│ ├── signing-keys.json
│ └── dummy
│ ├── terraform-provider-dummy_0.1.0_SHA256SUMS
│ ├── terraform-provider-dummy_0.1.0_SHA256SUMS.sig
│ ├── terraform-provider-dummy_0.1.0_linux_amd64.zip
│ └── terraform-provider-dummy_0.1.0_linux_arm64.zip
└── mirror
└── providers
└── terraform.example.com
└── acme
├── signing-keys.json
└── random
├── terraform-provider-random_0.1.0_SHA256SUMS
├── terraform-provider-random_0.1.0_SHA256SUMS.sig
└── terraform-provider-random_0.1.0_linux_amd64.zip
Example Terraform configuration using a module referenced from the registry:
module "tls-private-key" {
source = "boring-registry.example.com/acme/tls-private-key/aws"
version = "~> 0.1"
}
Modules can be published to the registry with the upload
command.
The command expects a directory as argument, which is then walked recursively in search of boring-registry.hcl
files.
The boring-registry.hcl
file should be placed in the root directory of the module and should contain a metadata
block like the following:
metadata {
namespace = "acme"
name = "tls-private-key"
provider = "aws"
version = "0.1.0"
}
When running the upload command, the module is then packaged up and published to the registry.
Walking the directory recursively is the default behavior of the upload
command.
This way all modules underneath the current directory will be checked for boring-registry.hcl
files and modules will be packaged and uploaded if they not already exist
However, this can be unwanted in certain situations e.g. if a .terraform
directory is present containing other modules that have a configuration file.
The --recursive=false
flag will omit this behavior.
By default the upload command will silently ignore already uploaded versions of a module and return exit code 0
.
For tagging mono-repositories this can become a problem as it is not clear if the module version is new or already uploaded.
The --ignore-existing=false
parameter will force the upload command to return exit code 1
in such a case.
In combination with --recursive=false
the exit code can be used to tag the Git repository only if a new version was uploaded.
for i in $(ls -d */); do
printf "Operating on module \"${i%%/}\"\n"
# upload the given directory
./boring-registry upload --type gcs -gcs-bucket=my-boring-registry-upload-bucket --recursive=false --ignore-existing=false ${i%%/}
# tag the repo with a tag composed out of the boring-registry.hcl if not already exist
if [ $? -eq 0 ]; then
# git tag the repository with the version from boring-registry.hcl
# hint: use mattolenik/hclq to parse the hcl file
fi
done
The --version-constraints-semver
flag lets you specify a range of acceptable semver versions for modules.
It expects a specially formatted string containing one or more conditions, which are separated by commas.
The syntax is similar to the Terraform Version Constraint Syntax.
In order to exclude all SemVer pre-releases, you can e.g. use --version-constraints-semver=">=v0"
, which will instruct the boring-registry cli to only upload non-pre-releases to the registry.
This would for example be useful to restrict CI to only publish releases from the main
branch.
The --version-constraints-regex
flag lets you specify a regex that module versions have to match.
In order to only match pre-releases, you can e.g. use --version-constraints-regex="^[0-9]+\.[0-9]+\.[0-9]+-|\d*[a-zA-Z-][0-9a-zA-Z-]*$"
.
This would for example be useful to prevent publishing releases from non-main
branches, while allowing pre-releases to test out pull requests for example.
For general information on how to build and publish providers for Terraform see the official documentation.
The boring-registry expects a file named signing-keys.json
to be placed under the <namespace>
level in the storage backend.
More information about the purpose of this file can be found in the Provider Registry Protocol.
The file should have the following format:
{
"gpg_public_keys": [
{
"key_id": "51852D87348FFC4C",
"ascii_armor": "-----BEGIN PGP PUBLIC KEY BLOCK-----\nVersion: GnuPG v1\n..."
}
]
}
Multiple public keys are supported by extending the gpg_public_keys
array.
The v0.10.0
and previous releases of the boring-registry only supported a single signing key in the following format:
{
"key_id": "51852D87348FFC4C",
"ascii_armor": "-----BEGIN PGP PUBLIC KEY BLOCK-----\nVersion: GnuPG v1\n..."
}
- Manually prepare the provider release artifacts according to the documentation from hashicorp
- Publish the artifacts with the following (minimal) command:
boring-registry upload provider \ --storage-s3-bucket <bucket_name> \ --namespace <namespace> \ --filename-sha256sums /absolute/path/to/terraform-provider-<name>_<version>_SHA256SUMS
Example Terraform configuration using a provider referenced from the registry:
terraform {
required_providers {
dummy = {
source = "boring-registry.example.com/acme/dummy"
version = "0.1.0"
}
}
}
Note
The Provider Network Mirror feature is available starting from v0.12.0
.
The Network Mirror is enabled by default, but can be disabled with --network-mirror=false
.
The boring-registry implements the Provider Network Mirror Protocol to provide an alternative installation source for providers.
Check the Terraform CLI documentation to learn how to configure Terraform to use the provider network mirror.
In the following is an example for a .terraformrc
:
provider_installation {
network_mirror {
url = "https://boring-registry.example.com:5601/v1/mirror/"
}
}
To populate the mirror, the provider release artifacts need to be uploaded to the storage backend.
Refer to the Internal Storage Layout section for an overview of the required structure.
The terraform providers mirror
command is a good starting point for collecting the necessary files.
As part of the Provider Network Mirror, a pull-through mirror can optionally be activated with --network-mirror-pull-through=true
.
The pull-through functionality makes it possible that the providers do not have to be uploaded upfront to the storage backend.
Instead, boring-registry serves the providers of the origin registry and mirrors them automatically to the storage backend on the first download.
On the subsequent download request, boring-registry serves the providers directly from the storage backend.
This can significantly speed up the terraform init
phase and in some cases save additional traffic costs.