td-agent-builder is a new build system for td-agent which aims to replace the traditional build system omnibus-td-agent since it has several problems due to Omnibus's limitations.

• Use system libraries: e.g. openssl
• Remove libraries for 3rd party gems: e.g. postgresql
• Remove embedded directory by omnibus
  • Use /opt/td-agent/bin/fluent-cat instead of /opt/td-agent/embedded/bin/fluent-cat
• Update core components: ruby, jemalloc and more

See also this issue for omnibus problems.

• Any host OS that is supported by Docker
  • Debian buster or Ubuntu 18.04 are recommended
• Docker
• Ruby 2.4 or later
• Git

• Windows OS (10 Pro or 2019 are verified)
• Docker Desktop for Windows
  • You need to switch to "Windows containers" before using it.
• RubyInstaller 2.4 or later.
• Git for Windows

After installed above software, you need to enable additional features from powershell (as admin).

dism.exe /online /enable-feature /featurename:Microsoft-Hyper-V /all /norestart
dism.exe /online /enable-feature /featurename:Containers /all /norestart

Then restart Windows.

% rake yum:build

By default, yum repositories for following platforms will be built under td-agent/yum/repositories/ directory:

• CentOS 6 (x86_64)
• CentOS 7 (x86_64)
• CentOS 8 (x86_64)

You can choose target platforms by YUM_TARGETS environment variable like this:

% rake yum:build YUM_TARGETS="centos-6,centos-7,centos-8"

You can find other supported platforms under td-agent/yum directory.

You can also build packages for AArch64 platforms like this:

% rake yum:build YUM_TARGETS="centos-8-aarch64"

But if you use older GNU/Linux platforms (e.g. Ubuntu 18.04 or before) as your host OS, you need to copy qemnu-aarch64-static into the base directory of the target:

% export TARGET_BASE="centos-8"
% sudo apt install qemu-user-static
% cd /path/to/td-agent-builder
% cp /usr/bin/qemu-aarch64-static td-agent/yum/${TARGET_BASE}
% rake yum:build YUM_TARGETS="${TARGET_BASE}-aarch64"

% rake apt:build

By default, apt repositories for following platforms will be built under td-agent/apt/repositories/ directory:

• Debian 10 "Buster" (x86_64)
• Ubuntu 18.04 LTS "Bionic Beaver" (x86_64)

You can choose target platforms by APT_TARGETS environment variable like this:

% rake apt:build APT_TARGETS="debian-buster,ubuntu-bionic"

You can find other supported platforms under td-agent/apt directory.

You can also built packages for AArch64 platforms like this:

% rake apt:build APT_TARGETS="ubuntu-bionic-arm64"

But if you use older GNU/Linux platforms (e.g. Ubuntu 18.04 or before) as your host OS, you need to copy qemnu-aarch64-static into the base directory of the target:

% export TARGET_BASE="ubuntu-bionic"
% sudo apt install qemu-user-static
% cd /path/to/td-agent-builder
% cp /usr/bin/qemu-aarch64-static td-agent/apt/${TARGET_BASE}
% rake apt:build APT_TARGETS="${TARGET_BASE}-arm64"

% rake msi:build

A td-agent-${version}-x64.msi package will be built under td-agent/msi directory.

• Edit td-agent/config.rb to choose Ruby & Fluentd versions
• Edit td-agent/core_gems.rb & td-agent/plugin_gems.rb to choose bundled gems
• Bump up the versions of rpm & deb packages by the following command:

% cd td-agent
% rake version:update
% git diff  # Check the change log
% git commit -a

• Build packages

% rake apt:build
% rake yum:build
% rake msi:build