Comments (9)
Hi folks! 👋
The systemd way to do this is probably using a template instance specific drop-in file.
[email protected]
is a service template. [email protected]
is an instance with the instance name "custom".
[email protected]
inherits the config from the default at /lib/systemd/system/[email protected]
. We can add instance specific config in a drop-in (describe in the systemd.unit man page) named after the template instance at /etc/systemd/system/[email protected]/custom-config.conf
with contents like:
[Service]
Environment=BUILDKITE_AGENT_CONFIG=/etc/buildkite-agent/buildkite-agent-custom.cfg
This will override the config location for this one particular service instance.
You can verify this is working using systemctl cat ...
:
root@468ee23db373:/# systemctl cat [email protected]
# /lib/systemd/system/[email protected]
[Unit]
Description=Buildkite Agent (%i)
Documentation=https://buildkite.com/agent
After=syslog.target
After=network.target
[Service]
Type=simple
User=buildkite-agent
Environment=HOME=/var/lib/buildkite-agent
ExecStart=/usr/bin/buildkite-agent start
RestartSec=5
Restart=on-failure
TimeoutStartSec=10
TimeoutStopSec=5min
KillMode=mixed
[Install]
WantedBy=multi-user.target
DefaultInstance=1
# /etc/systemd/system/[email protected]/custom-config.conf
[Service]
Environment=BUILDKITE_AGENT_CONFIG=/etc/buildkite-agent/buildkite-agent-custom.cfg
from docs.
What about a scenario where you want to run a few agents per host, plus an agent dedicated to pipeline uploads on a separate queue?
$ systemctl edit [email protected]
# opens /etc/systemd/system/[email protected]/override.conf in $EDITOR
[Service]
Environment=BUILDKITE_AGENT_NAME=%%hostname-pipeline-uploader
Environment=BUILDKITE_AGENT_TAGS=queue=pipeline-uploader
$ systemctl cat [email protected]
# /lib/systemd/system/[email protected]
[Unit]
Description=Buildkite Agent (%i)
Documentation=https://buildkite.com/agent
After=syslog.target
After=network.target
[Service]
Type=simple
User=buildkite-agent
Environment=HOME=/var/lib/buildkite-agent
ExecStart=/usr/bin/buildkite-agent start
RestartSec=5
Restart=on-failure
TimeoutStartSec=10
TimeoutStopSec=5min
KillMode=mixed
[Install]
WantedBy=multi-user.target
DefaultInstance=1
# /etc/systemd/system/[email protected]/override.conf
[Service]
Environment=BUILDKITE_AGENT_NAME=%%hostname-pipeline-uploader
Environment=BUILDKITE_AGENT_TAGS=queue=pipeline-uploader
$ systemctl enable --now buildkite-agent@pipeline-uploader
Created symlink from /etc/systemd/system/multi-user.target.wants/[email protected] to /lib/systemd/system/[email protected].
$ systemctl enable --now buildkite-agent@1
Created symlink from /etc/systemd/system/multi-user.target.wants/[email protected] to /lib/systemd/system/[email protected].
$ systemctl enable --now buildkite-agent@2
Created symlink from /etc/systemd/system/multi-user.target.wants/[email protected] to /lib/systemd/system/[email protected].
$ systemctl enable --now buildkite-agent@3
Created symlink from /etc/systemd/system/multi-user.target.wants/[email protected] to /lib/systemd/system/[email protected].
$ systemctl enable --now buildkite-agent@4
Created symlink from /etc/systemd/system/multi-user.target.wants/[email protected] to /lib/systemd/system/[email protected].
The default systemd description makes sure the instance name is interpolated, so it looks nice:
from docs.
@toolmantim is this still the only way to have multiple config files? Does the config path env var/flag not allow you to specify different files?
from docs.
I'm not exactly sure of the steps… I think there's a few different ways to do it. At a base level the agent let's you customise the config path, and create separate configuration files, but the Ubuntu-specific instructions here https://buildkite.com/docs/agent/v3/ubuntu#running-multiple-agents don't tell you how to do it — I believe it only shows how to run multiple agents from the same config file, because it's using a systemd template: https://www.digitalocean.com/community/tutorials/understanding-systemd-units-and-unit-files#creating-instance-units-from-template-unit-files
@sj26 is our go-to systemd expert!
from docs.
Looks like this is related to #186!
@sj26 is there a 'recommended' way to do separate config files with systemd templates?
from docs.
It'd be great to figure out a real life example, so we could use that instead of "custom". If you're doing a custom agent config, it's probably also on a separate queue? Or perhaps a separate organization?
Would you also probably want to override the unit’s Description=Buildkite Agent (%i)
value too?
from docs.
What if I wanted to make let's say 5 copies of a custom service? I could use BUILDKITE_AGENT_SPAWN
but then they all have the same name and can't run at the same time because they all want to work from the same directory. I can't seem to do [email protected]
or anything of the sort because it just launches the default [email protected]
. Do I have to make N custom services? Can I somehow get the names to be decorated by the spawn count?
from docs.
@hofstee ah, in that case you might like to add %n
to the agent name which will be substituted by a number which ensures the agent name is unique across all your agents (docs). So for a custom service drop-in, it's as above but with:
[Service]
Environment=BUILDKITE_AGENT_NAME=%%hostname-pipeline-uploader-%%n
Environment=BUILDKITE_AGENT_TAGS=queue=pipeline-uploader
Environment=BUILDKITE_AGENT_SPAWN=5
You could probably also use systemd's %i
and friends.
from docs.
(The default buildkite agent configuration uses the name %hostname-%n
.)
from docs.
Related Issues (20)
- inconsistent examples on conditional build notifications
- New UI - Can't tell easily if the pipeline failed
- AOTVUNLEEVONG HOT 1
- Fail fast example doesn't show how to fail fast!
- The `agent lifecycle hooks` link under `Creating job lifecycle hooks` points to `Job lifecycle hooks` HOT 1
- “Running multiple agents” documentation should use --spawn HOT 2
- Glob pattern dist/**/* doesn't discover top level (within dist/) files
- Artifacts doc page should mention `--include-retried-jobs` flag HOT 2
- How can we get the value of user input HOT 1
- Documentation does not work for github merge queue
- Block step is always active, would be nice to be inactive until depends_on is complete HOT 1
- Conditional queue assignment HOT 6
- Block and wait behaving unexpectedly HOT 1
- There is no documentation for the allowed-repositories configuration setting HOT 2
- Confusing wording about SSO Organisations
- Permissions to delete a pipeline HOT 2
- bkua_1b3d70714487ab03c5281a2cdc9c9ef18e71e6e8
- Mr HOT 2
- API documentation HOT 1
- Incorrect description of BUILDKITE_REBUILT_FROM_BUILD_NUMBER HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from docs.