voxpupuli / puppet-gitlab_ci_runner Goto Github PK

https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-session_server-section

Affected Puppet, Ruby, OS and module versions/distributions

• Puppet: 5.5.8
• Ruby: n/a
• Distribution: Voxpopuli
• Module version: Github head

How to reproduce (e.g Puppet code you use)

Feature request

What are you seeing

No option to omit URL and registration-token

What behaviour did you expect instead

Possibility to omit URL and registration-token

Output log

Notice: /Stage[main]/Gitlab::Cirunner/Gitlab::Runner[test_runner]/Exec[Register_runner_test_runner]/returns: PANIC: The url needs to be entered
Error: '/usr/bin/gitlab-runner register -n --name test_runner' returned 1 instead of one of [0]
Error: /Stage[main]/Gitlab::Cirunner/Gitlab::Runner[test_runner]/Exec[Register_runner_test_runner]/returns: change from 'notrun' to ['0'] failed: '/usr/bin/gitlab-runner register -n --name test_runner' returned 1 instead of one of [0]
Notice: Applied catalog in 2.92 seconds

Any additional information you'd like to impart

The user may want to customize the runner and have control. If puppet controls these parameters exclusively then it becomes a non-option because puppet will set url and registration token on each run. I.e "I want a runner, I'll configure it myself".

A parameter should be able to envelop the options similar to "::gitlab::cirunner::manage_docker: false". It should be named something like "::gitlab::cirunner::configure_runner" and be a boolean.

Affected Puppet, Ruby, OS and module versions/distributions

• Puppet: any
• Ruby: any
• Distribution: any
• Module version: latest

How to reproduce (e.g Puppet code you use)

Try to add two gitlab_ci_runner instances to a single node, where each runner connectst o a different gitlab server with different CA signed certificates.

What are you seeing

gitlab_ci_runner register_to_file SSL error: "unable to get local issuer certificate"

What behaviour did you expect instead

I want to set the ca_file using the runners hash on a per runner basis.

Right now the listen_interface type enforces a port number, which is actually optional. it seems better to leave it out as it has a sane default too, registered in the prometheus port list and everything:

it seems to me this should be some sort of hybrid type, a concatenation of Stdlib::Host and Stdlib::Port, with the latter optional. I'm not sure how we can do this without creating a new type, so maybe that's what should be done here, but then again I can't help but feel this might actually live better straight in stdlib. Surely someone else has had this problem before, right?

Note: Migrated from voxpupuli/puppet-gitlab#120, opened by @cdenneen

Currently the method of checking config.toml for runner exists or not needs to be fixed.
While this works for the existence of a runner it doesn't allow for any changes to the runner itself.
The addition of options, tags, image, etc won't generate updated config.toml

It appears somehow the runner hash needs to be compared to current found hash in the config.toml to determine if the runner has updated or not.

To reproduce:

gitlab_ci_runners:
  ruby2.1:
    docker-image: "ruby:2.1"
gitlab_ci_runners_defaults:
  url: "https://gitlab.host/ci"
  registration-token: "XXXX"
  executor: "docker"
  docker-image: "ubuntu:trusty"

to

gitlab_ci_runners:
  ruby2.1:
    docker-image: "ruby:2.FOO"
    tag-list:
      - "ruby2.1"
gitlab_ci_runners_defaults:
  url: "https://gitlab.host/ci"
  registration-token: "XXXX"
  executor: "docker"
  docker-image: "ubuntu:trusty"

Will not update the config.toml to the following:

concurrent = 4
check_interval = 0

[[runners]]
  name = "ruby2.1"
  url = "https://gitlab.host/ci"
  token = "XXXX"
  executor = "docker"
  [runners.docker]
    tls_verify = false
    image = "ruby:2.FOO"
    privileged = false
    disable_cache = false
  [runners.cache]

Also the tag update for the runner isn't something that gets written to config.toml but updated in gitlab so that needs to be checked as well.

Only solution so far would be to empty config.toml and delete the runners in Gitlab UI... then run puppet and it will recreate the runners with current configuration and proper tags.

The last release nearly a year ago. Are you willing to release a new version. Because of this we can't update to the latest stdlib.

Thanks!

Affected Puppet, Ruby, OS and module versions/distributions

• Puppet: 5
• Ruby:
• Distribution: CentOS
• Module version: git master

How to reproduce (e.g Puppet code you use)

Use with a CentOS host.

What are you seeing

The package xz-utils is xz on CentOS. This needs to be set manually for CentOS installs it seems.

What behaviour did you expect instead

Module should use xz or xz-utils based on the OS, rather than this needing to be set manually.

Output log

Let me know if you need this.

Any additional information you'd like to impart

This was probably going to be done at some point, but this should help track it.

for some reason I've hit a docker issue with DNS lookups.. so I need to add:
extra_hosts = ["mygiturl:mygithostip"] - to config.toml - under runners.docker.

The module does not support this currently.. and does not handle if config.toml file is missing :(

I'd also like to set priviledged to true.

(issue migrated from voxpupuli/puppet-gitlab#121 )

License was copied from puppet-gtilab module, and License information is outdated and incorrect.

metadata.json indicates BSD-3 when voxpupuli modules prefer Apache 2.

• Add Apache 2 LICENSE file
• Update metadata.json.license to indicate Apache 2 license

Gitlab is rolling out a new registration workflow: https://docs.gitlab.com/ee/architecture/blueprints/runner_tokens/index.html#using-the-authentication-token-in-place-of-the-registration-token

This module should adapt to the new flow, since it's already confusing where you get the right token.

Currently, the gitlab-ci-runner module manages the config.toml file for setting up the runners themselves. However, it would also be nice to manage the systemd/sysv service file as well to do things like set a different user to run the gitlab-runner service, set a different working directory, etc.

Affected Puppet, Ruby, OS and module versions/distributions

• Puppet: 5.5.8
• Ruby: 2.3.3
• Distribution: debian/stretch
• Module version: master

How to reproduce (e.g Puppet code you use)

gitlab_ci_runner::runners:
  "shared":
    name: "shared_%{facts.hostname}"
    docker-image: "debian:stretch"

What are you seeing

subsequent Puppet runs keep registering new runners

What behaviour did you expect instead

no change

Output log

Debug: Exec[Register_runner_shared](provider=posix): Executing check '/bin/grep '"shared_runner1"' /etc/gitlab-runner/config.toml'
Debug: Executing: '/bin/grep '"shared_runner1"' /etc/gitlab-runner/config.toml'
Debug: Exec[Register_runner_shared](provider=posix): Executing '/usr/bin/gitlab-runner register -n --name=shared-runner1 --paused --registration-token=abcdefg --executor=docker --url=https://git.my.tld --docker-image=debian:stretch --docker-disable-entrypoint-overwrite=false --docker-oom-kill-disable=false --docker-disable-cache=false --docker-shm-size=0 --docker-privileged=false --docker-tlsverify=false --docker-volumes=/cache'
Debug: Executing: '/usr/bin/gitlab-runner register -n --name=shared-runner1 --paused --registration-token=abcdefg --executor=docker --url=https://git.my.tld --docker-image=debian:stretch --docker-disable-entrypoint-overwrite=false --docker-oom-kill-disable=false --docker-disable-cache=false --docker-shm-size=0 --docker-privileged=false --docker-tlsverify=false --docker-volumes=/cache'
Notice: /Stage[main]/Gitlab_ci_runner/Gitlab_ci_runner::Runner[shared]/Exec[Register_runner_shared]/returns: executed successfully
Debug: /Stage[main]/Gitlab_ci_runner/Gitlab_ci_runner::Runner[shared]/Exec[Register_runner_shared]: The container Gitlab_ci_runner::Runner[shared] will propagate my refresh event
Debug: Gitlab_ci_runner::Runner[shared]: The container Class[Gitlab_ci_runner] will propagate my refresh event

Any additional information you'd like to impart

$parameters_array_no_underscores = regsubst($parameters_array, '_', '-', 'G')
changes the _ to - in the name, which causes the grep to fail, as it is looking for the original value $runner_name = $_config['name']

Workaround for now is to adjust the names with dashes instead of underscores.

Affected Puppet, Ruby, OS and module versions/distributions

• Puppet: 6.26.0
• Ruby: 2.5.1p57
• Distribution: Ubuntu 18.04(bionic)
• Module version: 4.3.0

How to reproduce (e.g Puppet code you use)

Here is my yaml config.

docker::version: "5:19.03.12~3-0~ubuntu-%{facts.os.distro.codename}"
docker::root_dir: /data/docker
docker::storage_driver: overlay2

gitlab_ci_runner::concurrent: 4
gitlab_ci_runner::manage_docker: true

gitlab_ci_runner::config_path: "/etc/gitlab-runner/config.toml"
gitlab_ci_runner::package_ensure: "14.5.0"

gitlab_ci_runner::runners:
  "%{::hostname}":
    url: "url"
    registration-token: "token"
    tag-list: "testing"
    ensure: present

gitlab_ci_runner::runner_defaults:
  url: "url"
  registration-token: "token"
  ensure: present

What are you seeing

gitlab-runner can't start, and i got config.toml parsing error.

gitlab-runner[22542]: #033[31;1mFATAL: Service run failed   #033[0;m  #033[31;1merror#033[0;m=Near line 2 (last key parsed ''): bare keys cannot contain '('
gitlab-runner.service: Main process exited, code=exited, status=1/FAILURE
gitlab-runner.service: Failed with result 'exit-code'.

The wrong config.toml rendered by puppet.

# MANAGED BY PUPPET
concurrent = 4
Deferred({'name' => 'gitlab_ci_runner::to_toml', 'arguments' => [{'runners' => [{'url' => 'url', 'executor' => 'docker', 'docker' => {'image' => 'ruby:2.1'}, 'builds_dir' => '/tmp', 'cache_dir' => '/tmp', 'name' => 'gitlab-testing-runner', 'token' => Deferred({'name' => 'gitlab_ci_runner::register_to_file', 'arguments' => ['url', 'token', {'tag_list' => 'testing'}, undef, undef]})}]}]})

What behaviour did you expect instead

I expect gitlab-runner config look like this.

[[runners]]
  name = "gitlab-runner-testing"
  url = "url"
  token = "token"
  executor = "docker"
  builds_dir = "/cache/builds"
  environment = ["DOCKER_DRIVER=overlay2"]
  [runners.custom_build_dir]

Output log

Any additional information you'd like to impart

Affected Puppet, Ruby, OS and module versions/distributions

• Puppet: 6.10.1
• Ruby:
• Distribution: ubuntu bionic
• Module version: current master (18.11.2019)

How to reproduce (e.g Puppet code you use)

If the registration_token contains an underscore, it gets converted into a "minus", and then of course the registration fails

What behaviour did you expect instead

Should be able to register :)

Add note to README that you should provide your own package manager module for non-RHEL family distros.

Affected Puppet, Ruby, OS and module versions/distributions

• Puppet: ?
• Ruby: 2.5
• Distribution: Mac
• Module version: Current default branch

How to reproduce (e.g Puppet code you use)

$ bundle exec rake lint
manifests/runner.pp:71:parameter_documentation:WARNING:missing documentation for defined type parameter gitlab_ci_runner::runner::http_proxy
manifests/runner.pp:71:parameter_documentation:WARNING:missing documentation for defined type parameter gitlab_ci_runner::runner::http_proxy

What are you seeing

1. missing documentation for defined type parameter gitlab_ci_runner::runner::http_proxy
2. Why isn't lint failing?

What behaviour did you expect instead

lint should fail.

Output log

Any additional information you'd like to impart

How to reproduce (e.g Puppet code you use)

Hiera code:

gitlab_ci_runner::runners:
  test_runner3:
    url: "https://git.alternative.org/ci"
    registration-token: "abcdef1234567890"
    tag-list:
      - tag1
      - tag2

What are you seeing

gitlab-runner is created with the last tag in the list (tag2) only.

What behaviour did you expect instead

Would expect to see all the tags in the list ( tag1 and tag2)

Version 4.0.0 is a new major release of this module. It has many improvements but also significant breaking changes that you should read about and test before deploying into a production environment. Specifically Puppet 6 is required, your code will probably need updating and existing runners will reregister.

The README has further details.

Huge thanks to all our contributors and especially to Matthias Baur for his excellent contributions to this release.

Affected Puppet, Ruby, OS and module versions/distributions

• Puppet:
• Ruby:
• Distribution:
• Module version:

How to reproduce (e.g Puppet code you use)

What are you seeing

gitlab_ci_runner::repo_keyserver does not accept valid keyserver url's like:
hkp://keyserver.ubuntu.com:80

This is needed in some environments, where outbound port 11371 (hkp protocol) is blocked and keys needs to be fetched over the regular http port.

What behaviour did you expect instead

Output log

Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Error while evaluating a Resource Statement, Class[Gitlab_ci_runner]: parameter 'repo_keyserver' expects a match for Stdlib::Fqdn = Pattern[/\A(([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\-]*[a-zA-Z0-9])\.)*([A-Za-z0-9]|[A-Za-z0-9][A-Za-z0-9\-]*[A-Za-z0-9])\z/], got 'hkp://keyserver.ubuntu.com:80'

Any additional information you'd like to impart

Affected Puppet, Ruby, OS and module versions/distributions

• Puppet:

puppet-agent-7.15.0-1.el8.x86_64
puppetserver-7.6.1-1.el7.noarch

• Ruby:

• Distribution:
  Puppet server: Red Hat Enterprise Linux Server release 7.9 (Maipo)
  Puppet agent: Red Hat Enterprise Linux release 8.1 (Ootpa)

• Module version:
  4.3.0

How to reproduce (e.g Puppet code you use)

Puppet manifest:

  # Resource ordering  THIS DOESN'T ORDER PROPERLY
  Class['gitlab'] -> Class['gitlab_ci_runner'] 
  include ::gitlab

  # Install Gitlab CI Runners
  include ::gitlab_ci_runner

Hiera data:

gitlab_ci_runner::concurrent: 4
gitlab_ci_runner::check_interval: 4
gitlab_ci_runner::metrics_server: "localhost:8888"
gitlab_ci_runner::manage_docker: true
gitlab_ci_runner::config_path: "/etc/gitlab-runner/config.toml"
gitlab_ci_runner::runners:
  ubi8:
    ensure: present
    url: "http://10.10.10.10/"
    registration-token: "token"
    tag-list: "aws,docker,example-tag"
    executor: "docker"
    docker:
      image: "ubi/ubi8:minimal"
  alpine:
    ensure: present
    url: "http://10.10.10.10/"
    registration-token: "token"
    tag-list: "alpine"
    executor: "docker"
    docker:
      image: "alpine:latest"
 
gitlab_ci_runner::runner_defaults:
  url: "https://localhost/ci"
  registration-token: "token"
  executor: "docker"
  docker:
    image: "ubi/ubi7:minimal"

What are you seeing

When running the puppet manifest, the gitlab_ci_runner https://github.com/voxpupuli/puppet-gitlab_ci_runner/blob/master/manifests/init.pp#L139 tries to connect to the local machine before gitlab is installed from the include ::gitlab code.

What behaviour did you expect instead

I expected Gitlab class module to run and install Gitlab CE and then the gitlab_ci_runner class module register the runners.

Output log

Info: Using environment '23_create_gitlab_ce_role_profile_for_installing_and_managing_gitlabce'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Error: Failed to apply catalog: Failed to open TCP connection to 10.100.100.10:80 (Connection refused - connect(2) for "10.100.100.119" port 80)

Any additional information you'd like to impart

Of course, since Gitlab isn't installed there's no way to configure the runners.

I've been out of the Puppet world for about four years so I'm rusty. I'm not familiar with Deferred Functions.

Is the resource ordering above not correct? Is there a way in the puppet module to ensure that the runners are registered later/last?

There are several possible use cases that require running gitlab-runner on windows platform, such as for executing utilities and proprietary software available only on windows, or administering windows systems themselves using ci triggers.

Would it be possible to provide support for gitlab-runner on windows platform?

Affected Puppet, Ruby, OS and module versions/distributions

• Puppet: 6.17.0
• Ruby: ruby 2.5.8p224 (2020-03-31 revision 67882) [x86_64-linux]
• Distribution: Ubuntu 18.04.5 LTS
• Module version: 4.0.0
• Gitlab version: 14.2.3-ee

How to reproduce (e.g Puppet code you use)

gitlab_ci_runner.yaml

---
gitlab_ci_runner::check_interval: 4
gitlab_ci_runner::concurrent: 4
gitlab_ci_runner::config_path: "/etc/gitlab-runner/config.toml"
gitlab_ci_runner::manage_docker: true
gitlab_ci_runner::metrics_server: localhost:8888
gitlab_ci_runner::runner_defaults:
  builds_dir: "/var/run/gitlab-runner/builds"
  cache_dir: "/var/run/gitlab-runner/cache"
  docker: 
    image: ubuntu:focal
  executor: docker
  registration-token: "%{hiera('gitlab::runner::token')}"
  tag-list: docker,%{::datacenter}
  url: https://gitlab.<domain>.com/ci
gitlab_ci_runner::runners:
  0ca6ceb2-9d56-4413-8fe7-60d9fc3110fb: {}
  4c156b10-5934-4843-a766-e39a1439f5c7: {}
  b9361143-8141-41ec-a1a8-c7843f2e5d24: {}
  e2139a8c-eb16-4a2d-ac09-2b48a0825ca2: {}
  e44c3545-e110-420d-85fe-7a42eb2d4c42: {}

glrunnr01.yml

---
classes:
- gitlab_ci_runner

What are you seeing

When puppet tried to apply and thus create the runner in the application.log of Gitlab, I see:

Completed 200 OK in 88ms (Views: 0.4ms | ActiveRecord: 13.7ms | Elasticsearch: 0.0ms | Allocations: 27723)
Started POST "/ci/api/v4/runners" for 10.5.200.10 at 2021-09-17 11:58:30 -0400
Processing by Gitlab::RequestForgeryProtection::Controller#index as HTML
Can't verify CSRF token authenticity.
Completed 422 Unprocessable Entity in 1ms (ActiveRecord: 0.0ms | Elasticsearch: 0.0ms | Allocations: 151)

In the puppet run, I see:

Error: Failed to apply catalog: Gitlab runner failed to register: Unprocessable Entity

What behaviour did you expect instead

I expect that the runners get applied.

Output log

Any additional information you'd like to impart

If I revert the module to v3.0.0, and change hiera (back to the old format), the runners are created.

This is the hiera that is working under v3.0.0 of the module:

---
gitlab_ci_runner::check_interval: 4
gitlab_ci_runner::concurrent: 4
gitlab_ci_runner::config_path: "/etc/gitlab-runner/config.toml"
gitlab_ci_runner::manage_docker: true
gitlab_ci_runner::metrics_server: localhost:8888
gitlab_ci_runner::runner_defaults:
  builds_dir: "/var/run/gitlab-runner/builds"
  cache_dir: "/var/run/gitlab-runner/cache"
  docker-image: ubuntu:focal
  executor: docker
  registration-token: "%{hiera('gitlab::runner::token')}"
  tag-list: docker,%{::datacenter}
  url: https://gitlab.<domain>.com/ci
gitlab_ci_runner::runners:
  0ca6ceb2-9d56-4413-8fe7-60d9fc3110fb: {}
  4c156b10-5934-4843-a766-e39a1439f5c7: {}
  b9361143-8141-41ec-a1a8-c7843f2e5d24: {}
  e2139a8c-eb16-4a2d-ac09-2b48a0825ca2: {}
  e44c3545-e110-420d-85fe-7a42eb2d4c42: {}

Affected Puppet, Ruby, OS and module versions/distributions

• Puppet: 6.19.1
• Ruby: ruby 2.5.5p157 (2019-03-15 revision 67260) [x86_64-linux-gnu]
• Distribution: debian/buster
• Module version: puppet-gitlab_ci_runner (v3.0.0)

How to reproduce (e.g Puppet code you use)

On a machine without internet access, I am trying to install gitlab-runner through one of my internal repos with a hiera conf like:

gitlab_ci_runner::manage_repo: true
gitlab_ci_runner::repo_base_url: 'https://myrepo/repository/debian-gitlab'

What are you seeing

The module tries to fetch apt key for the repo on the internet (keyserver.ubuntu.com). I do not have a mirror for this and do not wish to install one.

What behaviour did you expect instead

I would like to be able to use all options on apt::key, like providing source or content directly: https://forge.puppet.com/modules/puppetlabs/apt/reference#parameters-3

Output log

Debug: Prefetching apt_key resources for apt_key
Debug: Executing: '/usr/bin/apt-key adv --no-tty --list-keys --with-colons --fingerprint --fixed-list-mode'
Debug: Executing: '/usr/bin/apt-key adv --no-tty --keyserver keyserver.ubuntu.com --recv-keys F6403F6544A38863DAA0B6E03F01618A51312F3F'
Error: Execution of '/usr/bin/apt-key adv --no-tty --keyserver keyserver.ubuntu.com --recv-keys F6403F6544A38863DAA0B6E03F01618A51312F3F' returned 2: Warning: apt-key output should not be parsed (stdout is not a terminal)
Executing: /tmp/apt-key-gpghome.ZVaLy4HGvt/gpg.1.sh --no-tty --keyserver keyserver.ubuntu.com --recv-keys F6403F6544A38863DAA0B6E03F01618A51312F3F
gpg: keyserver receive failed: Connection timed out
Error: /Stage[main]/Gitlab_ci_runner::Repo/Apt::Source[apt_gitlabci]/Apt::Key[Add key: F6403F6544A38863DAA0B6E03F01618A51312F3F from Apt::Source apt_gitlabci]/Apt_key[Add key: F6403F6544A38863DAA0B6E03F01618A51312F3F from Apt::Source apt_gitlabci]/ensure: change from 'absent' to 'present' failed: Execution of '/usr/bin/apt-key adv --no-tty --keyserver keyserver.ubuntu.com --recv-keys F6403F6544A38863DAA0B6E03F01618A51312F3F' returned 2: Warning: apt-key output should not be parsed (stdout is not a terminal)
Executing: /tmp/apt-key-gpghome.ZVaLy4HGvt/gpg.1.sh --no-tty --keyserver keyserver.ubuntu.com --recv-keys F6403F6544A38863DAA0B6E03F01618A51312F3F
gpg: keyserver receive failed: Connection timed out

Since the current content of this module was mostly, but not entirely ported over from https://github.com/voxpupuli/puppet-gitlab, it's never actually been installed or run anywhere.

This is a good opportunity to with a fresh start to set up working acceptance tests to ensure the module works as expected with the v1.0 release.

Affected Puppet, Ruby, OS and module versions/distributions

• Puppet:
• Ruby:
• Distribution:
• Module version:

How to reproduce (e.g Puppet code you use)

bundle install
bundle exec rake release_checks

What are you seeing

An error occurred while loading ./spec/tasks/register_runner_spec.rb. - Did you mean?
                    rspec ./spec/tasks/unregister_runner_spec.rb

Failure/Error: require_relative '../../tasks/register_runner.rb'

LoadError:
  cannot load such file -- /home/paul/work/esa_auto/puppet/vagrant/vagrant-gitlabci/modules/ruby_task_helper/files/task_helper.rb
# ./tasks/register_runner.rb:7:in `require_relative'
# ./tasks/register_runner.rb:7:in `<top (required)>'
# ./spec/tasks/register_runner_spec.rb:2:in `require_relative'
# ./spec/tasks/register_runner_spec.rb:2:in `<top (required)>'
No examples found.

What behaviour did you expect instead

I Expected the relative paths to work on the command line too, and not only in the CI/CD pipeline.

Output log

bundle exec rake release_checks  
(in /home/paul/work/esa_auto/puppet/vagrant/vagrant-gitlabci/modules/puppet-gitlab_ci_runner)
---> syntax:manifests
---> syntax:templates
---> syntax:hiera:yaml
I, [2021-02-08T14:17:02.455330 #893550]  INFO -- : Creating symlink from spec/fixtures/modules/gitlab_ci_runner to /home/paul/work/esa_auto/puppet/vagrant/vagrant-gitlabci/modules/puppet-gitlab_ci_runner
4 processes for 4 specs, ~ 1 specs per process

An error occurred while loading ./spec/tasks/register_runner_spec.rb. - Did you mean?
                    rspec ./spec/tasks/unregister_runner_spec.rb

Failure/Error: require_relative '../../tasks/register_runner.rb'

LoadError:
  cannot load such file -- /home/paul/work/esa_auto/puppet/vagrant/vagrant-gitlabci/modules/ruby_task_helper/files/task_helper.rb
# ./tasks/register_runner.rb:7:in `require_relative'
# ./tasks/register_runner.rb:7:in `<top (required)>'
# ./spec/tasks/register_runner_spec.rb:2:in `require_relative'
# ./spec/tasks/register_runner_spec.rb:2:in `<top (required)>'
No examples found.


Finished in 0.00004 seconds (files took 0.87794 seconds to load)
0 examples, 0 failures, 1 error occurred outside of examples


An error occurred while loading ./spec/tasks/unregister_runner_spec.rb. - Did you mean?
                    rspec ./spec/tasks/register_runner_spec.rb

Failure/Error: require_relative '../../tasks/unregister_runner.rb'

LoadError:
  cannot load such file -- /home/paul/work/esa_auto/puppet/vagrant/vagrant-gitlabci/modules/ruby_task_helper/files/task_helper.rb
# ./tasks/unregister_runner.rb:7:in `require_relative'
# ./tasks/unregister_runner.rb:7:in `<top (required)>'
# ./spec/tasks/unregister_runner_spec.rb:2:in `require_relative'
# ./spec/tasks/unregister_runner_spec.rb:2:in `<top (required)>'
No examples found.


Finished in 0.00004 seconds (files took 0.88601 seconds to load)
0 examples, 0 failures, 1 error occurred outside of examples

^C
RSpec is shutting down and will print the summary report... Interrupt again to force quit.
RSpec is shutting down and will print the summary report... Interrupt again to force quit.


2 errors, 0 examples, 0 failures

Took 8 seconds
Tests Failed

Any additional information you'd like to impart

None

Affected Puppet, Ruby, OS and module versions/distributions

• Puppet: 7.14.0
• Ruby: 2.7.5p203
• Distribution: CentOS Stream 8
• Module version: 4.3.0

How to reproduce (e.g Puppet code you use)

include gitlab_ci_runner

What are you seeing

# dnf --refresh --disablerepo=\* --enablerepo=runner_gitlab-runner makecache
runner_gitlab-runner                            661  B/s | 862  B     00:01    
runner_gitlab-runner                             20 kB/s | 3.1 kB     00:00    
runner_gitlab-runner                            911  B/s | 862  B     00:00    
Error: Failed to download metadata for repo 'runner_gitlab-runner': repomd.xml GPG signature verification error: Bad GPG signature

What behaviour did you expect instead

The repo metadata signing is broken upstream, but we should be able to disable repo_gpgcheck. This is not possible in the current code.

Affected Puppet, Ruby, OS and module versions/distributions

• Puppet: 6.4.2
• Ruby: 2.5.3
• Distribution: CentOS 6.10
• Module version: master branch

How to reproduce (e.g Puppet code you use)

Add the gitlab_ci_runner class to your code specifying the builds_dir parameter, e.g.

class { 'gitlab_ci_runner':
  runner_defaults => {
    'url' => 'https://gitlab.example.com',
  },
  runners => {
    'example' => {
      'executor' => 'shell',
     },
  },
  builds_dir => '/path/to/builds',
}

What are you seeing

The builds_dir path is appended to config.toml outside of the [[runners]] section and is ignored:

concurrent = 1
check_interval = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "example"
  url = "https://gitlab.example.com"
  token = "foo"
  executor = "shell"
  [runners.custom_build_dir]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]
builds_dir = "/opt/sitespect/lib/perl/gitlab"

What behaviour did you expect instead

The builds_dir setting in config.toml should be in the [[runners]] section and properly indented in order to be correctly used by the gitlab-runner process.
E.G.

[[runners]]
  name = "example"
  url = "https://gitlab.example.com"
  token = "foo"
  executor = "shell"
  builds_dir = "/opt/sitespect/lib/perl/gitlab"
  [runners.custom_build_dir]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]

Any additional information you'd like to impart

The simple workaround for this is to add a 'builds_dir' key to the runners hash, specifying the value, e.g.

runners => {
  'example-runner' => {
    'executor' => 'shell',
    'builds_dir' => '/path/to/dir',
   },
}

Affected Puppet, Ruby, OS and module versions/distributions

• Puppet: 5.5.8
• Ruby: 2.5.1
• Distribution: Ubuntu Bionic
• Module version: 3029b6e

How to reproduce (e.g Puppet code you use)

The existing module.

What are you seeing

I was attempting a manual upgrade to gitlab runner along with needing to reregister the runner against gitlab, the easiest way at the seemed to be to just delete /etc/gitlab-runner/config.toml

When I then ran puppet the module failed because it tried to ensure a file_in_line on the config file with out it existing.

What behavior did you expect instead

I expected the module to:

1. have all file_line resources require that config.toml exists.
   and
2. create a blank file if it doesn't exist so the configuration can be populated

Output log

/opt/puppetlabs/puppet/bin/puppet agent --server server1 --verbose --test
....

Error: /Stage[main]/Gitlab_ci_runner/File_line[gitlab-runner-concurrent]: Could not evaluate: No such file or directory @ rb_sysopen - /etc/gitlab-runner/config.toml
Notice: /Stage[main]/Gitlab_ci_runner/Exec[gitlab-runner-restart]: Dependency File_line[gitlab-runner-concurrent] has failures: true
Warning: /Stage[main]/Gitlab_ci_runner/Exec[gitlab-runner-restart]: Skipping because of failed dependencies

touch /etc/gitlab-runner/config.toml
/opt/puppetlabs/puppet/bin/puppet agent --server server1 --verbose --test
...

Notice: /Stage[main]/Gitlab_ci_runner/File_line[gitlab-runner-concurrent]/ensure: created
Info: /Stage[main]/Gitlab_ci_runner/File_line[gitlab-runner-concurrent]: Scheduling refresh of Exec[gitlab-runner-restart]
Notice: /Stage[main]/Gitlab_ci_runner/Exec[gitlab-runner-restart]: Triggered 'refresh' from 1 event
Notice: Applied catalog in 2.89 seconds

Any additional information you'd like to impart

Above output has been trimmed. As you can see simply touching the file allowed the module to work as expected.

Hi,

on Debian and its derivatives the repo.pp manifest tells Puppet to attempt to download the Gitlab Apt signing key with the fingerprint F6403F6544A38863DAA0B6E03F01618A51312F3F and to add it to the trusted keys. The source for this key is $repo_keyserver, which is an optional parameter for the init.pp and is set to undef by default.
https://github.com/voxpupuli/puppet-gitlab_ci_runner/blob/master/manifests/repo.pp#L19
https://github.com/voxpupuli/puppet-gitlab_ci_runner/blob/master/manifests/init.pp#L69

The repo.pp manifest uses a defined type from the puppetlabs-apt module (apt::source) to actually download the key. That module has some default values, and if no value for $keyserver is provided it uses a default value of "keyserver.ubuntu.com"
https://github.com/puppetlabs/puppetlabs-apt/blob/main/manifests/params.pp#L21

without going into more detail, in the end Puppet attempts to do this:
/usr/bin/apt-key adv --keyserver keyserver.ubuntu.com --recv-keys F6403F6544A38863DAA0B6E03F01618A51312F3F

The problem is that the Ubuntu keyserver doesn't seem to have this key any more. In fact, the Gitlab documentation only asks users to download the key from packages.gitlab.com and then add the local key file with apt-key.
https://docs.gitlab.com/omnibus/update/package_signatures#package-repository-metadata-signing-keys

If I haven't missed anything this module effectively won't work at all on Debian-based systems until changes are made to it.

The puppetlabs-apt module's Apt::Key defined type has a parameter "source" that would probably fix this problem, as the key could be added directly from packages.gitlab.com:
https://github.com/puppetlabs/puppetlabs-apt/blob/main/manifests/key.pp#L24

If you could confirm my findings (maybe I just missed the key on keyserver.ubuntu.com and it's actually there?) then I can create a PR to fix this issue.

Migrated from voxpupuli/puppet-gitlab#150, originally opened by @gdubicki

Pleae make this module optionally do step 4. from https://docs.gitlab.com/ce/ci/docker/using_docker_build.html#use-shell-executor in case you want to use the runner for building Docker images.

You can do it with a hacky-but-working:

exec {'add gitlab-runner to docker group':
    command => '/sbin/usermod -aG docker gitlab-runner',
    unless  => '/bin/grep -q "docker\\S*gitlab-runner" /etc/group',
}

Affected Puppet, Ruby, OS and module versions/distributions

• Puppet: 6.24.0
• Ruby:
• Distribution: Ubuntu 20.04.3 LTS
• Module version: v4.0.0

How to reproduce (e.g Puppet code you use)

class { 'gitlab_ci_runner':
    concurrent => 1,
    runners => {
      'my-runner' => {
        'registration-token' => '...',
        'url'                => 'https://...',
        'tag-list'           => 'mytag',
        'executor'           => 'docker+machine',
        'docker-image'       => 'ubuntu:focal',
     },
   },
}

What are you seeing

Error: Failed to apply catalog: SSL_connect returned=1 errno=0 state=error: certificate verify failed (self signed certificate in certificate chain)

What behaviour did you expect instead

...
Notice: Applied catalog in x seconds

Output log

Any additional information you'd like to impart

• Downgrading to gitlab_runner v3.0.0 fixes it, which is why I am posting it here.
• The gitlab instance is behind a https proxy which is properly signed using a custom CA certificate which is installed on the host.

should be using Ruby 2.5

How to reproduce (e.g Puppet code you use)

Hiera Code:

gitlab_ci_runner::runners:
  test_runner3:
    url: "https://git.alternative.org/ci"
    registration-token: "abcdef1234567890"
    locked: "false"

What are you seeing

At the command line it is represented:

--locked "false"

What behaviour did you expect instead

According to https://gitlab.com/gitlab-org/gitlab-runner/merge_requests/758
Boolean values must be declared with equal sign e.g. --flag=x

Any additional information you'd like to impart

This does work but at the same time it's awful hack:

locked=false:

See #46 (comment)

Gitlab provides an interesting case where it only supports the last 3 releases, (and they release monthly on the 22nd. With the puppet-gitlab module, being fairly stable, usually new releases for our module don't provide new features, they just accommodate new config options in the omnibus config file.

That allows us to tell users to use an older version of the module if they're using an older (non-supported) version of gitlab.

We should probably think about how this plays out with gitlab-runners. Runner releases are loosely versioned along with GitLab, but updating gitlab does not update runners. It is not uncommon for a gitlab installation following zero day releases to have runners at least one major version behind registered with it.

Should this module adopt a support strategy similar to the puppet-gitlab module, or should we strive for a greater period of backwards compatibility as much as possible?

Note: Gitlab will likely be incrementing major versions from v11 to v12 in July.

Gitlab Runner Metrics server option rename

metrics_server is being depreciated, and will be replaced with the listen_address option.

See here: https://docs.gitlab.com/runner/monitoring/#configuration-of-the-metrics-http-server

GitLab session_server config

The session_server section should be configurable through the module. I need to fully investigate if this has been implemented but it seems not from a cursory look over the code.
Here is a link to the config section in the GitLab docs: https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-session_server-section

Affected Puppet, Ruby, OS and module versions/distributions

• Puppet: 6.24.0
• Ruby: ruby 2.5.9p229 (2021-04-05 revision 67939) [x86_64-linux]
• Distribution: Ubuntu 20.04.3 LTS
• Module version: v4.0.0

How to reproduce (e.g Puppet code you use)

contain ::gitlab_ci_runner

---
gitlab_ci_runner::concurrent: 4

gitlab_ci_runner::runner_defaults:
  url: 'https://...'
  registration-token: '...'
  executor: 'docker'
  docker:
    image: 'ubuntu:focal'
  builds_dir: '/tmp'
  cache_dir: '/tmp'

gitlab_ci_runner::runners:
  my_test_repo:
    registration-token: '...'

What are you seeing

# puppet agent -t
Info: Using configured environment '...'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Retrieving locales
Info: Loading facts
Error: Failed to apply catalog: Gitlab runner failed to register: Forbidden

What behaviour did you expect instead

Notice: Applied catalog in XX.xx seconds

Output log

# puppet agent -t --debug
[...]
Debug: Caching connection for https://compiler:port
Error: Failed to apply catalog: Gitlab runner failed to register: Forbidden
Debug: Resolving service 'report' using Puppet::HTTP::...
[...]

Any additional information you'd like to impart

• There is no additional debug output printed when using puppet agent -t --debug or puppet agent -t --http_debug
• The interactive gitlab-runner register command succeds (same URI and token as in hieradata)

This module should manage the build_dir and cache_dir config file directives as well as correctly populating the command line args string during runner registration if used.

Affected Puppet, Ruby, OS and module versions/distributions

• Puppet: 6.25.1
• Ruby: 2.5.9
• Distribution: Ubuntu 20.04
• Module version: 4.3

How to reproduce (e.g Puppet code you use)

Specify a custom CA file for an internal CA. Which does not exist. (Maybe Puppet hasn't created it yet)

gitlab_ci_runner::ca_file: "/etc/ssl/certs/mycoolca.pem"

This error will also been seen if the Gitlab instance is running an untrusted CA.

What are you seeing

The following error is displayed. This is confusing to users and does not allow the Puppet run to continue as the deferred function failing stops the entire run.

root@ip-172-31-5-231:/home/ubuntu# puppet agent -t
Info: Using environment 'production'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Retrieving locales
Info: Loading facts
Error: Failed to apply catalog: SSL_connect returned=1 errno=0 state=error: certificate verify failed (unable to get local issuer certificate)
root@ip-172-31-5-231:/home/ubuntu# 

What behaviour did you expect instead

The deferred function should check to confirm if the file exists when ca_file is specified. If it does not it should return an empty token string.

Something like this

        if ca_file != nil
          if !File.exist?(ca_file)
            return 'CA file doesn\'t exist, not creating authtoken'
          end
        end

Any additional information you'd like to impart

I haven't tested this on any other platforms but I assume it would affect them all.

We should check for the existence of the ca_file on disk in the register_to_file.rb. If it doesn't exist we should return an empty token and assume the Puppet run will configure it. Failing the entire run is not optimal as it prevents users from running Puppet.

This is the first release of puppet/gitlab_ci_runner. The functionality in this module was previously part of puppet/gitlab

Hi!

since the module puppet/gitlab released 3.0.0 without runner management this module should be released to cover the missing functionality.

These are removed in #104

Hi,

this module has not seen a new release for a year now, despite the fact that there have been quite a few updates and fixes since then:

v1.0.0...master

Thank you for maintaining this module.

Can some1 point me out how can I add sysctls with hiera. I have tryed many ways and always fails. I cant figure out the right syntax.

if I put like this
["vm.swappiness=0","vm.overcommit_memory=1"] in the config it apears
"vm.swappiness=0" = ""
"vm.overcommit_memory=1" = ""

or:
docker-sysctls:
vm.swappiness: 0
will make this registration parameter and fails.
--docker-sysctls={vm.swappiness => 0} --registration-token

any help would be greatly appreciated

Affected Puppet, Ruby, OS and module versions/distributions

• Puppet: 8.0.0
• Ruby: 3.2.2 (packaged with puppet)
• Distribution: Debian 11.6
• Module version: 4.4.0

How to reproduce (e.g Puppet code you use)

class { 'profile::docker':
  manage_iptables => false,
}

ensure_packages(['git-lfs', ], { 'ensure' => 'present' })

class { 'gitlab_ci_runner':
    runners => {
      "${facts['networking']['fqdn']}" => {
        'registration-token' => 'myregtoken',
        'url'                => 'mygitlabhost',
        'tag-list'           => "docker,${facts['os']['family']}",
        'executor'           => 'docker',
        'docker'             => {
          'image'      => 'debian:latest',
          'privileged' => true,
        },
      },
    },
  }
}

What are you seeing

When executing the agent, it fails when using the Deferred function:

Error: /Stage[main]/Gitlab_ci_runner::Config/Concat[/etc/gitlab-runner/config.toml]/Concat_file[/etc/gitlab-runner/config.toml]: Failed to generate additional resources using 'eval_generate': no implicit conversion of Puppet::Pops::Evaluator::DeferredValue into String

What behaviour did you expect instead

It should not abort. Pre-Puppet v8.0 this has been working fine

Output log

See above.

Any additional information you'd like to impart

I was able to pin the issue to the Deferred calls for register_to_file (and to_toml):

https://github.com/voxpupuli/puppet-gitlab_ci_runner/blob/master/manifests/runner.pp#L92
https://github.com/voxpupuli/puppet-gitlab_ci_runner/blob/master/manifests/runner.pp#L107

But I did not (yet) find a better way on how to handle this. At least to me, it seems that this should/must work.

I'm not sure whether it's an issue in the underlying Concat-Module or if its the way Deferred functions are used here.