puppetlabs / beaker-puppet

Beaker's puppet-specific helper library

beaker-puppet: The Puppet-Specific Beaker Library

The purpose of this library is to hold all puppet-specific info & DSL methods. This includes all helper & installer methods.

It might not be up to that state yet, but that's the goal for this library. If you see anything puppet-specific that you'd like to pull into this library out of beaker, please do, we would love any help that you'd like to provide.

How Do I Use This?

You will need to include beaker-puppet alongside Beaker in your Gemfile or project.gemspec. E.g.

# Gemfile
gem 'beaker', '~>4.0'
gem 'beaker-puppet', '~>1.0'
# project.gemspec
s.add_runtime_dependency 'beaker', '~>4.0'
s.add_runtime_dependency 'beaker-puppet', '~>1.0'

For DSL Extension Libraries, you must also ensure that you require the library in your test files. You can do this manually in individual test files or in a test helper (if you have one). You can use Bundler.require() to require the library automatically. To explicitly require it:

require 'beaker-puppet'

Doing this will include (automatically) the beaker-puppet DSL methods in the beaker DSL. Then you can call beaker-puppet methods, exactly as you did before.

Running Puppet in debug mode

When using apply_manifest to run Puppet, it is common that you need debug output. To achieve this, the debug option can be passed.

apply_manifest_on(host, manifest, { debug: true })

This has the downside that Puppet always runs in debug mode, which is very verbose. Of course you can modify the spec, but that's tedious. An easier alternative is to use the environment variable BEAKER_PUPPET_DEBUG.

BEAKER_PUPPET_DEBUG=1 rspec spec/acceptance/my_spec.rb

How Do I Test This?

Unit / Spec Testing

You can run the spec testing using our rake task test:spec:run. You can also run rspec directly. If you'd like to see a coverage report, then you can run the test:spec:coverage rake task.

Acceptance Testing

Acceptance testing can be run using the acceptance rake test namespace. For instance, to test using our package installation, you can run the acceptance:pkg task.

Note in the above rake tasks that there are some environment variables that you can use to customize your run. For specifying your System Under Test (SUT) environment, you can use BEAKER_HOSTS, passing a file path to a beaker hosts file, or you can provide a beaker-hostgenerator value to the TEST_TARGET environment variable. You can also specify the tests that get executed with the TESTS environment variable.

License

This gem is licensed under the Apache-2 license.

Release information

To make a new release, please do:

• update the version in lib/beaker-puppet/version.rb
• Install gems with bundle install --with release --path .vendor
• generate the changelog with bundle exec rake changelog
• Check if the new version matches the closed issues/PRs in the changelog
• Create a PR with it
• After it got merged, push a tag. GitHub actions will do the actual release to rubygems and GitHub Packages

beaker-puppet's Issues

Let me inject the cmdline_args parameter for the install_package method

I want to use the cmdline_args parameter of the host.install_package method on install_puppet_agent_on to inject the argument "-o Dpkg::Options::='--force-confold'".
I have some files already present that are also inside the puppet agent package and right now beaker gets stuck on the interactive apt install dialog where it asks me if it is okay to overwrite these files.

It is the second parameter of this method call:
https://github.com/voxpupuli/beaker-puppet/blob/0d4080c8f3c57dbcb87cc296a3164b006b7b3eea/lib/beaker-puppet/install_utils/foss_utils.rb#L437

Recent changes broke install_puppet_agent_pe_promoted_repo_on

We use the install_puppet_agent_pe_promoted_repo_on method to test installing older puppet-agents and test them out against new PE environments setup with newer puppetservers.
A recent change to the install_puppet_agent_pe_promoted_repo_on recently broke installing older agents (pre PE 2019.0 at least).
I believe the break down is in the method we changed how the puppet_collection is set.
We use to use the install path as this when the agent version is puppet_agent_version=>"5.5.3":
/root/repos/el/7/PC1/x86_64/puppet-agent-*.rpm
Which use to work, but now when we attempt to install we're using this:
/root/repos/el/7/puppet5/x86_64/puppet-agent-*.rpm
Notice the change from PC1 to puppet5, which looks to break things.

`ci:test:aio` skips the `post-suite` if any tests fail, but `ci:test:aio[true]` doesn't

Describe the Bug

When running the ci:test:aio rake task (with retries disabled), if any test fails, then the post-suite is skipped:

beaker-puppet/tasks/ci.rake

Lines 320 to 323 in 5f0984a

	beaker(:exec, ENV.fetch('TESTS', nil))
	beaker(:exec, 'post-suite')
	ensure
	preserve_hosts = ENV['OPTIONS'].include?('--preserve-hosts=always') if ENV['OPTIONS']

This is because beaker(:exec, ...) raises if any test fails and the post-suite isn't in an ensure block.

However, ci:test:aio[true] (with retries enabled) does execute the post-suite, because it's been moved to the ensure block.

beaker-puppet/tasks/ci.rake

Lines 348 to 349 in 5f0984a

	ensure
	beaker(:exec, 'post-suite')

Expected Behavior

The post-suite should always be executed irrespective of whether retries are enabled or not.

Also if the post-suite fails/raises, then we should ensure hosts are still destroyed too

Steps to Reproduce

Steps to reproduce the behavior:

git clone https://github.com/puppetlabs/facter
cd facter
patch -p1 << 'EOF' 
diff --git a/acceptance/tests/load_libfacter.rb b/acceptance/tests/load_libfacter.rb
index 1bc61ffd1..fe93a25bb 100644
--- a/acceptance/tests/load_libfacter.rb
+++ b/acceptance/tests/load_libfacter.rb
@@ -2,6 +2,8 @@
 test_name 'C100161: Ruby can load libfacter without raising an error' do
   tag 'risk:high'

+  raise "Whoops"
+
   require 'puppet/acceptance/common_utils'
   extend Puppet::Acceptance::CommandUtils
EOF
cd acceptance
bundle install
bundle exec rake ci:test:aio SHA=8.4.0 TESTS=tests/load_libfacter.rb HOSTS=redhat8-64a
...
...notice `bundle exec beker exec post-suite` is never executed
...

But when retries are enabled, the post-suite is executed:

bundle exec rake 'ci:test:aio[true]' SHA=8.4.0 TESTS=tests/load_libfacter.rb HOSTS=redhat8-64a

Environment

bundle exec gem list

*** LOCAL GEMS ***

ansi (1.5.0)
ast (2.4.2)
bcrypt_pbkdf (1.1.0)
beaker (4.41.2)
beaker-abs (0.11.0)
beaker-hostgenerator (2.9.0)
beaker-puppet (1.29.0)
beaker-vagrant (0.7.1)
beaker-vmpooler (1.4.0)
bundler (2.4.19)
commander (4.6.0)
deep_merge (1.2.2)
ed25519 (1.3.0)
faraday (1.10.3)
faraday-em_http (1.0.0)
faraday-em_synchrony (1.0.0)
faraday-excon (1.1.0)
faraday-httpclient (1.0.1)
faraday-multipart (1.0.4)
faraday-net_http (1.0.1)
faraday-net_http_persistent (1.2.0)
faraday-patron (1.0.0)
faraday-rack (1.0.0)
faraday-retry (1.0.3)
highline (2.0.3)
hocon (1.4.0)
in-parallel (0.1.17)
inifile (3.0.0)
minitar (0.9)
minitest (5.21.2)
multi_json (1.15.0)
multipart-post (2.3.0)
net-scp (4.0.0)
net-ssh (7.2.1)
oga (3.4)
rake (13.1.0)
rexml (3.2.6)
rsync (1.0.9)
ruby-ll (2.1.3)
ruby2_keywords (default: 0.0.5)
stringify-hash (0.0.2)
thor (1.3.0)
vmfloaty (1.8.1)

Useless 'PATH' string prepended to PATH environment variable

Currently the string PATH is being prepended to the PATH environment variable on SUTs.

This can be seen in pry:

> fact_on(host, 'path')
=> "PATH:/opt/puppetlabs/bin:/opt/puppetlabs/puppet/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin"

I believe I've tracked this down to https://github.com/voxpupuli/beaker-puppet/blob/ce5afbd0c239c99d963b63ac61558733f5dc9da6/lib/beaker-puppet/install_utils/puppet_utils.rb#L59.

Drop inifile dependency

The inifile gem is used a few places (puppet-helpers and tk-helpers) in beaker-puppet to generate INI files.

This gem is unmaintained and quite old (last commits were ~8 years ago). inifile uses taint methods, which were deprecated in Ruby 2.7 and completely removed in Ruby 3.2 (see https://bugs.ruby-lang.org/issues/16131).

beaker-puppet should drop its dependency on inifile to avoid incompatibility with these methods and any potential future issues.

#clone_git_repo_on is able to fail silently

If the git clone operation fails this method does not raise an exception.

Migrate gem back to puppetlabs

This gem was migrated together with all other beaker gems. Back then it was used by Vox Pupuli, but now we've migrated to beaker_puppet_helpers which is a really stripped down version of this gem.

beaker-puppet contains a lot of logic to test puppetserver and the only user of that is puppetlabs. That's why I'm proposing to migrate it back.