petems / cucumber-puppet

Specifying Puppet catalog behavior with Cucumber

cucumber-puppet

The project was discontinued. I'm seeing if I can resurect it

Currently cucumber-puppet doesnt support Ruby 1.9+ or any puppetversion beyond 2.7...

Cucumber-puppet is a tool for specifying Puppet catalog behavior. It uses Cucumber features as specifications and provides the necessary glue to access a node's catalog from Cucumber's step definitions.

Quickstart

Install cucumber-puppet

$ gem install cucumber-puppet

Go to your Puppet directory

$ cd puppet

Initialise the infrastructure

$ cucumber-puppet-gen world

Create a default policy and adapt it to your needs

$ cucumber-puppet-gen policy
$ vim features/catalog/policy.feature

Copy YAML node files into place

$ mkdir features/yaml
$ cp /var/lib/puppet/yaml/node/localhost.example.com.yaml features/yaml

Apply your policy

$ cucumber-puppet features/catalog/policy.feature

An example use case can be found at https://github.com/nistude/cucumber-puppet.example.

Further documentation is available in the wiki.

Acknowledgements

Many thanks to Bernd Ahlers for getting me started with cucumber, rspec and ruby.

cucumber-puppet was inspired by Lindsay Holmwood's cucumber-nagios.

cucumber-puppet's Issues

Debugging advice

Nikolay,

This is likely not a bug (tell me if I had to ask you somewhere else). I need an advice how to resolve one issue.

I have a Puppet catalog, it compiles and works in production just fine. I have a cucumber-puppet's "feature", very simple one:

$ cat features/catalog/policy.feature 
Feature: General policy for all catalogs
  In order to ensure applicability of a host's catalog
  As a manifest developer
  I want a catalog apply to a real node

  Scenario Outline: Compile and verify catalog for r4
    Given a node specified by "features/yaml/r4.osinka.int.yaml"
    When I compile its catalog
    Then compilation should succeed
    And all resource dependencies should resolve

    Examples:
      | hostname  | operatingsystem | kernel | is_virtual | domain        |
      | r4        | Debian          | Linux  | false      | osinka.int |

Unfortunately, it does not compile using cucumber-puppet and I have no idea what's wrong. The transcript:

$ cucumber-puppet -v features/catalog/policy.feature 
Code:
  * /Users/alaz/projects/confmgmt/features/support/hooks.rb
  * /Users/alaz/projects/confmgmt/features/support/world.rb
  * /Users/alaz/projects/confmgmt/features/steps/cron.rb
  * /Users/alaz/projects/confmgmt/features/steps/exec.rb
  * /Users/alaz/projects/confmgmt/features/steps/file.rb
  * /Users/alaz/projects/confmgmt/features/steps/package.rb
  * /Users/alaz/projects/confmgmt/features/steps/puppet.rb
  * /Users/alaz/projects/confmgmt/features/steps/service.rb
  * /Users/alaz/projects/confmgmt/features/steps/user.rb
Failed to load 'yaml' programming language for file /Users/alaz/projects/confmgmt/features/yaml/r4.osinka.int.yaml: no such file to load -- cucumber/yaml_support/yaml_language
  * /Users/alaz/projects/confmgmt/features/yaml/r4.osinka.int.yaml [NOT SUPPORTED]

Features:
  * features/catalog/policy.feature
Parsing feature files took 0m0.003s

Feature: General policy for all catalogs
  In order to ensure applicability of a host's catalog
  As a manifest developer
  I want a catalog apply to a real node

  Scenario Outline: Compile and verify catalog for r4            # features/catalog/policy.feature:6
    Given a node specified by "features/yaml/r4.osinka.int.yaml" # cucumber-puppet-0.3.3/lib/cucumber-puppet/steps.rb:1
    When I compile its catalog                                   # cucumber-puppet-0.3.3/lib/cucumber-puppet/steps.rb:14
    Then compilation should succeed                              # cucumber-puppet-0.3.3/lib/cucumber-puppet/steps.rb:43
    And all resource dependencies should resolve                 # cucumber-puppet-0.3.3/lib/cucumber-puppet/steps.rb:23

    Examples: 
      | hostname | operatingsystem | kernel | is_virtual | domain     |
      |warning: You cannot collect exported resources without storeconfigs being set; the collection will be ignored on line 15 in file /Users/alaz/projects/confmgmt/modules/munin/manifests/host.pp
warning: You cannot collect without storeconfigs being set on line 62 in file /Users/alaz/projects/confmgmt/modules/munin/manifests/client.pp
warning: You cannot collect without storeconfigs being set on line 71 in file /Users/alaz/projects/confmgmt/modules/munin/manifests/client.pp
warning: You cannot collect without storeconfigs being set on line 62 in file /Users/alaz/projects/confmgmt/features/support/../../modules/munin/manifests/client.pp
warning: You cannot collect without storeconfigs being set on line 71 in file /Users/alaz/projects/confmgmt/features/support/../../modules/munin/manifests/client.pp
 r4       | Debian          | Linux  | false      | osinka.int |
      Catalog compilation failed. (RuntimeError)
      features/catalog/policy.feature:9:in `Then compilation should succeed'

Failing Scenarios:
cucumber features/catalog/policy.feature:6 # Scenario: Compile and verify catalog for r4

1 scenario (1 failed)
4 steps (1 failed, 1 skipped, 2 passed)
0m0.973s

What should I do to find the reason?

Get Running in Travis

Currently only runs on 1.8.7, so that'll need to be in the .travis.yml

Trace and debug

Sometimes cucumber fails annoyingly - it'd be great if the runner supported --verbose and --backtrace option being passed to cucumber.

Error running features

I get the following:

cucumber-puppet features/modules/test/init.feature 
Code:
  * features/support/hooks.rb
  * features/support/world.rb

Features:
  * features/modules/test/init.feature
Parsing feature files took 0m0.096s

Code:
  * features/steps/cron.rb
  * features/steps/exec.rb
  * features/steps/file.rb
  * features/steps/package.rb
  * features/steps/puppet.rb
  * features/steps/service.rb
  * features/steps/user.rb

Feature: init
  In order to test my test module
  As an admin
  I want the test cron job installed
  And I want the /usr/bin/test  exec resource

  Scenario: resources present                       # features/modules/test/init.feature:7
    Given a node of class "pelin.lovedthanlost.net" # features/steps/puppet.rb:1
uninitialized constant Puppet::Node::Catalog
    When I compile the catalog                      # features/steps/puppet.rb:17
      exit (SystemExit)
      /usr/lib/ruby/gems/1.8/gems/cucumber-puppet-0.0.3/lib/cucumber-puppet/puppet.rb:52:in `exit'
      /usr/lib/ruby/gems/1.8/gems/cucumber-puppet-0.0.3/lib/cucumber-puppet/puppet.rb:52:in `compile_catalog'
      /etc/puppet/features/steps/puppet.rb:18:in `__instance_exec0'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/core_ext/instance_exec.rb:26:in `send'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/core_ext/instance_exec.rb:26:in `instance_exec'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/core_ext/instance_exec.rb:48:in `cucumber_instance_exec'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/core_ext/instance_exec.rb:69:in `cucumber_run_with_backtrace_filtering'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/core_ext/instance_exec.rb:36:in `cucumber_instance_exec'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/rb_support/rb_step_definition.rb:55:in `invoke'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/step_match.rb:24:in `invoke'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/ast/step_invocation.rb:59:in `invoke'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/ast/step_invocation.rb:38:in `accept'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/ast/tree_walker.rb:115:in `visit_step'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/ast/tree_walker.rb:180:in `broadcast'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/ast/tree_walker.rb:114:in `visit_step'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/ast/step_collection.rb:15:in `accept'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/ast/step_collection.rb:14:in `each'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/ast/step_collection.rb:14:in `accept'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/ast/tree_walker.rb:109:in `visit_steps'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/ast/tree_walker.rb:180:in `broadcast'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/ast/tree_walker.rb:108:in `visit_steps'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/ast/scenario.rb:47:in `accept'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/step_mother.rb:307:in `before_and_after'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/ast/scenario.rb:45:in `accept'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/ast/scenario.rb:100:in `with_visitor'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/ast/scenario.rb:39:in `accept'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/ast/tree_walker.rb:51:in `visit_feature_element'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/ast/tree_walker.rb:180:in `broadcast'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/ast/tree_walker.rb:50:in `visit_feature_element'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/ast/feature.rb:26:in `accept'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/ast/feature.rb:25:in `each'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/ast/feature.rb:25:in `accept'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/ast/tree_walker.rb:20:in `visit_feature'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/ast/tree_walker.rb:180:in `broadcast'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/ast/tree_walker.rb:19:in `visit_feature'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/ast/features.rb:29:in `accept'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/ast/features.rb:17:in `each'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/ast/features.rb:17:in `each'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/ast/features.rb:28:in `accept'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/ast/tree_walker.rb:14:in `visit_features'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/ast/tree_walker.rb:180:in `broadcast'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/ast/tree_walker.rb:13:in `visit_features'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/cli/main.rb:61:in `execute!'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/../lib/cucumber/cli/main.rb:20:in `execute'
      /usr/lib/ruby/gems/1.8/gems/cucumber-0.6.4/bin/cucumber:8
      /usr/bin/cucumber:19:in `load'
      /usr/bin/cucumber:19
      features/modules/test/init.feature:9:in `When I compile the catalog'
    Then cron job "test" should be "present"        # features/steps/cron.rb:1
    Then exec "/usr/bin/test" should be present     # features/steps/exec.rb:1

Failing Scenarios:
cucumber features/modules/test/init.feature:7 # Scenario: resources present

1 scenario (1 failed)
4 steps (1 failed, 2 skipped, 1 passed)
0m0.020s

Haven't had a chance to look at it this in any more detail - I'm running Puppet 0.24.4.

steps aren't easily usable without running generator

Ideally, these sweet puppet steps should be usable in an existing Cucucmber suite without running the generator. I've made some changes to cucumber-nagios in a fork that make this possible here:

http://github.com/jnewland/cucumber-nagios/commit/93e30bebd256d4fa384fa2ceb34d8c796b935453

Let me know if you have any questions, I can do this in a fork if needed.

Add support for defines in Given

Right now we can say
Given a node of class "foo"
...

But we can't say
Given a define named "baz" of type "foo::bar" with parameters
| name | value |
| derp | herp |
| zort | science |

Is this feature addable?
Thanks!

Fact hash

Hey - it'd be useful to load a YAML file containing facts rather than specify a hash. You can look at how the --compile code for puppetmasterd to see how Puppet does this.

Only 1.8.7 supported

Should support 1.8.7, 1.9.3 and 2.0.0

Find puppet configuration

This code:

before do
  # local configuration
  @confdir = File.dirname(__FILE__) + "/../../"
  # adjust facts like this
  @facts['architecture'] = "i386"
end

Is kind of ugly - since you know you need Puppet why not something like:

require 'puppet'
Puppet[:config] = "/etc/puppet/puppet.conf"
Puppet.parse_config

Then pull out any settings with: Puppet.settings[:settingname]

Support Puppet 3

Currently only supports Puppet 2.6 and 2.7