voxpupuli / puppet-extlib

This module provides functions that are out of scope for stdlib.

Extlib module for Puppet

Table of Contents

1. Overview
2. Setup - The basics of getting started with extlib
3. Usage - Configuration options and additional functionality
4. Limitations - OS compatibility, etc.
5. Development - Guide for contributing to the module

Overview

This module provides functions and facts that are out of scope for stdlib. Some of them are even intrinsically tied to stdlib.

Setup

 % puppet module install puppet-extlib

Usage

Since puppet/extlib version 3, all functions are provided under the extlib namespace. Non namespaced versions were removed in version 5.

All functions are documented in REFERENCE.md

Facts

All facts in this module are namespaced and begin extlib__. Facter 4 users can also find all facts under a single extlib toplevel structured fact.

extlib__puppet_config (or extlib['puppet_config'] when using facter 4)

A fact to expose puppet.conf settings. These are resolved on the agent, (unlike $settings::<setting_name> which is resolved on the puppet master).

The following sections/settings are included.

{
  main => {
    hostpubkey,
    hostprivkey,
    hostcert,
    localcacert,
    ssldir,
    vardir,
    server,
  },
  master => {
    localcacert,
    ssldir,
  }
}

Limitations

Some functions require puppetlabs-stdlib (>= 4.6.0) and all functions are only compatible with Puppet 4.7 and later.

Development

We highly welcome new contributions to this module, especially those that include documentation, and rspec tests ;) but will happily guide you through the process, so, yes, please submit that pull request!

Reference documentation is generated using puppet-strings. To regenerate it, please run the rake task as follows.

bundle exec rake reference

puppet-extlib's Issues

Move the gem provider logic from puppet-hiera to extlib

I'm working on voxpupuli/puppet-network#168, which requires using the gem provider. As @bastelfreak pointed out, there are more than one gem providers and the correct one needs to be selected. This is already implemented in puppet-hiera:
https://github.com/voxpupuli/puppet-hiera/blob/master/manifests/params.pp
I wouldn't like to just copy this into puppet-network, plus it might be needed again in the future in another module.
So, would it be possible to move it from puppet-hiera to extlib? Is it the right place to move it to or is there a better one instead?

document default_content function

we should document default_content in README.md

Convert all functions to 4.x API

• default_content
• echo
• ip_to_cron
• extlib::has_module
• extlib::sort_by_version
• random_password PR
• cache_data PR
• dump_args PR
• resources_deep_merge PR

Before a new (breaking/major) release, we could/should also...

• Namespace all functions PR
• Remove all function documentation from README and link to REFERENCE.MD PR
• Rewrite Limitations section PR
• Rewrite Development section PR

cache_data() stores data locally on puppetserver

Affected Puppet, Ruby, OS and module versions/distributions

• Puppet: server 2.7 in multi-puppetserver configuration with load-balancer
• Ruby: AIO version
• Distribution: community
• Module version:

How to reproduce (e.g Puppet code you use)

$password = cache_data('foreman_cache', 'postgres_db_passwd', getrandomstring())

What are you seeing

Cached data is stored in the puppetserver's local file system (/opt/puppetlabs/server/data/....) from where it is only available to the same puppet server. If a subsequent puppet agent run on the same client is served by another puppet server from the load-balancing pool, the supposedly cached data is not there, resulting in another (most likely, inconsistent) copy of data.

What behaviour did you expect instead

For this function to be useful in multi-puppetserver environment, the data cache should located in a location accessible to all servers, for instance on common shared storage or in a PuppetDB table.

Output log

Any additional information you'd like to impart

puppet_config / puppet_settings fact confined to Linux

in #149 (comment), @vchepkov very correctly notices:

why the fact is confined to Linux only? Seems universal to me

i agree.

Add encrypted_cache_data function

We would like a cache_data function that acts exactly the same, but also encrypts the yaml files with eyaml. This would be useful for us in a multi puppet master setup, so we can store the passwords on a nfs / shared storage without worrying about exposing passwords.

Would such a PR be desirable? Interest? Thoughts?

Release summary for 3.0.0

In this release, all functions have been moved to the extlib:: namespace and converted to the 4.X function API. Non name-spaced versions are still available, but deprecated and will be removed in a future major release.

During the conversion work, a couple of functions have minor but technically breaking changes. These are documented below.

Solving conflicting resources in mkdir_p

The mkdir_p function can be quite useful but it doesn't take much to make it unusable in the catalog due to double resource definitions. I basically recreated mkdir_p as a defined resource before finding extlib and solved this issue by adding a filter to the resulting array. Add:

.filter|$_dir| {!defined(File[$_dir])} # filter out File resources already defined in the catalog

to

puppet-extlib/functions/mkdir_p.pp

Line 16 in 37b738b

}.flatten.unique.sort

I've tested it and it works, it filters out already elsewhere defined resources (in my case some root directories which are created as mount points).

spec tests to 3.0

cue transpec

Module conflicts with puppetlabs/puppet_agent

fact puppet_config is already provided by puppetlabs/puppet_agent module and represents path to the configuration file itself. IMHO fact should be renamed to puppet_settings

https://github.com/puppetlabs/puppetlabs-puppet_agent/blob/master/lib/facter/settings.rb#L9-L13

import ip_to_cron

Does it make sense to integrate https://github.com/theforeman/puppet-puppet/blob/master/lib/puppet/parser/functions/ip_to_cron.rb ?

RFC: one or more of CIDR/IP address manipulation functions

I'd like to propose having extlib::parse_cidr() function which returns a Structure similar to what ipcalc.pl returns (with bit values stripped):

➜  ipcalc 192.168.0.0/24
Address:   192.168.0.0
Netmask:   255.255.255.0
Wildcard:  0.0.0.255
Network:   192.168.0.0/24
HostMin:   192.168.0.1
HostMax:   192.168.0.254
Broadcast: 192.168.0.255
Hosts/Net: 254

My original problem was to get first usable IP in a routable range I have.

Please add your ideas and use-cases in comments below

Document ip_to_cron in README

This function was imported from https://github.com/theforeman/puppet-puppet, but never documented in the README.

Unit tests are spread across 2 directories

Decide on spec/unit/puppet/parser/functions or spec/functions and move tests to single location.

ip_to_cron needs unit tests

Currently, the only test is

it 'should exist' do
    expect(Puppet::Parser::Functions.function('ip_to_cron')).to eq('function_ip_to_cron')
end

please make a fresh release

After importing #48, we should make a fresh release.

---

Should we declare this module stable, releasing 1.0.0?

Poor documentation of default_content()

Current documentation states that the function:
"Takes an optional content and an optional template name and returns the contents of a file."

It does not return "the contents of a file". It returns the first argument (if not empty or undef) otherwise it uses the second argument as the file name of a template, and returns the evaluated template. If both arguments are empty or undef, the function returns undef.

The example show that one intended use case is to use the result as the content attribute of a file type. Maybe this is why the documentation ended up like it is?

deployment, how does it work!?

thanks to @nibalizer's #5 we've setup auto-deployment.

we've now reduced the amount of noise #6 & #8, and, set our currect forge name #7 (we've got the /puppet namespace. isn't that cool ;)

i've pushed out new tags: (v0.9.3), and travis "built" and deployed it to the forge.

the problem is, that it built and attempted to deploy this thing seven times:

https://travis-ci.org/puppet-community/puppetcommunity-extlib/jobs/54677916
https://travis-ci.org/puppet-community/puppetcommunity-extlib/jobs/54677915
https://travis-ci.org/puppet-community/puppetcommunity-extlib/jobs/54677914
https://travis-ci.org/puppet-community/puppetcommunity-extlib/jobs/54677913
https://travis-ci.org/puppet-community/puppetcommunity-extlib/jobs/54677912
https://travis-ci.org/puppet-community/puppetcommunity-extlib/jobs/54677911
https://travis-ci.org/puppet-community/puppetcommunity-extlib/jobs/54677910
https://travis-ci.org/puppet-community/puppetcommunity-extlib/jobs/54677909

and, i have no idea what a successful attempt looks like.

import add sorted_json function from solarkennedy/puppet-consul

can we add the sorted_json function here so that we can reference is from other module. aka the nomad module also depends on it. but instead of declaring the consul module as a dependency the code was copied as is but now trails with its updated

New Release

Any chance for a new release with the new functionality that has been added?

[Feature] `cache_data` expire time

Affected Puppet, Ruby, OS and module versions/distributions

• Puppet: 7.14
• Ruby: 3.0
• Distribution: Fedora
• Module version: 6.0.0

What behaviour did you expect instead

Could an additional parameter be added to the cache_data function setting the max lifetime of the file? This would let me trivially rotate certain resources to a new value over time.

Any additional information you'd like to impart

This could have interesting interactions with facter's new caching infrastructure.

Add optional version parameter to has_module()

Currently extlib::has_module($name) can be used to check if a module is present. It could benefit from an optional parameter to check for a specific version (range). This could allow easy soft dependency checks with version numbers:

if $git {
  unless extlib::has_module('theforeman-git', '>= 5.0.0 < 8.0.0') {
    fail('theforeman-git 5.x, 6.x or 7.x is needed for this functionality')
  }
}

If the parameter is not specified, it should use the current behavior. It would be equivalent to having a default value of >= 0.