GithubHelp home page GithubHelp logo

puppetlabs / puppetlabs-stdlib Goto Github PK

View Code? Open in Web Editor NEW
353.0 193.0 580.0 4.35 MB

Puppet Labs Standard Library module

Home Page: http://forge.puppetlabs.com/puppetlabs/stdlib

License: Apache License 2.0

Ruby 96.04% Puppet 2.39% Pascal 1.52% Dockerfile 0.04%
module supported hacktoberfest

puppetlabs-stdlib's Introduction

stdlib

Table of Contents

  1. Overview
  2. Module Description
  3. Setup
  4. Usage
  5. Reference
    1. Data Types
    2. Facts
  6. Limitations
  7. License
  8. Development
  9. Contributors

Overview

This module provides a standard library of resources for Puppet modules.

Module Description

Puppet modules make heavy use of this standard library. The stdlib module adds the following resources to Puppet:

  • Stages
  • Facts
  • Functions
  • Defined types
  • Data types
  • Providers

Note: As of version 3.7, Puppet Enterprise no longer includes the stdlib module. If you're running Puppet Enterprise, you should install the most recent release of stdlib for compatibility with Puppet modules.

Setup

Install the stdlib module to add the functions, facts, and resources of this standard library to Puppet.

If you are authoring a module that depends on stdlib, be sure to specify dependencies in your metadata.json.

Usage

Most of stdlib's features are automatically loaded by Puppet. To use standardized run stages in Puppet, declare this class in your manifest with include stdlib.

When declared, stdlib declares all other classes in the module. This currently consists of stdlib::manage and stdlib::stages.

The stdlib::stages class declares various run stages for deploying infrastructure, language runtimes, and application layers. The high level stages are (in order):

  • setup
  • main
  • runtime
  • setup_infra
  • deploy_infra
  • setup_app
  • deploy_app
  • deploy

Sample usage:

node default {
  include stdlib
  class { java: stage => 'runtime' }
}

The stdlib::manage class provides an interface for generating trivial resource declarations via the create_resources parameter. Depending on your usage, you may want to set hiera's lookup_options for the stdlib::manage::create_resources: element.

---
stdlib::manage::create_resources:
  file:
    /etc/somefile:
      ensure: file
      owner: root
      group: root
  package:
    badpackage:
      ensure: absent

Reference

For information on the classes and types, see the REFERENCE.md.

Data types

Stdlib::Absolutepath

A strict absolute path type. Uses a variant of Unixpath and Windowspath types.

Acceptable input examples:

/var/log
/usr2/username/bin:/usr/local/bin:/usr/bin:.
C:\\WINDOWS\\System32

Unacceptable input example:

../relative_path

Stdlib::Ensure::Service

Matches acceptable ensure values for service resources.

Acceptable input examples:

stopped
running

Unacceptable input example:

true
false

Stdlib::HTTPSUrl

Matches HTTPS URLs. It is a case insensitive match.

Acceptable input example:

https://hello.com

HTTPS://HELLO.COM

Unacceptable input example:

httds://notquiteright.org`

Stdlib::HTTPUrl

Matches both HTTPS and HTTP URLs. It is a case insensitive match.

Acceptable input example:

https://hello.com

http://hello.com

HTTP://HELLO.COM

Unacceptable input example:

httds://notquiteright.org

Stdlib::MAC

Matches MAC addresses defined in RFC5342.

Stdlib::Unixpath

Matches absolute paths on Unix operating systems.

Acceptable input example:

/usr2/username/bin:/usr/local/bin:/usr/bin:

/var/tmp

Unacceptable input example:

C:/whatever

some/path

../some/other/path

Stdlib::Filemode

Matches octal file modes consisting of one to four numbers and symbolic file modes.

Acceptable input examples:

0644
1777
a=Xr,g=w

Unacceptable input examples:

x=r,a=wx
0999

Stdlib::Windowspath

Matches paths on Windows operating systems.

Acceptable input example:

C:\\WINDOWS\\System32

C:\\

\\\\host\\windows

Valid values: A windows filepath.

Stdlib::Filesource

Matches paths valid values for the source parameter of the Puppet file type.

Acceptable input example:

http://example.com

https://example.com

file:///hello/bla

Valid values: A filepath.

Stdlib::Fqdn

Matches paths on fully qualified domain name.

Acceptable input example:

localhost

example.com

www.example.com

Valid values: Domain name of a server.

Stdlib::Host

Matches a valid host which could be a valid ipv4, ipv6 or fqdn.

Acceptable input example:

localhost

www.example.com

192.0.2.1

Valid values: An IP address or domain name.

Stdlib::Port

Matches a valid TCP/UDP Port number.

Acceptable input examples:

80

443

65000

Valid values: An Integer.

Stdlib::Port::Privileged

Matches a valid TCP/UDP Privileged port i.e. < 1024.

Acceptable input examples:

80

443

1023

Valid values: A number less than 1024.

Stdlib::Port::Unprivileged

Matches a valid TCP/UDP Privileged port i.e. >= 1024.

Acceptable input examples:

1024

1337

65000

Valid values: A number more than or equal to 1024.

Stdlib::Base32

Matches paths a valid base32 string.

Acceptable input example:

ASDASDDASD3453453

asdasddasd3453453=

ASDASDDASD3453453==

Valid values: A base32 string.

Stdlib::Base64

Matches paths a valid base64 string.

Acceptable input example:

asdasdASDSADA342386832/746+=

asdasdASDSADA34238683274/6+

asdasdASDSADA3423868327/46+==

Valid values: A base64 string.

Stdlib::Ipv4

This type is no longer available. To make use of this functionality, use Stdlib::IP::Address::V4.

Stdlib::Ipv6

This type is no longer available. To make use of this functionality, use Stdlib::IP::Address::V6.

Stdlib::Ip_address

This type is no longer available. To make use of this functionality, use Stdlib::IP::Address

Stdlib::IP::Address

Matches any IP address, including both IPv4 and IPv6 addresses. It will match them either with or without an address prefix as used in CIDR format IPv4 addresses.

Examples:

'127.0.0.1' =~ Stdlib::IP::Address                                # true
'10.1.240.4/24' =~ Stdlib::IP::Address                            # true
'52.10.10.141' =~ Stdlib::IP::Address                             # true
'192.168.1' =~ Stdlib::IP::Address                                # false
'FEDC:BA98:7654:3210:FEDC:BA98:7654:3210' =~ Stdlib::IP::Address  # true
'FF01:0:0:0:0:0:0:101' =~ Stdlib::IP::Address                     # true

Stdlib::IP::Address::V4

Match any string consisting of an IPv4 address in the quad-dotted decimal format, with or without a CIDR prefix. It will not match any abbreviated form (for example, 192.168.1) because these are poorly documented and inconsistently supported.

Examples:

'127.0.0.1' =~ Stdlib::IP::Address::V4                                # true
'10.1.240.4/24' =~ Stdlib::IP::Address::V4                            # true
'192.168.1' =~ Stdlib::IP::Address::V4                                # false
'FEDC:BA98:7654:3210:FEDC:BA98:7654:3210' =~ Stdlib::IP::Address::V4  # false
'12AB::CD30:192.168.0.1' =~ Stdlib::IP::Address::V4                   # false

Valid values: An IPv4 address.

Stdlib::IP::Address::V6

Match any string consisting of an IPv6 address in any of the documented formats in RFC 2373, with or without an address prefix.

Examples:

'127.0.0.1' =~ Stdlib::IP::Address::V6                                # false
'10.1.240.4/24' =~ Stdlib::IP::Address::V6                            # false
'FEDC:BA98:7654:3210:FEDC:BA98:7654:3210' =~ Stdlib::IP::Address::V6  # true
'FF01:0:0:0:0:0:0:101' =~ Stdlib::IP::Address::V6                     # true
'FF01::101' =~ Stdlib::IP::Address::V6                                # true

Valid values: An IPv6 address.

Stdlib::IP::Address::Nosubnet

Match the same things as the Stdlib::IP::Address alias, except it will not match an address that includes an address prefix (for example, it will match '192.168.0.6' but not '192.168.0.6/24').

Valid values: An IP address with no subnet.

Stdlib::IP::Address::V4::CIDR

Match an IPv4 address in the CIDR format. It will only match if the address contains an address prefix (for example, it will match '192.168.0.6/24' but not '192.168.0.6').

Valid values: An IPv4 address with a CIDR provided eg: '192.186.8.101/105'. This will match anything inclusive of '192.186.8.101' to '192.168.8.105'.

Stdlib::IP::Address::V4::Nosubnet

Match an IPv4 address only if the address does not contain an address prefix (for example, it will match '192.168.0.6' but not '192.168.0.6/24').

Valid values: An IPv4 address with no subnet.

Stdlib::IP::Address::V6::Full

Match an IPv6 address formatted in the "preferred form" as documented in section 2.2 of RFC 2373, with or without an address prefix as documented in section 2.3 of RFC 2373.

Stdlib::IP::Address::V6::Alternate

Match an IPv6 address formatted in the "alternative form" allowing for representing the last two 16-bit pieces of the address with a quad-dotted decimal, as documented in section 2.2.1 of RFC 2373. It will match addresses with or without an address prefix as documented in section 2.3 of RFC 2373.

Stdlib::IP::Address::V6::Compressed

Match an IPv6 address which may contain :: used to compress zeros as documented in section 2.2.2 of RFC 2373. It will match addresses with or without an address prefix as documented in section 2.3 of RFC 2373.

Stdlib::IP::Address::V6::Nosubnet

Alias to allow Stdlib::IP::Address::V6::Nosubnet::Full, Stdlib::IP::Address::V6::Nosubnet::Alternate and Stdlib::IP::Address::V6::Nosubnet::Compressed.

Stdlib::IP::Address::V6::Nosubnet::Full

Match an IPv6 address formatted in the "preferred form" as documented in section 2.2 of RFC 2373. It will not match addresses with address prefix as documented in section 2.3 of RFC 2373.

Stdlib::IP::Address::V6::Nosubnet::Alternate

Match an IPv6 address formatted in the "alternative form" allowing for representing the last two 16-bit pieces of the address with a quad-dotted decimal, as documented in section 2.2.1 of RFC 2373. It will only match addresses without an address prefix as documented in section 2.3 of RFC 2373.

Stdlib::IP::Address::V6::Nosubnet::Compressed

Match an IPv6 address which may contain :: used to compress zeros as documented in section 2.2.2 of RFC 2373. It will only match addresses without an address prefix as documented in section 2.3 of RFC 2373.

Stdlib::IP::Address::V6::CIDR

Match an IPv6 address in the CIDR format. It will only match if the address contains an address prefix (for example, it will match 'FF01:0:0:0:0:0:0:101/32', 'FF01::101/60', '::/0', but not 'FF01:0:0:0:0:0:0:101', 'FF01::101', '::').

Stdlib::ObjectStore

Matches cloud object store uris.

Acceptable input example:

s3://mybucket/path/to/file

gs://bucket/file

Valid values: cloud object store uris.

Stdlib::ObjectStore::GSUri

Matches Google Cloud object store uris.

Acceptable input example:

gs://bucket/file

gs://bucket/path/to/file

Valid values: Google Cloud object store uris.

Stdlib::ObjectStore::S3Uri

Matches Amazon Web Services S3 object store uris.

Acceptable input example:

s3://bucket/file

s3://bucket/path/to/file

Valid values: Amazon Web Services S3 object store uris.

Stdlib::Syslogfacility

An enum that defines all syslog facilities defined in RFC5424. This is based on work in the voxpupuli/nrpe module.

Facts

package_provider

Returns the default provider Puppet uses to manage packages on this system.

is_pe

Returns whether Puppet Enterprise is installed. Does not report anything on platforms newer than PE 3.x.

pe_version

Returns the version of Puppet Enterprise installed. Does not report anything on platforms newer than PE 3.x.

pe_major_version

Returns the major version Puppet Enterprise that is installed. Does not report anything on platforms newer than PE 3.x.

pe_minor_version

Returns the minor version of Puppet Enterprise that is installed. Does not report anything on platforms newer than PE 3.x.

pe_patch_version

Returns the patch version of Puppet Enterprise that is installed.

puppet_vardir

Returns the value of the Puppet vardir setting for the node running Puppet or Puppet agent.

puppet_environmentpath

Returns the value of the Puppet environment path settings for the node running Puppet or Puppet agent.

puppet_server

Returns the Puppet agent's server value, which is the hostname of the Puppet server with which the agent should communicate.

root_home

Determines the root home directory.

Determines the root home directory, which depends on your operating system. Generally this is '/root'.

service_provider

Returns the default provider Puppet uses to manage services on this system

Limitations

As of Puppet Enterprise 3.7, the stdlib module is no longer included in PE. PE users should install the most recent release of stdlib for compatibility with Puppet modules.

For an extensive list of supported operating systems, see metadata.json

License

This codebase is licensed under the Apache2.0 licensing, however due to the nature of the codebase the open source dependencies may also use a combination of AGPL, BSD-2, BSD-3, GPL2.0, LGPL, MIT and MPL Licensing.

Development

Puppet modules on the Puppet Forge are open projects, and community contributions are essential for keeping them great. We can’t access the huge number of platforms and myriad hardware, software, and deployment configurations that Puppet is intended to serve. We want to keep it as easy as possible to contribute changes so that our modules work in your environment. There are a few guidelines that we need contributors to follow so that we can have a chance of keeping on top of things. For more information, see our module contribution guide.

To report or research a bug with any part of this module, please go to https://github.com/puppetlabs/puppetlabs-stdlib/issues.

Contributors

The list of contributors can be found at: https://github.com/puppetlabs/puppetlabs-stdlib/graphs/contributors.

puppetlabs-stdlib's People

Contributors

adrienthebo avatar alexjfisher avatar b4ldr avatar binford2k avatar bmjen avatar chelnak avatar cmurphy avatar cyberious avatar daianamezdrea avatar david22swan avatar davids avatar eimlav avatar ekohl avatar eputnam avatar hail9000 avatar hlindberg avatar hunner avatar jordanbreen28 avatar kbarber avatar kwilczynski avatar lionce avatar lukasaud avatar martyewings avatar pmcmaw avatar ramesh7 avatar sanfrancrisko avatar sheenaajay avatar smortex avatar tphoney avatar transifex-bot avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

puppetlabs-stdlib's Issues

Attempt to redefine entity 'http://puppet.com/2016.1/runtime/type/package'

Describe the Bug

Puppet agent runs on my Puppet server (and only there) now produce the following error:

Info: Redefining package in Puppet::Type
Error: Facter: error while resolving custom facts in /opt/puppetlabs/puppet/cache/lib/facter/package_provider.rb Attempt to redefine entity 'http://puppet.com/2016.1/runtime/type/package'. Originally set at file:///opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/type/package.rb?line=11.

Both the agent and the server were restarted today, one of these restarts triggered the error, which now occurs on every run.

More restarts and reboots have not made the error go away, but commenting out require 'puppet/type/package' in lib/facter/package_provider.rb does.

The error only occurs on Puppet agent runs, running 'puppet facts' is not affected.

Environment

  • Debian Bullseye
  • Puppetserver 7.9.1-1bullseye
  • Puppet-agent 7.19.0-1bullseye
  • Puppetlabs-stdlib 8.4.0

Version release

Hi,

Do you have an ETA of when the next version will be released? Still having the issue of fqdn_rotate causing changes on each run in Puppet 8 (fixed in #1341) and it's hard to find where actual changes are occurring.

Thanks
Ian

Calls to `Deprecation` function cause catalog compilation to fail if `strict` setting is set to `error`

In stdlib 9.0 many functions were namespaced and non-namespaced deprecated shims were introduced to maintain compatibility with modules that haven't yet switched over to using the namespaced versions.

This largely works as intended, but (originally in #1365) it's been noted that the deprecation function will actually fail a catalog compile when strict is set to error, and that this is also the default in the upcoming Puppet 8.

This makes it very difficult for users to upgrade to stdlib 9 unless they're prepared to turn the strict setting back to off or warning (and there are many other good reasons for setting strict to error). Since lots of modules are now also being updated to use the namespaced versions (requiring stdlib 9.0), upgrading a single affected component module now means having to upgrade all modules that use stdlib functions.

Possible solutions include:

  • Still leaving the shims documented as deprecated, but removing the calls to the deprecation function
  • Adjusting deprecation so that it never raises an error

(This issue replaces #1365 where 2 separate issues were being discussed)

loadjson returns StringIO data which doesn't work in Puppet 8

Describe the Bug

When using loadjson in a puppet manifest with small json files, the returning Data values come in as a StringIO instead of String value which Puppet 8 can't cast properly.

Expected Behavior

Return a hash of Strings.

Steps to Reproduce

Steps to reproduce the behavior:

  1. Use loadjson as documented to pull in a json file that you want to convert to hash.
  2. Run puppet agent and see the error.

Environment

  • RedHat Enterprise Linux 8
  • Puppet 8.3.0

Additional Context

I was able to fix this by editing the loadjson ruby code lines 56 and 63 changing from
JSON.parse(content) || args[1]
to
JSON.parse(content.string) || args[1]

The `deprecation` function and Puppet 8 are not compatible

Describe the Bug

I wanted to discuss the issue of puppet's strict mode and the deprecation function in the puppetlabs-stdlib module. This was discussed previously in:

#1365
#1373
https://tickets.puppetlabs.com/browse/PUP-11868

As described in those issues, the problem is that all deprecations become hard errors when "strict=error", which is the default in puppet8. In practice, this means you can't really use "strict=error" and need to relax the setting to "strict=warning".

Expected Behavior

I would expect the deprecation function to behave similarly to puppet when its Puppet.deprecation_warning method is called. That is, the strict setting should not control whether the deprecation warning is displayed or not. Instead, it should be controlled by the disable_warnings=deprecations setting, similar to how -Wno-deprecated-declarations can silence warnings in GCC.

From my perspective, the problem is that both puppet and stdlib are trying to decide how to handle deprecation warnings and the two approaches are not compatible. To understand the disconnect, suppose you call Puppet.deprecation_warning('message', 'key'). The warning is always displayed when strict is set to error, warning and off, respectively:

$ bundle exec ruby -rpuppet -e " \
  Puppet.initialize_settings; \
  Puppet::Util::Log.newdestination(:console); \
  Puppet[:strict] = 'error'; \
  Puppet.deprecation_warning('message', 'key')"
Warning: message
   (location: -e:1:in `<main>')

$ bundle exec ruby -rpuppet -e " \
  Puppet.initialize_settings; \
  Puppet::Util::Log.newdestination(:console); \
  Puppet[:strict] = 'warning'; \
  Puppet.deprecation_warning('message', 'key')"
Warning: message
   (location: -e:1:in `<main>')

$ bundle exec ruby -rpuppet -e " \
  Puppet.initialize_settings; \
  Puppet::Util::Log.newdestination(:console); \
  Puppet[:strict] = 'off'; \
  Puppet.deprecation_warning('message', 'key')"
Warning: message
   (location: -e:1:in `<main>')

In order to silence the deprecation, you can set disable_warnings=deprecations and this works even when strict=error:

$ bundle exec ruby -rpuppet -e " \
  Puppet.initialize_settings; \
  Puppet::Util::Log.newdestination(:console); \
  Puppet[:strict] = 'error'; \
  Puppet[:disable_warnings] = 'deprecations'; \
  Puppet.deprecation_warning('message', 'key')"
$ 

Steps to Reproduce:

I would expect the following to print a deprecation warning instead of failing compilation:

$ bundle exec puppet apply --strict=error puppet apply -e "deprecation('key', 'message')" 
Error: Evaluation Error: Error while evaluating a Function Call, deprecation. key. message at ["unknown", 1]:["unknown", 0] (line: 1, column: 1) on node localhost

I would expect the deprecation warning to be silenced and for compilation to succeed:

$ bundle exec puppet apply --strict=error --disable_warnings=deprecations puppet apply -e "deprecation('key', 'message')"
Error: Evaluation Error: Error while evaluating a Function Call, deprecation. key. message at ["unknown", 1]:["unknown", 0] (line: 1, column: 1) on node localhost

Proposal

  1. If the deprecation function is called with use_strict_setting=true and strict != :off, then only raise if disable_warnings doesn't include deprecations.

OR

  1. A bigger change would be to remove this line:
    raise("deprecation. #{key}. #{message}") if use_strict_setting && Puppet.settings[:strict] == :error

and always just call Puppet.deprecation_warning. It's possible that folks could be relying on the raise behavior? Though I think it's doubtful because strict=error wasn't actually usable prior to puppet8, see https://github.com/puppetlabs/puppet/wiki/Puppet-8-Compatibility#strict-mode

`Stdlib::Port` should (probably) not allow 0

Describe the Bug

README description of Stdlib::Port:
Matches a valid TCP/UDP Port number.
Port 0 is reserved for both TCP and UDP and most implementations will reject it. It is now used when binding to tell the kernel to find an available port. Hence, configurations that are pedantic will want to disable it to avoid misconfiguration.

Expected Behavior

Either the description should say "any TCP/UDP port" and there is a type to represent valid port numbers (Interger[1, 65535]) or the lower bound is changed from 0 to 1. The latter change should apply to any derived type too.

Additional Context

Technically (by the original RFC), port 0 was reserved, but the behavior it now has when calling bind makes allowing the value slightly dangerous: https://www.rfc-editor.org/rfc/rfc1340#page-7

Convert the Legacy Puppet functions to the Modern format

Since the release of the Modern Puppet function format several of the functions and all new ones to be added have been written in it, however the vast majority of the original functions remain in the Legacy format.

While they are still functional and will not currently cause any errors in their use, they are still outdated and using a format that has been fazed out, potential causing potential problems down the line.

As such it would be best to update them now before any issues can arise.

"is_email_address" deprecated function

Environment

  • Puppet: 7.9.5
  • Ruby: ruby 2.7.7p221 (2022-11-24 revision 168ec2b1e5) [x86_64-linux]
  • Distribution: Ubuntu 20.04 LTS

Additional Context

Please advice, in our code we use next block:

  unless is_email_address($admin_email) {
    fail('Invalid admin email address!')
  }

I can see that it was removed in v9.0.0 as "deprecated" function (f5c2710)

In comments to that PR:
"This PR remove a bunch of legacy functions that are now part of Puppet or which where added to stdlib to allow parameters validation prior to Puppet 4 introduction of data types."

But when checking Puppet 8 documentation i cannot find anything that can replace this function:
https://www.puppet.com/docs/puppet/8/lang_functions.html
https://www.puppet.com/docs/puppet/8/function.html

In theory this function was useful and was doing it's job, to make sure that email that is passed is valid. And there seems to be no substitution for it as of now.

Main reason why

We had an issue when every time we would execute "puppet agent -t" there would be applied "corrective" actions related to PPA.
We did search and saw that to get it fixed we need to update module "puppetlabs-apt".
and we decided to upgrade to latest version at that time:
mod 'puppetlabs-apt', '9.2.0'

But 'puppetlabs-apt', '9.2.0' depends on version puppetlabs-stdlib (>= 9.0.0 < 10.0.0).

So i would keep getting error:

Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Unknown function: 'is_email_address'. (file: /etc/puppetlabs/code/environments/xxx/site/profile/manifests/reverseproxy.pp, line: 40, column: 10) on node grafana

For now we upgraded to mod 'puppetlabs-apt', '9.0.1' which is still supporting older version:
puppetlabs/stdlib (>= 4.16.0 < 9.0.0)

cleanups after puppetlabs-functions merge

Based on discussions with Jeff - a few things need to be done here:

  • load_yaml, load_json becomes parseyaml & parsejson to distinguish between file load functions and string parsing functions
  • remove validate_resource - this is not ready
  • rename is_valid_* functions and drop the 'valid' part to be clear about what is to 'validate' and what returns a boolean

pw_hash used in exported ressource return a base64 entry

Describe the Bug

We use pw_hash to generate password for user creation. When we use it directly in puppet, the user is created with a correct password.
We have create an exported ressource user and when we retrieve it, we get a base64 as password on the system.

Expected Behavior

When using pw_hash in an exported ressource, we must retrieve a correct hash and not a base64 entry from the puppetdb.

Steps to Reproduce

Steps to reproduce the behavior:
Use this code:
@@user { $newuser: ensure => present, home => "/home/${newuser}", managehome => true, password => pw_hash('password', 'SHA-512', 'salt'), tag => 'mytag', }
And get the result with:
User <<| tag == 'mytag' |>>

And then look your /etc/shadow

Environment

  • Version 8.5
  • Platform Centos 7

Additional Context

I found that there is a problem if an exported ressource is encoded in 8bit ASCII: voxpupuli/hiera-eyaml#273

pw_hash use crypt and on the FAQ of this function (http://crypt.finalstep.com.au/faq.html):

_Does this work for unicode?

Yes it does, but you have to take care of setting the desired encoding. When you provide a unicode string, it is automatically converted into 8-bit ascii for purposes of encryption (we need to deal with bytes). But when you decrypt, you will get an 8-bit ascii string and if you want unicode, you will need to force the encoding like this:

decrypted = decrypted.force_encoding("UTF-8")_

It may be a simple encoding problem.

stdlib should include spool

The puppetlabs-stdlib module should define a default spool directory for use with the file fragment pattern.

Deprecated Ruby 3.2 method exists? is still used

Describe the Bug

Ruby 3.2 (used by Puppet-Agent 8) has removed the deprecated methods {File,Dir}.exists?: https://github.com/puppetlabs/puppet/wiki/Puppet-8-Compatibility

Currently File.exists? is still used by three parser functions: loadyaml, loadjson and load_module_metadata.
Using one of these functions with Puppet 8 will probably not work as expected.

Expected Behavior

The functions should work when Puppet 8 is installed.

Environment

  • 8.0.0
  • Debian-11

Stdlib::Compat::Ipv4 and Stdlib::Compat::Ipv6 are not consistent about validation of IP networks

Use Case

  • Stdlib::Compat::Ipv4 considers 192.168.0.1 and 192.168.0.0/24 to be valid
  • Stdlib::Compat::Ipv6 however only accepts single addresses like 2001:db8:1:2::3, but not a network like 2001:db8:1:2::/64

The Apache module uses Stdlib::Compat::Ipv4 and Stdlib::Compat::Ipv6 to validate IP addresses before including them into the mod_remoteip trusted proxy list. Therefore adding an entire IPv4 network works fine, but an IPv6 network does not. Both are valid settings for mod_remoteip itself.

Describe the Solution You Would Like

The current behavior does not make sense to me, because the IPv6 space is much larger and therefore one would benefit much more from the possibility to use entire IPv6 networks as opposed to IPv4.
I'll submit a PR later that will add support for /xxx at the end of an IPv6 address and would be really delighted to see it included.

Describe Alternatives You've Considered

  1. Add all possible addresses individually - this would work if it's just a few, but even adding an entire /112 network is obviously out of the question, let alone larger networks
  2. Ask/modify the Apache module to use a different validation method. Doesn't seem sensible to me - why write an own validation method if stdlib provides one that just needs to be tweaked?

namespacing ensure_packages breaks a lot of modules

The recent release of stdlib 9 namespaced a lot of functions.
However, namespacing ensure_packages breaks a lot of existing puppet forge modules.

Other namespaced functions like merge and the dropped has_keys have good alternatives.
I don't think ensure_packages does. So I don't think its a good idea to namespace it if the non namespaced version generates
an error.

So I think it would be better to namespace it, but leave the non namespaced version alone or at worst make it a warning.
Possibly the same reasoning applies to ensure_resources. But that doesn't seem to be as much of a problem.

has_key deprecated shim.

As far as I can tell, in version 8.6.0 the has_key function was documented as being deprecated, but didnt
produce a deprecated warning.
But its been removed in version 9+.

So its gone from working silently to missing with no runtime notification to users.
I think it would be better to add a shim with a warning for the moment, so modules have a chance to convert to the "in" operator.

validate_email_address doesn't recognize all valid emails

Describe the Bug

Some domains aren't recognized as valid by validate_email_address

Expected Behavior

Every valid email should be recognized as such.

Steps to Reproduce

Steps to reproduce the behavior:

Use validate_email_address('[email protected]')

Environment

  • Version: puppet 7.x
  • Platform: independent

Additional Context

  • A colleague discovered this.
  • the problem is that a dash -sign is recognized in a subdomain, but not in the domain. Example '[email protected]' but '[email protected]'
  • Likely the email Type in Stdlib, is better, would be possible to either use that in validate_email_address or deprecate the function as outdated, so that one looks elsewhere?

to_toml function is not working

Describe the Bug

With stdlib 8.2.0, when using the to_toml function, introduced in 8.1.0, I get the following error message:

Internal Server Error: org.jruby.exceptions.LoadError: (LoadError) no such file to load -- puppet_x/stdlib

looking up the trace in puppetserver, I ended up on this line where the problem would be. Googling a potential solution, I found that @mrbanzai solved the issue on a fork of stdlib and it works for me as well.

If the proposed fix is considered okay, could we upstream that fix perhaps?

Expected Behavior

With the fix applied, the catalog just applies and TOML is properly formatted.

Steps to Reproduce

Steps to reproduce the behavior:

  1. Use to_toml while on stdlib 8.2.0

Environment

  • Version: 8.2.0
  • Platform: Ubuntu 20.04
  • Puppetserver: 7.4.2

slow custom facts

The package_provider.rb facts can be optimized simply by moving the require lines into the setcode block. This reduces stand-alone puppet runs by 0.5s on modern systems. (On puppet-agent runs, no such savings are observed, probably because the libraries are already required by the puppet agent itself. Apparently, the facter command evaluates the outer and inner blocks in different contexts.

Interestingly, moving the corresponding lines in service_provider.rb has no measurable performance benefit.

Current version

# bench 3 "facter -p >/dev/null"
Run 1
Run 2
Run 3
1665392723.165300291
1665392730.070952629
2.3018

With recommended change

# bench 3 "facter -p >/dev/null"
Run 1
Run 2
Run 3
1665392652.861524963
1665392661.744424272
2.9609

pw_hash bcrypt-a does not work anymore

Describe the Bug

in stdblib 8.6.0 i could do something like this:

  user { 'bcrypt_user':
    ensure   => present,
    password => pw_hash('password', 'bcrypt-a', '10$ABCDE.bcrypt.fixedsalt'),
  }

in stdlib 9.3.0 i get this:

Info: Loading facts
Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Error while evaluating a Method call, Invalid salt value: $2a$10$ABCDE.bcrypt.fixedsalt (file: /etc/puppetlabs/code/environments/production/manifests/site.pp, line: 28, column: 45) on node puppet.example.com
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run

https://github.com/puppetlabs/puppetlabs-stdlib/blob/main/lib/puppet/parser/functions/pw_hash.rb
when i change this block, it works again

9.3.0 not working

  # handle weak implementations of String#crypt
  # dup the string to get rid of frozen status for testing
  if RUBY_PLATFORM == 'java'
    # puppetserver bundles Apache Commons Codec
    org.apache.commons.codec.digest.Crypt.crypt(password.to_java_bytes, salt)
  elsif (+'test').crypt('$1$1') == '$1$1$Bp8CU9Oujr9SSEw53WV6G.'
    password.crypt(salt)
  else
    # JRuby < 1.7.17
    # MS Windows and other systems that don't support enhanced salts
    raise Puppet::ParseError, 'system does not support enhanced salts'
  end

8.6.0 working

  # handle weak implementations of String#crypt
  # dup the string to get rid of frozen status for testing
  if ('test'.dup).crypt('$1$1') != '$1$1$Bp8CU9Oujr9SSEw53WV6G.'
    # JRuby < 1.7.17
    # MS Windows and other systems that don't support enhanced salts
    raise Puppet::ParseError, 'system does not support enhanced salts' unless RUBY_PLATFORM == 'java'
    # puppetserver bundles Apache Commons Codec
    org.apache.commons.codec.digest.Crypt.crypt(password.to_java_bytes, salt)
  else
    password.crypt(salt)
  end

'stdlib::fqdn_rand_string' parameter 'charset' expects a String value, got Undef

Describe the Bug

Passing undef as the second argument to fqdn_rand_string results in a compilation error. This worked (unintentionally or not) in previous versions of stdlib.

Expected Behavior

Function returns a random string (as it did in 8.x)

Steps to Reproduce

Steps to reproduce the behavior:

  1. Install stdlib 9.0.0
[root@40776c2909e5 /]# puppet module install puppetlabs-stdlib -v 9.0.0
Notice: Preparing to install into /etc/puppetlabs/code/environments/production/modules ...
Notice: Downloading from https://forgeapi.puppet.com ...
Notice: Installing -- do not interrupt ...
/etc/puppetlabs/code/environments/production/modules
└── puppetlabs-stdlib (v9.0.0)
  1. Call fqnd_rand_string with undef as the second argument
[root@40776c2909e5 /]# puppet apply -e 'notify { fqdn_rand_string(10, undef, "custom seed"): }'
Warning: This function is deprecated, please use stdlib::fqdn_rand_string instead. at ["unknown", 1]:["unknown", 0]
   (location: /etc/puppetlabs/code/environments/production/modules/stdlib/lib/puppet/functions/deprecation.rb:35:in `deprecation')
Error: Evaluation Error: Error while evaluating a Function Call, 'stdlib::fqdn_rand_string' parameter 'charset' expects a String value, got Undef (line: 1, column: 10) on node 40776c2909e5

Environment

  • Version: 9.0.0
  • Platform: Rocky Linux 8 (shown here for example, others verified via rspec)

Additional Context

This code worked as recently as stdlib 8.6.0.

[root@865c93eff098 /]# puppet module install puppetlabs-stdlib -v 8.6.0
Notice: Preparing to install into /etc/puppetlabs/code/environments/production/modules ...
Notice: Downloading from https://forgeapi.puppet.com ...
Notice: Installing -- do not interrupt ...
/etc/puppetlabs/code/environments/production/modules
└── puppetlabs-stdlib (v8.6.0)
[root@865c93eff098 /]# puppet apply -e 'notify { fqdn_rand_string(10, undef, "custom seed"): }'
Notice: Compiled catalog for 865c93eff098 in environment production in 0.02 seconds
Notice: OhaNI3TjnN
Notice: /Stage[main]/Main/Notify[OhaNI3TjnN]/message: defined 'message' as 'OhaNI3TjnN'
Notice: Applied catalog in 0.00 seconds

fqdn_rotate function output often changes since v9

Commit e2d8b18 broke the fqdn_rotate function as it no longer takes in only the fqdn as seed, instead it now takes in all the facts, which can change frequently.

There is already a PR open that would resolve this: #1341
If this PR would take too long to merge, please consider reverting the mentioned commit.

RFC: Convert all Puppet 4.x API functions to namespaced variants

IMO, only core puppet functions should not have namespaces, and all stdlib functions should start stdlib::.

I'd like to suggest that all current 'modern' (non 3.x style) functions get converted to namespaced versions with a non-namespaced deprecated shim replacing the original file in the same way as is done here.

This was also the approach we took in [extlib](https://github.com/voxpupuli/puppet-extlib/pull/120) and [postgresql](https://github.com/puppetlabs/puppetlabs-postgresql/commit/700d2c5bb54b7ea91d518de96e2c7a22318d0afa).

In a future major release we could drop the shims similar to how we did in [extlib 5.0.0](https://github.com/voxpupuli/puppet-extlib/pull/153)

Stdlib::Fqdn type accepts ipv4 addresses, also allows truncated addresses

Describe the Bug

Specifying a variable as type Stdlib::Fqdn allows ipv4 addresses. This issue is compounded when using Stdlib::Host as with the inclusion of Fqdn it allows truncated ipv4 addresses such as 10.10.10.10.10.

Expected Behavior

IPv4 addresses should be validated when using Stdlib::Host and not accepted within the Stdlib:Fqdn type

Steps to Reproduce

Steps to reproduce the behavior:

  1. Create a variable with type Stdlib::Fqdn

  2. Submit valid or truncated ipv4 address, which passes (the regex)

  3. Create a variable with type Stdlib::Host

  4. Submit a truncated ipv4 address, which passes (the regex)

Environment

  • stdlib v8.4.0
  • Puppet 6x

Additional Context

Unsure if this is a flaw in catering for numerical fqdn's, and how it can be addressed. TL:DR is that using Stdlib::Host does not ensure ipv4 addresses are valid

Stdlib::Fqdn does not allow leading _

Describe the Bug

Stdlib::Fqdn does not support leading _ characters in hostnames. These are valid for most if not all SRV record for a domain.

eg: '_kerberos'

Expected Behavior

The leading _ should be allowed.

Steps to Reproduce

Create a type using Stdlib::Fqdn and pass a value that begins with _ (ie: _kerberos)

Environment

  • Stdlib 9.4

Additional Context

None applicable

stdlib should provide an anchor resource for composite classes

This is largely based on a discussion on the internal tech mailing list.

There is a need to make modules easy for end users to establish relationships to.

For example:

class { 'ntp': } -> class { 'mcollective': }

OR

class { 'mcollective': } -> class { 'ntp': }

If the ntp or mcollective classes are compositions of other classes, such as {ntp,mcollective}::{package,config,service}, then the composed classes must be somehow contained within the ntp or mcollective class.

We can work around this in 2.6.x today using resources as anchors:

class ntp {
  class { 'ntp::package': }
  -> class { 'ntp::config': }
  -> class { 'ntp::service': }
  # These two resources "anchor" the composed classes
  # such that the end user may use "require" and "before"
  # relationships with Class['ntp']
  notify { 'ntp::begin': } -> class { 'ntp::package': }
  class { 'ntp::service': } -> notify { 'ntp::end': }
}

A resource similar to a Whit should be added to the standard library to provide these anchor points. The resource should do nothing and always be in sync.

uriescape function deprecated on P7 - no alternative on P8

Describe the Bug

With puppet 8

# --- Caused by: ---
      # Puppet::ParseError:
      #   Puppet: This function is not available in Puppet 8. URI.escape no longer exists as of Ruby 3+.
      #   ./spec/fixtures/modules/stdlib/lib/puppet/parser/functions/uriescape.rb:23:in `block in <module:Functions>'

Environment

  • Puppet 8

I don't see any alternative function to use ?

file_line extension for multiple matches

Use Case

My request is to introduce another parameter, that ensures that there is only one occurrence of a line that can be matched by the pattern and deletes the rest.
file_line with multiple = true will replace all lines, but doesn't do a cleanup.

Describe the Solution You Would Like

A new attribite for file_line called "unique".
The attribute unique could have several values to control its behaviour, for example
first … changes the first occurrence, all matches after the first
last … deletes all ocurrences but the last, the last one is updated

Describe Alternatives You've Considered

Nothing

Additional Context

Because config files have been managed by hand until now, inconsistencies and errors the parameter naming (small vs. capital letters, type-errors, double entries) happened all over the place, so that we need a strict way of enforcing the profiles correctness.

Error: Facter: Error while resolving custom fact fact='root_home', resolution='<anonymous>': undefined method `dir' for nil:NilClass

Reading this code it looks like it takes windows into account.

# frozen_string_literal: true

Facter.add(:root_home) do
  setcode do
    require 'etc'
  rescue LoadError
  # Unavailable on platforms like Windows
  else
    Etc.getpwnam('root').dir
  end
end

But I always get this error, whilst running on windows.

Error: Facter: Error while resolving custom fact fact='root_home', resolution='<anonymous>': undefined method `dir' for nil:NilClass

ensure_package existing resource detection

Use Case

supported puppet modules use sometimes the ensure_package function, which leads to a duplicate declaration error if you have it already defined in another manifest which gets included during the catalog build.

Describe the Solution You Would Like

the function ensure_package could check, as it does already for the packages puppet and facter, if it is somewhere defined in the current build catalog, to allow the defined resources from the manifest to be used instead of the ensure_package function.

Describe Alternatives You've Considered

I have thought about patching the upstream puppetlabs-apt module by adding a if condition with something like if ! (defined(Package['<packagename>']) { but that would mean on each update of the modules I would need to take care of getting back my change.
Another solution could be, that I place an if condition in front of the include command based on the puppet role which is used.
Both are not a really nice to have, but would work.

Additional Context

Found it while using the puppetlabs-apt module which uses ensure_package(['gnupg']), also I have a package resource which is already defined in an existing manifest. As soon both are used for the same catalog build, it runs into a duplicate declaration error.

puppetlabs-apt: v8.4.1
puppetlabs-stdlib: v8.2.0

  package { 'gnupg':
    ensure  => installed,
    require => Exec['apt_update'],
  }

Error:

Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Error while evaluating a Resource Statement, Duplicate declaration: Package[gnupg] is already declared at (file: /data/puppet/environments/production/modules/apt/manifests/init.pp, line: 365); cannot redeclare (file: /data/puppet/environments/production/modules/defaults/manifests/packages/gnupg.pp, line: 6) (file: /data/puppet/environments/production/modules/defaults/manifests/packages/gnupg.pp, line: 6, column: 3) on node puppettest1.internal.testdomain

With this extension of ensure_package, puppet would make sure that the defined attributes on the package resource are getting used instead of "just" an ensure => present/installed as things like that are sometimes required.

Non namespaced `ensure_packages` function behaviour changed in 9.0.0

Describe the Bug

As originally discussed in #1365 the behaviour of ensure_packages is subtly different depending on whether the namespaced or non-namespaced version is called.

Expected Behavior

Package resources created by the function should be identically 'contained' whether they were created via the namespaced or non-namespaced function shim.

Additional Context

This issue supercedes #1365 where 2 separate issues are discussed and conflated.
A fix for this specific issue has been submitted in #1366

missing merge 1192

i just noticed that #1192 seems to be missing. I don't see anything removing it and also don't see the commit in the history. Wondering if I'm missing something, if i need to submit again or if it was removed because of some other issues thanks

Parser Functions using legacy facts

Describe the Bug

The two functions has_interface_with and fqdn_rotate do not work correctly when running the Puppet Agent with the --no-include_legacy_facts option that seems to be the default in Puppet 8.

  • has_interface_with uses lookupvar(interfaces) which is no longer available
  • fqdn_rotate calls lookupvar(::fqdn) which is also no longer available

Expected Behavior

Both functions should work as before.

Environment

  • Version 7.24.0 with option --no_include_legacy_facts
  • Platform Debian-11

Upgrading Puppetlabs-stdlib

Dear Puppetlabs Team,

Hope you are doing well and thank you for your continuous support making puppet better. It is always much appreciated.

Describe the Bug

Trying to update puppetlabs-stdlib, update fails with

Error: Could not upgrade module 'puppetlabs-stdlib' (v7.1.0 -> latest)
  Downgrading is not allowed.

Expected Behavior

Should update to the latest version

Steps to Reproduce

Steps to reproduce the behavior:

root@puppet:/etc/puppetlabs/code/environments/production/manifests# puppet module upgrade puppetlabs-stdlib
Notice: Preparing to upgrade 'puppetlabs-stdlib' ...
Notice: Found 'puppetlabs-stdlib' (v7.1.0) in /etc/puppetlabs/code/environments/production/modules ...
Notice: Downloading from https://forgeapi.puppet.com ...
Error: Could not upgrade module 'puppetlabs-stdlib' (v7.1.0 -> latest)
  Downgrading is not allowed.

Environment

Puppet

puppet --version
6.25.1
uname -a
Linux puppet 4.19.0-21-cloud-amd64 #1 SMP Debian 4.19.249-2 (2022-06-30) x86_64 GNU/Linux

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.