voxpupuli / puppet-ipset

Puppet module to manage ipset tooling and actual sets

puppet-ipset

Table of Contents

• Overview
• Usage
  • Array
  • String
  • Module file
  • Local file
• Reference
• Limitations
• Changelog
• Development and Contribution
• Thanks

Overview

This module manages Linux IP sets.

• Checks for current ipset state, before doing any changes to it.
• Applies ipset every time it drifts from target state, not only on config file change.
• Handles type changes.
• Autostart support for RHEL 6 and RHEL 7 family (upstart, systemd).

Usage

Array

IP sets can be filled from an array data structure. Typically passed from Hiera.

ipset::set { 'foo':
  ensure => present,
  set    => ['1.2.3.4', '5.6.7.8'],
  type   => 'hash:ip',
}

String

You can also pass a pre-formatted string directly, using one entry per line (with \n as a separator). This pattern is practical when generating the IP set entries using a template.

ipset::set { 'foo':
  ensure => present,
  set    => "1.2.3.4\n5.6.7.8",
  type   => 'hash:ip',
}

Module file

IP sets content can also be stored in a module file:

ipset::set { 'foo':
  ensure => present,
  set    => "puppet:///modules/${module_name}/foo.ipset",
}

Local file

Or using a plain text file stored on the filesystem:

file { '/tmp/bar_set_content':
  ensure  => present,
  content => "1.2.3.0/24\n5.6.7.8/32",
}

ipset::set { 'bar':
  ensure    => present,
  set       => 'file:///tmp/bar_set_content',
  type      => 'hash:net',
  subscribe => File['/tmp/bar_set_content'],
}

Unmanaged ipsets

Declare an IP set, without managing its content:

ipset::unmanaged { 'baz':
  ensure => present,
  type   => 'hash:net',
}

Useful when you have a dynamic process that generates an IP set content, but still want to define and use it from Puppet.

Warning: When changing IP set attributes (type, options) contents won't be kept, set will be recreated as empty.

Reference

The module uses puppet-strings for documentation. The result is the REFERENCE.md file.

Limitations

• Tested on Debian and RedHat-like Linux distributions
• Only hash ipsets are supported (this excludes bitmap and list:set)

Changelog

See CHANGELOG

Development and Contribution

See development

Thanks

This module is a complete rewrite of sl0m0ZA/ipset, which is a fork of pmuller/ipset, which was forked from mighq/ipset, which was based on thias/ipset.

puppet-ipset's Issues

Potential for systemd dependency cycles

We see occasional systemd dependency cycles on instances which run cloud-init that result in the ipset start job being deleted at system startup, delaying the firewall configuration until Puppet has run post-boot.

Affected Puppet, Ruby, OS and module versions/distributions

• Puppet: 6.26.0
• Ruby: 2.5.9p229
• Distribution: Debian 11
• Module version: 2.1.0

How to reproduce (e.g Puppet code you use)

We set firewall_service => 'netfilter-persistent.service',

Our server instances are provisioned with cloud-init.

What are you seeing

We see a systemd dependency cycle that sometimes results in the ipset start job being deleted, knocking on to the iptables-persistent service failing as the rules use ipsets. This means that the server instance firewalls aren't configured at boot until Puppet runs subsequently and sorts it all out, leaving a vulnerable window of time.

What behaviour did you expect instead

No dependency cycles and a firewall before the networking comes up.

Output log

network-pre.target: Job ipset.service/start deleted to break ordering cycle starting with network-pre.target/start

Any additional information you'd like to impart

As it stands, the service unit file shipped with this module sets:

Before=network-pre.target iptables.service ip6tables.service
Wants=network-pre.target

It has no setting for DefaultDependencies which means it gets the defaults, one of which is After=sysinit.target.

This is good enough to get the ipsets configured before networking is started in many cases... until you introduce another unit that wants to run after networking, but before sysinit.target. You then end up with dependency cycles which sometimes result in the ipset job being deleted, which might mean no firewall if that relies upon ipsets (which ours does via puppetlabs-firewall module and the iptables-persistent service).

One example of a service that clashes in this way is cloud-init at some of our providers, which has some constraints over when the various stages need to run, and often has DefaultDependencies=no, After=networking.service and Before=sysinit.target to meet them.

I understand that this issue is going to be somewhat environment specific, but given that it seems to me that you will always want your ipsets configured early in the boot process and before networking, I think the implicit dependency of After=sysinit.target should be dropped by setting DefaultDependencies=no. The systemd documentation linked above says "Only services involved with early boot or late system shutdown should disable this option" - which seems to be the case here.

To work around this, we've added a drop-in file with the following (similar to what's in the netfilter-persistent.service unit - the other settings ensure some of the other settings implied by DefaultDependencies are handled sensibly):

[Unit]
DefaultDependencies=no
After=systemd-modules-load.service local-fs.target
Wants=systemd-modules-load.service local-fs.target
Before=shutdown.target
Conflicts=shutdown.target

This appears to work well for us and I suspect that these settings could go into the main unit file safely, or at least are worth documenting as something to keep in mind.

Happy to send a PR to this effect if you agree, but I thought I'd open this issue first to allow for discussion.

Make the 'set' parameter optional

Affected Puppet, Ruby, OS and module versions/distributions

• Puppet: 6.12.0
• Ruby: 2.5.7p206 (2019-10-01 revision 67816) [x86_64-linux]
• Distribution: Debian 10
• Module version: 1.2.0

How to reproduce (e.g Puppet code you use)

    ipset::set { 'dynamicblocklist':
      ensure => present,
      type   => 'hash:ip',
    }
    -> firewall { '500 allow trafic on port 1234':
      proto => tcp,
      dport => 1234,
      ipset => 'dynamicblocklist src',
    }

Reason is we want to ensure a ipset exists in order to be able to use it in a firewall statement, but the list content itself is supposed to be managed on the system itself (for example to dynamically block malicious clients). For this we need an option to have the module ignore the content of the ipset.

What are you seeing

Error 500 on SERVER: Server Error: Evaluation Error: Error while evaluating a Resource Statement, Ipset::Set[setclients]: expects a value for parameter 'set'

What behaviour did you expect instead

An empty ipset is created and and on later runs the content of the ipset is ignored.

The module doesn't work if ipset in use

Affected Puppet, Ruby, OS and module versions/distributions

• Puppet: 5.5.14
• Ruby: ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux-gnu]
• Distribution: Ubuntu 22.04 LTS
• Module version: 1.2.3 (I see this bug in code 2.1.0)

How to reproduce (e.g Puppet code you use)

Create ipset and add iptables rule for use this ipset
Example:
iptables -A INPUT -m set --match-set test src -j ACCEPT

What are you seeing

We found that when using the ipset firewall or ubuntu 20 and older, the address addition does not occur.
The error occurs while running the ipset_sync script

What behaviour did you expect instead

Output log

ipset v7.15: Set cannot be destroyed: it is in use by a kernel component
ipset v7.15: Error in line 1: Set cannot be created: set with the same name already exists

Any additional information you'd like to impart

We create new function in ipset_sync

function import_ipset_new() {
  local id=$1
  local alias=${2:-${id}}
  local no_content=${3:-0}
  local f_content=$(get_content_file ${id})
  if [ ! -f "${f_content}" ]; then
    echo "Set configuration file '${f_content}' does not exist!" >&2
    exit 6
  else
    while read LINE; do
            ipset add ${id} $LINE
    done < ${f_content}
  fi

}

and change lines
ipset destroy ${set_id} to ipset flush ${set_id}
import_ipset ${set_id} ${set_id} ${ignore_contents} to
import_ipset_new ${set_id} ${set_id} ${ignore_contents}

Enable creating `ipset::set` resources via Hiera

Enable creating ipset::set resources via Hiera.

Example: 'puppet-ferm' enables creating e.g. ferm::rule resources via Hiera (https://github.com/voxpupuli/puppet-ferm).

ipset_sync error on noop for the first time

Hello

I encounter an issue when I'm using ipset::set for the first time on a server in noop.
I believe unless command of Exec[sync_] is evaluated and fails as ipset_sync is absent from my host. Don't know if it's my fault or a bug.

Regards,

Affected Puppet, Ruby, OS and module versions/distributions

• Puppet: 5.5.10
• Ruby: 2.5.5p157
• Distribution: Debian/10.3
• Module version: v1.2.3

How to reproduce (e.g Puppet code you use)

Add this in manifest:

    ipset::set { "test":
        ensure => present,
        set    => ['1.1.1.1'],
        type   => 'hash:ip',
    }

And run puppet in noop for the first time with this module (ie. ipset_sync does not exist).

What are you seeing

Puppet noop output:

Notice: /Stage[main]/Ipset/Package[ipset]/ensure: current_value 'purged', should be 'present' (noop)
Error: /Stage[main]/Role::Sdev_docker/Ipset::Set[test]/Exec[sync_ipset_test]: Could not evaluate: Could not find command 'ipset_sync'
Notice: /Stage[main]/Role::Sdev_docker/Ipset::Set[test]/Exec[sync_ipset_test]: Would have triggered 'refresh' from 2 events
Info: Ipset::Set[test]: Unscheduling all events on Ipset::Set[test]
Notice: /Stage[main]/Ipset/Systemd::Unit_file[ipset.service]/Service[ipset.service]/ensure: current_value 'stopped', should be 'running' (noop)
Info: /Stage[main]/Ipset/Systemd::Unit_file[ipset.service]/Service[ipset.service]: Unscheduling refresh on Service[ipset.service]
Notice: Systemd::Unit_file[ipset.service]: Would have triggered 'refresh' from 2 events
Notice: Class[Ipset]: Would have triggered 'refresh' from 5 events

And especially:

Error: /Stage[main]/Role::Sdev_docker/Ipset::Set[test]/Exec[sync_ipset_test]: Could not evaluate: Could not find command 'ipset_sync'

What behaviour did you expect instead

Error on Exec[sync_ipset_sync] should be absent

subnet 32 should not be stripped in ipset_sync

Affected Puppet, Ruby, OS and module versions/distributions

• Puppet: n/a
• Ruby: n/a
• Distribution: n/a
• Module version: 2.1.0

How to reproduce (e.g Puppet code you use)

use an IPv6 network with 32 bits subnet

What are you seeing

/32 is being stripped from the script ipset_sync even on IPv6 networks

What behaviour did you expect instead

/32 should not be stripped on IPv6 because it's a legit subnet for IPv6 network

Any additional information you'd like to impart

this is the line in the script where the subnet is being stripped:
https://github.com/voxpupuli/puppet-ipset/blob/master/files/ipset_sync#L77

we are preparing a Pull Request

ipsync error will never be retried

In module 4.1.0 version (or current code as of writing) there is an issue in the exec statement for ipset_sync.

    # sync if needed by helper script
    exec { "sync_ipset_${title}":
      path        => ['/sbin', '/usr/sbin', '/bin', '/usr/bin', '/usr/local/bin', '/usr/local/sbin'],
      # use helper script to do the sync
      command     => "ipset_sync -c '${config_path}'    -i ${title}${ignore_contents_opt}",
      # only when difference with in-kernel set is detected
      unless      => "ipset_sync -c '${config_path}' -d -i ${title}${ignore_contents_opt}",
      require     => [Package['ipset'], File['/usr/local/bin/ipset_sync']],
      refreshonly => true,
    }

If there is an error in ipset_sync (such as a duplicate ip in the sync. this can happen for example having a hostname and a load balancer address that points to the hostname) the refresh only will prevent the unless from running so you will end up with just an empty ip set with no errors and no retries on subsequent puppet runs. The problem is the use of refreshonly here, either it could be removed, or the command could remove the file on failure.

hashsize is dynamic and produces errors on large sets

Affected Puppet, Ruby, OS and module versions/distributions

• Puppet: 6.13.0
• Ruby: ruby 2.5.7p206 (2019-10-01 revision 67816) [x86_64-linux]
• Distribution: Debian 9
• Module version: 1.2.0

How to reproduce (e.g Puppet code you use)

  file { '/var/lib/ipsets/cn-hk.ipset':
      ensure => file,
      mode   => '0644',
      source => 'puppet:///modules/profiles/ipsets/cn-hk.ipset',
  }

  -> ipset::set { 'cn':
    ensure       => 'present',
    type         => 'hash:net',
    set          => 'file:///var/lib/ipsets/cn-hk.ipset',
    keep_in_sync => true,
  }

  -> firewall { '000 Block CN-HK':
    proto => 'all',
    ipset => 'cn src',
    jump  => 'LOGDROP',
  }

The set created has > 10k networks. This results in the hash size growing. We could set this at the beginning, but as it may vary dynamically this doesn't seem like the correct approach.

What are you seeing

Puppet thinks that the ipset has changed because of the hashsize mismatch and attempts to delete it and recreate it but this fails as it is in-use by the kernel.

What behaviour did you expect instead

Nothing - the contents of the set have not changed but the hashsize has since it was created.

Output log

Notice: /Stage[main]/Profiles::Ipset::Cn/Ipset::Set[cn]/Exec[sync_ipset_cn]/returns: ipset v6.30: Set cannot be destroyed: it is in use by a kernel component
Notice: /Stage[main]/Profiles::Ipset::Cn/Ipset::Set[cn]/Exec[sync_ipset_cn]/returns: ipset v6.30: Error in line 1: Set cannot be created: set with the same name already exists
Error: 'ipset_sync -c '/etc/ipset.d/'    -i cn' returned 1 instead of one of [0]
Error: /Stage[main]/Profiles::Ipset::Cn/Ipset::Set[cn]/Exec[sync_ipset_cn]/returns: change from 'notrun' to ['0'] failed: 'ipset_sync -c '/etc/ipset.d/'    -i cn' returned 1 instead of one of [0] (corrective)
Notice: /Stage[main]/Profiles::Ipset::Cn/Firewall[000 Block CN-HK]: Dependency Exec[sync_ipset_cn] has failures: true

Actual header: Header: family inet hashsize 8192 maxelem 65536

Contents of /etc/ipset.d/cn.hdr: create cn hash:net family inet hashsize 1024 maxelem 65536

Any additional information you'd like to impart

This issue was raised in mighq/puppet-ipset and resolved there.

I have a forked copy of this repository with a branch with the commit from mighq cherry-picked and can confirm that it appears to fix the problem for me.

Add support for comments

http://ipset.netfilter.org/ipset.man.html#lbAI mentions that ipset supports comments for each entry. This would be really helpful in an environment with a lot of networks/prefixes.

/32 is incorrectly stripped from IPv6 networks

ipset_sync incorrectly strips /32 from IPv6 networks (e.g. fe80::/32).

This happens here: https://github.com/voxpupuli/puppet-ipset/blob/master/files/ipset_sync#L77

/32 should only be stripped if family is inet. Similarly, /128 should only be stripped if family is inet6 (but this is not a problem in practice).

Set file for unmanaged ipsets is emptied on each puppet run

Affected Puppet, Ruby, OS and module versions/distributions

• Puppet: 6.19
• Ruby: 2.5.8
• Distribution: CentOS 7
• Module version: All tagged releases

How to reproduce (e.g Puppet code you use)

  ipset::unmanaged { 'example_ipset':
    ensure => present,
    type   => 'hash:net',
  }

What are you seeing

On each run the unmanaged ipset's set file is emptied, though the ipset in memory is retained.

What behaviour did you expect instead

The set file contents should be retained for unmanaged sets, in addition to the ipset in memory.

Output log

Any additional information you'd like to impart