- Overview
- Module Description - What the module does and why it is useful
- Setup - The basics of getting started with splunk
- Usage - Configuration options and additional functionality
- Reference - An under-the-hood peek at what the module is doing and how
- Limitations - OS compatibility, etc.
- Development - Guide for contributing to the module
This module provides a method to deploy Splunk Server or Splunk Universal Forwarder with common configurations and ensure the services maintain a running state. It provides types/providers to interact with the various Splunk/Forwarder configuration files.
This module does not configure firewall rules. Firewall rules will need to be configured separately in order to allow for correct operation of Splunk and the Splunk Universal Forwarder. Additionally, this module does not supply Splunk or Splunk Universal Forwarder installation media. Installation media will need to be aquired seperately, and the module configured to use it. Users can use yum or apt to install these components if they're self-hosted.
- Installs the Splunk/Forwarder package and manages their config files. It does not purge them by default.
- The module will set up both Splunk and Splunkforwarder to run as the 'root' user on POSIX platforms.
To begin using this module, use the Puppet Module Tool (PMT) from the command line to install this module:
puppet module install puppet-splunk
This will place the module into your primary module path if you do not utilize the --target-dir directive.
You can also use r10k or code-manager to deploy the module so ensure that you have the correct entry in your Puppetfile.
Once the module is in place, there is just a little setup needed.
First, you will need to place your downloaded splunk installers into the files
directory, <module_path>/splunk/files/
. If you're using r10k or code-manager
you'll need to override the splunk::params::src_root
parameter to point at a
modulepath outside of the Splunk module because each deploy will overwrite the
files.
The files must be placed according to directory structure example given below.
The expected directory structure is:
$root_url/
└── products/
├── universalforwarder/
│ └── releases/
| └── $version/
| └── $platform/
| └── splunkforwarder-${version}-${build}-${additl}
└── splunk/
└── releases/
└── $version/
└── $platform/
└── splunk-${version}-${build}-${additl}
A semi-populated example files directory might then contain:
$root_url/
└── products/
├── universalforwarder/
│ └── releases/
| └── 7.0.0/
| ├── linux/
| | ├── splunkforwarder-7.0.0-c8a78efdd40f-linux-2.6-amd64.deb
| | ├── splunkforwarder-7.0.0-c8a78efdd40f-linux-2.6-intel.deb
| | └── splunkforwarder-7.0.0-c8a78efdd40f-linux-2.6-x86_64.rpm
| ├── solaris/
| └── windows/
| └── splunkforwarder-7.0.0-c8a78efdd40f-x64-release.msi
└── splunk/
└── releases/
└── 7.0.0/
└── linux/
├── splunk-7.0.0-c8a78efdd40f-linux-2.6-amd64.deb
├── splunk-7.0.0-c8a78efdd40f-linux-2.6-intel.deb
└── splunk-7.0.0-c8a78efdd40f-linux-2.6-x86_64.rpm
Second, you will need to supply the splunk::params
class with three critical
pieces of information.
- The version of Splunk you are using
- The build of Splunk you are using
- The root URL to use to retrieve the packages
In the example given above, the version is 7.0.0, the build is c8a78efdd40f, and the root URL is puppet:///modules/splunk. See the splunk::params class documentation for more information.
Once the Splunk packages are hosted in the users repository or hosted by the Puppet Server in the modulepath the module is ready to deploy.
If a user is installing Splunk with packages provided from their modulepath, this is the most basic way of installing Splunk Server with default settings:
include ::splunk
This is the most basic way of installing the Splunk Universal Forwarder with default settings:
class { '::splunk::params':
server => $my_splunk_server,
}
include ::splunk::forwarder
Once both Splunk and Splunk Universal Forwarder have been deployed on their respective nodes, the Forwarder is ready to start sending logs.
In order to start sending some log data, users can take advantage of the
Splunkforwarder_input
type. Here is a basic example of adding an input to
start sending Puppet Server logs:
@splunkforwarder_input { 'puppetserver-sourcetype':
section => 'monitor:///var/log/puppetlabs/puppetserver/puppetserver.log',
setting => 'sourcetype',
value => 'puppetserver',
tag => 'splunk_forwarder'
}
This virtual resource will get collected by the ::splunk::forwarder
class if
it is tagged with splunk_forwarder
and will add the appropriate setting to
the inputs.conf file and refresh the service.
This module has the ability to install and upgrade the splunk and splunkforwarder packages. All you have to do is declare package_ensure => 'latest'
when calling the ::splunk
or ::splunk::forwarder
classes.
The following code will install the 6.6.8 version of the splunk forwarder. Then comment out the 6.6.8 version and build values and uncomment the 7.1.2 version and build values. Running puppet again will perform the following:
- splunk forwarder package is upgraded
- splunk service is stopped as part of the package upgrade process
- new license agreement is automatically accepted
- license agreement must be accepted or the splunk service will fail to start
- splunk service is started
# Tell the module to get packages directly from Splunk.
class { '::splunk::params':
version => '6.6.8',
build => '6c27a8439c1e',
#version => '7.1.2',
#build => 'a0c72a66db66',
src_root => 'https://download.splunk.com',
}
# Specifying package_ensure => 'latest' will ensure that the splunk and
# splunkforwarder packages will be upgraded when you specify newer values for
# version and build.
class { '::splunk::forwarder':
package_ensure => 'latest',
}
-
splunk_config
: This is a meta resource used to configur defaults for all the splunkforwarder and splunk types. This type should not be declared directly as it is declared insplunk::params
and used internally by the types and providers. -
splunk_authentication
: Used to manage ini settings in authentication.conf -
splunk_authorize
: Used to manage ini settings in authorize.conf -
splunk_deploymentclient
: Used to manage ini settings in deploymentclient.conf -
splunk_distsearch
: Used to manage ini settings in distsearch.conf -
splunk_indexes
: Used to manage ini settings in indexes.conf -
splunk_input
: Used to manage ini settings in inputs.conf -
splunk_limits
: Used to mange ini settings in limits.conf -
splunk_metadata
: Used to manage ini settings in default.meta -
splunk_output
: Used to manage ini settings in outputs.conf -
splunk_props
: Used to manage ini settings in props.conf -
splunk_server
: Used to manage ini settings in server.conf -
splunk_serverclass
: Used to manage ini settings in serverclass.conf -
splunk_transforms
: Used to manage ini settings in transforms.conf -
splunk_web
: Used to manage ini settings in web.conf -
splunkforwarder_deploymentclient
: Used to manage ini settings in deploymentclient.conf -
splunkforwarder_input
: Used to manage ini settings in inputs.conf -
splunkforwarder_output
:Used to manage ini settings in outputs.conf -
splunkforwarder_props
: Used to manage ini settings in props.conf -
splunkforwarder_transforms
: Used to manage ini settings in transforms.conf -
splunkforwarder_web
: Used to manage ini settings in web.conf -
splunkforwarder_limits
: Used to manage ini settings in limits.conf -
splunkforwarder_server
: Used to manage ini settings in server.conf
All of the above types use puppetlabs/ini_file
as a parent and are declared in
an identical way, and accept the following parameters:
section
: The name of the section in the configuration filesetting
: The setting to be managedvalue
: The value of the setting
Both section and setting are namevars for the types. Specifying a single string
as the title without a forward slash implies that the title is the section to be
managed (if the section attribute is not defined). You can also specify the
resource title as section/setting
and ommit both section
and setting
params
for a more shortform way of declaring the resource. Eg:
splunkforwarder_output { 'useless title':
section => 'default',
setting => 'defaultGroup',
value => 'splunk_9777',
}
splunkforwarder_output { 'default':
setting => 'defaultGroup',
value => 'splunk_9777',
}
splunkforwarder_output { 'default/defaultGroup':
value => 'splunk_9777',
}
The above resource declarations will all configure the following entry in outputs.conf
[default]
defaultGroup=splunk_9997
Note: if the section contains forward slashes you should not use it as the resource
title and should explicitly declare it with the section
attribute.
Optional Specifies the version of Splunk Enterprise that the module should install.
Optional Specifies the build of Splunk Enterprise that the module should use.
Optional The root path that the staging module will use to find packages for splunk and splunk::forwarder.
Optional The splunkd port. Used as a default for both splunk and splunk::forwarder.
Optional The port on which to send and listen for logs. Used as a default for both splunk and splunk::forwarder.
Optional The fqdn or IP address of the Splunk server. Used for setting up the default TCP output and input.
The source URL for the splunk installation media (typically an RPM, MSI, etc). If a $src_root parameter is set in splunk::params, this will be automatically supplied. Otherwise it is required. The URL can be of any protocol supported by the nanliu/staging module. On Windows, this can be a UNC path to the MSI.
The name of the package(s) Puppet will use to install Splunk.
Ensure parameter which will get passed to the Splunk package resource. Default to the value in splunk::params
The port to receive TCP logs on. Default to the port specified in splunk::params.
The user to run Splunk as. Default to the value set in splunk::params.
The management port for Splunk. Default to the value set in splunk::params.
The port on which to service the Splunk Web interface. Default to 8000.
Optional If set to true, inputs.conf will be purged of configuration that is no longer managed by the splunk_input type. Default to false.
Optional If set to true, outputs.conf will be purged of configuration that is no longer managed by the splunk_output type. Default to false.
Optional If set to true, authentication.conf will be purged of configuration that is no longer managed by the splunk_authentication type. Default to false.
Optional If set to true, authorize.conf will be purged of configuration that is no longer managed by the splunk_authorize type. Default to false.
Optional If set to true, distsearch.conf will be purged of configuration that is no longer managed by the splunk_distsearch type. Default to false.
Optional If set to true, indexes.conf will be purged of configuration that is no longer managed by the splunk_indexes type. Default to false.
Optional If set to true, limits.conf will be purged of configuration that is no longer managed by the splunk_limits type. Default to false.
Optional If set to true, props.conf will be purged of configuration that is no longer managed by the splunk_props type. Default to false.
Optional If set to true, server.conf will be purged of configuration that is no longer managed by the splunk_server type. Default to false.
Optional If set to true, transforms.conf will be purged of configuration that is no longer managed by the splunk_transforms type. Default to false.
Optional If set to true, web.conf will be purged of configuration that is no longer managed by the splunk_web type. Default to false.
Optional The fqdn or IP address of the Splunk server. Default to the value in ::splunk::params.
The source URL for the splunk installation media (typically an RPM, MSI, etc). If a $src_root parameter is set in splunk::params, this will be automatically supplied. Otherwise it is required. The URL can be of any protocol supported by the nanliu/staging module. On Windows, this can be a UNC path to the MSI.
The name of the package(s) Puppet will use to install Splunk Universal Forwarder.
Ensure parameter which will get passed to the Splunk package resource. Default to the value in ::splunk::params
Optional The port on which to send and listen for logs. Default to the value in ::splunk::params.
The management port for Splunk. Default to the value set in splunk::params.
This variable is passed to the package resources' install_options parameter. Default to the value in ::splunk::params.
The user to run Splunk as. Default to the value set in splunk::params.
The address on which splunkd should listen. Defaults to 127.0.0.1.
Optional If set to true, inputs.conf will be purged of configuration that is no longer managed by the splunkforwarder_input type. Default to false.
Optional If set to true, outputs.conf will be purged of configuration that is no longer managed by the splunk_output type. Default to false.
Optional If set to true, props.conf will be purged of configuration that is no longer managed by the splunk_props type. Default to false.
Optional If set to true, transforms.conf will be purged of configuration that is no longer managed by the splunk_transforms type. Default to false.
Optional If set to true, web.conf will be purged of configuration that is no longer managed by the splunk_web type. Default to false.
Optional This will override the default package provider for the package resource. Default to undef.
The root directory where Splunk Universal Forwarder is installed. Default to the value in ::splunk::params.
Used to override the default forwarder_input type defined in ::splunk::params.
Used to override the default forwarder_output type defined in ::splunk::params.
Not yet implemented.
- Currently tested manually on Centos 7, but we will eventually add automated testing and are targeting compatibility with other platforms.
- Tested with Puppet 4.x
TBD
TBD