GithubHelp home page GithubHelp logo

golja-influxdb's Introduction

InfluxDB

Table of Contents

  1. Overview
  2. Module Description - What the module does and why it is useful
  3. Installation
  4. Setup - The basics of getting started with influxdb
  5. Usage - Configuration options and additional functionality
  6. Reference - An under-the-hood peek at what the module is doing and how
  7. Limitations - OS compatibility, etc.
  8. Development - Guide for contributing to the module
  9. License

Overview

This module manages InfluxDB installation.

Build Status Puppet Forge

Module Description

The InfluxDB module manages both the installation and configuration of InfluxDB. I am planning to extend it to allow management of InfluxDB resources, such as databases, users, and privileges.

Deprecation Warning

Notes for version 5.0.0+:

This module was a refactor of the 4.x version to handle influxdb >= 1.x. Due to the changes in influxdb 1.x, this module should now support future change more easily due to thew way the configuration files are now managed.

Highlights

  • The module layout out has changed significantly from previous versions.
  • A new fact was added influxdb_version.
  • The influxdb.conf.erb file was refactored.
  • Added and fixed a lot of rspec puppet tests.
  • Fixed all the beaker tests, they work now.
  • This module now supports influxdb >= 1.x < 2.x
  • Major change to the original class parameters now hashes vs individual items.

Notes for version 4.0.0+:

influxdb 1.0.0 contains breaking changes which require changing the data_logging_enabled config attribute to trace_logging_enabled. The other configuration changes are managed by the influxdb.conf.erb template already.

Notes for versions older than 3.1.1:

This release is a major refactoring of the module which means that the API changed in backwards incompatible ways. If your project depends on the old API and you need to use influxdb prior to 0.10.X, please pin your module dependencies to 0.1.2 (0.8.X) or 2.2.2 (0.9.X) version to ensure your environments don't break.

NOTE: Until influxdb 1.0.0 is releases the API of this module may change, however I will try my best to avoid it.

Installation

puppet module install golja/influxdb

Setup

What InfluxDB affects

  • InfluxDB packages
  • InfluxDB configuration files
  • InfluxDB service

Beginning with InfluxDB

If you just want a server installed with the default options you can run include '::influxdb'.

Usage

All interaction for the server is done via influxdb::server.

Install influxdb

class {'influxdb':}
# These are defaults, but demonstrates how you can change sections of data
$global_config => {
  'bind-address'       => ':8088',
  'reporting-disabled' => false,
}

class {'influxdb':
  global_config  => $global_config,
  manage_repos   => true,
  manage_service => true,
  version        => '1.2.0',
}

Enable Graphite plugin with one database

# Most of these will be defaults, unless otherwise noted.
$graphite_config = {
  'default' => {
    'enabled'           => true, # not default
    'database'          => "graphite",
    'retention-policy'  => '',
    'bind-address'      => ':2003',
    'protocol'          => 'tcp',
    'consistency-level' => 'one',
    'batch-size'        => 5000,
    'batch-pending'     => 10,
    'batch-timeout'     => '1s',
    'udp-read-buffer'   => 0,
    'separator'         => '.',
    'tags'              => [ "region=us-east", "zone=1c"],
    'templates'         => [ "*.app env.service.resource.measurement" ],
  }
}

class { 'influxdb':
  manage_repos    => true,
  graphite_config => $graphite_config,
}

Enable Collectd plugin

# most of these are defaults, unless otherwise noted
$collectd_config = {
  'default' => {
    'enabled'          => true, # not default
    'bind-address'     => ':25826',
    'database'         => 'collectd',
    'retention-policy' => '',
    'typesdb'          => '/usr/share/collectd/types.db',
    'batch-size'       => 5000,
    'batch-pending'    => 10,
    'batch-timeout'    => '10s',
    'read-buffer'      => 0,
  }
}

class {'influxdb':
  manage_repos    => true,
  collectd_config => $collectd_config,
}

Enable UDP listener

# most of these are defaults unless otherwise noted.
$udp_config = {
  'default' => {
    'enabled'          => true, # not default
    'bind-address'     => ':8089',
    'database'         => 'udp',
    'retention-policy' => '',
    'batch-size'       => 5000,
    'batch-pending'    => 10,
    'batch-timeout'    => '1s',
    'read-buffer'      => 0,
  }
}

class {'influxdb':
  manage_repos => true,
  udp_config   => $udp_config
}

Enable opentsdb

# most of these are defaults unless otherwise noted
$opentsdb_config = {
  'default' => {
    'enabled'           => true, # not default
    'bind-address'      => ':4242',
    'database'          => 'opentsdb',
    'retention-policy'  => '',
    'consistency-level' => 'one',
    'tls-enabled'       => false,
    'certificate'       => '/etc/ssl/influxdb.pem',
    'log-point-errors'  => true,
    'batch-size'        => 1000,
    'batch-pending'     => 5,
    'batch-timeout'     => '1s'
  }
}


class {'influxdb':
  manage_repos    => true,
  opentsdb_config => $opentsdb_config,
}

Reference

Classes

Public classes

  • influxdb: Installs and configures InfluxDB.

Private classes

  • influxdb::install: Installs packages.
  • influxdb::config: Configures InfluxDB.
  • influxdb::repo: Manages install repo.
  • influxdb::service: Manages service.

Parameters

influxdb

ensure

Allows you to install or remove InfluxDB. Can be 'present' or 'absent'.

version

Version of InfluxDB. Default: installed

NOTE: installed (will install the latest version if the package repo if not already installed). It is highly recommended that you manage this param with a specific version.

config_file

Path to the config file. Default: /etc/influxdb/influxdb.conf

conf_template

The path to the template file that puppet uses to generate the influxdb.conf Default: influxdb/influxdb.conf.erb

startup_conf_template

The path to the template file that puppet uses to generate the start config. Default: influxdb/influxdb_default.erb

service_enabled

Boolean to decide if the service should be enabled. Default: true

service_ensure

String to decide if the service should be running|stopped. Default: running

manage_service

Boolean to decide if the service should be managed with puppet or not. Default: true

manage_repos

Boolean to decide if the package repos should be managed by puppet. Default: false

manage_install

Boolean to decide if puppet should manage the install of packages. Default: true

influxdb_stderr_log

Where influx will log stderr messages Default: /var/log/influxdb/influxd.log

influxdb_stdout_log

Where influx will log stdout messages Default: /var/log/influxdb/influxd.log

influxd_opts

String of startup config options that need to be present. Default: undef

global_config

A hash of global configuration options for influxdb.conf

NOTE: The default for this hash is what is in 1.2.0 of the influx docs.

Influx Global Options

params.pp

meta_config

A hash of meta configuration options for influxdb.conf

NOTE: The default for this hash is what is in 1.2.0 of the influx docs

Influx Meta Options

params.pp

data_config

A hash of data configuration options for influxdb.conf

NOTE: The default for this hash is what is in 1.2.0 of the influx docs

Influx Data Options

params.pp

logging_config

A hash of logging configuration options for influxdb.conf

NOTE: The default for this hash is what is in 1.5.x of the influx docs

Influx Logging Options

params.pp

coordinator_config

A hash of coordinator configuration options for influxdb.conf

NOTE: The default for this hash is what is in 1.2.0 of the influx docs

Influx Coordinator Options

params.pp

retention_config

A hash of retention configuration options for influxdb.conf

NOTE: The default for this hash is what is in 1.2.0 of the influx docs

Influx Retention Options

params.pp

shard_precreation_config

A hash of shard_precreation configuration options for influxdb.conf

NOTE: The default for this hash is what is in 1.2.0 of the influx docs

Influx Shard Precreation Options

params.pp

monitor_config

A hash of monitor configuration options for influxdb.conf

NOTE: The default for this hash is what is in 1.2.0 of the influx docs

Influx Monitor Options

params.pp

admin_config

A hash of admin configuration options for influxdb.conf

NOTE: The default for this hash is what is in 1.2.0 of the influx docs

Influx Admin Options

params.pp

http_config

A hash of http configuration options for influxdb.conf

NOTE: The default for this hash is what is in 1.2.0 of the influx docs

Influx HTTP Options

params.pp

subscriber_config

A hash of subscriber configuration options for influxdb.conf

NOTE: The default for this hash is what is in 1.2.0 of the influx docs

Influx Subscriber Options

params.pp

graphite_config

A hash of graphite configuration options for influxdb.conf

NOTE: The default for this hash is what is in 1.2.0 of the influx docs

Influx Graphite Options

params.pp

collectd_config

A hash of collectd configuration options for influxdb.conf

NOTE: The default for this hash is what is in 1.2.0 of the influx docs

Influx Collectd Options

params.pp

opentsdb_config

A hash of opentsdb configuration options for influxdb.conf

NOTE: The default for this hash is what is in 1.2.0 of the influx docs

Influx Opentsdb Options

params.pp

udp_config

A hash of udp configuration options for influxdb.conf

NOTE: The default for this hash is what is in 1.2.0 of the influx docs

Influx Udp Options

params.pp

continuous_queries_config

A hash of continuous queries configuration options for influxdb.conf

NOTE: The default for this hash is what is in 1.2.0 of the influx docs

Influx Continuous Queries Options

params.pp

hinted_handoff_config

This is depcreated as of influxdb >= 1.0 Default: {}

Limitations

This module has been tested on:

  • Ubuntu 12.04
  • Ubuntu 14.04
  • CentOS 6/7

Development

In order to better facilitate beaker testing, the Gemfile was modified to support environment variables. You must set this enviroment variable before running bundle exec *. The Gemfile will automatically set this for you but you can also override it.

export BEAKER_VERSION=3.10.0 # assumes ruby >= 2.2.5

Please see CONTRIBUTING.md

Todo

  • Add native types for managing users and databases
  • Add more rspec tests
  • Add beaker/rspec tests

License

See LICENSE file

golja-influxdb's People

Contributors

andyroyle avatar axxelg avatar bastelfreak avatar booo avatar bovy89 avatar codylane avatar danielpalstra avatar dgolja avatar dhoppe avatar dsbaars avatar egeland avatar johnf avatar jschlyter avatar lswith avatar lukebigum avatar madddi avatar niteman avatar rcousens avatar sammcj avatar shashank-te avatar sorrison avatar wdschei avatar xaniasd avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

golja-influxdb's Issues

Is this code still maintained?

Hallo,

is this module still maintained?
It has the most downloads in the forge for influxdb, but i don't see any commits since 2019?

Greetings,
Robert

support InfluxDB 1.0

InfluxDB 1.0 was officially released today. Looking at the release notes the only relevant changes seem to be in the config file, any plans about it?

Is it possible to define more than one graphite database? (BUG)

Hi there,

Is it possible to define more than one graphite database?
And if it is, what should be the syntax for graphite_options?
I tried this code:

    graphite_options        => [ {
      'enabled'           => true,
      'database'          => 'graphite',
      'bind-address'      => ':2003',
      'protocol'          => tcp,
      'consistency-level' => 'one',
      'name-separator'    => '.',
      'batch-size'        => 1000,
      'batch-pending'     => 5,
      'batch-timeout'     => '1s',
      'udp-read-buffer'   => 0,
    },
    {
      'enabled'           => true,
      'database'          => 'graphite2',
      'bind-address'      => ':2011',
      'protocol'          => tcp,
      'consistency-level' => 'one',
      'name-separator'    => '.',
      'batch-size'        => 1000,
      'batch-pending'     => 5,
      'batch-timeout'     => '1s',
      'udp-read-buffer'   => 0,
    } ],

But I've got an error:

Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Failed to parse template influxdb/influxdb.conf.erb:
  Filepath: /etc/puppet/modules/influxdb/templates/influxdb.conf.erb
  Line: 128
  Detail: undefined method `key' for nil:NilClass
 at /etc/puppet/modules/influxdb/manifests/server/config.pp:96 on node influxdbnode-96jxf36fl7tdfn7
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run

Please advice. Thank you!

Error: Could not set 'present' on ensure

Hi

I am trying to use your module with Vagrant and receiving the following error:

==> default: Info: Applying configuration version '1449750519'
==> default: Error: Could not set 'present' on ensure: No such file or directory - /etc/opt/influxdb/influxdb.conf20151210-8384-1nkkxh5.lock at 84:/tmp/vagrant-puppet/modules-511dbe0177ab2cd68e9692b8ddd398e8/influxdb/manifests/server/config.pp
==> default: Error: Could not set 'present' on ensure: No such file or directory - /etc/opt/influxdb/influxdb.conf20151210-8384-1nkkxh5.lock at 84:/tmp/vagrant-puppet/modules-511dbe0177ab2cd68e9692b8ddd398e8/influxdb/manifests/server/config.pp
==> default: Wrapped exception:
==> default: No such file or directory - /etc/opt/influxdb/influxdb.conf20151210-8384-1nkkxh5.lock
==> default: Error: /Stage[main]/Influxdb::Server::Config/File[/etc/opt/influxdb/influxdb.conf]/ensure: change from absent to present failed: Could not set 'present' on ensure: No such file or directory - /etc/opt/influxdb/influxdb.conf20151210-8384-1nkkxh5.lock at 84:/tmp/vagrant-puppet/modules-511dbe0177ab2cd68e9692b8ddd398e8/influxdb/manifests/server/config.pp
==> default: Notice: /Stage[main]/Influxdb::Server::Service/Service[influxdb]: Dependency File[/etc/opt/influxdb/influxdb.conf] has failures: true
==> default: Warning: /Stage[main]/Influxdb::Server::Service/Service[influxdb]: Skipping because of failed dependencies
==> default: Notice: /Stage[main]/Influxdb::Server/Anchor[influxdb::server::end]: Dependency File[/etc/opt/influxdb/influxdb.conf] has failures: true
==> default: Warning: /Stage[main]/Influxdb::Server/Anchor[influxdb::server::end]: Skipping because of failed dependencies

Tried version 0.2.1 and Master with InfluxDB 0.9.5 and 0.9.6
Thanks

make a fresh release

Hi,
could you please make a fresh release? I would love to use the new archlinux support.

Fix puppetlabs/stdlib and puppetlabs/apt dependencies

the metadata.json requires the old stdlib version <4.13.0

make sure that this module supports the latest stdlib version.
otherwise it blocks people from upgrading

[R10K::TaskRunner - ERROR] Task #<R10K::Task::Module::Sync:0x00000001de1858> failed while running: "puppet module --modulepath '/home/vagrant/puppet/modules' install --version=4.25.1 --ignore-dependencies puppetlabs/stdlib" returned with non-zero exit value #<Process::Status: pid 17534 exit 1>

update: same for the apt package which is 3.0.0 (latest version on forge: 4.5.1)

is also related to #65 and #70

influxdb2

Should we create a PR to be able to move to influxdb2? or should influxdb puppet module not support influxdb2? (like new module called influxdb2)

if so should this module switch to the v2 automatically or just have the forge version bumped a major version with a warning?

Empty config file on RHEL6

Hi!

On RHEL6 with the following config:

class {'influxdb::server':
    collectd_options => {
      enabled => true,
      bind-address => '127.0.0.1:25826',
      database => 'collectd',
      typesdb => '/usr/share/collectd/types.db',
      batch-size => 1000,
      batch-pending => 5,
      batch-timeout => '1s',
      read-buffer => 0,
    },
  }

I'm getting a config file with lots of values missing:

Leading to errors:

###
### [meta]
###
### Controls the parameters for the Raft consensus group that stores metadata
### about the InfluxDB cluster.
###
[meta]

  dir = "/var/lib/influxdb/meta"
  hostname = "metrics-stack-puppet-profile"
  bind-address = ":8088"
  retention-autocreate = true
  election-timeout = "1s"
  heartbeat-timeout = "1s"
  leader-lease-timeout = "500ms"
  commit-timeout = "50ms"
  cluster-tracing =
  raft-promotion-enabled =
2016/01/26 18:55:38 InfluxDB starting, version 0.9.6.1, branch 0.9.6, commit 6d3a8603cfdaf1a141779ed88b093dcc5c528e5e, built 2015-12-10T23:40:23+0000
2016/01/26 18:55:38 Go version go1.4.2, GOMAXPROCS set to 1
2016/01/26 18:55:38 Using configuration at: /etc/influxdb/influxdb.conf
run: parse config: Near line 22 (last key parsed 'meta.cluster-tracing'): Expected value but found '\n' instead.

This is using 58fe0e7c9060bd9f03b95aac8a8ed9db48a87fe7

UDP not configurable

To reproduce:

class { "influxdb::server":
    udp_enabled => true,
    udp_database => "helios",
    udp_bind_address => ":8089",
}

expected result:

[[udp]]
  enabled = true
  bind-address = ":8089"
  database = "helios"

actual result

[udp]
  enabled = true
  bind-address = ":8089"
  database = "helios"

this leads to influxdb 0.9.3 not starting:

run: parse config: Type mismatch for 'run.Config.udp': Expected slice but found 'map[string]interface {}'.

Publish new versions to the forge

Hi.

Could you please publish new versions of your module to the forge?

It seems that there has been some cool changes since last year.

Service influxdb does not refresh on config file change

When supplying a collectd config as follows

  $collectd_config = {
    'default' => {
      'enabled'          => true, # not default
      'bind-address'     => ':25826',
      'database'         => 'collectd',
      'retention-policy' => '',
      'typesdb'          => '/usr/share/collectd/types.db',
      'batch-size'       => 5000,
      'batch-pending'    => 10,
      'batch-timeout'    => '10s',
      'read-buffer'      => 0,
    }
  }

  $global_config = {
    'reporting-disabled' => false,
  }

  class {'influxdb':
    manage_repos    => true,
    global_config => $global_config,
    collectd_config => $collectd_config,
  }

and using a systemd repo managed version of influxdb, the config is written after the package install which has already started the service. The config write should refresh the service so it will restart and pull in the new config. I will send a pull request momentarily, thank you!

Creation of config file is not deterministic

In my environment /etc/influxdb/influxdb.conf changes on every puppet run, which triggers a service restart. I patched the template file to iterate the config hash maps with a sorted key order.

Influxdb 0.13

The settings provided by your module are no longer necessary for Influxdb 0.13. The template file should reflect this.

Create database with hiera

hi,

sorry for my question: I try to create a database for icinga2, but don't now how can I do it. Can somebody show me, how the hiera yaml should looks like?

cu

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.