GithubHelp home page GithubHelp logo

chef-ganglia's Introduction

Build Status

Description

Installs and configures Ganglia.

Originally written by Heavy Water (http://hw-ops.com/), Now maintained by the Ganglia team.

Requirements

  • SELinux must be disabled on CentOS
  • iptables must allow access to port 80

Attributes

  • node['ganglia']['grid_name'] - the name to use for the ganglia grid - displayed in the web UI
  • node['ganglia']['server_role'] - the name of the role used for the ganglia collector and web server
  • node['ganglia']['server_host'] - (optional) If present, overrides server_role and uses the hostname provided as the ganglia server
  • node['ganglia']['unicast'] - True indicates ganglia should use unicast; false indicates it should use multicast
  • node['ganglia']['clusterport'] - a hash with clustername => portnum pairs for all the clusters in the ganglia grid
  • node['ganglia']['host_cluster'] - a hash with clustername => 1/0 pairs, where 1 indicates the node should be a member of the cluster
  • node['ganglia']['enable_rrdcached'] - Default true. Enables rrdcached on the gmetad server.
  • node['ganglia']['enable_two_gmetads'] - Default false. Setting to true runs two copies of gmetad on the server; one writes out RRDs and the web UI talks to the other. This improves web UI performance for large installations.
  • node['ganglia']['spoof_hostname'] - Default false. Setting to true configures gmond to force the use of its hostname as the node name rather than the default ganglia behavior of using reverse DNS. Useful for cloud environments such as EC2.

Usage

Terminology: the ganglia grid is made up of multiple clusters, each cluster has multiple hosts. It is common to group hosts by function or some other useful designation into a cluster.

Adding the default recipe to your runlist will install gmond. This recipe should be included on all nodes in your grid to get ganglia to graph metrics for them.

There should be one or more hosts running under the role indicated by node['ganglia']['server_role']; these hosts will serve as the web UI and central collection point for all your metrics. It should run at least the ganglia::gmetad and ganglia::web recipes. It may also run the ganglia::gmond_collector recipe if you have multiple clusters in your grid. Adding the ganglia::graphite recipe will enable graphite monitoring in addition to the standard ganglia graphing.

The aggregator recipe will install aggregator.py and run it every minute from cron. The aggregation recipe should be run on the same node as runs your gmond collectors. It will look for attributes set in other recipes indicating what metrics to aggregate and how to aggregate them. It will connect ot each of the collector gmond processes, get all the relevant metrics, aggregate them, and re-submit them to the same gmond under the pseudo-host "all_${clustername}".

LWRP

gmetric

Installs a gmetric plugin.

The plugin is composed of two templates:

  • One for the script
  • One for the cron job that will call the script

The templates must be in the caller cookbook.

Example:

ganglia_gmetric 'memcache' do
    options :port => 11211
end

templates:
cookbooks/memcache/templates/default/memcache.gmetric.erb
cookbooks/memcache/templates/default/memcache.cron.erb

The content of 'options' will be passed to the templates

logtailer

The logtailer LWRP makes it easy to configure the ganglia-logtailer package with a custom module to consume a log file and report statistics to ganglia. If you are using one of the modules that came with ganglia-logtailer (look in /usr/share/ganglia-logtailer), don't use the LWRP - instead create a crontab entry in your recipe.

In order to use the logtailer from cron or the LWRP, you must

When using the LWRP, you must include the python ganglia-logtailer module you want to use in a directory called 'ganglia' in the calling cookbook's templates dir.

For example, if my cookbook configures and installs nginx and I wish to use the ganglia-logtailer with a custom nginx module I would:

  • put the following LWRP invocation in the nginx recipe: include_recipe "ganglia::logtailer" ganglia_logtailer "NginxLogtailer" do action :enable log_file "/var/log/nginx/access.log" end
  • place the python module in mynginx/templates/ganglia/NginxLogtailer.py.erb

python

Installs a python plugin.

The plugin is composed of two templates:

  • One for the python module
  • One for the configuration of the module

The templates must be in the caller cookbook.

Example:

ganglia_python 'memcache' do
    options :port => 11211
end

templates:
cookbooks/memcache/templates/default/memcache.py.erb
cookbooks/memcache/templates/default/memcache.pyconf.erb

The content of 'options' will be passed to the templates

Caveats

This cookbook has been tested on Ubuntu 12.

Search seems to takes a moment or two to index. You may need to converge again to see recently added nodes.

Testing

This cookbook is set up to test using

  • knife test
  • foodcritic
  • chefspec
  • test-kitchen with minitest

To launch all the tests, run:

  • bundle install
  • bundle exec strainer test

For individual tests, examine the Strainerfile for the relevant commands to run.

Continuous tests are run using Travis CI. Travis only runs foodcritic and chefspec; knife test is broken and test kitchen doesn't work with Travis. You are encouraged to run those tests on your own branch.

chef-ganglia's People

Contributors

brugidou avatar cyberflow avatar dje avatar gilles avatar gwaldo avatar humboldtux avatar jamesonjlee avatar jbuchbinder avatar jkugler avatar justizin avatar kamaradclimber avatar kplimack avatar luckymike avatar majuscule avatar mrdimka avatar rbugajewski avatar vvuksan avatar wico avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

chef-ganglia's Issues

Berkshelf install failure for ganglia-0.2.1

This issue is about distribution of ganglia through the use of Berkshelf. Somewhere in the chain of custody between github and a machine running Berkshelf your cookbook is being mangled. This issue probably needs to be routed through another project but I have been unable to figure out who precisely. Please bare with me. I figure this project has a vested interest in proper distribution to Chef users. I am hopeful someone on this project can help redirect me to the proper place to log an issue, or someone will know enough to help convey the problem to the right people.

The issue:
when using ganglia-0.2.1 with Berkshelf the cookbook becomes damaged and cannot be uploaded to a Chef server because it fails validation.

Environment:
berkshelf: 3.1.4
chef server: 11.1.0

Analysis:
Uploading ganglia-0.2.1 with the command berks upload fails because of a validation error. The error reports as follows:

E, [2014-07-23T16:28:27.255594 #10823] ERROR -- : Cookbook file attributes/PaxHeader/ganglia.rb has a ruby syntax error:
E, [2014-07-23T16:28:27.256008 #10823] ERROR -- : /home/vagrant/.berkshelf/cookbooks/ganglia-0.2.1/attributes/PaxHeader/ganglia.rb:1: syntax error, unexpected tIDENTIFIER, expecting $end
E, [2014-07-23T16:28:27.256121 #10823] ERROR -- : 18 gid=1876110778
E, [2014-07-23T16:28:27.256721 #10823] ERROR -- :       ^
E, [2014-07-23T16:28:27.263077 #10823] ERROR -- : Ridley::Errors::CookbookSyntaxError: Invalid ruby files in cookbook: ganglia (0.2.1).
E, [2014-07-23T16:28:27.266391 #10823] ERROR -- : /var/lib/gems/1.9.1/gems/ridley-4.0.0/lib/ridley/chef/cookbook.rb:198:in `validate'

Viewing the contents of the ganglia cookbook on disk it is clear that there has been some error in the tar package handling. This may have occurred at github, at the Opscode supermarket intermediary, or in the Berkshelf code. Within the cookbook every directory is mirrored with a PaxHeader directory, which is a bookkeeping artifact from tar packaging. The PaxHeader directories should not show up in the output from tar. Here is a list of the files as they appear in the downloaded cookbook: https://gist.github.com/4ba91ca3463844aa16d4.git

In that output you can see that, for example, the attributes directory is mirrored by an attributes/PaxHeader directory. attributes contains the file ganglia.rb which has the actual ruby code to set cookbook attributes. attributes/PaxHeader contains the file ganglia.rb which includes metadata tar uses to track the file. Since attributes/PacHeader/ganglia.rb represents itself as a ruby file (by its file extension .rb) the Chef server attempts to validate the ruby contained within. Since the file is metadata, and not actual ruby code, the validation fails and uploading ganglia to the Chef server fails.

So the question is, where do things break down?

My project downloads 62 cookbooks in addition to ganglia. Ganglia is the only cookbook which manifests this particular problem. I am happy to attempt to find a forum in Opscode, or on GitHub to replay this issue, but I am uncertain whether and how the cookbooks provided through supermarket.getchef.com (where Berkshelf pulls Opscode blessed cookbooks) come from. Perhaps they have a cached copy of the project tar that was provided manually from this project?

For my part I am able to script deletion of PaxHeader directories and can move along smoothly. But I sense this may be a problem for other ganglia users who employ Berkshelf. I bring it up here because I suspect folks on this project will be the best advocates for this being straightened out. If I can be of assistance I am happy to do so, but will require input as to where the proper venue is for lodging this issue.

Thanks!

Love & hugs

There's quite a few open pull requests. Could someone merge them or close them?

Merge to Chef Supermarket?

Hello, there is allot of bugfixes and work that has been done on this cookbook since the 0.2.3 release. Basically the version that is on https://supermarket.chef.io/cookbooks/ganglia is broken for rpm based systems and the fixes are already incorporated into the current release. Any chance this can be bumped up to a current version in supermarket?

Add ability to install extended packages

Ubuntu 14.04 there are packages like ganglia-modules-linux or ganglia-monitor-python.
Will be great to have ability install this packages with gmond daemon if i need it.

Build from source needed on e.g. centos?

Hi,

on centos (and probably redhat) we could fetch all the packages from epel:

Name        : ganglia
Arch        : x86_64
Version     : 3.1.7
Release     : 6.el6
Size        : 150 k
Repo        : epel
Summary     : Ganglia Distributed Monitoring System
URL         : http://ganglia.sourceforge.net/
License     : BSD
Description : Ganglia is a scalable, real-time monitoring and execution environment
            : with all execution requests and statistics expressed in an open
            : well-defined XML format.

Is there any reason why we build it from source instead?

gmond does not load modules on Ubuntu 12.04

All gmond templates don’t have a path to all modules and assume that they’re statically linked. When you try to load a dynamically compiled module (as it is the case by default on Ubuntu nodes) you’ll just get an error that the module was not found.

There should be an attribute (module_path or similar) that can be empty be default. Setting the attribute would prefix all modules with that path.

New release?

Version 0.2.3 was released back in 2014 and there has been quite a bit of development on this cookbook since that time. Any chance of a 0.2.4 release?

The biggest thing I need is the package support on Centos and Redhat, we don't want to have to build from source every time we install gmond on a host.

Need solo mode.

Hello,
I really need use cookbook in chef-solo, but some parts of recipes have search. I know how fix it, but i don't know how fix chefspec test for it.

upload to chef community site?

could 0.2.1 be stamped (or tagged?) and be uploaded to the community site, there was a patch that fixes an issue with ganglia and a unknown configuration variable in ubuntu 12.04.

oracle linux support

can you switch from using node['platform'].include?('redhat', 'fedora', 'centos') to node['platform_family'] == 'rhel' so that Oracle Linux gets supported?

ganglia-monitor restart error on ubuntu 14.04

in package ganglia-monitor (3.6.0) on ubuntu 14.04 initscript return exit code 1 when restart because upstart run process fastly than initscript. Generally it error of initscript which return exit code 1 when application already run.

Documentation help

It would help if the attributes were broken down by global, server and client.

I'm having a hard time getting this cookbook to work without trial and error.

Also which recipe needs to be run on which type of host could use a bit more clarification. If I have a server that is running gmetad and the webUI, do that host need the default gmond as well?

ganglia-webfrontend not working on Ubuntu 12.04

All symlinks have to be placed in /etc/apache/sites-enabled/*.conf in order to be read by the default Ubuntu Apache configuration. The current symlink doesn’t conform to Ubuntu file system naming standards and is just /etc/apache/sites-enabled/ganglia. It should be /etc/apache/sites-enabled/001-ganglia.conf instead.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.