GithubHelp home page GithubHelp logo

voxpupuli / puppetboard Goto Github PK

View Code? Open in Web Editor NEW
706.0 92.0 240.0 23.34 MB

Web frontend for PuppetDB

Home Page: https://pypi.org/project/puppetboard/

License: Apache License 2.0

Python 64.02% CSS 6.35% JavaScript 3.53% HTML 23.12% Smarty 2.15% Dockerfile 0.48% Ruby 0.35%
hacktoberfest dashboard puppetdb reporting

puppetboard's Introduction

Puppetboard

PyPI Version PyPI Downloads Tests Status codecov By Voxpupuli

Puppetboard is a web interface to PuppetDB aiming to replace the reporting functionality of Puppet Enterprise console (previously: Puppet Dashboard) for the open source Puppet.

Overview

See more screenshots here.

Table of Contents

Requirements

  • PuppetDB v. 5.2-8.*
    • (Note: PuppetDB 8.1.0 is not supported because of this bug. Please update to 8.1.1+.)
  • Python 3.8-3.11 or Docker

Installation

Puppetboard is packaged and available on PyPI.

With Puppet module

There is a Puppet module originally written by Spencer Krum and currently maintained by Voxpupuli that takes care of installing the Puppetboard for you.

To see how to get it working with RedHat/Centos 7 check out these docs.

Using Docker

We provide an official Docker image in the GitHub Container Registry.

You must provide a secret key! Generate one for example by running ruby -e "require 'securerandom'; puts SecureRandom.hex(32)".

You can run the app on your PuppetDB host with this command:

docker run -it \
  -e PUPPETDB_HOST=localhost \
  -e PUPPETDB_PORT=8080 \
  -e SECRET_KEY=XXXXXXXX \
  --net=host \
  ghcr.io/voxpupuli/puppetboard

Optionally you can set PUPPETBOARD_URL_PREFIX env variable to a value like /puppetboard to run the app under a URL prefix.

You can use the following Puppet Code to have Puppetboard managed by Puppet:

include docker

docker::image { 'ghcr.io/voxpupuli/puppetboard': }

docker::run { 'puppetboard':
  image => 'ghcr.io/voxpupuli/puppetboard',
  env   => [
    'PUPPETDB_HOST=127.0.0.1',
    'PUPPETDB_PORT=8080',
    'PUPPETBOARD_PORT=8088',
    'SECRET_KEY=XXXXXXXX',
  ],
  net   => 'host',
}

If you want to have all features enabled, you must use SSL talking to PuppetDB:

file { '/etc/puppetboard':
  ensure => directory,
}
file { '/etc/puppetboard/key.pem':
  ensure => file,
  mode   => '0644',
  source => "/etc/puppetlabs/puppet/ssl/private_keys/${facts['networking']['fqdn']}.pem",
}
file { '/etc/puppetboard/cert.pem':
  ensure => file,
  mode   => '0644',
  source => "/etc/puppetlabs/puppet/ssl/certs/${facts['networking']['fqdn']}.pem",
}

include docker

docker::image { 'ghcr.io/voxpupuli/puppetboard': }

docker::run { 'puppetboard':
  image   => 'ghcr.io/voxpupuli/puppetboard',
  volumes => ['/etc/puppetboard:/etc/puppetboard:ro'],
  env     => [
    'PUPPETDB_HOST=puppet', # this must be the certname or DNS_ALT_NAME of the PuppetDB host
    'PUPPETDB_PORT=8081',
    'PUPPETBOARD_PORT=8080',
    'ENABLE_CATALOG=true',
    'PUPPETDB_SSL_VERIFY=false',
    'PUPPETDB_KEY=/etc/puppetboard/key.pem',
    'PUPPETDB_CERT=/etc/puppetboard/cert.pem',
    'SECRET_KEY=XXXXXXXX',
    'DEFAULT_ENVIRONMENT=*',
  ],
  net     => 'host',
}

Within an air gapped environment you want to load all content from your local puppetboard web service. Add: 'OFFLINE_MODE=true', to the env parameter list of the docker::run Puppet type.

We also provide the Dockerfile, so you can build the image yourself:

docker build -t puppetboard .

From a package

Actively maintained packages:

Manually

You can also install the package from PyPI and configure a WSGI-capable application server to serve it.

We recommend using virtualenv to provide a separate environment for the app.

virtualenv -p python3 venv
. venv/bin/activate
pip install puppetboard

Please see an article about more deployment setups here.

Configuration

Puppet agents

The default value of usecacheonfailure = true configuration setting for Puppet agents causes Puppet runs to always succeed, event if there are catalog compilation failures f.e. because of a syntax error in your code. This is because in such cases with this setting Puppet will just use a cached working catalog and report the run to PuppetDB as successful. (Although with an error visible in the Puppet run log.)

Therefore, to show the nodes with a catalog compilation as failed in Puppetboard you need to set usecacheonfailure = false in your nodes' puppet.conf.

PuppetDB

Of course you need to configure your Puppet Server to store the Puppet run reports in PuppetDB. If you haven't done that already please follow the PuppetDB documentation about this.

If you run Puppetboard on a different host than PuppetDB then you may want to configure the certificate allow-list for which certificates are allowed to access data from PuppetDB. Please read more about this feature in the PuppetDB documentation here.

App settings

Puppetboard will look for a file pointed at by the PUPPETBOARD_SETTINGS environment variable. The file has to be identical to default_settings.py but should only override the settings you need changed.

If you run PuppetDB and Puppetboard on the same machine the default settings provided will be enough to get you started and you won't need a custom settings file.

Assuming your webserver and PuppetDB machine are not identical you will at least have to change the following settings:

  • PUPPETDB_HOST
  • PUPPETDB_PORT

By default PuppetDB requires SSL to be used when a non-local client wants to connect. Therefore you'll also have to supply the following settings:

  • PUPPETDB_SSL_VERIFY = True
  • PUPPETDB_KEY = /path/to/private/keyfile.pem
  • PUPPETDB_CERT = /path/to/public/keyfile.crt

When using the Puppetboard Docker image, you may also pass Puppetboard it's certificate contents via these environment variables, either as a multiline string or pre-base64 encoded. This can be useful where the certificate is stored in a secrets store i.e. AWS SSM Parameter Store.

PUPPETDB_CERT="-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----"
PUPPETDB_CERT=LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQouLi4KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQ==

For information about how to generate the correct keys please refer to the pypuppetdb documentation. Alternatively it is possible to explicitly specify the protocol to be used setting the PUPPETDB_PROTO variable.

Other settings that might be interesting, in no particular order:

  • SECRET_KEY: set this to a long string, the same for each application replica, and keep it secret. Refer to Flask documentation, section "How to generate good secret keys" for more info. Cannot be an empty string, which is the default.
  • FAVORITE_ENVS: an ordered list of Puppet environment names that will be shown immediately after "All Environments" and before other environments (which are sorted by name) in the dropdown for choosing the environment shown in the top-right of the UI. Environments listed here that do not really exist in your deployment are silently ignored.
  • SHOW_ERROR_AS: friendly or raw. The former makes Puppet run errors in Report and Failures views shown in a modified, (arguably) more user-friendly form. The latter shows them as they are. Defaults to friendly.
  • CODE_PREFIX_TO_REMOVE: what code path that should be shortened in "Friendly errors" to "…" for readability. A regexp. Defaults to /etc/puppetlabs/code/environments(/.*?/modules)?.
  • PUPPETDB_TIMEOUT: Defaults to 20 seconds, but you might need to increase this value. It depends on how big the results are when querying PuppetDB. This behaviour will change in a future release when pagination will be introduced.
  • UNRESPONSIVE_HOURS: The amount of hours since the last check-in after which a node is considered unresponsive.
  • LOGLEVEL: A string representing the loglevel. It defaults to 'info' but can be changed to 'warning' or 'critical' for less verbose logging or 'debug' for more information.
  • ENABLE_QUERY: Defaults to True causing a Query tab to show up in the web interface allowing users to write and execute arbitrary queries against a set of endpoints in PuppetDB. Change this to False to disable this. See ENABLED_QUERY_ENDPOINTS to fine-tune which endpoints are allowed.
  • ENABLED_QUERY_ENDPOINTS: If ENABLE_QUERY is True, allow to fine tune the endpoints of PuppetDB APIs that can be queried. It must be a list of strings of PuppetDB endpoints for which the query is enabled. See the QUERY_ENDPOINTS constant in the puppetboard.app module for a list of the available endpoints.
  • GRAPH_TYPE: Specify the type of graph to display. Default is pie, other good option is donut. Other choices can be found here: C3.js documentation
  • GRAPH_FACTS: A list of fact names to tell PuppetBoard to generate a pie-chart on the fact page. With some fact values being unique per node, like ipaddress, uuid, and serial number, as well as structured facts it was no longer feasible to generate a graph for everything.
  • INVENTORY_FACTS: A list of tuples that serve as the column header and the fact name to search for to create the inventory page. If a fact is not found for a node then undef is printed.
  • INVENTORY_FACT_TEMPLATES: A mapping between fact name and jinja template to customize display
  • ENABLE_CATALOG: If set to True allows the user to view a node's latest catalog. This includes all managed resources, their file-system locations and their relationships, if available. Defaults to False.
  • REFRESH_RATE: Defaults to 30 the number of seconds to wait until the index page is automatically refreshed.
  • DEFAULT_ENVIRONMENT: Defaults to 'production', as the name suggests, load all information filtered by this environment value.
  • REPORTS_COUNT: Defaults to 10 the limit of the number of reports to load on the node or any reports page.
  • OFFLINE_MODE: If set to True load static assets (jquery, semantic-ui, etc) from the local web server instead of a CDN. Defaults to False.
  • DAILY_REPORTS_CHART_ENABLED: Enable the use of daily chart graphs when looking at dashboard and node view.
  • DAILY_REPORTS_CHART_DAYS: Number of days to show history for on the daily report graphs.
  • DISPLAYED_METRICS: Metrics to show when displaying node summary. Example: 'resources.total', 'events.noop'.
  • TABLE_COUNT_SELECTOR: Configure the dropdown to limit number of hosts to show per page.
  • LITTLE_TABLE_COUNT: Default number of reports to show when when looking at a node.
  • NORMAL_TABLE_COUNT: Default number of nodes to show when displaying reports and catalog nodes.
  • LOCALISE_TIMESTAMP: If set to True then timestamps are shown using your browser's timezone. Otherwise UTC is used. Defaults to True.
  • WITH_EVENT_NUMBERS: If set to True then Overview and Nodes list shows exact number of changed resources in the last report. Otherwise shows only 'some' string if there are resources with given status. Setting this to False gives performance benefits, especially in big Puppet environments (more than few hundreds of nodes). Defaults to True.
  • ENABLE_CLASS: If set to True allows the user to view the number of resource events (number of changed resources in the last report) grouped by class. The resource events are grouped by their status ('failure', 'success', 'noop').
  • CLASS_EVENTS_STATUS_COLUMNS: A mapping between the status of the resource events and the name of the columns of the table to display.
  • CACHE_TYPE: Specifies which type of caching object to use when SCHEDULER_ENABLED is set to True. The cache is used for the classes view (ENABLE_CLASS is set to True) which requires parsing the events of all the latest reports to group them by Puppet class. If the last report is present in the cache, we do not parse the events, which avoids unnecessary processing. If you configure more than one worker, you must use a shared backend (e.g. MemcachedCache) to allow the sharing of the cache between the processes. Indeed, the SimpleCache type does not allow sharing the cache between processes, it uses the process memory to store the cache. Defaults to SimpleCache.
  • CACHE_DEFAULT_TIMEOUT: Cache lifetime in second. Defaults to 3600.
  • SCHEDULER_ENABLED: If set to True then a scheduler instance is created in order to execute scheduled jobs. Defaults to False.
  • SCHEDULER_JOBS: List of the scheduled jobs to trigger within a worker. A job can for example be used to compute a result to be cached. This is the case for the classes view which uses a job to pre-compute at regular intervals the results to be displayed. Each scheduled job must contain the following fields: id, func, trigger, seconds.
  • SCHEDULER_LOCK_BIND_PORT: Specifies an available port that allows a single worker to listen on it. This allows to configure scheduled jobs in a single worker. Defaults to 49100.

Getting Help

For questions or bug reports you can file an issue.

Contributing

Development

Puppetboard relies on the pypuppetdb library to fetch data from PuppetDB and is built with the help of the Flask microframework.

If you wish to hack on Puppetboard you should fork/clone the Github repository and then install the requirements through:

pip install --upgrade wheel setuptools
python setup.py develop
pip install --upgrade -r requirements-test.txt
mypy --install-types --non-interactive puppetboard/ test/

You're advised to do this inside a virtualenv specifically created to work on Puppetboard as to not pollute your global Python installation.

You can run the tests with:

pytest --cov=. --cov-report=xml --strict-markers --mypy puppetboard test
pylint --errors-only puppetboard test

You can run the app it in development mode by simply executing:

flask run

You can specify listening host and port with environment variables or command line otions:

export FLASK_RUN_HOST=0.0.0.0
export FLASK_RUN_PORT=8000

flask run

or

flask run --host '0.0.0.0' --port '8000'

Use PUPPETBOARD_SETTINGS to change the different settings or patch default_settings.py directly. Take care not to include your local changes on that file when submitting patches for Puppetboard. Place a settings.py file inside the base directory of the git repository that will be used, if the environment variable is not set.

We welcome contributions to this project. However, there are a few ground rules contributors should be aware of.

License

This project is licensed under the Apache v2.0 License. As such, your contributions, once accepted, are automatically covered by this license.

Commit messages

Write decent commit messages. Don't use swear words and refrain from uninformative commit messages as 'fixed typo'.

The preferred format of a commit message:

docs/quickstart: Fixed a typo in the Nodes section.

If needed, elaborate further on this commit. Feel free to write a
complete blog post here if that helps us understand what this is
all about.

Fixes #4 and resolves #2.

If you'd like a more elaborate guide on how to write and format your commit messages have a look at this post by Tim Pope.

Build a release

please see: RELEASE.md

More Screenshots

  • Overview / Index / Homepage

Overview / Index / Homepage

  • Nodes view, all active nodes

Nodes view, all active nodes

  • Single node page / overview

Single node page / overview

  • Report view

Report view

  • Facts view

Facts view

  • Single fact, with graphs

Single fact, with graphs

  • All nodes that have this fact with that value

All nodes that have this fact with that value

  • Query view - results as table

Query view

  • Query view - results as JSON

Query view

  • Metrics view

Metrics view

  • Single metric

Single metric

  • Inventory view

Inventory view

  • Classes view, group the resource events of the last reports by Puppet class

Classes view

  • Class view, list the nodes with almost one resource event for a given class

Class view

Legal

The app code is licensed under the Apache License, Version 2.0.

The favicon has been created based on the icon created by Jonathan Coutiño under the Attribution 3.0 Unported (CC BY 3.0) license, downloaded from the Noun Project.

puppetboard's People

Contributors

alexjfisher avatar arthurwutw avatar bastelfreak avatar corey-hammerton avatar daenney avatar dependabot[bot] avatar dom-nie avatar fgimian avatar gdubicki avatar gertvdijk avatar juliushaertl avatar kenyon avatar kirkins avatar larsnaesbye avatar lcharreau avatar melck avatar mterzo avatar nibalizer avatar nmaludy avatar octomike avatar petems avatar qkponton avatar raphink avatar rfletcher avatar roidelapluie avatar rwaffen avatar sebastianrakel avatar smortex avatar stack72 avatar tuxmea avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

puppetboard's Issues

Show run/report status

When puppetboard shows a report - it does not show its status (ie. from puppet-dasboard: failed, changed, ok - red/blue/green).

That - and a summary "block" with all my hosts last status (ie. latest report from each host, divided into failed, changed, ok and unresponsive counts).

Problem with passenger_wsgi.py on CentOS 6.4 and Passenger 4.0.41

I installed the latest version of puppetboard on CentOS 6.4 and Passenger 4.0.41 but was getting errors when I tried to access the web page. After some investigation and a lot of Googling I was able to modify the passenger_wsgi.py to get it working. I thought I would share what I changed in case this helps any one. Please understand I am not good with python. I am sure there is a better way to do this.
Louis
Replaced the line:
logging.basicConfig(filename=/path/to/file/for/logging, level=logging.INFO)
With these lines
logfilename = os.path.join('/var/log', 'puppetboard.log')
logging.basicConfig(filename=logfilename, level=logging.INFO)

Overview layout breaks with lots of changes

When there are machines that have more than 9 changes or errors the width given to that column can't contain the red and green labels anymore because they become a few pixels wider.

We need to fix that in such a way that the column will be at least 220px, not fixed to 220px.

ImportError: cannot import name stream_with_context

After cloning 0.0.4 running dev.py (or the former wsgi script for that matter) results in:

Traceback (most recent call last):
  File "dev.py", line 4, in <module>
    from puppetboard.app import app
  File "/tmp/puppetboard/puppetboard/app.py", line 10, in <module>
    from flask import (
ImportError: cannot import name stream_with_context

This is with py-flask-0.10.1 and py-flask-wtf-0.9.4 (not 0.8.4), however this used to work before. Did anything change, or what am I doing wrong? :)

Structured facts display could be improved

Facter 2.0 introduced structured facts (compared to only key-value pairs in 1.x). In the 2.0.x series of Facter, no structured facts were provided by default. However, as stated in the Facter 2.1.0 announcement, now partitions is the first standard structured fact (hash).

Please add support for this in Puppetboard. By support I mean one should think of a way it can be displayed 'nicely', rather than for example:

partitions  {"vda1"=>{"uuid"=>"af740304-b2c6-440f-877a-5fb15cf86c39", "size"=>"12578816", "filesystem"=>"ext4"}}

Unable to launch application TypeError: nodes() got an unexpected keyword argument 'unreported'

I am getting the following stack trace error when I launch the console from the browser. Please help

Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/flask/app.py", line 1836, in call
return self.wsgi_app(environ, start_response)
File "/usr/lib/python2.6/site-packages/flask/app.py", line 1820, in wsgi_app
response = self.make_response(self.handle_exception(e))
File "/usr/lib/python2.6/site-packages/flask/app.py", line 1403, in handle_exception
reraise(exc_type, exc_value, tb)
File "/usr/lib/python2.6/site-packages/flask/app.py", line 1817, in wsgi_app
response = self.full_dispatch_request()
File "/usr/lib/python2.6/site-packages/flask/app.py", line 1477, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/lib/python2.6/site-packages/flask/app.py", line 1381, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/lib/python2.6/site-packages/flask/app.py", line 1475, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/lib/python2.6/site-packages/flask/app.py", line 1461, in dispatch_request
return self.view_functionsrule.endpoint
File "/root/puppetdashboard/puppetboard/puppetboard/app.py", line 108, in index
with_status=True)
TypeError: nodes() got an unexpected keyword argument 'unreported'

multiple puppetdb support

first i wanted just to make a simple proxy that connects to each of my puppetdb (per-datacenter) and merges resulting json, then passing it to puppetboard.

but i've noticed that ie. pagination is supported on puppetdb level, so it won't work (we'd receive multiples of results, with likely issues on sorting).

so, are there any plans to make single puppetboard to query multiple puppetdbs?
or maybe select a backend that we want to talk to from a predefined list with some pull-down?
or should i look for anything else?

Strange layout problem

Hi,

I installed puppetboard on a CentOS 6.5 box by following the instructions on https://github.com/nedap/puppetboard/. Although everything seems to work allright, I am experiencing some strange layout issues where text is cut off at the top. (Both in Firefox 29.0 and Internet Explorer 11).

unbenannt

unbenannt2

Kind regards
Florian

Facts pie should be readable

The facts piechart is hard to see when slices are very thin. Being able to hover over thin slices or click on the slices to drill down would be helpful.

csv export

Just got a question to deliver a list for a boss, who wanted to "play with the results" in a spreadsheet.

Found myself missing puppet-dashboard's CSV export feature :)

It would be great, if one could export all lists (complete list of nodenames, and also resultsets under facts) as csv.

Provide an option to not display value for some events?

Within my puppet manifests I have code like so. The actual password is coming from encrypted storage though.

user{'localadmin'
    password=>'theplaintextpassword',
    provider=>'windows_adsi';
}

The problem of course is that when the above is applied to a system, it gets logged into puppetdb. Then it gets displayed for everyone with access puppetboard to see. The json for the specific event is below.

{
  "containment-path" : [ "Stage[main]", "Localauth", "Localauth::User[localadmin]", "User[localadmin]" ],
  "new-value" : "theplaintextpassword",
  "containing-class" : "Localauth",
  "report-receive-time" : "2014-07-02T22:08:31.286Z",
  "report" : "1892d84ef0bbbd6d909e572782f360d21598e1eb",
  "resource-title" : "localadmin",
  "property" : "password",
  "file" : "/srv/puppet/environments/win_production/modules/localauth/manifests/user.pp",
  "old-value" : "absent",
  "run-start-time" : "2014-07-02T22:06:42.286Z",
  "line" : 22,
  "status" : "success",
  "run-end-time" : "2014-07-02T22:08:15.428Z",
  "resource-type" : "User",
  "timestamp" : "2014-07-02T22:08:02.318Z",
  "configuration-version" : "1404338814",
  "certname" : "thecomputer.example.org",
  "message" : "created password"
},

It sure would be nice to have some way in puppetboard to set some kind of filter on the server if an event has "resource-type" : "User", and "property" : "password", then the old and new values are hidden from the display.

I suspect this would also be useful for other types of resources where a secret would be displayed as part of the new/old value.

Unsafe content warning when deploying over HTTPS

The warning is because the 'flatly' theme for bootswatch includes a google font via a HTTP link. The file in question is the following, the import statement is at the very top.

https://netdna.bootstrapcdn.com/bootswatch/2.3.2/flatly/bootstrap.min.css

I know, this is not an issue with puppetboard itself, but I thought to report it anyway, maybe somebody knows an easy fix! (Not loading the font file is an easy fix, but produces some layout glitches.)

publish changes via rss

Would be awesome to have the feature to access a few things via RSS.
Things that come into my mind:

  • failed runs
  • node changes
  • new nodes

Optional:

  • resource changes

Nodes page fails to load all nodes

I have 1200 nodes in puppetdb. Going to the nodes page ends up failing to get all the nodes with:

INFO:werkzeug:17.153.45.123 - - [21/Aug/2013 21:38:25] "GET /nodes HTTP/1.1" 200 -
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): seovm57.apple.com
Debugging middleware caught exception in streamed response at a point where response headers were already sent.
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/werkzeug/wsgi.py", line 682, in next
return self._next()
File "/Library/Python/2.7/site-packages/werkzeug/wrappers.py", line 81, in _iter_encoded
for item in iterable:
File "/Library/Python/2.7/site-packages/flask/helpers.py", line 122, in generator
for item in gen:
File "/Library/Python/2.7/site-packages/Jinja2-2.7-py2.7.egg/jinja2/environment.py", line 1186, in next
return self._next()
File "/Library/Python/2.7/site-packages/Jinja2-2.7-py2.7.egg/jinja2/environment.py", line 1168, in generator
c = next()
File "/Library/Python/2.7/site-packages/Jinja2-2.7-py2.7.egg/jinja2/environment.py", line 993, in generate
yield self.environment.handle_exception(exc_info, True)
File "/Library/Python/2.7/site-packages/Jinja2-2.7-py2.7.egg/jinja2/environment.py", line 742, in handle_exception
reraise(exc_type, exc_value, tb)
File "/Users/blysik/Documents/git/puppetboard/puppetboard/templates/nodes.html", line 1, in top-level template code
{% extends 'layout.html' %}
File "/Users/blysik/Documents/git/puppetboard/puppetboard/templates/layout.html", line 39, in top-level template code
{% block container %}
File "/Users/blysik/Documents/git/puppetboard/puppetboard/templates/layout.html", line 42, in block "container"
{% block row_fluid %}
File "/Users/blysik/Documents/git/puppetboard/puppetboard/templates/layout.html", line 44, in block "row_fluid"
{% block content %} {% endblock content %}
File "/Users/blysik/Documents/git/puppetboard/puppetboard/templates/nodes.html", line 30, in block "content"
{% for node in nodes %}
File "/Users/blysik/Documents/git/puppetboard/puppetboard/utils.py", line 47, in yield_or_stop
yield next(generator)
File "/Library/Python/2.7/site-packages/pypuppetdb/api/v2.py", line 69, in nodes
facts_timestamp=node['facts_timestamp'],
File "/Library/Python/2.7/site-packages/pypuppetdb/types.py", line 239, in init
self.catalog_timestamp = catalog_timestmap
NameError: global name 'catalog_timestmap' is not defined

This is just a test instance running on Mac OS X 10.8.4.

Add "pending" nodes status, also in overview

I've always been using the Pupper Dashboard in order to get a quick overview of all nodes which have pending changes, since I run all puppet agents with --noop.

This doesn't seem to be available in Puppetboard. Is it because of what PuppetDB stores, making it impossible to get that information?

Currently, my nodes with pending changes appear with status None in the Overview and Nodes pages. I would really like them to show up (in orange for instance) with "with status pending". Would that be possible?

Radiotor

Similar in purpose as Dashboard's radiotor we need a view that shows us the amount of nodes checking in, the amount of failing nodes etc etc.

wish: counters in facts

I was just asked: how many windows hosts do we actually have, running Puppet?
I had to cut'n'paste to a shell, to count with wc.. :)

I could have built a query and counted with curl+wc..

But puppetboard was so easy to use.. that it would have given me the answer, if only it had give me a counter for the resultset size.. :)

show last puppet run status

When puppetboard shows a report - it does not show its status (ie. from puppet-dasboard: failed, changed, ok - red/blue/green).

That - and a summary "block" with all my hosts last status (ie. latest report from each host, divided into failed, changed, ok and unresponsive counts).

Facts page has issue with complicated Fact values

This causes all kinds of havoc on the /facts page:

mwapp_entitlement_0_status=<html><head><title>apache tomcat/6.0.26 - error report</title><style><!--h1 {font-family:tahoma,arial,sans-serif;color:white;background-color:#525d76;font-size:22px;} h2 {font-family:tahoma,arial,sans-serif;color:white;background-color:#525d76;font-size:16px;} h3 {font-family:tahoma,arial,sans-serif;color:white;background-color:#525d76;font-size:14px;} body {font-family:tahoma,arial,sans-serif;color:black;background-color:white;} b {font-family:tahoma,arial,sans-serif;color:white;background-color:#525d76;} p {font-family:tahoma,arial,sans-serif;background:white;color:black;font-size:12px;}a {color : black;}a.name {color : black;}hr {color : #525d76;}--></style> </head><body><h1>http status 401 - </h1><hr size="1" noshade="noshade"><p><b>type</b> status report</p><p><b>message</b> <u></u></p><p><b>description</b> <u>this request requires http authentication ().</u></p><hr size="1" noshade

Insecure permissions recommended for mod_wsgi

The README contains suggestions such as

$ chown www-data:www-data /var/www/puppetboard

and

Make sure this file [wsgi.py] is owned by the user and group the webserver runs as.

Unless I'm missing something this seems like a bad idea. The webserver needs to read wsgi.py, but it does not need to write to it. If there was a bug in puppetboard that allowed an attacker to write to an arbitrary file, they could write new code under /var/www/puppetboard and rewrite wsgi.py to run that code.

Querying the Nodes Page

When a user goes to the nodes page, they need to type the search term in the search bar. When a page refresh happens, it doesnt respect that there is a term in it and it clears the search term

Do you think a query string param would help here?

e.g. /nodes?term=x

Thoughts?

Conenction or version problem?

Hi,

I've just tried out puppetboard and I get the following messages inside the log:

[Tue Nov 18 15:58:00.296193 2014] [:error] [pid 24445:tid 140375343712000] DEBUG:pypuppetdb.api:_query called with endpoint: mbean, path: com.puppetlabs.puppetdb.query.population:type=default,name=num-nodes, query: None, limit: None, offset: None, summarize_by None, count_by None, count_filter: None
[Tue Nov 18 15:47:54.158739 2014] [:error] [pid 24446:tid 140375385675520] DEBUG:pypuppetdb.api:_url called with endpoint: mbean and path: com.puppetlabs.puppetdb.query.population:type=default,name=num-nodes
[Tue Nov 18 15:47:54.159698 2014] [:error] [pid 24446:tid 140375385675520] INFO:urllib3.connectionpool:Starting new HTTPS connection (1): puppet-test01.smaatolabs.net
[Tue Nov 18 15:47:54.170741 2014] [:error] [pid 24446:tid 140375385675520] ERROR:pypuppetdb.api:Could not reach PuppetDB on puppet-test01.smaatolabs.net:8081 over HTTPS.

I'm wondering if there is really a connection problem or what I assume something like a change in the puppetdb API.

Currently I'm using version 0.0.4 of puppetboard and version 1.6.2 for puppetdb.

Could you confirm something? Or have any other hints what is going wrong here?

Pagination

Next release really needs to support pagination, this is getting ridiculous.

Docs incorrect

In your readme, under the section UWSGI + NGINX you have us starting uwsgi like so:

uwsgi --http :9090 --wsgi-file /var/www/puppetboard/wsgi.py

This does not open a UWSGI socket, so the proxy via NGINX does not work.

==> /var/log/nginx/error.log <== 2014/06/05 17:25:51 [error] 27833#0: *7 upstream timed out (110: Connection timed out) while reading response header from upstream, client: x.x.x.x, server: x.x.net, request: "GET / HTTP/1.1", upstream: "uwsgi://x.x.x.x:9090", host: "x.x.net"

You want something like:

uwsgi --socket :9090 --wsgi-file /var/www/puppetboard/wsgi.py

More info available for the report page?

The last thing that's keeping us from turning off Puppet Dashboard and just using PuppetBoard is the extra detail that's provided in the Dashboard display of a report:

  • Log entries; which I think are not available in PuppetDB's API (maybe also not stored there at all?)
  • Time counters for each resource type; again, not seeing anywhere in the PuppetDB API to fetch this
  • List of all resources (I guess "events" in the API?), including unchanged, which is handy for troubleshooting of "did this resource not get put in the catalog like I expected, or is it there but just unchanged?" - it seems like these would be available, maybe collapse or hide them by default?

Are my guesses at API availability for these correct? We'd like to avoid losing useful info when we put Dashboard out to pasture, so adding any of these would be handy for us. Great work on PuppetBoard by the way! 👍

Question: Do you only show 10 reports for a reason?

HI,

I have just migrated from the old puppetdashboard to puppetboard. I love the project. Unfortunately, I got told that the last 10 reports was not enough from one of the engineers.

If I sent a PR to update that to a variable would this be supported? Or is 10 the limit of what we can do easily?

Thanks

Paul

Installation issues.

Hi
I followed instructions on installing puppetboard by doing pip install puppetboard and Apache + mod_passenger, but i am getting error in puppetboard.errror.log saying missing puppetboard.app

Not sure what i am doing wrong ?

Many thanks.

enhancement: /fact/FACT-NAME/FACT-VALUE

A new route like:

@app.route('/fact/fact name/fact value')

Returns the list of the nodes matching that fact lookup. Can probably just use the nodes.html template.

Sorting by catalog or report timestamp does not work

Hey,

Sorting nodes by catalog or report timestamp on the "Nodes" page does not work for some reason. The table keeps getting sorted by Hostname. This is quite an important functionality when it comes to tracking change propagation and rolling updates.

I tried bump-ing the tablesorter version and playing with the date formatting without success.

I am using the latest master and Firefox 31.0. Let me know if you need any additional info.

Status for pending changes - noop

It would be really nice if the status button could show Pending instead of None when there are changes pending. This is really useful when the puppet agents run in noop mode per default.

If the color of the button could also change to something else than grey when this happens it would be really nice.

install latest version

sorry for the nob question but when I install via pip or the puppet forge module with the "latest" param set, I end up with a very old version of puppetboard. I can tell because many part of the UI are broken and in the bottom right of the screen I see 2013 not 2013-2014.
I'm installing on Centos 6.6/python is 2.6
Any suggestions on what to look at, what I might be doing wrong?

Thanks for the great work on this tool by the way !

/nodes timing out

I installed puppetboard on a new machine, set it up with wsgi.

/nodes was timing out, even though I upped PUPPETDB_TIMEOUT to 60:

[Mon Sep 16 15:04:30 2013] [error] [client 17.203.15.7] Timeout: HTTPConnectionPool(host='XXXX.com', port=8080): Request timed out. (timeout=10), referer: http://puppetboard.XXXX.com/facts

I was able to modify pypuppetdb/api/init.py and by hand set timeout=60 at:

def init(self, api_version, host='localhost', port=8080,
ssl=False, ssl_key=None, ssl_cert=None, timeout=60):
"""Initialises our BaseAPI object passing the parameters needed in
order to be able to create the connection strings, set up SSL and
timeouts and so forth."""

Obviously just a hack, but I'm not much of a python guy. Looking at app.py, it appears to be creating the puppetdb object properly, so I really don't know what's going on.

Metric route not working with Apache + mod_passenger

When running puppetboard with Apache and Passenger on Ubuntu 12.04 with Python 2.7.3 the metric route doesn't work.

It seems like there is a problem with the url encoding with the pypuppetdb metric() method, but it works fine running dev.py and with mod_wsgi.

Log output of the failed HTTP call:

[ pid=11081, time=2014-02-03 09:32:19,168 ]: Exception on /metric/com.jolbox.bonecp%253Atype%253DBoneCP [GET]
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1817, in wsgi_app 
    response = self.full_dispatch_request()
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1477, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1381, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1475, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1461, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "puppetboard/app.py", line 353, in metric
    metric = puppetdb.metric(metric)
  File "/usr/local/lib/python2.7/dist-packages/pypuppetdb/api/__init__.py", line 321, in metric
    return self._query('mbean', path=metric)
  File "/usr/local/lib/python2.7/dist-packages/pypuppetdb/api/__init__.py", line 267, in _query
    r.raise_for_status()
  File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 683, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
HTTPError: 404 Client Error: Not Found

Nodes not reporting as errors

I have a weird bug that my node throws an error but it doesnt show up on the dashboard.

root@ci-resloc-03:~# puppet agent -t
Info: Retrieving plugin
Info: Loading facts in /etc/puppet/lib/facter/facter_dot_d.rb
Info: Loading facts in /etc/puppet/lib/facter/pe_version.rb
Info: Loading facts in /etc/puppet/lib/facter/iptables_persistent_version.rb
Info: Loading facts in /etc/puppet/lib/facter/unix_default_gateway.rb
Info: Loading facts in /etc/puppet/lib/facter/otenvironment.rb
Info: Loading facts in /etc/puppet/lib/facter/python_version.rb
Info: Loading facts in /etc/puppet/lib/facter/root_home.rb
Info: Loading facts in /etc/puppet/lib/facter/environment.rb
Info: Loading facts in /etc/puppet/lib/facter/esx_version.rb
Info: Loading facts in /etc/puppet/lib/facter/rabbitmq_erlang_cookie.rb
Info: Loading facts in /etc/puppet/lib/facter/pper_installed.rb
Info: Loading facts in /etc/puppet/lib/facter/jenkins.rb
Info: Loading facts in /etc/puppet/lib/facter/datacenter.rb
Info: Loading facts in /etc/puppet/lib/facter/iptables_version.rb
Info: Loading facts in /etc/puppet/lib/facter/ip6tables_version.rb
Info: Loading facts in /etc/puppet/lib/facter/os_maj_version.rb
Info: Loading facts in /etc/puppet/lib/facter/vmwaretools_version.rb
Info: Loading facts in /etc/puppet/lib/facter/concat_basedir.rb
Info: Loading facts in /etc/puppet/lib/facter/ipaddress4.rb
Info: Loading facts in /etc/puppet/lib/facter/pip_version.rb
Info: Loading facts in /etc/puppet/lib/facter/puppet_vardir.rb
Info: Loading facts in /etc/puppet/lib/facter/virtualenv_version.rb
Info: Loading facts in /etc/puppet/lib/facter/agent_configuration.rb
Info: Caching catalog for ci-resloc-03.qasql.opentable.com
Info: Applying configuration version '1395079566'
Error: Could not start Service[jmeter-agent-upstart]: Execution of '/sbin/start jmeter-agent-upstart' returned 1:
Error: /Stage[main]/Application_jmeter_agent/Upstart::Job[jmeter-agent-upstart]/Service[jmeter-agent-upstart]/ensure: change from stopped to running failed: Could not start Service[jmeter-agent-upstart]: Execution of '/sbin/start jmeter-agent-upstart' returned 1:
Error: Could not start Service[jmeter-agent-upstart]: Execution of '/sbin/start jmeter-agent-upstart' returned 1:
Error: /Stage[main]/Application_jmeter_agent/Upstart::Job[jmeter-agent-upstart]/Service[jmeter-agent-upstart]/ensure: change from stopped to running failed: Could not start Service[jmeter-agent-upstart]: Execution of '/sbin/start jmeter-agent-upstart' returned 1:
Notice: Finished catalog run in 3.20 seconds

But yet when i look at the board, i see the following

screen shot 2014-03-17 at 18 24 55

Has this been seen before? I am using puppet agent 3.4.2, puppetmaster 3.4.3 and puppetdb 1.6

Paul

Facts page is badly formatted

See the attached image. No matter what browser I use (Safari, Chrome, Firefox) my facts page looks like the image where the top of the frame is cut off. You can't even scroll up to see what's been cut off.

puppetoard_and_issues__nedap_puppetboard

puppetboard under passenger

Just wanted to write a short howto make puppetboard work under passenger.

I would recommend the following simply be part of puppetboard:
standing in puppetboard checkout:
mkdir public
mkdir tmp
ln -s wsgi.py passenger_wsgi.py

And the apache vhost is as you want it, but this as a minimum (port 443 or 80 - your choice):
<VirtualHost *:80>
ServerName puppetboard.example.tld
DocumentRoot /var/www/puppetboard/public

RackAutoDetect On
Alias /static /var/www/puppetboard/static
<Directory /var/www/puppetboard/>
Options None
Order allow,deny
allow from all

Also - if you are using CentOS 6 (or RHEL6) all python modules are in the EPEL repo - except pypuppetdb-0.0.2 and python-jinja2 needs to be v2.7+

Ability to configure timezone

Our puppetboard users are all in different timezones, but we like to be able to talk to each other about "the report at such and such time". All our servers use UTC, and that's what we usually use when talking to each other, so it would be great if we could configure puppetboard to use UTC in the web UI regardless of the user's browser timezone.

Query without parameters

Hey,

Would be cool to be able to use query just by checking an andpoint but not writing a query.

It does make sense for factnames for example.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.