pommi / cgp Goto Github PK

Hi,

Without double quotes to protect DEF image rendering and javascript rendering aren't working
Patching rrd_escape to return str_replace(':', ':', $value); make image rendering working but not javascript rendering
Adding double quote (or single quote) (and remove rrd_espape) make image rendering working but not javascript rendering

Demo for this bug :
http://web.quakelive.fr/CGP/host.php?h=host.quakelive.fr:vmadmin&p=libvirt

I didn''t find how to fix this correctly ...

Best regards

javascript rrd is fixed : http://sourceforge.net/projects/javascriptrrd/files/javascriptrrd/1.1.1/

I'm not sure this would be at all possible, but would it be possible to combine data from multiple hosts?

Basically, we have a bunch of identical servers. They all collect the same data. Rather than viewing each of them individually, for some items, we would want to "sum" or "average" the data accross all of the servers.

Is that something we could achieve with CGP?

I am running CGP with the revision 355a313. When I load the page, I get the following message in the http server log (lighttpd):

(mod_fastcgi.c.2676) FastCGI-stderr: PHP Notice:  Undefined index: cat in /[...]/index.php on line 12

In the file index.php (in the root directory of CGP), I added isset() to the if statement in order to check if $CONFIG['cat'] is set in the config.php.

# show all categorized hosts
if (isset($CONFIG['cat']) && is_array($CONFIG['cat'])) {
    ...
}

I am new to php and I don't now if this is the right way to do it. However, with this change, I get rid of the message :)

Something strange happens when visualizing a derived value (ipt_read). The values are only shown properly in the 2 hours view. There is a particular spike from a speed test that is shown properly as 4MBytes/s in the 2hour view (which is correct). Switching to the 8hour view the y-axis gets a different scale and the spike is only 1.2MBytes/s. Day scale is the same. On the week-view the same value is (according to the y-axis scale) 350k/s.

collection.cgi shows everything properly, so I'd say the problem must be with cgp.

For the first time, we're trying to monitor the df of our servers using CGP and we can't get anything to be displayed :/ Everything else works, and the df output works well in collection3 so I'm thinking this is a CGP thing.

When loading the <img>'s src in my browser (Firefox) I get an error like The image "http://...." cannot be displayed because it contains errors.

No message in the lighty logs :/ Any idea how I could debug further?

I use the latest version of CGP (v4) and some graphs are shown as broken images because RRD files contain spaces in their filenames ex:
if_packets-vmxnet3 Ethernet Adapter.rrd
if_octets-vmxnet3 Ethernet Adapter.rrd

Is there any way to avoid/fix this?

ps: CGP v3 works fine with those RRDs.

NOTICE messages are logged becouse get_host_rrd_files returns all rrd files that have been found. I use a setup where I have rrdcached runing and is is used by collectd and pnp4nagios. All pnp4nagios stats go into same directory as collectd data.

pnp4nagios:

/srv/rrd/HOSTNAME/_HOST_.rrd

collectd:

/srv/rrd/HOSTNAME/snmp/if_octets-ether1.rrd

It is not a big problem. CGP would need to filter data returned by get_host_rrd_files so it will match only files from collectd.

When using the default template for instance, filenames are passed to Type_Base::parse_filename before serving as command arguments. All arguments are escaped by escapeshellargs before the command is build.

Thus, every space and parenthesis is first escaped by Type_Base::parse_filename (backslashed) then quoted by escapeshellargs. The rrdtools command fails due to the erroneous path :

ERROR: opening '/var/lib/collectd/rrd/xxx/ipmi/temperature-System\ Temp\ system_board\ \(7.1\).rrd'

Sticking to escapeshellargs only sounds reasonable.

ProcessMatch "Foo Bar" "process to grep" will fail, while:
ProcessMatch "Foo_Bar" "process to grep" won't.

I am looking for support of a mongo db plugin.
https://github.com/sebest/collectd-mongodb (because it works with 4.x as well as with 5.x of collectd)

Will this be implemented soon?

I found this script https://gist.github.com/gunkaaa/9501723 which is nice to keep an eye on queries and recursiverespones I tried for 2 hours to make a plugin for CGP but failed ^^'

When i switched from png to canvas mode and try to display graphs for dns, I see message box with text:

Invalid RRD: "Wrong magic id."
undefined

In apache and php logs there is no information about error. I tried delete some rrd files in dns direstory with names like dns_qtype-#18466.rrd - no effect for me.
In png mode graphs are fine and there is no error message boxes.

Can you give some ideas for suggestions for debug this problem?

From the http server log, I get the following message

(mod_fastcgi.c.2676) FastCGI-stderr: PHP Notice:  ob_clean(): failed to delete buffer. No buffer to delete in /[...]/CGP/rrd.php on line 11

Canvas type graph is great, but can't click on it to view it on detailed page (bigger size).
Maybe it will be configureable, which graph type I need on host page and on detailed page.
I'll use png on host page and canvas on detailed page. Png is faster, and informal, canvas view is better (and bigger) on detailed page.
Is it possible?

Sry for my bad english :)

Hi

I have pulled b6ff5ed@master and noticed that when setting:

$CONFIG['graph_type'] = 'canvas';
$CONFIG['graph_type'] = 'hybrid';

Results in box being displayed with:

Unknow argument: 800
undefined

May it be related to #87 ?

There is no graphing support for the Collectd Serial plugin

lost 2 file
ps_state-idle.rrd
ps_state-wait.rrd

Hello!
I believe it would really amazingly convenient if CGP's graphs (generated by rrdgraph I believe) showed things like 95' percentiles and/or medians.

I looked into the PHP code and I believe this should be something like
sprintf('DEF:min_%s%s=%s:%s:MIN', crc32hex($sources[$i]), $raw, $this->parse_filename($this->files[$tinstance]), $ds which defines and get the minimum (?) that we'd need to use to get the percetiles.

Unfortunately, my rrd knowledge is close to 0... any help would be greatly appreciated!

Hey
This commit (when enable) seem to broke Canvas,
d21f4a7

TypeError: this.gdes[vidx] is undefined
this.gdes[i].yrule = this.gdes[vidx].vf.val;

Best regards,

I configured collectd to get if_octects of a TPlink access point via snmp, collectd seems to have correctly gather data (rrd dump files seems ok) but cgp returns those errors if I try to render some graphs.

PHP Warning: Creating default object from empty value in /cgp2/plugin/snmp.php on line 12, referer: https://server.local/cgp2/host.php?h=tplink.local&p=snmp PHP Notice: Undefined property: stdClass::$args in /cgp2/plugin/snmp.php on line 21, referer: https://server.local/cgp2/host.php?h=tplink.local&p=snmp PHP Notice: Undefined property: stdClass::$identifiers in /cgp2/plugin/snmp.php on line 33, referer: https://server.local/cgp2/host.php?h=tplink.local&p=snmp PHP Fatal error: Call to undefined method stdClass::rrd_graph() in /cgp2/plugin/snmp.php on line 34, referer: https://server.local/cgp2/host.php?h=tplink.local&p=snmp

Collectd versions is 5.1.0, cgp is updated to the last git commit from github and all is running os Debian Wheezy.

Thanks for any help

Hi,

There is a notice about PHP Notice: Undefined index: type on graph.php switch ($plugin_json[$type]['type']) {

Perhaps we can have a default.json and do an array_replace on this one with current_plugin.json to avoid these notices ?

Structure is very similar to sensors (besides having multiple sensors):

# ls ipmi/
fanspeed-FAN 1 fan_cooling (29.1).rrd
fanspeed-FAN 2 fan_cooling (29.2).rrd
fanspeed-FAN 3 fan_cooling (29.3).rrd
fanspeed-FAN A fan_cooling (29.5).rrd
temperature-Peripheral Temp system_board (7.2).rrd
temperature-System Temp system_board (7.1).rrd
voltage-12V system_board (7.17).rrd
voltage--12V system_board (7.34).rrd
voltage-3.3VCC system_board (7.32).rrd
voltage-5VCC system_board (7.33).rrd
voltage-AVCC system_board (7.36).rrd
voltage-VBAT system_board (7.18).rrd
voltage-Vcore processor (3.2).rrd
voltage-VDIMM memory_device (32.1).rrd
voltage-VSB system_board (7.35).rrd

All rrd files are gauges.

memory.json lost var Cache for osx/bsd

Lets get it in. I'll do a pull req as soon as collectd/collectd#464 is in a collectd release since that's when I will start using this. Just thought I'd do an issue in case someone else wants to add the lines needed.

As collectd supports multiple types db, we can separate our own types db with the one shipped with the collectd. I suggests that CGP can also supports multiple types db.

Can we get a dbi.php or generic type plugin so i can get some ideas. I have started writing a dbi plugin on my own for now.

Is it possible to style the interface to a more modern UI ?
I tested twitter bootstrapp and it is very nice now!

If you want, i can implement this...

1. df: It just shows as df on <hostname> without specification of the mountpoint. So all the mountpoints for the same host have the same title. It happens if the plugin doesn't have ReportReserved true in configuration
2. df: graphs for Free/Reserved/Used are split to 3 graphs otherwise
3. interface: it shows up as Interface Packets () on <hostname>, without interface name (yep, just 2 empty parentheses)

It's collectd 4 being used if it is important, but another UIs show the given metric names correctly. So the data presumably is stored correctly in rrd files. Also some similar graphs, like disk still have correct titles.

I've started working on adding stuff that requires jQuery in my 'jquery' branch. tgharold@3cf2046

It's not ready for a pull into the master, but it is working for auto-refreshing the graph on the details page. Current version is hard-coded to reload the graph every 10 seconds.

My plan is that there will be a set of config file boolean parameters that can be set to enable this behavior. The refresh interval would be tuned by an "aggressiveness" parameter (default of 200). Higher "aggressiveness" values would result in more frequent refreshes of the graph. For a graph which is 86400 seconds wide, the default refresh would be (86400 / 200) or every 432 seconds. For a graph that is 7200 seconds wide, refresh time would be (7200/200) = 36 seconds.

Minimum aggressiveness value would be 1, maximum aggressiveness value would be 1000.

In cases where auto-refresh is enabled, the 'cache' option will need to be changed to match the refresh interval.

Other possible uses of jQuery that I'm thinking about:

• On a host page with multiple graphs, add a filter box at the top which can quickly hide any graphs that don't match the filter. It would be a simple text filter that matches on p= value, pi= value and t= value. The initial version might just mash those (3) parameter values together and use them as the <div> id value.
• Changing the graph size on-the-fly.
• Being able to select multiple types in the left-hand column for display at the same time on the host page.
• Quick filtering of hosts on the index page.
• Other ideas?

I don't feel a need to completely restructure CGP as I really like the design and the use of URLs that can be bookmarked. The UI is very good for exploring all of the possible graphs. Hopefully you like the upcoming changes that I'm doing on the jquery branch.

Hi, Collectd Wiki mentions CGP license to be GPLv3. Please put in a README to that effect.

Hey,

For requests such as this one: /rrd.php/d31bcb1edd53/cpu-0/cpu-idle.rrd I get the Collectd Graph Panel page saying "Forbidden, Return home ...". And consequently I don't see any graphs. For graph_type = 'png' things work perfectly. Is there something apart from graph_type that should be set?

I use the cloned version of the project and the os is ubuntu 12.x.

Thanks

is it impossible to implement?
and a option for size and turn off in config.php
:)

The index-page is very slow (I have a list of 150+ servers) even if I disable $CONFIG['showload']. It can take up to a minute to generate the page.

Turns out the heavy assignment $cores = count(group_plugindata(collectd_plugindata($host, 'cpu'))); in host_summary() is called before the if statement, instead of within thus not effectively disabling the showload.

I moved the line and the index page is now rendered in a jiffy.

Hi.

I did make a reproducable example. Please see merge request #44

Since commit 0a547ad, plugin/ contains JSON files instead of PHP files. graph.php has been updated accordingly but not detail.php. This file still contains:

if ($CONFIG['graph_type'] == 'canvas') {
    chdir($CONFIG['webdir']);
    include $CONFIG['webdir'].'/plugin/'.$plugin.'.php';
} else {
    ...

Hence graph.php fails to load the needed plugin to draw the requested graph and hence no graph is drawn.

As image describe i have:

TCP Connections () at ${my_hostname}

the port number in describe is missing.

my collectd conf:

LoadPlugin tcpconns
<Plugin tcpconns>
       LocalPort "22"
       LocalPort "25"
</Plugin>

The df plugin functionality with collectd 4 was broken by this commit:
0c50eff

Hi

Here's a plugin to integrate yoctopuce sensors (at least the few I have)
stackmagic/CGP@pommi:master...8b80b93

More information on how to setup everything is here:
https://github.com/stackmagic/collectd-yoctopuce

Cheers

memory-active.rrd
memory-cache.rrd
memory-free.rrd
memory-inactive.rrd
memory-wired.rrd

Hi

Here's a little update to the snmp plugin so it supports data retrieved via the IF-MIB (RFC2233: https://tools.ietf.org/html/rfc2233 ):
stackmagic@68edff3

It supports several pieces of information such as in/out for octetc, errors, unicast, multicast, broadcast for each port of a switch (or other device that supports the IF-MIB).

The collectd configuration required is as follows:

<Plugin snmp>
    <Data "traffic">
        Type "net_if_octets"
        Table true
        Instance "IF-MIB::ifDescr"
        Values "IF-MIB::ifInOctets" "IF-MIB::ifOutOctets"
    </Data>

    <Data "errors">
        Type "net_if_errors"
        Table true
        Instance "IF-MIB::ifDescr"
        Values "IF-MIB::ifInErrors" "IF-MIB::ifOutErrors"
    </Data>

    <Data "discards">
        Type "net_if_discards"
        Table true
        Instance "IF-MIB::ifDescr"
        Values "IF-MIB::ifInDiscards" "IF-MIB::ifOutDiscards"
    </Data>

    <Data "unicast">
        Type "net_if_unicast_pkts"
        Table true
        Instance "IF-MIB::ifDescr"
        Values "IF-MIB::ifInUcastPkts" "IF-MIB::ifOutUcastPkts"
    </Data>

    <Data "broadcast">
        Type "net_if_broadcast_pkts"
        Table true
        Instance "IF-MIB::ifDescr"
        Values "IF-MIB::ifInBroadcastPkts" "IF-MIB::ifOutBroadcastPkts"
    </Data>

    <Data "multicast">
        Type "net_if_multicast_pkts"
        Table true
        Instance "IF-MIB::ifDescr"
        Values "IF-MIB::ifInMulticastPkts" "IF-MIB::ifOutMulticastPkts"
    </Data>

    <Host "YOUR_SWITCH">
        Address   "YOUR_SWITCH"
        Version   2
        Community "public"
        Collect   "traffic" "errors" "discards" "unicast" "broadcast" "multicast"
    </Host>

You'll also need additional collectd types and reference them from the collectd config:

net_if_octets         ifInOctets:DERIVE:0:U,        ifOutOctets:DERIVE:0:U
net_if_errors         ifInErrors:DERIVE:0:U,        ifOutErrors:DERIVE:0:U
net_if_discards       ifInDiscards:DERIVE:0:U,      ifOutDiscards:DERIVE:0:U
net_if_unicast_pkts   ifInUcastPkts:DERIVE:0:U,     ifOutUcastPkts:DERIVE:0:U
net_if_broadcast_pkts ifInBroadcastPkts:DERIVE:0:U, ifOutBroadcastPkts:DERIVE:0:U
net_if_multicast_pkts ifInMulticastPkts:DERIVE:0:U, ifOutMulticastPkts:DERIVE:0:U

On debian, the mib can be installed via aptitude install snmp-mibs-downloader the mid definition is then available at /usr/share/mibs/ietf/IF-MIB

Manual querying would work like this `snmpwalk -v 2c -c YOUR_COMMUNITY -m ALL YOUR_SWITCH ifInUcastPkts?

Cheers!

We are using the GenericStacked graphs to display our stats.
However, I think we found a bug.
Basically, if we have at least one number for each value in the data set, then, we're fine. However, when for one key in the data set, we have no value, then, the previous value is not displayed. These 2 graphs should be clearer:

First graph: everything is fine (ignore the gaps):

As you can see we have values for each of the series 200,304 and 404.
Now, when we add the 201 to that series, here is what happens:

AS you can the 201 series shows Nan for its values which is expected since there was no 201 in this time frame... however, the 200 values are gone from the graph!

Any idea why? How can we fix it?

I am collecting queue metrics via snmp plugin. I have added a custom types.db in collectd witch has client_queue rx:DERIVE:0:U, tx:DERIVE:0:U. It is exactly the same as if_octets. Problem is that CGP does not graph any of these.

plugins/snmp.json

{
    "if_octets": {
        "title": "Interface Traffic ({{TI}})",
        "vertical": "{{ND}} per second",
        "type": "io",
        "legend": {
            "rx": {
                "name": "Receive",
                "color": "0000ff"
            },
            "tx": {
                "name": "Transmit",
                "color": "00b000"
            }
        },
        "legend_format": "%5.1lf%s",
        "datasize": true
    },
    "client_queue": {
        "title": "Client queue ({{TI}})",
        "vertical": "{{ND}} per second",
        "type": "io",
        "legend": {
            "rx": {
                "name": "Receive",
                "color": "0000ff"
            },
            "tx": {
                "name": "Transmit",
                "color": "00b000"
            }
        },
        "legend_format": "%5.1lf%s",
        "datasize": true
    }
}

Errors:

[Thu Jul 24 15:15:57 2014] [error] [client x.x.x.x] PHP Notice:  Undefined offset: 1 in /var/www/cgp/type/GenericIO.class.php on line 44, referer: https://somehost.com/cgp/host.php?h=R2&p=snmp
[Thu Jul 24 15:15:57 2014] [error] [client 178.16.113.2] PHP Notice:  Undefined offset: 1 in /var/www/cgp/type/GenericIO.class.php on line 55, referer: https://somehost.com/cgp/host.php?h=R2&p=snmp
[Thu Jul 24 15:15:57 2014] [error] [client 178.16.113.2] PHP Notice:  Undefined index:  in /var/www/cgp/type/GenericIO.class.php on line 55, referer: https://somehost.com/cgp/host.php?h=R2&p=snmp
[Thu Jul 24 15:15:57 2014] [error] [client 178.16.113.2] PHP Notice:  Undefined offset: 1 in /var/www/cgp/type/GenericIO.class.php on line 44, referer: https://somehost.com/cgp/host.php?h=R2&p=snmp
[Thu Jul 24 15:15:57 2014] [error] [client 178.16.113.2] PHP Notice:  Undefined offset: 1 in /var/www/cgp/type/GenericIO.class.php on line 55, referer: https://somehost.com/cgp/host.php?h=R2&p=snmp
[Thu Jul 24 15:15:57 2014] [error] [client 178.16.113.2] PHP Notice:  Undefined index:  in /var/www/cgp/type/GenericIO.class.php on line 55, referer: https://somehost.com/cgp/host.php?h=R2&p=snmp
ERROR: invalid rpn expression in: avg_e4d3c1cc,avg_0_neg,LT,avg_e4d3c1cc,avg_0_neg,IF
ERROR: invalid rpn expression in: avg_bd6a43a4,avg_0_neg,LT,avg_bd6a43a4,avg_0_neg,IF

Hello,
Title describes the issue.

To reproduce:

1. add some rrdtool_opts in conf/config.php (for example --disable-rrdtool-tag)
2. verify graph generated by the plugin which adds own options (like cpu.json)
   apparently the options defined in conf/config.php are overwritten.

observed at 786f0e7

BR

CGP is really great and fast, which is a blessing.
However in our app, we use a lot of custom metrics that we're tracking with collectd. At this point, it looks like they're not rendered by CGP. Is it something we can do/add? If so, how?

Thanks!
Ps: sorry, I looked for docs but couldn't find anhthing about adding a new plugin.

Not entirely sure what the cause of this is. Choosing canvas graphs in the config file generates this message: "the RRD does not contain an RRA matching the chosen CF" followed by "undefined". Looks like the "undefined" comes from "e.stack", with the error coming from RrdDatafile.js.

PNG graphs work fine. Any suggestions?

Setting config graph_type to 'canvas' will give me

while png continues to work just fine. Any chance that I'm overlooking something here?

The system uses collectd 5.

uptime from osx/bsd not show , i run php from php-cgi, no see any error ...

Hi, after commit cc28f86 I noticed an error on GenericIO.class.php.
The problem on snmp.php persists but with this different error:
[Wed Jun 26 23:48:39 2013] [error] [client 192.168.1.200] PHP Notice: Undefined index: ath0-rx in /cgp2/type/GenericIO.class.php on line 46, referer: https://server.local/cgp2/host.php?h=tplink.local&p=snmp ERROR: Could not parse color in ''

The error is repeated for each interface checked using snmp.
I found something about php isset() function, I hope it can help, I have no dev skills to change code in GenericIO.class.php --> http://notesofgenius.com/how-fix-php-notice-undefined-index/
I tried changing php.ini file adding "error_reporting(E_ALL ^ E_NOTICE);"; problem persists but all errors disappeared except last line (ERROR: Could not parse color in ''), I know it's not a solution but I tried.

Please let me know if I can help in some ways or with other infos.