GithubHelp home page GithubHelp logo

collectsphere's People

Contributors

dennis1987 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

collectsphere's Issues

VM Name Length Issue

In my lab

collectd.collectsphere.collectsphere.gauge-lab_cluster.vm.vse-customer-router1_(5c4131)-0.cpu.all.usage.perc

was truncated to

collectd.collectsphere.collectsphere.gauge-lab_cluster.vm.vse-customer-router1_(5c4131)-0.cpu.all.usage.pe

which means the string is still too long for collectd to handle. Since we have control of the string from element 4, the only idea I have to truncate the VM name even more.

restriction in collectd's type_instance variable

When we dispatch the values to collectd, we set type_instance to

clustername."vm".vmname.countergroup.counterinstance.countername.unit

which can easily exceed the 63 character limitation. See https://collectd.org/wiki/index.php/Naming_schema

NAAs and VMFS uuids are already shortened introducing the probablity of collision. When monitoring vCloud Director controlled VMs we regularly exceed that limit as VMs are named in the

(<35 character uuid>)

pattern.

The only idea I have to solve this is to detect this naming pattern and shorten the uuid to e.g. 6 characters as we do with NAAs and VMFS uuids.

Any ideas?

Metrics show in Graphite but not Values

I found out that vCloud Director controlled VMs are shown in Graphite but the chart says "no data" when you try to render it. All non vCD VMs work fine, so I assume it has something to do with the name under which they appear.

Permission denied error in journalctl starting collectd

I have recently installed collectsphere and it works correctly running test.py against my test esxi host. However running collectd via systemctl gives the following trace viewable through journalctl with a error: [Errno 13] Permission denied towards the bottom of the trace. Does anybody know why this happens?

`Unit collectd.service has begun starting up.
Jun 09 16:38:42 esxicollector.technolog.com collectd[2675]: plugin_load: plugin "syslog" successfully loaded.
Jun 09 16:38:42 esxicollector.technolog.com collectd[2675]: plugin_load: plugin "network" successfully loaded.
Jun 09 16:38:42 esxicollector.technolog.com collectd[2675]: plugin_load: plugin "python" successfully loaded.
Jun 09 16:38:42 esxicollector.technolog.com collectd[2675]: collectsphere plugin: Unknown config key: logtraces.
Jun 09 16:38:42 esxicollector.technolog.com collectd[2675]: configure_callback: Loaded config: name=Jabbars_esxi, host=172.16.0.37, port=443, verbose=True, username=root, password=******, host_metrics=3, vm_metrics=2, inventory_refresh_interval=600
Jun 09 16:38:47 esxicollector.technolog.com collectd[2675]: Systemd detected, trying to signal readyness.
Jun 09 16:38:47 esxicollector.technolog.com collectd[2675]: set_thread_name("python interpreter"): name too long
Jun 09 16:38:47 esxicollector.technolog.com polkitd[583]: Unregistered Authentication Agent for unix-process:2658:689439 (system bus name :1.45, object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_GB.UTF-8) (disconnected from bus)
Jun 09 16:38:47 esxicollector.technolog.com systemd[1]: Started Collectd statistics daemon.
Subject: Unit collectd.service has finished start-up
Defined-By: systemd
Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Unit collectd.service has finished starting up.

The start-up result is done.
Jun 09 16:38:47 esxicollector.technolog.com collectd[2675]: python: Interactive interpreter exited, stopping collectd ...
Jun 09 16:38:47 esxicollector.technolog.com collectd[2675]: >>>
Jun 09 16:38:47 esxicollector.technolog.com collectd[2675]: Unhandled python exception in init callback: error: [Errno 13] Permission denied
Jun 09 16:38:47 esxicollector.technolog.com collectd[2675]: Traceback (most recent call last):
Jun 09 16:38:47 esxicollector.technolog.com collectd[2675]: File "build/bdist.linux-x86_64/egg/collectsphere.py", line 144, in init_callback
env = create_environment(config)
Jun 09 16:38:47 esxicollector.technolog.com collectd[2675]: File "build/bdist.linux-x86_64/egg/collectsphere.py", line 401, in create_environment
pwd=config.get("password"))
Jun 09 16:38:47 esxicollector.technolog.com collectd[2675]: File "/usr/lib/python2.7/site-packages/pyVim/connect.py", line 836, in SmartConnect
sslContext)
Jun 09 16:38:47 esxicollector.technolog.com collectd[2675]: File "/usr/lib/python2.7/site-packages/pyVim/connect.py", line 718, in __FindSupportedVersion
sslContext)
Jun 09 16:38:47 esxicollector.technolog.com collectd[2675]: File "/usr/lib/python2.7/site-packages/pyVim/connect.py", line 638, in __GetServiceVersionDescription
path + "/vimServiceVersions.xml", sslContext)
Jun 09 16:38:47 esxicollector.technolog.com collectd[2675]: File "/usr/lib/python2.7/site-packages/pyVim/connect.py", line 604, in __GetElementTree
conn.request("GET", path)
Jun 09 16:38:47 esxicollector.technolog.com collectd[2675]: File "/usr/lib64/python2.7/httplib.py", line 1017, in request
self._send_request(method, url, body, headers)
Jun 09 16:38:47 esxicollector.technolog.com collectd[2675]: File "/usr/lib64/python2.7/httplib.py", line 1051, in _send_request
self.endheaders(body)
Jun 09 16:38:47 esxicollector.technolog.com collectd[2675]: File "/usr/lib64/python2.7/httplib.py", line 1013, in endheaders
self._send_output(message_body)
Jun 09 16:38:47 esxicollector.technolog.com collectd[2675]: File "/usr/lib64/python2.7/httplib.py", line 864, in _send_output
self.send(msg)
Jun 09 16:38:47 esxicollector.technolog.com collectd[2675]: File "/usr/lib64/python2.7/httplib.py", line 826, in send
self.connect()
Jun 09 16:38:47 esxicollector.technolog.com collectd[2675]: File "/usr/lib64/python2.7/httplib.py", line 1227, in connect
HTTPConnection.connect(self)
Jun 09 16:38:47 esxicollector.technolog.com collectd[2675]: File "/usr/lib64/python2.7/httplib.py", line 807, in connect
self.timeout, self.source_address)
Jun 09 16:38:47 esxicollector.technolog.com collectd[2675]: File "/usr/lib64/python2.7/socket.py", line 571, in create_connection
raise err
Jun 09 16:38:47 esxicollector.technolog.com collectd[2675]: error: [Errno 13] Permission denied
Jun 09 16:38:47 esxicollector.technolog.com collectd[2675]: Initialization complete, entering read-loop.
Jun 09 16:38:47 esxicollector.technolog.com collectd[2675]: Exiting normally.
Jun 09 16:38:47 esxicollector.technolog.com collectd[2675]: collectd: Stopping 5 read threads.
Jun 09 16:38:47 esxicollector.technolog.com collectd[2675]: collectd: Stopping 5 write threads.
Jun 09 16:38:47 esxicollector.technolog.com collectd[2675]: Exception KeyError: KeyError(139864518674240,) in <module 'threading' from '/usr/lib64/python2.7/threading.pyc'> ignored`

No Module Named pyVM.connect

Good afternoon,

I'm attempting to set up collectsphere to collect against my vSphere 6 Update2 Infrastructure (using the VCSA Appliance).. All my hosts are also esxi 6 update2

My collector VM is ubuntu server 16.. I ran through the wiki documentation on here..

When I attempt to run the run-test.py script i get the following error.

root@feeder204:/etc/collectsphere# python run-test.py
Traceback (most recent call last):
File "run-test.py", line 12, in
import collectsphere
File "/etc/collectsphere/collectsphere.py", line 23, in
from pyVim.connect import SmartConnect, Disconnect
ImportError: No module named pyVim.connect
root@feeder204:/etc/collectsphere#

When i saw this error I did a pip install pyVim which installed additional packages but it still comes back with the same ImportError.. Thoughts?

UnboundLocalError: local variable 'cluster_name' referenced before assignment

I'm following the installation guide and when I run the test, it reports the following:

Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
self.run()
File "/root/collectsphere/collectsphere.py", line 669, in run
len(inventory[cluster_name]['hosts'].keys()),
UnboundLocalError: local variable 'cluster_name' referenced before assignment

I'm running this on Ubuntu 14.04 with python 2.7.6.

Unhandled python exception in read callback: TypeError

Hi,

we have strange Error messages from collectd:

Unhandled python exception in read callback: TypeError: Dataset VirtualMachine.virtualDisk.average.totalWriteLatency.millisecond not found.

adding the entry in vmware-types.db worked fine for some missing objekts but this on will be ignored.

grep VirtualMachine.virtualDisk.average.totalWriteLatency.millisecond /etc/collectd/vmware-types.db
VirtualMachine.virtualDisk.average.totalWriteLatency.millisecond value:ABSOLUTE:0:U

maybe pathlength too long ?

best regards
robert

Memory Usage during Iventory Discovery

I just ran collectsphere agains an environment with 14 ESXi hosts and 1990 VMs: I constantly run out of memory during the inventory discovery even with 8GB of RAM for the VM. I dont know what exactly uses so much space.

Datacenter Objects: Include them or not?

At the moment collectsphere creates trees like this:

VC NAME -> CLUSTER -> HOST -> ...

Question is: Should we replace the VC_NAME with the name of the Datacenter Object in vSphere? This would abstract multiple environments from the vCenter that manages them.

What do you think?

VM counter filtering..

Hi this plugin works beautifully, but I am finding that 'VM_Counters "cpu.usage,mem.usage" doesn't collect enough counters for our needs, but VM_Counters "all" is a bit too much information.. how do we fine tune the counter filters?

For example I am interested in cpu, virtualDisk, mem and net.

Thanks

got error while install

when i run python setup.py install i got this :

File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 870, in obtain
return installer(requirement)
File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 292, in fetch_build_egg
dist = self.class({'script_args':['easy_install']})
File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 243, in init
_Distribution.init(self,attrs)
File "/usr/lib/python2.7/distutils/dist.py", line 287, in init
self.finalize_options()
File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 276, in finalize_options
ep.require(installer=self.fetch_build_egg)
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2100, in require
working_set.resolve(self.dist.requires(self.extras),env,installer)))
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 620, in resolve
dist = best[req.key] = env.best_match(req, ws, installer)
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 858, in best_match
return self.obtain(req, installer) # try and download/install
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 870, in obtain
return installer(requirement)
File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 292, in fetch_build_egg
dist = self.class({'script_args':['easy_install']})
File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 243, in init
_Distribution.init(self,attrs)
File "/usr/lib/python2.7/distutils/dist.py", line 287, in init
self.finalize_options()
File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 276, in finalize_options
ep.require(installer=self.fetch_build_egg)
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2100, in require
working_set.resolve(self.dist.requires(self.extras),env,installer)))
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 620, in resolve
dist = best[req.key] = env.best_match(req, ws, installer)
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 858, in best_match
return self.obtain(req, installer) # try and download/install
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 870, in obtain
return installer(requirement)
File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 292, in fetch_build_egg
dist = self.class({'script_args':['easy_install']})
File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 243, in init
_Distribution.init(self,attrs)
File "/usr/lib/python2.7/distutils/dist.py", line 287, in init
self.finalize_options()
File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 276, in finalize_options
ep.require(installer=self.fetch_build_egg)
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2100, in require
working_set.resolve(self.dist.requires(self.extras),env,installer)))
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 613, in resolve
env = Environment(self.entries)
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 782, in init
self.scan(search_path)
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 811, in scan
for dist in find_distributions(item):
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 1785, in find_eggs_in_zip
metadata = EggMetadata(importer)
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 1750, in init
self.zipinfo = build_zipmanifest(importer.archive)
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 1517, in build_zipmanifest
zfile = zipfile.ZipFile(path)
File "/usr/lib/python2.7/zipfile.py", line 770, in init
self._RealGetContents()
File "/usr/lib/python2.7/zipfile.py", line 844, in _RealGetContents
x = ZipInfo(filename)

Datacenter contains "ComputeResource" instead of "ClusterComputeResource"

In read_callback() the code looks for objects with _wsdlName of "ClusterComputeResource". However in my case it contains resources with the name "ComputeResource" instead.

My setup consists of three ESXi 6.5 nodes connected to a VCSA 6.5.

This is what I had to change to be able to discover hosts / VMs:

for datacenter in serviceInstance.RetrieveServiceContent().rootFolder.childEntity:
            collectd.info("DC name: " + datacenter._wsdlName)
            if datacenter._wsdlName == "Datacenter":
                for cluster in datacenter.hostFolder.childEntity:
                    collectd.info("Cluster name: " + cluster._wsdlName)
-                   if cluster._wsdlName == "ClusterComputeResource":
+                   if cluster._wsdlName == "ComputeResource":

'Friendly Names' for Clusters, Hosts and VMs

Hi, great project, exactly what I was looking for, however when I run against my test VCenter, I get some internal name for the cluster, host and VM eg:

gauge-domain-c151.HostSystem.host-51.cpu.0.average.usage.perc
gauge-domain-c151.VirtualMachine.vm-216.cpu.all.average.usage.perc

Any way of getting the names that show up in the web client as tags?

Execution Time

Read_callback still takes too long to execute: In this environment with 16 hosts and 451 VMs, it takes 80 to 95 seconds. Does anybody see any more potential to speed it up?

Wiki

The following wiki articles should be created:

  1. Installation of Collectd with collectsphere
  2. Installation of Graphite and Grafana with a quick example of how to access the data from collectsphere
  3. Example dashboards for various use cases

No Filtering in Counters

Hi,

I was trying to get this to work with some counters like:

  • Bandwidth usage
  • Disk usage
  • CPU usage (pct)
  • Memory usage (pct)

I tried with a config, with only 1 setting. However all the counters where processed.
I am using the network config for pushing the data into a logstash pipeline.

I am using only these values now:
Host_Counters ""
VM_Counters "mem.usage,cpu.usage"

But got many many statistics to my logstash instance.

vmodl.fault.InvalidArgument

Been Looking at this for a bit and can't see an issue, built it on different boxes.

Was wondering if anyone had seen this?

This occurs on both vCenter 5.5 and vCenter 6.0 with full admin privs.

[2017-11-07 19:59:30] Unhandled python exception in init callback: vmodl.fault.InvalidArgument: (vmodl.fault.InvalidArgument) {
   dynamicType = <unset>,
   dynamicProperty = (vmodl.DynamicProperty) [],
   msg = 'A specified parameter was not correct: ',
   faultCause = <unset>,
   faultMessage = (vmodl.LocalizableMessage) [],
   invalidProperty = <unset>
}
[2017-11-07 19:59:30] Traceback (most recent call last):
[2017-11-07 19:59:30]   File "build/bdist.linux-x86_64/egg/collectsphere.py", line 144, in init_callback
    env = create_environment(config)
[2017-11-07 19:59:30]   File "build/bdist.linux-x86_64/egg/collectsphere.py", line 479, in create_environment
    performance_manager.UpdatePerfInterval(performance_interval)
[2017-11-07 19:59:30]   File "/usr/lib/python2.7/site-packages/pyVmomi/VmomiSupport.py", line 580, in <lambda>
    self.f(*(self.args + (obj,) + args), **kwargs)
[2017-11-07 19:59:30]   File "/usr/lib/python2.7/site-packages/pyVmomi/VmomiSupport.py", line 386, in _InvokeMethod
    return self._stub.InvokeMethod(self, info, args)
[2017-11-07 19:59:30]   File "/usr/lib/python2.7/site-packages/pyVmomi/SoapAdapter.py", line 1366, in InvokeMethod
    raise obj # pylint: disable-msg=E0702
[2017-11-07 19:59:30] vmodl.fault.InvalidArgument: (vmodl.fault.InvalidArgument) {
   dynamicType = <unset>,
   dynamicProperty = (vmodl.DynamicProperty) [],
   msg = 'A specified parameter was not correct: ',
   faultCause = <unset>,
   faultMessage = (vmodl.LocalizableMessage) [],
   invalidProperty = <unset>
}

Simple config:

TypesDB "/usr/share/collectd/vmware-types.db"


<LoadPlugin "python">
    Globals true
</LoadPlugin>

<Plugin "python">
    Import "collectsphere"
    <Module "collectsphere">
        Name "NAME"
        Host "VCENTER_FQDN"
        Port "443"
        Verbose true
        VerifyCertificate false
        UseFriendlyName false
        Username "USERNAME@DOMAIN"
        Password "PASSWORD"
        Host_Counters "cpu.usage,mem.usage,disk.usage"
        VM_Counters "cpu.usage,mem.usage"
    </Module>
</Plugin>

Any insight would be awesome! Excited to use this!

'vim.Folder' object has no attribute 'host'

I'm getting below error in collectd logs.

Unhandled python exception in init callback: AttributeError: 'vim.Folder' object has no attribute 'host'

Name: pyvmomi
Version: 6.0.0.2016.6

Bug - If Username has a "\" Character Login Attempt Fails

Hi,

I would like to thank you for the awesome tool to pull vSphere data.

Bug:
If the username has

\

character in it, for example:

"cool\awesome".

The tool would just accept

"coolawesome"

and miss

\

character, thus resulting into failed attempts.

Adding another

\

solves the issue though. In our case cool\\awesome

would work instead of "cool\awesome".

Let me know if you need more information.

Thanks.

Multiple hosts simultaneously

Is it possible to support multiple simultaneous collections?

E.g

[root@zabbix-a collectd.d]# cat collectsphere.conf
TypesDB "/etc/collectd.d/vmware-types.db"
 
<LoadPlugin "python">
    Globals true
</LoadPlugin>
 
<Plugin "python">
    Import "collectsphere"
 
    <Module "collectsphere">
        Port "443"
        Verbose true
        VerifyCertificate false
        UseFriendlyName true
        Host_Counters "cpu.usage,mem.usage,net.usage"
        VM_Counters "all"
    </Module>
 
    <Module "collectsphere">
        Port "443"
        Verbose true
        VerifyCertificate false
        UseFriendlyName true
        Host_Counters "cpu.usage,mem.usage,net.usage"
        VM_Counters "all"
    </Module>
 
</Plugin>

collectd logging tons of 'uc_update: Value too old'

There's still something strange going on, even when using the master branch code as of 2/10/2017. collectd logs the following sorts of things constantly. In fact, I have 305,000 log lines matching 'uc_update' since 4 days ago on our host that is running collectsphere under collectd:

...
Feb 10 12:24:14 gazoo collectd[82312]: uc_update: Value too old: name = gazoo.our.org/collectsphere/gauge-domain-c7.HostSystem.host-15.cpu.all.average.usage.perc; value time = 1486747360.000; last cache update = 1486747440.000;
Feb 10 12:24:14 gazoo collectd[82312]: uc_update: Value too old: name = gazoo.our.org/collectsphere/gauge-domain-c7.HostSystem.host-15.cpu.all.average.usage.perc; value time = 1486747380.000; last cache update = 1486747440.000;
Feb 10 12:24:14 gazoo collectd[82312]: uc_update: Value too old: name = gazoo.our.org/collectsphere/gauge-domain-c7.HostSystem.host-15.cpu.all.average.usage.perc; value time = 1486747400.000; last cache update = 1486747440.000;
Feb 10 12:24:14 gazoo collectd[82312]: uc_update: Value too old: name = gazoo.our.org/collectsphere/gauge-domain-c7.HostSystem.host-15.cpu.all.average.usage.perc; value time = 1486747420.000; last cache update = 1486747440.000;
Feb 10 12:24:14 gazoo collectd[82312]: uc_update: Value too old: name = gazoo.our.org/collectsphere/gauge-domain-c7.HostSystem.host-15.cpu.all.average.usage.perc; value time = 1486747440.000; last cache update = 1486747440.000;
...

This situation is mentioned in the collectd FAQ: https://collectd.org/faq.shtml

I have confirmed that we are not running 2 collectd processes on the same host and we are not altering the system time.

My configuration is:

<LoadPlugin "python">
    Globals true
</LoadPlugin>

<Plugin "python">
    Import "collectsphere"
    <Module "collectsphere">
        Name "R701-VMware"
        Host "vcenter.our.org"
        Port "443"
        Verbose "True"
        VerifyCertificate "True"
        Username "OURDOMAIN\\ad-user-name"
        Password "REDACTED"
        Host_Counters "cpu.usage,mem.usage,disk.usage"
    </Module>
</Plugin>

vsphere read-only user does not work

Hi,

when I configured a read-only user in vsphere for the monitoring, I'll get a error message from python script
collectd[10786]: Unhandled python exception in init callback: vim.fault.NoPermission: (vim.fault.NoPermission) {#12 dynamicType = ,#12 dynamicProperty = (vmodl.DynamicProperty) [],#12 msg = 'Permission to perform this operation was denied.',#12 faultCause = ,#12 faultMessage = (vmodl.LocalizableMessage) [],#12 object = 'vim.Folder:group-d1',#12 privilegeId = 'Performance.ModifyIntervals'#12}

the same user works fine with administrator rights. It would be fine to use an account with read-only permissions.

Another issue are the counters. Is the an overview available what types of counters are available and what they means ?
at some points there areonly read values but no write (eg. HostSystem-Datastore-average-read - but no write counter ...)

best regards
robert

Issue with collectsphere

Oct 18 11:16:27 prlesljmp001 collectd[8907]: configure_callback: Loaded config: name=######, host=######, port=443, verbose=True, username=######, password=******, host_metrics=3, vm_metrics=2, inventory_refresh_interval=600
Oct 18 11:16:28 prlesljmp001 collectd[8907]: create_environment: vCenter ###### does not contain any hosts. Cannot continue
Oct 18 11:16:28 prlesljmp001 collectd[8907]: Initialization complete, entering read-loop.
Oct 18 11:16:28 prlesljmp001 collectd[8907]: tcpconns plugin: Reading from netlink succeeded. Will use the netlink method from now on.
Oct 18 11:16:28 prlesljmp001 collectd[8907]: read_callback: entering environment: ######
Oct 18 11:16:28 prlesljmp001 collectd[8907]: Unhandled python exception in read callback: TypeError: 'NoneType' object has no attribute 'getitem'

conf:
TypesDB "/usr/share/collectd/vmware-types.db"

<LoadPlugin "python">
Globals true

<Plugin "python">
Import "collectsphere"
<Module "collectsphere">
Name "######"
Host "#######"
Port "443"
Verbose true
VerifyCertificate false
UseFriendlyName false
Username "####"
Password "####"
Host_Counters "cpu.usage,mem.usage,disk.usage"
VM_Counters "cpu.usage,mem.usage"

Many problems with collectd namespace use

There are huge issues with how this plugin uses the collectd namespace.

I've made some changes that result in it being:

For VirtualMachine statistics:
collectsphere,host=,instance=,type=VirtualMachine.cpu.summation.system,type_instance=1

For vCenter statistics:
collectsphere,host=,instance=,type=HostSystem.sys.average.resourceCpuUsage,type_instance=host_vim_vmvisor_net-daemons

I've also fixed the types.db file by replacing "value" with the "unit" name. Type names cannot be more than 63 characters.

The attached files are totally incompatible with the existing script but will make things much easier when it comes to grafana.

Summary of Datastores

Hi,

It is possible know the usage of datastores? Capacity, Free Space, Usage,...
I dont find the succesfully metric.

Thanks!!

Name Length Issue

I managed to test the plugin in a big environment and found that with the current form of truncation, there is a very high probablilty of collitions for VMFS UUIDs and NAA canonical names.

Suggestion: Use 12 characters instead of 6.

Metrics appear in graphite on disk, but can't graph

I'm stumped. In case anyone has ideas... We've been using Graphite, collectd, and Grafana for years. I can't get Grafana (or the native Graphite web UI even) to display any of the data coming in from collectd for this plugin (and only this plugin).

whisper-dump.py shows there are values

#  /opt/graphite/bin/whisper-dump.py gauge-domain-c7_HostSystem_host-10_cpu_all_average_usage_perc.wsp
...
1: 1486421640,       1048
2: 1486421700,       1061
3: 1486421760,       1037
4: 1486421820,       1096
5: 1486421880,       1256
6: 1486421940,       1047
7: 1486422000,       2211
8: 1486422060,       2024
9: 1486422120,       1527
10: 1486422180,       1289
11: 1486422240,       1091
12: 1486422300,       1068
13: 1486422360,       1165
...

No metrics visible in Grafana

Thanks for sharing this. I've been trying to implement collectsphere for our vsphere setup but so far I can't get the metrics to show up in grafana. I was trying to use your demo grafana, but it seems to be down. It would have been a great way to see how the queries need to be structured.

We're not using graphite but influxdb, so the path is slightly different:

collectsphere -> collectd -> network plugin -> influxdb -> grafana.

Using run-test.py as per wiki and running collectd in debug mode show that data is actually gathered by collectpshere/collectd. Another test with https://github.com/llambiel/collectd-vcenter did produce results in grafana but it seems older and less up-to-date/capable than collectsphere and I had to hack around in the code to get the ssl connection working (untrusted cert). I'd love to use collectsphere, but so far I'm at a loss and would appreciate any feedback or tips to get it working.

Timezone aware

collectsphere.py isn't really timezone aware. It tries to be with:
timestamp += time.localtime().tm_isdst * (3600)
but that's not working for me.

I used calendar.timegm() instead of time.mktime() and it works perfectly.

Basically it looks like this

import calendar
...
# Get the timestamp of value. Because of an issue by VMware the
# has to be add an hour if you're at DST
timestamp = float(time.mktime(time.localtime(calendar.timegm(metricsOfEntities[p].sampleInfo[i].timestamp.timetuple()))))
cd_value.time = timestamp

Well after that the comment makes no sense, but it's easier to find...

EDIT: not it actually works... :)

Missing Metrics

CollectSphere is configured to get metrics from machnines every 20 seconds, so he should get metrics 3 times every minute. But he usually get only 2 metrics every minute. The firewall is down, selinux the same, this problem occurs only with disks and interfaces metrics. What could be the problem?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.