GithubHelp home page GithubHelp logo

n27051538 / solaris_exporter Goto Github PK

View Code? Open in Web Editor NEW
19.0 6.0 8.0 880 KB

SPARC solaris exporter for Prometheus

License: MIT License

Python 88.09% Shell 11.91%
prometheus solaris-exporter solaris-zones sparc ansible-role solaris monitoring grafana dashboard exporter

solaris_exporter's Introduction

solaris_exporter

Description

SPARC solaris exporter for Prometheus.
Written by Alexander Golikov for collecting SPARC Solaris metrics for Prometheus.
Tested on Solaris 11.3.25, 11.4.4, 11.4.41, 10u11(limited) SPARC.
Also work on x86 platform, community-tested with Openindiana (x86) (OI-hipster-minimal-20201031.iso) and Solaris10u11.

versions

  • 2020 Jan 31. Initial
  • 2020 Feb 04. Added UpTime in UpTimeCollector.
  • 2020 Feb 09. Added DiskErrorCollector, ZpoolCollector, FmadmCollector, SVCSCollector, FCinfoCollector
  • 2020 Dec 17. Added PrtdiagCollector, MetaStatCollector, MetaDBCollector
  • 2021 Jan 05. Added TextFileCollector, SVCSCollector now enabled for all zones (Thanks to Marcel Peter)
  • 2021 Mar 01. Fixed psutil version to 5.7.0 (something changed in the newer versions, have to time to look at)
  • 2022 Jan 24. Added support for Python 3.
  • 2022 Feb 04. Documentation update for support of Solaris 11.4.41.
  • 2022 Feb 05. Fixed support of Python 2.7 for Solaris 11.4.41.
  • 2022 May 04. Added LdomsLsCollector due to discussion.

Provides info about:

  • Solaris Zones CPU Usage with processor sets info (PerZoneCpuCollector);
  • Solaris Zones Virtual Memory (SWAP) Resource Capping (PerZoneCapsCollector);
  • Common CPU stats (CpuTimeCollector);
  • Avg Load (CpuLoadCollector);
  • Disk IO (DiskIOCollector);
  • Disk Errors (DiskErrorCollector);
  • Disk Space (DiskSpaceCollector, requires 'file_dac_search' priv for solaris zones)
  • Memory Usage, swap-in, swap-out (MemCollector);
  • Network Interfaces (NetworkCollector);
  • Node time, uptime (CurTimeCollector, UpTimeCollector);
  • FC links Multipath (FCinfoCollector, /usr/sbin/mpathadm list lu)
  • System Services health via 'svcs -x' command (SVCSCollector);
  • Whole system health via 'fmadm faulty' (FmadmCollector), requires pfexec of '/usr/sbin/fmadm'.
  • Zpool devices health via 'zpool status' command (ZpoolCollector)
  • prtdiag -v return code(PrtdiagCollector)
  • Solaris Volume Manager disk status (MetaStatCollector, MetaDBCollector).
  • Get info from text files *.prom in folder provided by text_file_path var (TextFileCollector).
  • LDOM info via 'ldm list' (LdomsLsCollector), requires auth 'solaris.ldoms.read'.

Grafana dashboard.

Dashboard config is located in file grafana-dashboard-solaris.json
Pic1 Pic2

Installation.

To use this exporter you need Python2.7 or Python3.x and its modules prometheus_client, psutil.

Solaris 10u11:

# Setup proxy vars to have access to internet  
    export http_proxy=http://proxy.example.com:3128  
    export https_proxy=http://proxy.example.com:3128  
# Install pkgutil  
    wget http://get.opencsw.org/now   
    pkgadd -d ./now   
# Update repo list  
    /opt/csw/bin/pkgutil -U  
# Install Python 2.7 or Python 3.3 (it works on both)  
    # Python 2.7 (preferred)  
        /opt/csw/bin/pkgutil -y -i python27  
        /opt/csw/bin/pkgutil -y -i python27_dev  
        /opt/csw/bin/pkgutil -y -i py_pip  
        /usr/sbin/pkgchk -L CSWpy-pip               # list installed files if you need  
    # or Python 3.3  
        /opt/csw/bin/pkgutil -y -i python33  
        /opt/csw/bin/pkgutil -y -i python33_dev  
        # pip3 is not included in pkgutil, we need to install it by hands  
        # download pip3.3 installer [https://bootstrap.pypa.io/pip/3.3/get-pip.py] and run it with python3.3  
        /opt/csw/bin/python3.3 get-pip.py  
# Install gcc5core  
    /opt/csw/bin/pkgutil -y -i gcc5core  
# Install Python module prometheus_client  
    # Python 2.7  
        /opt/csw/bin/pip2.7 install prometheus_client==0.7.1
        # Note than current versions of prometheus_client (0.14.1 +) for some reasons not works with Python 2.7, 
        # that is why I fixed its version to 0.7.1, it is tested.                      
    # or Python 3.3  
        /opt/csw/bin/pip3.3 install prometheus_client  
# Install Python module psutil, it have to compile some libs, but we preinstalled all that needed  
    ln -s /opt/csw/bin/gcc-5.5 /opt/csw/bin/gcc-5.2  
    # Python 2.7  
        # note that the latest version of psutil not supports Python2.7,  
        # that is why version of psutil is fixed to '5.7.0'  
            /opt/csw/bin/pip2.7 install psutil==5.7.0  
    # or Python 3.3  
        /opt/csw/bin/pip3.3 install psutil  
# Run exporter, check http://ip:9100  
    # Python 2.7  
    export LANG=C  
    /opt/csw/bin/python2.7 solaris_exporter.py  
    # or Python 3.3  
    export LANG=C  
    /opt/csw/bin/python3.3 solaris_exporter.py  

Solaris 11.4.4 (this way works with Python 2.7):

# Setup proxy vars to have access to internet  
    export http_proxy=http://proxy.example.com:3128  
    export https_proxy=http://proxy.example.com:3128  
# Install Python 2.7 module prometheus_client  
    pip-2.7 install prometheus_client==0.7.1
    # Note than current versions of prometheus_client (0.14.1 +) for some reasons not works with Python 2.7, 
    # that is why I fixed its version to 0.7.1, it is tested.  
# Install Python 2.7 module psutil, it have to compile some libs  
# Also you could get psutil for Python 2.7 via 'pkg install library/python/psutil-27',  
# but it returns wrong Network statistics, tested from Solaris 11.4.4 repo.  
# The latest version of psutil not supports Python2.7, that is why version of psutil is fixed on '5.7.0'  
    pkg install pkg:/developer/gcc/gcc-c-5  
    ln -s /usr/bin/gcc /usr/bin/cc  
    export CFLAGS=-m32  
    pip-2.7 install psutil==5.7.0  
    # if you have troubles with compilation, try to switch to gcc-c-9 and Python 3.7   
# Run exporter, check http://ip:9100  
    export LANG=C  
    python2.7 solaris_exporter.py  

Solaris 11.4.41 (this way works with Python 3.7):

# Setup proxy vars to have access to internet  
        export http_proxy=http://proxy.example.com:3128  
        export https_proxy=http://proxy.example.com:3128  
# Install Python 3.7 module prometheus_client  
        pip-3.7 install prometheus_client  
        # tested with version 0.13.1, to install this version you may run
        # pip-3.7 install prometheus_client==0.13.1
# Install Python 3.7 module psutil  
# Also you could get psutil for Python 3.7 via 'pkg install library/python/psutil-37',  
# but its old version '5.6.7' not adapted for Sol11.4.41 changes, fails at 'swap -l' output, have network dev inaccuracy. 
# The best way is to install actual version of psutil (tested on '5.9.0')   
    pkg install pkg:/developer/gcc/gcc-c-9  
    ln -s /usr/bin/gcc /usr/bin/cc  
    pip-3.7 install psutil==5.9.0  
# Run exporter, check http://ip:9100  
    export LANG=C  
    python3.7 solaris_exporter.py  

SMF, Roles, Deployment.

  • Create user and group 'monitor'
  • Run './solaris_exporter_smf.sh' to create 'prometheus/solaris_exporter' service.
  • The best way to do all installation tasks on all nodes is Ansible. See tasks file install_solaris_exporter.yml as an example for creating role. You have to create Role and add all declared variables to it.

Prometheus configuration

rule_files:
   - 'alerts.solaris.yml'
scrape_configs:
  - job_name: 'solaris_exporter'
    scrape_interval: 60s
    scrape_timeout:  50s
    metrics_path: /
    scheme: http
    file_sd_configs:
      - files:
        - 'inventory/static/hosts.yml'
        - 'inventory/static/hosts/*.yml'
        - 'inventory/dynamic/solaris/*.yml'
    relabel_configs:
      - source_labels: [os]                      #force this job if os label is 'solaris'
        regex: '^solaris.*'
        target_label: jobs
        replacement: solaris_exporter
      - source_labels: [jobs]                    #pick up config if it has 'solaris_exporter' in comma-separated list in label 'jobs', drop if not
        regex: '(.*,|^)solaris_exporter(,.*|$)'
        action: keep
      - regex: '^jobs$'                          #drop unused label 'jobs'
        action: labeldrop
      - source_labels: [__address__]             #save ip address into 'ip' label
        regex: '(.*)(:.*)?'
        replacement: '$1'
        target_label: ip
      - source_labels: [__address__]             #add port if it is absent in target, save ip:port to '__param_target' label
        regex: (.*)(:.*)?
        replacement: ${1}:9100
        target_label: __param_target
      - source_labels: [__param_target]          #copy '__param_target' label to 'instance' label
        target_label: instance
      - source_labels: [__param_target]          #copy '__param_target' label to '__address__' label
        target_label: __address__

solaris_exporter's People

Contributors

n27051538 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

solaris_exporter's Issues

unable to setup solaris_exporter

followed all the instructions but when I tryied to launch solaris_exporter by
python2.7 solaris_exporter.py

Traceback (most recent call last):
File "solaris_exporter.py", line 133, in
from prometheus_client.core import REGISTRY, Counter, Gauge, GaugeMetricFamily, CounterMetricFamily, UntypedMetricFamily
File "/usr/lib/python2.7/site-packages/prometheus_client/init.py", line 3, in
from . import (
File "/usr/lib/python2.7/site-packages/prometheus_client/exposition.py", line 113
def make_wsgi_app(registry: CollectorRegistry = REGISTRY, disable_compression: bool = False) -> Callable:
^
SyntaxError: invalid syntax

root@solaris-test:/opt/solaris_exporter# uname -a
SunOS solaris-test 5.11 11.4.0.15.0 i86pc i386 i86pc

Can some one help me to get this running.

thanks

Unfortunately does not work in Solaris 11.4.35

I had such high hopes..
Unfortunately needs a touch up to get up to date I expect.. Certainly python 2.7 is an issue at least...

DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.

I will have a dig around, but unlikely to get it sorted myself unfortunately

Issue running solaris_exporter

Hi,

I have run into a similar issue as seen previously by others with traceback errors when running solaris_exporter.py.

The errors seen are: -

admin@XXXXXX:/solaris_exporter$ python2 solaris_exporter.py
couldn't set locale correctly
Traceback (most recent call last):
File "solaris_exporter.py", line 1040, in
REGISTRY.register(c)
File "/export/home/admin/.local/lib/python2.7/site-packages/prometheus_client/registry.py", line 26, in register
names = self._get_names(collector)
File "/export/home/admin/.local/lib/python2.7/site-packages/prometheus_client/registry.py", line 66, in _get_names
for metric in desc_func():
File "solaris_exporter.py", line 441, in collect
swap = psutil.swap_memory()
File "/export/home/admin/.local/lib/python2.7/site-packages/psutil/init.py", line 2000, in swap_memory
return _psplatform.swap_memory()
File "/export/home/admin/.local/lib/python2.7/site-packages/psutil/_pssunos.py", line 160, in swap_memory
free += int(int(f) * 512)
ValueError: invalid literal for int() with base 10: 'yes'
admin@XXXXXX:
/solaris_exporter$

In terms of the system I am trying to run this on, the following shows the environment and the installation process for the various components installed: -

admin@XXXXXX:$ cat /etc/release
Oracle Solaris 11.4 SPARC
Copyright (c) 1983, 2021, Oracle and/or its affiliates.
Assembled 15 December 2021
admin@XXXXXX:
$ uname -a
SunOS XXXXXX 5.11 11.4.41.107.2 sun4v sparc sun4v non-virtualized
admin@XXXXXX:~$

admin@XXXXXX:$ export https_proxy=http://www-proxy.us.oracle.com:80
admin@XXXXXX:
$ python2 -m pip install prometheus_client
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Defaulting to user installation because normal site-packages is not writeable
couldn't set locale correctly
Collecting prometheus_client
Downloading prometheus_client-0.12.0-py2.py3-none-any.whl (57 kB)
|################################| 57 kB 993 kB/s
Installing collected packages: prometheus-client
Successfully installed prometheus-client-0.12.0
admin@XXXXXX:~$ sudo pkg install pkg:/developer/gcc
Password:
Failed to set locale: unsupported locale setting. Falling back to C.
pkg: Unable to set locale 'en_GB.UTF-8'; locale package may be broken or
not installed. Reverting to C locale.
Packages to install: 19
Mediators to change: 2
Services to change: 1
Create boot environment: No
Create backup boot environment: No

DOWNLOAD PKGS FILES XFER (MB) SPEED
Completed 19/19 1974/1974 677.7/677.7 35.4M/s

PHASE ITEMS
Installing new actions 2553/2553
Updating package state database Done
Updating package cache 0/0
Updating image state Done
Creating fast lookup database Done
Updating package cache 1/1
admin@XXXXXX:$ which gcc
/usr/bin/gcc
admin@XXXXXX:
$ sudo ln -s /usr/bin/gcc /usr/bin/cc
Password:
admin@XXXXXX:$ export CFLAGS=-m32
admin@XXXXXX:
$ python2 -m pip install psutil==5.7.0
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Defaulting to user installation because normal site-packages is not writeable
couldn't set locale correctly
Collecting psutil==5.7.0
Downloading psutil-5.7.0.tar.gz (449 kB)
|################################| 449 kB 6.4 MB/s
Using legacy 'setup.py install' for psutil, since package 'wheel' is not installed.
Installing collected packages: psutil
Running setup.py install for psutil ... done
Successfully installed psutil-5.7.0
admin@XXXXXX:~$

Can you help and let me know what I've done wrong ?

Steve Dennis

openindiana support

Greetings, just letting you know that I have tested this on Openindiana (x86) (OI-hipster-minimal-20201031.iso) and it seems to work. I haven't gotten any errors or stack-traces anyways.

pkgadd with proxy authentification...

pkgadd not support user and password in proxy environement variable http_proxy and https_proxy.

you must download pkgutil package with wget/curl before.

# export http_proxy=http://<user>:<password>@proxy.example.com:3128  
# export https_proxy=https://<user>:<password>@proxy.example.com:3128  
# wget http://get.opencsw.org/now 
# pkgadd -d ./now

The last pkgutil work fine with this proxy variable syntax, you can do csw pkg install with it.

can't solaris_exporter on solars11.3.36

root@xxxxxxxxxx # /opt/csw/bin/pip install psutil==5.7.0
/opt/csw/bin/gcc-5.2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -m32 -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_SIZEOF_PID_T=4 -DPSUTIL_VERSION=570 -DPSUTIL_SUNOS=1 -DNEW_MIB_COMPLIANT=1 -I/opt/csw/include/python2.7 -c psutil/_psutil_common.c -o build/temp.solaris-2.11-sun4v.32bit-2.7/psutil/_psutil_common.o
unable to execute '/opt/csw/bin/gcc-5.2': No such file or directory
error: command '/opt/csw/bin/gcc-5.2' failed with exit status 1

----------------------------------------

Command "/opt/csw/bin/python2.7 -u -c "import setuptools, tokenize;file='/tmp/pip-build-TxNAvo/psutil/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-KW04Mu-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-TxNAvo/psutil/
You are using pip version 8.1.2, however version 22.0.4 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
root@xxxxxxxx # ls -ld /opt/csw/bin/gcc-5.2
/opt/csw/bin/gcc-5.2: No such file or directory
root@xxxxxxxx # gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/gcc/5/lib/gcc/sparcv9-sun-solaris2.11/5.4.0/lto-wrapper
Target: sparcv9-sun-solaris2.11
Configured with: /builds/ul11u3sru-gate/components/gcc5/gcc-5.4.0/configure CC='/usr/gcc/4.7/bin/gcc -m64' CXX='/usr/gcc/4.7/bin/g++ -m64' --prefix=/usr/gcc/5 --mandir=/usr/gcc/5/share/man --bindir=/usr/gcc/5/bin --libdir=/usr/gcc/5/lib --sbindir=/usr/gcc/5/sbin --infodir=/usr/gcc/5/share/info --libexecdir=/usr/gcc/5/lib --enable-languages=c,c++,fortran,objc --enable-shared --enable-initfini-array --disable-rpath --with-system-zlib --with-build-config=no --with-gmp-include=/usr/include/gmp --with-mpfr-include=/usr/include/mpfr --without-gnu-ld --with-ld=/usr/bin/ld --with-gnu-as --with-as=/usr/gnu/bin/as 'BOOT_CFLAGS=-g -O2' sparcv9-sun-solaris2.11
Thread model: posix
gcc version 5.4.0 (GCC)
root@xxxx #

root@xxxxxxx # pkg info entire
Name: entire
Summary: entire incorporation including Support Repository Update (Oracle Solaris 11.3.36.24.0).
Description: This package constrains system package versions to the same
build. WARNING: Proper system update and correct package
selection depend on the presence of this incorporation.
Removing this package will result in an unsupported system.
For more information see:
https://support.oracle.com/rs?type=doc&id=2433413.1
Category: Meta Packages/Incorporations
State: Installed
Publisher: solaris
Version: 0.5.11 (Oracle Solaris 11.3.36.24.0)
Build Release: 5.11
Branch: 0.175.3.36.0.24.0
Packaging Date: Wed Jan 13 20:40:04 2021
Size: 5.46 kB
FMRI: pkg://solaris/[email protected],5.11-0.175.3.36.0.24.0:20210113T204004Z
root@xxxxxxxxxx #

Issue with running solaris exporter

I've installed everything as described in readme. No issues found during packages installation. However when I try to run exporter, I get following error:

bash-3.2# /opt/csw/bin/python2.7 solaris_exporter.py  
Traceback (most recent call last):
  File "solaris_exporter.py", line 1039, in <module>
    REGISTRY.register(c)
  File "/opt/csw/lib/python2.7/site-packages/prometheus_client/registry.py", line 26, in register
    names = self._get_names(collector)
  File "/opt/csw/lib/python2.7/site-packages/prometheus_client/registry.py", line 66, in _get_names
    for metric in desc_func():
  File "solaris_exporter.py", line 440, in collect
    swap = psutil.swap_memory()
  File "/opt/csw/lib/python2.7/site-packages/psutil/__init__.py", line 1982, in swap_memory
    return _psplatform.swap_memory()
  File "/opt/csw/lib/python2.7/site-packages/psutil/_pssunos.py", line 158, in swap_memory
    t, f = line[3:4]
ValueError: need more than 1 value to unpack


bash-3.2# cat /etc/release 
                    Oracle Solaris 10 1/13 s10x_u11wos_24a X86
  Copyright (c) 1983, 2013, Oracle and/or its affiliates. All rights reserved.
                            Assembled 17 January 2013

bash-3.2# uname -a
SunOS myhost.com 5.10 Generic_147148-26 i86pc i386 i86pc

Ask about your GCC installation

When executing pkg install pkg:/developer/gcc/gcc-c-5

Creating Plan (Solver setup): - pkg install: No matching version of developer/gcc/gcc-c-5 can be installed: Reject: pkg://solaris/developer/gcc/[email protected] Reason: No version for 'require' dependency on library/[email protected],5.11-0.175.3.0.0.30.0 can be found

Metrics solaris_exporter_per_zone_usage_total can't collect

Hi
Look like solaris_exporter_per_zone_usage_total didn't collect only I see some metrics for zones as below.

solaris_exporter_per_zone_caps_errors_created
solaris_exporter_per_zone_caps_errors_total
solaris_exporter_per_zone_caps_processing
solaris_exporter_per_zone_caps_timeouts_created
solaris_exporter_per_zone_caps_timeouts_total
solaris_exporter_per_zone_caps_tota
lsolaris_exporter_per_zone_cpu_errors_created
solaris_exporter_per_zone_cpu_errors_total
solaris_exporter_per_zone_cpu_processing
solaris_exporter_per_zone_cpu_timeouts_created
solaris_exporter_per_zone_cpu_timeouts_total


# TYPE solaris_exporter_cpu_time_processing gauge
solaris_exporter_cpu_time_processing 0.01242818683385849
# HELP solaris_exporter_per_zone_usage_total kstat counters
# TYPE solaris_exporter_per_zone_usage_total counter
# HELP solaris_exporter_MemCollector_processing Time spent processing request
# TYPE solaris_exporter_MemCollector_processing gauge
solaris_exporter_MemCollector_processing 0.028274312615394592
-
How to collect other metrics for zone?

OS: Oracle Solaris 11.3.36.24.0
HW: Sparc T7

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.