mykolq / zabbix_lsi_template

Template is for discovering and monitoring LSI (Avago, Broadcom) storage controllers

zabbix_lsi_template

Description

This template is for discovering and monitoring LSI based (Avago, Broadcom, Perc, Lenovo) storage controllers by using json outputs of storcli64 and perccli64 tool. Now it works only with zabbix 5.2 and higher

Main features

• Discovery of controllers, logical discs, physical discs, batteries (bbu and cv) without scripts on agent side (it uses parsing of json and javascript preprocessing on zabbix server side)
• Monitoring controllers, logical, physical discs, batteries
• Useful with OS, where storcli64 and perccli64 works
• Overrides for reducing non supported items
• Comfortable changing of time intervals by macroses.

Plans

• In some cases we can get all data (without detailed information about batteries) in one command (/call show all j)

Installation

Zabbix server

• Import template
• Create and configure global or template macroses:
  • {$ADAP_DISCOVERY_PERIOD} - adapters discovery period. I think you can set it nearly 1d (daily)
  • {$ADAP_HISTORY_PERIOD} - period of saving history for adapters data. For example 30d
  • {$ADAP_REQUEST_PERIOD} - period of requesting storage adapters data ( adapter,battery state, etc). 1h
  • {$LD_DISCOVERY_PERIOD} - logical discs discovery period. 6h
  • {$LD_HISTORY_PERIOD} - period of saving history for logical discs data. 30d
  • {$LD_REQUEST_PERIOD} - period of requesting logical discs data. 5m
  • {$PD_DISCOVERY_PERIOD} - physical discs discovery period. 30m
  • {$PD_HISTORY_PERIOD} - period of saving history for physical discs data. 30d
  • {$PD_REQUEST_PERIOD} - period of requesting physical discs data. 5m
  • {$ADAP_THROTTLING_HB_PERIOD} - period of heartbit for throttling for adapter data
  • {$LD_THROTTLING_HB_PERIOD} - period of heartbit for throttling for logical discs data
  • {$PD_THROTTLING_HB_PERIOD} - period of heartbit for throttling for physical discs data.
• Connect template to agent, configure {$LSI_CLI} = 'perccl64' host macro if this host has Perc controller. All other controllers use storcli64 and it is default option. {$LSI_PD_COMMAND} has default option '/call/eall/sall' and can be counted by template *For automating changes of {$LSI_PD_COMMAND} you have to do instructions below:
  • Create user for API. This user must have write access for changing settings of servers, monitoring by this template.
  • Install pyzabbix module for you default python version (Something like pip install pyzabbix)
  • Copy updatehostmacro.py script to your zabbix external script folder
  • Than you have to create macroses (i use global) for login and password of this user. Create macros for zabbix server URL. For example: {$ZBX_API_WRITER_USER}, {$ZBX_API_WRITER_PASSWORD}, {$ZBX_URL}. Zabbix url macros must equal something like that: http://zabbix_server_address/zabbix
  • Create script 'Update hostmacro' (Administration-Scripts) with command (check your file path) /usr/bin/python3 /zabbixexternalscripts/updatehostmacro.py "--zbxurl" "{$ZBX_URL}" "--zbxuser" "{$ZBX_API_WRITER_USER}" "--zbxpass" '{$ZBX_API_WRITER_PASSWORD}' "--host" "{HOST.HOST}" "--macroname" "{EVENT.TAGS.MACRONAME}" "--macrovalue" '{EVENT.TAGS.MACROVALUE}' "--valuetype" "{EVENT.TAGS.VALUETYPE}"
  • Create action with name like "Change macro value on host" and condition "Value of tag ACTION equals Change macro value on host"
  • Add step for execution 'Update hostmacro' script

Windows

• Copy storcli64, perccli64 utility (you can use version in diskutils_windows.zip) in place where you store things like this
• Copy lsi_raid_win.conf in zabbix_agent configs folder
• Edit storcli paths in lsi_raid_win.conf.

Linux (tested with Centos 7 with disabled SELinux, Ubuntu 20.04)

• Copy storcli64, perccli64 utility (you can use version in diskutils_windows.zip) in place where you store things like this
• Copy lsi_raid_linux.conf in zabbix_agent configs folder (by default /etc/zabbix/zabbix_agentd.d/)
• Check and edit storcli paths in lsi_raid_linux.conf
• Copy sudoers_zabbix file to /etc/sudoers.d. Check path for storcli64, perccli64.

Later today or tomorrow i will add default macro values in template

zabbix_lsi_template's Issues

It is not working for Zabbix 5.0

Unfortunately, it doesn't work for Zabbix 5.0
During importing template I'm getting the errors like below:

Cannot read XML: (41) Specification mandate value for attribute crossorigin [Line: 15 | Column: 28].

Can you use both perccli64 and storcli64?

I haven't set this up yet, but am looking into it. I just installed Zabbix 7.0 and was going to see if it will work. We need to monitor servers using either perccli64 or storcli64 because they might be either one. Can you only choose one of these for the entire environment or can you select which one to use for each server that you monitor?

BBU failure false positive

I have some servers that I am using with this template, and noticed that a couple of servers triggers were showing the BBU/Cachevault module needed to be replaced with a value of 'OK'.

After digging further, it looks like the newer version of storcli adds a newer field to the response.

Here is a working result:

Storage Command Line Tool  Ver 1.21.06 Nov 18, 2016
{
"Controllers":[
{
        "Command Status" : {
                "Controller" : 0,
                "Status" : "Success",
                "Description" : "None"
        },
        "Response Data" : {
                "Cachevault_Info" : [
                        {
                                "Property" : "Type",
                                "Value" : "CVPM02"
                        },
                        {
                                "Property" : "Temperature",
                                "Value" : "48 C"
                        },
                        {
                                "Property" : "State",
                                "Value" : "Optimal"
                        }
                ],
                "Firmware_Status" : [
                        {
                                "Property" : "Replacement required",
                                "Value" : "No"
                        },
                        {
                                "Property" : "No space to cache offload",
                                "Value" : "No"
                        },
                        {
                                "Property" : "Module microcode update required",
                                "Value" : "No"
                        }
                ],

And here is the result that is causing the false positive.

{
"Controllers":[
{
        "Command Status" : {
                "CLI Version" : "007.0709.0000.0000 Aug 14, 2018",
                "Operating system" : "Linux 4.15.0-54-generic",
                "Controller" : 0,
                "Status" : "Success",
                "Description" : "None"
        },
        "Response Data" : {
                "Cachevault_Info" : [
                        {
                                "Property" : "Type",
                                "Value" : "CVPM02"
                        },
                        {
                                "Property" : "Temperature",
                                "Value" : "42 C"
                        },
                        {
                                "Property" : "State",
                                "Value" : "Optimal"
                        }
                ],
                "Firmware_Status" : [
                        {
                                "Property" : "NVCache State",
                                "Value" : "OK"
                        },
                        {
                                "Property" : "Replacement required",
                                "Value" : "No"
                        },
                        {
                                "Property" : "No space to cache offload",
                                "Value" : "No"
                        },
                        {
                                "Property" : "Module microcode update required",
                                "Value" : "No"
                        }
                ],

Not work BBU recovery triggers

I have BBU and not have cachevault.
And not work recovery trigger LSI controller {#CTLID} battery state is {ITEM.LASTVALUE} -

({Template LSI JSON discovery:lsi.ctl.bbustate.["{#CTLID}"].str(Optimal)}=0 and {Template LSI JSON discovery:lsi.ctl.bbustate.["{#CTLID}"].strlen()}>0) or ({Template LSI JSON discovery:lsi.ctl.cvstate.["{#CTLID}"].str(Optimal)}=0 and {Template LSI JSON discovery:lsi.ctl.cvstate.["{#CTLID}"].strlen()}>0)

because key lsi.ctl.cvdata.["0"] has status Not supported with error "Status" : "Failure"
and trigger expression lsi.ctl.cvstate.["{#CTLID}"].strlen()}>0 causes an error
Cannot evaluate expression: "Cannot evaluate function "SERVER:lsi.ctl.cvstate.["0"].strlen()": not enough data.".

Same errors with triggers:
LSI controller {#CTLID} battery was replaced
LSI controller {#CTLID} battery is need to be replaced

Zabbix server 4.2.3

Case-sensetive in postprocessing JavaScript in lsi.pd.state.["{#PDID}"] key

Trigger LSI physical disk "{#PDID}" is "{ITEM.LASTVALUE}" not work because case-sensetive in postprocessing JavaScript switch(value) in lsi.pd.state.["{#PDID}"] key:
case 'UGood':
case 'UBad':

You can also add:
case 'GHS':
value='Global Hot Spare';

case 'DHS':
value='Dedicated Hot Spare';

case 'Rbld':
value='Rebuild';

Hi. I`m getting JSON with trashline. How can I remove this with preprocessing? (I tried Regular expression with ^Error.*& as a pattern and \1 as output. I`t dosn`t work.

Error in IOCTL_SCSI_GET_INQUIRY_DATA
{
"Controllers":[
{
"Command Status" : {
"Status Code" : 0,
"Status" : "Success",
"Description" : "None"
},
"Response Data" : {
"Number of Controllers" : 1,
"Host Name" : "NG-CH3",
"Operating System " : "Windows Server 2008 R2",
"System Overview" : [
{
"Ctl" : 0,
"Model" : "LSIMegaRAIDSAS9260-8i",
"Ports" : 8,
"PDs" : 8,
"DGs" : 2,
"DNOpt" : 0,
"VDs" : 2,
"VNOpt" : 0,
"BBU" : "Fld",
"sPR" : "On",
"DS" : "1&2",
"EHS" : "Y",
"ASOs" : 2,
"Hlth" : "NdAtn"
}
],
"ASO" : [
{
"Ctl" : 0,
"Cl" : "X",
"SAS" : "U",
"MD" : "X",
"R6" : "U",
"WC" : "U",
"R5" : "U",
"SS" : "X",
"FP" : "X",
"Re" : "X",
"CR" : "X",
"RF" : "X",
"CO" : "X",
"CW" : "X",
"HA" : "X",
"SSHA" : "X"
}
]
}
}
]
}

Cannot get it working - need support

Question before we go in details:
What are this variables for as they are never used as i understand it:

• {$DEFAULT_KLR_PERIOD}
• {$LD_KLR_PERIOD}
• {$PD_KLR_PERIOD}

I have configured this values for the variables:

• {$ADAP_DISCOVERY_PERIOD} 1d adapters discovery period (1d)
• {$ADAP_HISTORY_PERIOD} 30d period of saving history for adapters data (30d)
• {$ADAP_REQUEST_PERIOD} 1h period of requesting storage adapters data (1h)
• {$ADAP_THROTTLING_HB_PERIOD} 10800 period of heartbeat for throttling for adapter data (10800=3h)
• {$LD_DISCOVERY_PERIOD} 6h logical discs discovery period. (6h)
• {$LD_HISTORY_PERIOD} 30d period of saving history for logical discs data (30d)
• {$LD_RECOVERY_STATES} Opt|OK (Opt|OK)
• {$LD_REQUEST_PERIOD} 5m period of requesting logical discs data (5m)
• {$LD_THROTTLING_HB_PERIOD} 10800 period of heartbeat for throttling for logical discs data (10800=3h)
• {$LSI_CLI} perccli64 (perccli64) storcli64
• {$LSI_PD_COMMAND} /call/eall/sall (/call/eall/sall)
• {$PD_DISCOVERY_PERIOD} 30m physical discs discovery period (30m)
• {$PD_HIGH_SEVERITY_STATES} Missing|Offline|Foreign|Failed (Missing|Offline|Foreign|Failed)
• {$PD_HISTORY_PERIOD} 30d period of saving history for physical discs data (30d)
• {$PD_REQUEST_PERIOD} 5m period of requesting physical discs data (5m)
• {$PD_THROTTLING_HB_PERIOD} 10800 period of heartbeat for throttling for physical discs data (10800=3h)

My problem is now after 15min All Pds LLD info is filled with a JSON which looks ok and i get the physical disks created but all values are empty !
After 24h still nothing changed !
Also the 3 other functions in UserParameters are never triggered :

• UserParameter=lsi.ctlsinfo
• UserParameter=lsi.ldslld
• UserParameter=lsi.batteryinfo

What i am doing wrong ?

Dependent item not update

Hello,
Dependent item not check after master item check, resulting in inadequate updating of status

pls help me

updatemacro.py not working

in 6.4.12 and at least the latest version of zabbixapi you cannot login with the current script.
you need to split the action in 2 parts

--snippit--
zabbix = ZabbixAPI(zbxurl)
zabbix.login(apiuser,apipass)
--snippit--