Posts on Jan 1970

Monitoring servers through HP iLO in Zabbix

Today i have found a lot of news about our Monitoring tool!
I have translated this very good article from a russian website, every award is for http://habrahabr.ru/, thanks so much.


Introduction

During the implementation of Zabbix in our very extensive infrastructure, I was faced with the need to monitor the hardware of a fairly large fleet of servers HP Proliant different models and generations regardless of operating system agents and HP. itself an idea suggested itself to realize all this through iLO, but the task proved to be far less trivial, what initially looked. As a result of its decision turned out pretty interesting design that:

  • Uses the discovery, saves us from having to manually set anything at all, except for the address iLO,
  • Monitors the temperature of coolers and food servers Proliant, ranging from 5 generations
  • Monitors the state of the memory and hard drive for servers Proliant, ranging from 7 generation
  • Collects general information for inventory – serial number, model number, firmware version.

Now exactly how this was done.
Seemingly simple: iLO can give data through IPMI, and Zabbix has native support for this protocol, but, as usual, was smooth on paper. When we look at the issue once there are three problems:

  1. Zabbix uses a library openipmi, in which there is a bug – a successful connection to iLO will only happen if it originated from the name of an account that has administrator privileges. From a security standpoint this is fundamentally wrong. You can solve this patch / update, but it does not eliminate the other,
  2. Removing information from discrete sensors via IPMI is not supported,
  3. And finally, for different models of servers keys, names and number of sensors differ. Make templates for each model by hand – very productive.

In connection with the foregoing, it was decided to write a separate mechanism for interaction with iLO, relying on scripts and other utilities to work with IPMI. As a programming language was chosen perl, as well as a data source – package freeipmi . On all servers in the wards iLO account was created to monitor read-only rights. Logically, the entire structure is divided into two parts:

  • Script detection data sources ilo_discovery.pl – iLO polls for the supported parameters and keys, parses them, and outputs a format understandable Zabbix,
  • Script retrieve data ipmi_proliant.pl – on request gives the value of a specific parameter.

Just want to note that perl programmer and I am not used to solve problems of those examples and designs that were clear to me, the end result was achieved – all this works successfully.

Detection script

This provides data in a script format zabbix discovery according to which class the data was requested – sensors, chassis information, and so forth. Such separation is due to the logic of the template that is used in conjunction with scripts.

ilo_discovery.pl

Script retrieve data

This script outputs the value of specific sensors – again, depending on what class of data has been requested. The obtained data is cached in a text file, so you do not accidentally zaddosit iLO simultaneous requests.

ipmi_proliant.pl

Template monitoring

Write scripts – half the battle. Had yet to properly configure the import of all of this information to Zabbix and configure the triggers. The result of this work was the monitoring template, which established rules for detecting all sensors and other data sources and automatically create a corresponding triggers and graphs.

Application in practice

For practical application of the above construction is necessary:

  1. Download the archive with the script and template import template Zabbix,
  2. Put scripts ilo_discovery.pl ipmi_proliant.pl and the folder specified as storage ExternalScripts config Zabbix, and make it executable,
  3. Download and install FreeIPMI (FAQ assembly and dependencies is here ):
# wget http://ftp.gnu.org/gnu/freeipmi/freeipmi-1.2.1.tar.gz
# tar -xvzf freeipmi-1.2.1.tar.gz # cd freeipmi-1.2.1 # ./configure –prefix=/usr –exec-prefix=/usr –sysconfdir=/etc –localstatedir=/var –mandir=/usr/share/man
# make install
For 64-bit systems configure line would be:
./configure –prefix=/usr –exec-prefix=/usr –sysconfdir=/etc –localstatedir=/var –mandir=/usr/share/man –libdir=/usr/lib64

  1. Create a user account for iLO Zabbix and prescribe its data in scripts ($ user and $ pass),
  2. Check that FreeIPMI successfully connects to iLO (address, username and password substitute your own):
# /usr/sbin/ipmi-sensors -D LAN2_0 -h 192.168.0.1 -u monitor -p P@$$w0rd -l USER -W discretereading –no-Header-output –quiet-cache –sdr-cache-recreate –comma-separated-output –entity-sensor-names
In response, we should get a list of sensors such as:
0,System Chassis 1 UID Light,OEM Reserved,N/A,N/A,‘OEM Event = 0000h’
1,System Chassis 2 Health LED,OEM Reserved,N/A,N/A,‘OEM Event = 0000h’
2,Processor Module VRM 1,Power Unit,N/A,N/A,‘Device Inserted/Device Present’
3,Power Supply Power Supply 1,Power Supply,N/A,N/A,‘Presence detected’
Check that the script successfully parses the data detection (substitute your address):
# /usr/lib/zabbix/externalscripts/ilo_discovery.pl 192.168.0.1 sensor temp numeric
In response, we should get something like this:
{
data“:[ { “{#CLASS}“:“sensor”, “{#KEY}“:“Air Inlet 01-Inlet Ambient”, “{#SECTION}“:“Temperature”, “{#TYPE}“:“numeric”, “{#MEASURE}“:“C”}, { “{#CLASS}“:“sensor”, “{#KEY}“:“Processor 02-CPU”, “{#SECTION}“:“Temperature”, “{#TYPE}“:“numeric”, “{#MEASURE}“:“C”},
{
{#CLASS}“:“sensor”,
{#KEY}“:“Air Inlet 01-Inlet Ambient”,
{#SECTION}“:“Temperature”,
{#TYPE}“:“numeric”,
{#MEASURE}“:“C”},
{
{#CLASS}“:“sensor”,
{#KEY}“:“Processor 02-CPU”,
{#SECTION}“:“Temperature”,
{#TYPE}“:“numeric”,
{#MEASURE}“:“C”},
  1. In Zabbix front-end server, which we want to interrogate through iLO, iLO enter the address in the macro {$ ILO} (in the address field ipmi interface is nothing to indicate it is not necessary)
  2. Bind to the server template monitoring iLO
  3. Wait until fulfills detection.
Will look something like this section lastest data for node monitoring iLO:
Zabbix HP ILO IPMI
Charts on data obtained included:
Zabbix HP Ilo Graps IPMI

Conclusion

This monitoring mechanism has been successfully tested with HP Proliant server series DL, ML and BL 5, 6, 7 and 8 generations.General recommendation – try before applying it to update to the latest versions of iLO firmware. As for the younger line of servers, having on board instead Lo100 iLO – with them all this will work too, but some of the information obtained from the older models of the same generation, will not be available because lo100 sends less data than iLO.

Read More