ITM Agent Insights: Using "krarloff" utility to view binary short term historical data

Body

IBM Tivoli Monitoring allows for collection of attribute data historically.

Historical collections are defined through the Historical Collection Configuration editor in the TEP, and the short term historical (STH) files are stored at either the TEMS or the TEMA before the data is warehoused to the Tivoli Data Warehouse (TDW) database.

STH data samples are written in binary to files named with the internal attribute group identifier, and a corresponding header file that has a .hdr extension.

In order to review the actual historical sample data before it is warehoused to the TDW database, the binary data must be translated.
On Windows platforms, this is done through the use of the "krarloff" utility.
On UNIX / Linux, it is possible to use "itmcmd history" command in addition to the "krarloff" utility.

ITM 6.23 documentation:
Converting short-term history files to delimited flat files
http://www.ibm.com/support/knowledgecenter/SSTFXA_6.2.3/com.ibm.itm.doc/itm_admin_62143.htm
***
The krarloff rolloff program can be run either at the Tivoli Enterprise Monitoring Server or in the directory in which the monitoring agent is running, from the directory in which the history files are stored. See Location of the Windows executables and historical data collection table files. 
***

ITM 6.3 documentation:
Converting short-term history files to delimited flat files
http://www.ibm.com/support/knowledgecenter/SSTFXA_6.3.0/com.ibm.itm.doc_6.3/adminuse/historyconvertflat_intro.htm

Converting files using the krarloff program
http://www.ibm.com/support/knowledgecenter/SSTFXA_6.3.0/com.ibm.itm.doc_6.3/adminuse/history_krarloffwindows.htm
***
Run the krarloff rolloff program from the directory in which the monitoring server or the monitoring agent is run by entering the following at the command prompt:
***

The "krarloff" command needs correct command syntax to work, and it is necessary to run the command from correct location / export environment variables so that it will work.

Attempting to run "krarloff" from the directory where the monitoring agent history files are stored fails:

# ls /opt/IBM/ITM63FP6/lx8266/77/hist
K77APPLIT3  K77APPLIT3.hdr k77.his  khdexp.cfg

# /opt/IBM/ITM63FP6/tmaitm6/lx8266/bin/krarloff -d ":" -o /krarloff.out -m /opt/IBM/ITM63FP6/lx8266/77/hist/K77APPLIT3.hdr -s /opt/IBM/ITM63FP6/lx8266/77/hist/K77APPLIT3

Current codepage is 1208
Source file is : /opt/IBM/ITM63FP6/lx8266/77/hist/K77APPLIT3
Definition file is : /opt/IBM/ITM63FP6/lx8266/77/hist/K77APPLIT3.hdr
Source file will be renamed to : /opt/IBM/ITM63FP6/lx8266/77/hist/K77APPLIT3.old
Destination file is : /krarloff.out
Delimiter set as : ":"
ATTRLIB was not defined.
Using working directory as default
Using ./ATTRLIB/ as ATTRLIB
ATTRLIB directory <./ATTRLIB/> does not exist
There were 0 entries for various attribute files
Error building the attribute table
Output will have wrong headers
#

The "krarloff" command above is failing trying to find "ATTRLIB" directory.  
If the environment variable is not set to point to the correct "ATTRLIB" location, the utility will default to trying to find ATTRLIB in the current working directory.

Performing a search on the system to find the location of the "ATTRLIB" directories:

# find / -name ATTRLIB
/opt/IBM/ITM63FP6/tmaitm6/lx8266/ax/tables/ATTRLIB
/opt/IBM/ITM63FP6/lx8266/78/tables/ATTRLIB
/opt/IBM/ITM63FP6/lx8266/77/tables/ATTRLIB
/opt/IBM/ITM63FP6/lx8266/lz/tables/ATTRLIB
/opt/IBM/ITM63FP6/lx8266/vm/tables/ATTRLIB
#

Notice there are "ATTRLIB" directories for each monitoring agent.

In order to convert the historical data for a particular agent, the "krarloff" utility should be run from the "tables" directory of that monitoring agent.
In this example, the "krarloff" utility is being used to convert historical data for the 77 product code:

# pwd
/opt/IBM/ITM63FP6/lx8266/77/tables
# /opt/IBM/ITM63FP6/tmaitm6/lx8266/bin/krarloff -d ":" -o /krarloff.out -m /opt/IBM/ITM63FP6/lx8266/77/hist/K77APPLIT3.hdr -s /opt/IBM/ITM63FP6/lx8266/77/hist/K77APPLIT3
Current codepage is 1208
Source file is : /opt/IBM/ITM63FP6/lx8266/77/hist/K77APPLIT3
Definition file is : /opt/IBM/ITM63FP6/lx8266/77/hist/K77APPLIT3.hdr
Source file will be renamed to : /opt/IBM/ITM63FP6/lx8266/77/hist/K77APPLIT3.old
Destination file is : /krarloff.out
Delimiter set as : ":"
Rename succeeded : /opt/IBM/ITM63FP6/lx8266/77/hist/K77APPLIT3 being renamed to /opt/IBM/ITM63FP6/lx8266/77/hist/K77APPLIT3.old
ATTRLIB was not defined.
Using working directory as default
Using ./ATTRLIB/ as ATTRLIB
There were 1 entries for various attribute files
#

Review the destination file to confirm the historical data was able to be converted for review:
# cat /krarloff.out | more
21600:"1161117134500000":"nc9053114067:77":"1161117134523000":"  Last Full Backup Time:":" 2016-10-10 18:0":10:60
21600:"1161117134500000":"nc9053114067:77":"1161117134523000":"  Last Backup Time:":" 2016-10-12 21:0":11:60
21600:"1161117134600000":"nc9053114067:77":"1161117134623000":"  Last Full Backup Time:":" 2016-10-10 18:0":10:60

Keywords:
drd401709