The format file describes the patterns that the agent looks for to match events in the monitored logs. The format file consists one or more format specifications.
You can change the format file while an agent instance is running. The file is read by the agent when it starts, and is monitored for changes to its timestamp every 60 seconds thereafter. If the timestamp of the file changes, the agent reinitializes its configuration dynamically, without requiring a restart. For more information, see Changing the agent configuration and format files.
To create new patterns to match an event, use the new regular expression syntax that consists of the following parts:
The format header contains the keyword REGEX, which informs the agent that you are using a regular expression to match the pattern in the monitored log.
REGEX REExample
REGEX *DISCARD*
As a pattern matched, nothing
is written to the unmatch log. The log file status records matched
include these discarded events.After the format header, the format content consists of a regular expression on the first line, followed by mappings. Each mapping is shown on a separate line and these mappings are described in the following example.
All lines that match the regular expressions are selected and sent to the monitoring server as events. The regular expression contains sub expressions. You can use the sub expressions to match specific parts of these lines that match, to a variable called a 'slot' in the Event Integration Facility. You can optionally map these slots to specific attributes in IBM® Tivoli® Monitoring.
Error: disk failure
Error: out of memory
WARNING: incorrect login
REGEX REExample
Error: (.*)
msg $1
END
Based on this format specification, and the preceding set of log data, the agent generates two events. Both events are assigned the REEXample event class. In the first event, the disk failure value is assigned to the msg slot. Also, in the second event, the out of memory value is assigned to the msg slot. As the Warning line did not match the regular expression, it is ignored and no event is generated.
When you assign the value of $1 to the msg slot, you assign it the value of the first sub expression.
Error: disk failure on device /dev/sd0: bad sector
Error: disk failure on device /dev/sd1: temperature out of range
you might want to assign these error messages to their own event
class so that you are informed immediately if there is a disk failure.
You can include a description of the disk on which the error occurred
and more specifically the disk error in the event. REGEX DiskFailure
Error: disk failure on device (/dev/sd[0-9]):(.*)
device $1 CustomSlot1
msg $2
END
You assign these two sub expressions to event slots. The two events that are generated contain the following values:
"device=/dev/sd0" and "msg=bad sector"
"device=/dev/sd1" and "msg=temperature out of range"
If you use EIF to generate the first event, it displays as shown in the following example:
DiskError;device='/dev/sd0';msg='bad sector';END
If the event is sent to IBM Tivoli Monitoring, the slot named msg is assigned to the IBM Tivoli Monitoring attribute of the same name. But there is no pre-defined IBM Tivoli Monitoring attribute for the slot that we called device.
If you need to see the value assigned to device directly on the Tivoli Enterprise Portal, or write situations against it, you must assign it to an IBM Tivoli Monitoring attribute.
Using these attribute names in the format file populates IBM Tivoli Monitoring attributes of the same name. Using these attributes does not affect the content of the EIF event sent directly to OMNIbus.
You assign a slot from the event definition to one of these custom IBM Tivoli Monitoring attributes in the format file.
REGEX DiskFailure
Error: disk failure on device (/dev/sd[0-9]):(.*)
device $1 CustomSlot1
msg $2
END
When the event displays in IBM Tivoli Monitoring, the value assigned to the device slot is assigned to the IBM Tivoli Monitoring attribute called CustomSlot1. You view this value on the Tivoli Enterprise Portal or use it to define situations. You can assign any slot in the event definition to any of the 10 custom IBM Tivoli Monitoring attributes in the same manner, by putting "CustomSlot<n>", where <n> is a number from 1 to 10, next to the slot definition.
In this example, the first sub expression is defined specifically as (/dev/sd[0-9]), but the second sub expression is defined generally as (.*). In defining the regular expression as specifically as possible, you improve performance. Therefore, if you enter a search for an error on a device that does not match the specific error message defined here, the search procedure stops immediately when the error is not found. Time is not wasted looking for a match.
REGEX REExample
Error:
msg $1
END <EOL>
<EOF>
The custom integer attributes CustomInteger1 to CustomInteger3 are 64-bit integer attributes. You can use them in the same manner as the string type CustomSlot attributes. You can use these attributes to map individual slots, or sub expressions, from the log file to individual IBM Tivoli Monitoring attributes. Because these attributes are numeric, you can use arithmetic comparisons on them, such as < and >, which is not possible with the string attributes.
Oct 24 11:05:10 jimmy fschecker[2165]: Filesystem /usr is 97% full.
REGEX FileSystemUsage
^([A-Z][a-z]{2}) ([ 0-9][0-9]) ([0-9]{2}:[0-9]{2}:[0-9]{2}) (.*?) (.*?):
Filesystem (.*?) is ([0-9]+)% full\.$
Month $1 CustomSlot1
Date $2 CustomSlot2
Time $3 CustomSlot3
Host $4 CustomSlot4
Service $5 CustomSlot5
Filesystem $6 CustomSlot6
PctFull $7 CustomInteger1
msg PRINTF("%s: %s% full", Filesystem, PctFull)
END
( Class == 'FileSystemUsage' AND CustomInteger1 >= 95)
A different event can then use CustomInteger1 for a different purpose and not trigger this situation accidentally. In summary, you can now write a situation in IBM Tivoli Monitoring that uses arithmetic operators on the CustomInteger attributes, which is not possible with the CustomSlots attributes.