APAR status
Closed as program error.
Error description
Environment: TEMS/TEPS/TEPD/WPA/AgentBuilder/AgentBuilderAgent Win2k3 ITM6.2+FP02 Problem Description: The customer created an agent in order to monitor a text log file by the Agent Builder.Then he tried to collect the history data from the agent.Initially, the monitored log file included 179,424 enties. The history collection file "K05SYSLOGC" under <ITMHOME>/tmaitm6/logs directory included all 179,424 entries successfully. However, after the warehouse interval, in this case one hour, only 169,000 entries were stored into WAREHOUS database. The missing records are not on the tail end or not at the beginning but in the middle. Detailed Recreation Procedure: I could recreate the problem in my test environment. 1) Install TEMS, TEPS, TEP Desktop Client, WPA, Windows OS agent and Agent Builder on a single win2k3 box. 2) Apply FP0002 for each component. 3) Launch the Agent Builder and click "Create New Agent" icon. 4) On "New IBM Tivoli Monitoring Agent Project" window, specify the project name "Project-K05" and click "Next". 5) On "Agent Information" window, specify the following values. Company identifier : IBM Agent identifier : K05 Display name : K05 SYSLOG Agent Product code : K05 Version : 620 Copyright : IBM Then click "Next". 6) On "Monitoring Information" window, check "This agent will gather data from an external data source." 7) On "Data Source Definition" window, click "New Data Source". 8) On "Data Source Location" window, select "Collect data from a Log File."Then click Next. 9) On "Log File Information" window, specify "C:\userfiles\SyslogCatchAll.txt.001" as Log file name and select "Tab separator".Then click "Advanced". 10) On "Advanced Log File Data Source Information", select the followings. Process new records appended to the file Detect new records when the file size increases Ignore existing records Single line Then click OK. 11) Back to "Log File Information" window, click finish. 12) Add the following attributes. Attribute name : datetime Maximum size : 64 Help text : datetime Recoed Field Information : Next field check "Display attribute in the Tivoli Enterprise Portal" check "Display string" Attribute name : priority Maximum size : 64 Help text : priority Recoed Field Information : Next field check "Display attribute in the Tivoli Enterprise Portal" check "Display string" Attribute name : hostname Maximum size : 64 Help text : hostname Recoed Field Information : Next field check "Display attribute in the Tivoli Enterprise Portal" check "Display string" Attribute name : message Maximum size : 2000 Help text : message Recoed Field Information : Remainder of record check "Display attribute in the Tivoli Enterprise Portal" check "Display string" Then click finish and generate this agent. 13) On TEP Desktop Client, click "History Configuration" bottun. 14) On "History Collection Configuration" window, select "K05 SYSLOG Agent" as Select a Product field and configure "K05 SYSLOGCATCHALL TXT 001" group with the following values. Collection Interval : 30 minutes Collection Location : TEMA Warehouse Interval : 1 hour Then click "Start Collection". 15) From MTEMS, start the agent. 16) Copy SyslogCatchAll.txt.001 sent from the customer to C:/userfiles directory. 17) Start WPA and wait for a few hours.Then a new table named "K05_SYSLOGCATCHALL_TXT_001" is created into WAREHOUS database. 18) After connecto to WAREHOUS database, run the following command. db2 -z select.out select * from itmuser.k05_syslogcatchall_txt_001 19) Having a look at select.out, only 169,000 entries of 179,424 were stored into WAREHOUS database.This is the problem. 20) Having a look at K05SYSLOGC under <ITMHOME>/tmaitm6/logs directory using krarloff command, all 179,424 records were included. Related Files and Output: - history collection file - krarloff command output - db2 corresponding table export output
Local fix
If WRITETIME value of n*1000 th row is different from WRITETIME value of n*1000+1 th row in the history collection file, this problem can be avoided.
Problem summary
An agent that is generating more than 999 events per second does not get all entries uploaded to the WAREHOUS database at the warehouse interval, even though all events were correctly created in the agent historical data binary file. The code was only processing the first 999 entries with the same timestamp and all events with the same timestamp after the initial 999 were deemed to be duplicates and were discarded.
Problem conclusion
The code was modified to increment the SAMPLES value for each row of data if its timestamp (up to seconds) is the same as that of the previous row. The combination of timestamp and SAMPLES would ensure that each record is treated as an individual one rather than as a duplicate. The fix has only been made for tables defined to use pure event sampling since the problem has been seen with pure events only. The fix for this APAR is contained in the following maintenance packages: | fix pack | 6.2.0-TIV-ITM-FP0003
Temporary fix
Comments
APAR Information
APAR number
IZ41189
Reported component name
TEMA
Reported component ID
5724C04TE
Reported release
620
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt
Submitted date
2009-01-13
Closed date
2009-04-22
Last modified date
2009-04-22
APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:
Fix information
Fixed component name
TEMA
Fixed component ID
5724C04TE
Applicable component levels
R620 PSY
UP
[{"Business Unit":{"code":"BU048","label":"IBM Software"},"Product":{"code":"SSCTLMR","label":"ITM Tivoli Enterprise Mgmt Agent V6"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"620","Edition":"","Line of Business":{"code":"","label":""}}]
Document Information
Modified date:
22 April 2009