A fix is available
APAR status
Closed as program error.
Error description
Environment: TEMS/TEPS/TEPD/WPA/AgentBuilder/AgentBuilderAgent Win2k3 ITM6.2+FP02 Problem Description: The customer created an agent in order to monitor a text log file by the Agent Builder.Then he tried to collect the history data from the agent.Initially, the monitored log file included 179,424 enties. The history collection file "K05SYSLOGC" under <ITMHOME>/tmaitm6/logs directory included all 179,424 entries successfully. However, after the warehouse interval, in this case one hour, only 169,000 entries were stored into WAREHOUS database. The missing records are not on the tail end or not at the beginning but in the middle. Detailed Recreation Procedure: I could recreate the problem in my test environment. 1) Install TEMS, TEPS, TEP Desktop Client, WPA, Windows OS agent and Agent Builder on a single win2k3 box. 2) Apply FP0002 for each component. 3) Launch the Agent Builder and click "Create New Agent" icon. 4) On "New IBM Tivoli Monitoring Agent Project" window, specify the project name "Project-K05" and click "Next". 5) On "Agent Information" window, specify the following values. Company identifier : IBM Agent identifier : K05 Display name : K05 SYSLOG Agent Product code : K05 Version : 620 Copyright : IBM Then click "Next". 6) On "Monitoring Information" window, check "This agent will gather data from an external data source." 7) On "Data Source Definition" window, click "New Data Source". 8) On "Data Source Location" window, select "Collect data from a Log File."Then click Next. 9) On "Log File Information" window, specify "C:\userfiles\SyslogCatchAll.txt.001" as Log file name and select "Tab separator".Then click "Advanced". 10) On "Advanced Log File Data Source Information", select the followings. Process new records appended to the file Detect new records when the file size increases Ignore existing records Single line Then click OK. 11) Back to "Log File Information" window, click finish. 12) Add the following attributes. Attribute name : datetime Maximum size : 64 Help text : datetime Recoed Field Information : Next field check "Display attribute in the Tivoli Enterprise Portal" check "Display string" Attribute name : priority Maximum size : 64 Help text : priority Recoed Field Information : Next field check "Display attribute in the Tivoli Enterprise Portal" check "Display string" Attribute name : hostname Maximum size : 64 Help text : hostname Recoed Field Information : Next field check "Display attribute in the Tivoli Enterprise Portal" check "Display string" Attribute name : message Maximum size : 2000 Help text : message Recoed Field Information : Remainder of record check "Display attribute in the Tivoli Enterprise Portal" check "Display string" Then click finish and generate this agent. 13) On TEP Desktop Client, click "History Configuration" bottun. 14) On "History Collection Configuration" window, select "K05 SYSLOG Agent" as Select a Product field and configure "K05 SYSLOGCATCHALL TXT 001" group with the following values. Collection Interval : 30 minutes Collection Location : TEMA Warehouse Interval : 1 hour Then click "Start Collection". 15) From MTEMS, start the agent. 16) Copy SyslogCatchAll.txt.001 sent from the customer to C:/userfiles directory. 17) Start WPA and wait for a few hours.Then a new table named "K05_SYSLOGCATCHALL_TXT_001" is created into WAREHOUS database. 18) After connecto to WAREHOUS database, run the following command. db2 -z select.out select * from itmuser.k05_syslogcatchall_txt_001 19) Having a look at select.out, only 169,000 entries of 179,424 were stored into WAREHOUS database.This is the problem. 20) Having a look at K05SYSLOGC under <ITMHOME>/tmaitm6/logs directory using krarloff command, all 179,424 records were included. Related Files and Output: - history collection file - krarloff command output - db2 corresponding table export output
Local fix
If WRITETIME value of n*1000 th row is different from WRITETIME value of n*1000+1 th row in the history collection file, this problem can be avoided.
Problem summary
**************************************************************** * USERS AFFECTED: All TEMS users. * **************************************************************** * PROBLEM DESCRIPTION: PURE EVENTS NOT WRITTEN TO THE * * WAREHOUSE DURING AN EVENT STORM * **************************************************************** * RECOMMENDATION: Apply this PTF. * **************************************************************** An agent that is generating more than 999 events per second and has historical data collection started on the agent, can cause entries to not be uploaded to the WAREHOUSE database. Any entries in excess of 999 are discarded as duplicates.
Problem conclusion
Increment the SAMPLES value for each row of data if its timestamp is the same as that of the previous row. The combination of timestamp and SAMPLES would ensure that each record is treated as unique rather than as a duplicate. The fix is only applicable to tables defined to use pure event sampling since the problem has been seen with pure events only.
Temporary fix
Comments
APAR Information
APAR number
OA28769
Reported component name
MGMT SERVER DS
Reported component ID
5608A2800
Reported release
620
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt
Submitted date
2009-04-25
Closed date
2009-05-05
Last modified date
2009-06-01
APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:
Modules/Macros
KRA
Fix information
Fixed component name
MGMT SERVER DS
Fixed component ID
5608A2800
Applicable component levels
R620 PSY UA47228
UP09/05/27 P F905
Fix is available
Select the PTF appropriate for your component level. You will be required to sign in. Distribution on physical media is not available in all countries.
[{"Business Unit":{"code":"BU054","label":"Systems w\/TPS"},"Product":{"code":"SSRJ5K","label":"Tivoli Management Server for Distributed Systems on z\/OS"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"620","Edition":"","Line of Business":{"code":"LOB17","label":"Mainframe TPS"}}]
Document Information
Modified date:
01 June 2009