Direct links to fixes
7.0.0-WS-WCServer-FP009
WebSphere Commerce Version 7.0.0.4 Fix Pack
WebSphere Commerce Version 7.0.0.5 Fix Pack
WebSphere Commerce Version 7.0.0.6 Fix Pack
WebSphere Commerce Version 7.0.0.7 Fix Pack
WebSphere Commerce Version 7.0.0.8 Fix Pack
WebSphere Commerce Version 7.0.0.9 Fix Pack
JR39837: Performance improvements for the ECEvent processing framework
APAR status
Closed as program error.
Error description
The three scheduler jobs used for the ECEvent framework have been updated to process the ECEvents more efficiently: RaiseECEvent, EventProcessor and CleanECEvent. Without this APAR, after an event is created and raised with the RaiseECEvent job, the event is set to -10 state (BUSEVENT.CHECKED column). -10 events are processed further with the EventProcessor command, which has a default batch size of 200 (configurable in wc-server.xml). After the events are processed by the EventProcessor, they are set to -20 state, so they can be cleaned with the CleanECEvent job. Since there are several events enabled by default, for example OrderCreation and OrderItemUpdate, the BUSEVENT grows very large and the EventProcessor job is not able to catch up. With the fix associated to this APAR, enhancements are made to the three commands so that they update and delete rows in the BUSEVENT table using multiple transactions, to allow it to update and delete millions of records, over time, without running into transaction time-outs or database locking. RaiseECEvent: ------------- This scheduled job, that runs by default every five minutes, raises a common base event for each of the events in the BUSEVENT table that have not been raised (CHECKED = 0). Raising the event will notify the registered event listeners. After the event is raised, the CHECKED column is updated to -10. It is the responsibility of the EventProcessor scheduled job to do further processing of events in -10 state: analytics events are sent to an external analytics system and then updated to -20 state, while non-analytics events are simply updated to the -20 state. Finally, the CleanECEvent job deletes the events with CHECKED = -20 from the BUSEVENT table. As an optimization, if there are no analytics events that need to be processed by EventProcessor, the "Analytics Event Configuration" component can be disabled in the instance configuration file and the EventProcessor scheduled job can be deleted. Then the parameter "noEventProcessing=true" must be passed to the RaiseECEvent command so that events are placed in state -20 after they are raised. A maximum of 5000 events are changed from state -10 to -20, per command execution. To determine if EventProcessor is sending analytics events to an external analytics system, look at the "Analytics Even Configuration" in the instance configuration file. If the "enable" attribute of the component node is set to false, or if there are no stores enabled, the Analytics component is not being used. <component compClassName="com.ibm.commerce.bi.events.AnalyticsComponentConf iguration" enable="true" name="Analytics Event Configuration"> <property display="false"> <supportedEvents inProcessState="-15" mapper="mapCommonEvents.xml" maxBatchSize="200" maxProcessTime="86400" processState="-10" purgeState="-20"> ... <stores> <store enabled="false" storeId="-1"> <configuration cmClientID="" password="" serviceURL="https://wscreceiver.coremetrics.com/Receiver/sen dEventData" sslKeyPassphrase="WebAS" sslKeyStore="/usr/WebSphere/AppServer/profiles/demo/etc/Dumm yServerKeyFile.jks" sslTrustPassphrase="WebAS" sslTrustStore="/usr/WebSphere/AppServer/profiles/demo/etc/Du mmyServerTrustFile.jks" transmitClassName="com.ibm.commerce.bi.events.transmit.CMWeb ServiceTransmitter" transmitEnabled="false" username="" /> </store> </stores> </property> </component> EventProcessor: --------------- This scheduled job, that runs by default every hour, does further processing on analytics events that have been raised and persisted in the BUSEVENT table so they can be transmitted in a batch to an external (non-WebSphere Commerce) analytics system. If the "Analytics Event Configuration" component is enabled, 200 events are fetched and updated from 'process' state (default -10) to 'inProcess' state (default -15). Any qualifying analytics events (events for stores that are enabled, with the correct type, transport and channel ID's) in this batch are transformed and then transmitted to the analytics system, and then updated to purge state. Any non-qualifying events in this batch are updated to purge state (default -20). This process of fetching and processing 200 events is repeated until the maxBatchSize analytics events are processed. If maxBatchSize is set to -1, then the batching will continue until all the events in the BUSEVENT table are processed. If the "Analytics Event Configuration" component is disabled or its stores are disabled, up 500,000 events in the process state (-10) are updated to be in the purge state (default -20). These are committed in batches of 5,000. CleanECEvent: --------------- This scheduled job, that runs by default every 30 minutes, deletes up to the specified maximum events from the BUSEVENT table that are in the purge state and are older than the specified expire time (default 30 minutes). Records are deleted and committed in batches of 5,000. The default maximum is 500,000. If "Analytics Event Configuration" component is enabled, any events that are older than maxProcessTime (default 1 day) and stuck in the -15 state are re-activated by changing them back to -10 state, so that they can be processed by the EventProcessor job. Before V6 FEP2, the default BUSEVENT.CHECKED value to indicate an event is ready to be purged was -10. However, as of V6 FEP2 with the cm-salescenter feature and in V7 base, the scheduler configuration of the CleanECEvent job has been changed to specify that -20 is the purge state: SCHCONFIG.SCCQUERY='purgeState=-20'. This was done so that the EventProcessor scheduled job could do further processing of -10 events, transmit the data to an external analytics system, and then put the event in -20 state.
Local fix
Problem summary
USERS AFFECTED: WebSphere Commerce 7.0 users PROBLEM ABSTRACT: Performance improvements for the ECEvent processing framework (BUSEVENT table) BUSINESS IMPACT: Without this APAR, and using the default configuration for the Business Event framework, the rows in the BUSEVENT table will grow unbounded. Once the table reaches a large size, it is possible for the site to experience high memory utilization or database locking problems resulting from the Event Framework commands trying to operate on millions of rows RECOMMENDATION:
Problem conclusion
Install the iFix associated to this APAR to obtain the code changes which make the Event Processor framework more efficient as described above ------------------------------------------------------------- The latest available maintenance information can be obtained from the Recommended Fixes for WebSphere Commerce technote: http://www.ibm.com/support/docview.wss?rs=3046&uid=swg21261296
Temporary fix
Comments
APAR Information
APAR number
JR39837
Reported component name
WC BUS EDITION
Reported component ID
5724I3800
Reported release
700
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
YesSpecatt / Pervasive
Submitted date
2011-06-13
Closed date
2011-08-09
Last modified date
2011-08-09
APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:
Fix information
Fixed component name
WC BUS EDITION
Fixed component ID
5724I3800
Applicable component levels
R700 PSY
UP
[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSYSYL","label":"WebSphere Commerce Enterprise"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"7.0","Edition":"","Line of Business":{"code":"","label":""}}]
Document Information
Modified date:
09 August 2011