Troubleshooting
Problem
A smaller number of indexing batches than configured leads to a noticeable degradation in the overall indexing throughput rate.
Symptom
Content Platform Engine (CPE) systems are configured with a specific number of indexing worker threads, which are used, along with the number of Content Engine indexing instances, to calculate the number of concurrent batches allowed per Content Search Service (CSS) server.
For example, if a system is configured with 12 indexing threads (Total Configured concurrent batches) and has 2 CE instances with dispatching enabled (CE instances configured for indexing) then 6 concurrent batches allowed would be calculated per CSS server. However, if the Total CE instances configured for indexing are too high a smaller number of concurrent batches allowed will be calculated. This scenario can be checked by reviewing the Concurrent indexing batch Semaphore for Text Search Server summary log message in the p8_server_trace log, for example:
2020-03-26T01:35:37.913 BA02B6CF CBR FNRCE0000D - DEBUG Concurrent indexing batch Semaphore for Text Search Server: css1 created with concurrent batches allowed: 2 Total Configured concurrent batches: 12 Total CE instances configured for indexing: 5
In the above example, the system actually had two CE instances configured for indexing, but 5 are (inaccurately) reported causing the concurrent batches allowed to be smaller (2) than the expected 6. This incorrect CE count can occur when the number of Virtual Servers is greater than the actual number of CE server instances (with dispatching enabled) on the system.
Cause
The count of CE instances used to calculate concurrent batches allowed is based on the number of Virtual Servers on the system. Because obsolete Virtual Servers in the Global Configuration Database (GCD) are not automatically cleaned up, that Virtual Server count can be higher than the actual number of CE instances with dispatching enabled. That inflated number will then result in a smaller number of concurrent batches per CSS server, negatively affecting indexing performance. The disparity between Virtual Server and actual CE indexing instances can be especially large in container-based environments.
Environment
This problem only occurs on Content Platform Engine (CPE) systems configured for indexing documents via CSS servers.
Diagnosing The Problem
Indications the problem occurs in an environment are noted by:
- Degradation in overall indexing performance caused by a smaller number of concurrent indexing batches on CSS servers
- The p8_server_trace log summary log Concurrent indexing batch Semaphore for Text Search Server summary log message has a Total CE instance configured for indexing higher than the number of CE instances enabled for indexing dispatching on the system.
- The list of Virtual Servers in the Administration Console for Content Platform Engine (ACCE) is inaccurate (too high).
Resolving The Problem
There are two ways to correct this situation; by using a JVM parameter, which requires a restart of the CPE server, or by deleting obsolete Virtual Server instances in Administration Console for Content Platform Engine (ACCE), which does not require restarting the CPE server:
1. In 5.5.5.0-P8CPE a new Java Virtual Machine (JVM) com.filenet.cbr.dispatcherEnabledCEServers parameter can be used to set the correct number of CE instances configured for dispatching in an environment. If configured, this parameter value is used to calculate the concurrent batches allowed for each CSS server. The JVM parameter must be configured in the Application Server with an integer value greater than zero, any other value is ignored. For example, to designate the system has two CE instances configured with indexing dispatching enabled:
-Dcom.filenet.cbr.dispatcherEnabledCEServers=2
Note: A restart of the CPE server is required.
2. In ACCE if any invalid or obsolete Virtual Servers are identified, they can be deleted by accessing the Domain tab then Global Configuration -> Administration -> Sites -> <Site> -> Virtual Servers
Note: Right click on the Virtual Server node to delete, but doing so first requires deletion of the server defined below the node. Restarting the CPE server is not required.
See APAR PJ46129 for details on the 5.5.5.0-P8CPE fix.
See APAR PJ46129 for details on the 5.5.5.0-P8CPE fix.
Document Location
Worldwide
[{"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Product":{"code":"SSNVNV","label":"FileNet Content Manager"},"ARM Category":[],"Platform":[{"code":"PF002","label":"AIX"},{"code":"PF016","label":"Linux"},{"code":"PF051","label":"Linux on IBM Z Systems"},{"code":"PF033","label":"Windows"}],"Version":"5.5.2;5.5.3;5.5.4","Edition":"","Line of Business":{"code":"LOB45","label":"Automation"}}]
Was this topic helpful?
Document Information
Modified date:
14 July 2021
UID
ibm16151797