Indexing work capacity

You can control the indexing work capacity of IBM® Content Search Services index servers and Content Platform Engine instances.

The work capacity is determined by the following properties:
Maximum concurrent index batches This property sets the maximum number of index batches that an IBM Content Search Services index server can work on concurrently. For more information, see Setting the indexing work capacity of index servers.
Maximum worker threads for indexing This property sets an upper limit to the maximum number of worker threads for a Content Platform Engine instance. For more information, see Setting the indexing work capacity of subsystem dispatchers.
Initially, you might want to make the following items equal:
  • The total number of concurrent batches
  • The total number of worker threads
For example, suppose the following situation:
Item Situation Total
Concurrent batches Two IBM Content Search Services index servers that are configured to work a maximum of 12 concurrent index batches 24 (2 * 12 = 24)
Worker threads Two Content Platform Engine instances that are configured to use an upper limit of eight worker threads 16 (2 * 8 = 16)

To make these totals equal, you might want to set the worker thread upper limit to 12.

If these totals match, the number of worker threads is still likely to be too low. Experimentally increase the number of worker threads while you monitor your system with IBM System Dashboard for Enterprise Content Management. Navigate to the object store counters and look under CBR > Indexing Batch and view the Indexing Wait Duration Avg ms entry. This entry reflects the amount of time that worker threads waited to send an index batch to an index server because of the concurrent batch limit. A consistent delay indicates that the configured work capacity of Content Platform Engine is too high relative to that of IBM Content Search Services.

To resolve a consistent delay, either decrease the work capacity of Content Platform Engine or increase the work capacity of IBM Content Search Services. You can do the former by decreasing the number of worker threads. You can do the latter by pursuing one or more of the following actions:
  • Increasing the maximum number of concurrent batches.
  • Verifying that the number of preprocessing and indexing threads for index servers is properly tuned. The default number of these threads is probably too low. For more information, see Troubleshooting queues.
  • Adding another index server.