Scaling an IBM Business Automation Insights system

To scale your IBM® Business Automation Insights system, you increase the parallelism of the processing jobs or you add Elasticsearch pod replicas.

About this task

When you first install IBM Business Automation Insights, as instructed in Configuring Business Automation Insights for the first time, you need to specify only what processing jobs you want to install and Business Automation Insights automatically creates the task managers that are necessary for the execution of the selected jobs. Likewise, when you scale a specific job, Business Automation Insights automatically creates the necessary task managers.

Procedure

New in 18.0.1 Increase the parallelism of the appropriate processing job as follows.
1. Make sure that <job>.install parameter for this job is set to true.
2. Increase the parallelism of the job by updating the value of the corresponding <job>.parallelism parameter.
  For more information, see also Updating your Business Automation Insights deployment and Monitoring job throughput and back pressure.
Increase the number of Kafka topic partitions as necessary.
The number of Kafka topic partitions should always be equal to, or greater than, the number of task slots that are assigned to each processing job.
If you use embedded Elasticsearch and Kibana, rather than an external Elasticsearch installation, add pods as necessary for proper scalability. For more information, see Changing the number of replicas.

What to do next

Higher parallelism requires more overall memory and CPU. For each job, calculate on the following basis.

Number of necessary CPUs = <job>.parallelism

Memory = 1.5 Gb X <job>.parallelism