BAI4S events are not synchronized with BAW, performance fine tuning.

Troubleshooting

Problem

BAI for Server BAI4S is not able to display current events created by Business Automation Workflow.

Symptom

When we see the Dashboards in the Kibana Discovery interface we notice new events are coming but with a long delay.

Cause

It is possible that the cause of the backpressure in Flink comes from Elasticsearch that cannot absorb the flow of the events from the Business Automation Workflow. The system can be in need of fine-tuning.

Environment

BAI for Server 20.0.3-21.0.3

Diagnosing The Problem

When we see the Dashboards in the Kibana Discovery interface we notice new events are coming but with a long delay.

Resolving The Problem

You can try to increase the memory value of Elastic Search (ES), the properties -Xmx and -Xms defined in
<bai4s-root-dir>/config/elasticsearch/jvm.options

Confirm the value in jvm.options, you can cat jvm.options then gradually increase the values. The default is 1G. It can be raised to 2G and so on depending on your system

The value for -Xmx and -Xms should be the same. The best path is to increase gradually and test the system.

For the ElasticSearch JVM parameters it is best to refer to ES documentation:
https://www.elastic.co/guide/en/elasticsearch/reference/current/advanced-configuration.html#set-jvm-heap-size

Set Xms and Xmx to no more than 50% of your total memory.
Set Xms and Xmx to no more than the threshold for compressed ordinary object pointers (oops). (follow the URL above for working oops threshold)

A sample might look as follows (after changes)

"jvm" : {

    "pid" : 96,

    "version" : "1.8.0_272",

    "vm_name" : "OpenJDK 64-Bit Server VM",

    "vm_version" : "25.272-b10",

    "vm_vendor" : "Red Hat, Inc.",

    "bundled_jdk" : true,

    "using_bundled_jdk" : false,

    "start_time_in_millis" : 1660307563154,

    "mem" : {

     "heap_init_in_bytes" : 2147483648,

     "heap_max_in_bytes" : 2112618496,

     "non_heap_init_in_bytes" : 2555904,

     "non_heap_max_in_bytes" : 0,

     "direct_max_in_bytes" : 1073741824

    },

If you are getting close to 50% of your total memory and have not seen improvement, you can try the following while keeping the JVM with the values that you already changed above.

The property TASKMANAGER_HEAP_MEMORY has an impact on the Java parameters of the task manager, which is what to test next.

Please try the following:

In the .env file, change
TASKMANAGER_HEAP_MEMORY=3072m
Restart BAI4S.
To check the Java process once the task manager is up and running

docker-compose -f data/bai.yml exec taskmanager ps auxww

See whether the events are still delayed.
If there is an improvement, but it is not enough then increment the value by 512m. Be careful, if the value is too high there is a risk that other containers start memory swapping, which will slow everything down again.

I ran the command out of

/root/bai-sn-master/bai-for-server-20.0.3-IF002

The command in this case was as follows:

docker-compose -f data/bai.yml exec taskmanager ps auxww

Related Information

elasticsearch reference advanced configuration

Document Location

Worldwide

[{"Type":"MASTER","Line of Business":{"code":"LOB45","label":"Automation"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSBYVB","label":"IBM Cloud Pak for Business Automation"},"ARM Category":[{"code":"a8m0z0000001iUIAAY","label":"Operate-\u003EBAI App Management"}],"ARM Case Number":"TS010020783","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"}]

Product Synonym

BAI; Performance, fine-tune, Business Automation Insights

Was this topic helpful?

Document Information

Modified date:
17 October 2022

UID

ibm16615325

Tips