Troubleshooting
Problem
Out Of Memory error in the CNEA Collator pod after disabling super grouping.
Symptom
The CNEA collator pod crashes.
Cause
The collater is creating a list of all events and groups which is causing the OOM.
Environment
To Reproduce:
Steps to reproduce the behavior:
- Using 1.6.8 deployment, add large number events and allow system to create policies and perform groupings.
- Follow documented steps to disable super grouping
- The collater pod grows in memory because the cache it keeps does not clear
Diagnosing The Problem
Just before the Out Of Memory, the collater prints the cache. The last few lines of the log will be like this:
2023-05-08 06:03:46 DEBUG Graph:260 - Graph--> Adding the mapping between groups and events for dismantling the supergroup.
2023-05-08 06:03:50 DEBUG PollerTask:71 - normalizerConnectionEnabled is enabled, getting the super grouping enablement and rank list from normalizer...
2023-05-08 06:03:50 INFO NormalizerAgg:52 - Connecting to normalizer with url: http://noihybrid-ibm-hdm-analytics-dev-normalizer-aggregationservice:5600/api/aggregation/v1/configuration
JVMDUMP039I Processing dump event "systhrow", detail "java/lang/OutOfMemoryError" at 2023/05/08 06:03:52 - please wait.
JVMDUMP032I JVM requested System dump using '/app/core.20230508.060352.1.0001.dmp' in response to an event
JVMDUMP030W Cannot write dump to file /app/core.20230508.060352.1.0001.dmp: Permission denied
Resolving The Problem
The issue has been resolved in NOI 1.6.9.
A hot fix is available for 1.6.8
Document Location
Worldwide
[{"Type":"MASTER","Line of Business":{"code":"LOB45","label":"Automation"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSTPTP","label":"Netcool Operations Insight"},"ARM Category":[{"code":"a8m0z0000001jZTAAY","label":"NOI Netcool Operations Insights-\u003ECNEA Cloud Native Event Analytics"}],"ARM Case Number":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"1.6.8"}]
Was this topic helpful?
Document Information
Modified date:
04 September 2023
UID
ibm17030482