Known Baseline VSZ increase in IBM App Connect Enterprise from 12.0.11.0 onwards

The baseline amount of virtual set size (VSZ) needed for IntegrationServer, DataFlowEngine, and bipbroker processes since IBM App Connect Enterprise (ACE) 12.0.11.0 has increased by 4 GiB. This change in expected behaviour is related to how ACE leverages an embedded Node.js runtime which powers a wide range of features including, but not limited to:

• The administration REST API and web user interface
• Remote callable flows and remote debugging
• Discovery connector nodes

To be able to continue to provide customers with security updates, internal changes were made to how ACE uses Node.js to allow consumption of ECMAScript modules (ES modules) which has resulted in this change in expected behaviour.

Background on process memory statistics

Memory usage of processes on Linux and IBM AIX is usually reported using two values:

• Virtual memory size (VSZ) - This represents the total amount of memory requested by a process, this can include memory that has been swapped out to disk, memory that has been allocated but not used, or memory occupied by shared libraries.
• Resident set size (RSS) - This represents how much memory allocated to the process is actively occupying pages in RAM, such as thread stacks, and actively or recently used heap memory. The RSS of a process is always lower than the VSZ.

Microsoft Windows reports the memory usage of a process in terms of its Working Set Size, which is equivalent to the RSS on Linux and AIX, and does not report an equivalent of VSZ by default.

On these operating systems, the kernel will lazily allocate real memory to back virtual memory pages when needed by the process. For example, if a process requests a 2 GiB block of memory but only touches the first 4 KiB page of the block, then the kernel will mark that the process is using 2 GiB more virtual memory but will only use one 4 KiB page of real memory to handle the request. This corresponds to the reported VSZ of the process increasing by 2 GiB but the reported RSS increasing only by 4 KiB.

Lazily backing virtual memory with physical manner in this way allows the system to overcommit memory resources to process in excess of the total amount of physical memory available. We have previously published a Tech Note on virtual memory, memory overcommit, and how it relates to IBM App Connect Enterprise here.

Baseline VSZ increase in ACE 12.0.11.0 and above

IBM App Connect Enterprise 12.0.11.0 included an update to an open-source JavaScript library used inside the administration API which resolved a potential security vulnerability. The open-source library is now only available as an ES module, meaning that ACE would have to consume ES module based JavaScript code for the first time.

When Node.js needs to load JavaScript code provided as an ES module, it first loads a helper library which handles translating JavaScript code written in the non-ES module format so that it can be understood by ES module JavaScript code. This helper library is implemeted as a WebAssembly (WASM) binary embedded within Node.js for improved startup times, CPU usage, and enhanced portability.

Unlike Java and JavaScript, which use garbage collection and configurable heap sizes, WASM executables are responsible for their own memory management and require a fixed, contiguous, 4 GiB block of memory. So, when Node.js loads this WASM executable a 4 GiB allocation of virtual memory is needed for the WASM heap. The actual amount of resident memory needed by this WASM executable is much smaller than 4 GiB.

This manifests itself as a 4 GiB increase in the baseline VSZ required an IntegrationServer, DataFlowEngine, or bipbroker process. Due to the fact that actual amount of memory needed is much smaller than 4 GiB, the baseline RSS of these processes does not increase significantly.

Changes to the baseline amount of VSZ needed by a process is not necessarily a cause for concern, if there is no significant change in the RSS needed by the process. It is not uncommon for the baseline VSZ for an ACE process to fluctuate with upgrades to the embedded Java Virtual Machine or Node.js runtime. Both of these languages use a garbage collected heap which appears as an initial large reservation of memory, and so large initial increase in VSZ, which their garbage collecting allocators then grow into over time, eventually causing the RSS to increase as real memory pressure increases. The garbage collectors in these runtimes are then responsible for freeing the used memory when it is no longer needed.

We do not currently believe that this change in baseline VSZ usage due to the addition of this WASM heap is cause for any undue concern. We do not expect that this should affect any capacity planning that a customer may need to perform, or the minimum system requirements for the product.