Optimizing the startup time for an independent integration server

You can optimize an independent integration server to speed up server startup, which is particularly relevant in cloud container environments.

Before you begin

  • You must install IBM® App Connect Enterprise 12.0.4.0 or later to use this feature.
  • Watch the following video, which demonstrates how to optimize the start-up time of an integration server: How to optimize integration server start-up time.
  • Create an integration server work directory by using the mqsicreateworkdir command as described in mqsicreateworkdir command.
  • Configure the integration server by modifying the server.conf.yaml file, as described in Configuring an integration server by modifying the server.conf.yaml file. You can switch off functions of the server that you do not need. For example:
    • activityLogEnabled: false # Switch to shut off all activity logging
    • cacheOn: false # Switch to shut off the Global Cache
    • recordReplayEnabled: false # Switch to shut off Record and Replay
  • If compilable resources are included, deploy the resources to the integration server by running the ibmint deploy command, including the --compile-maps-and-schemas parameter. For more information, see ibmint deploy command.
  • Apply any relevant overrides to the deployed applications flows by running the ibmint apply overrides command, as described in ibmint apply overrides command.

About this task

Server startup can be slower than is desirable, especially when you run in containers with restricted CPU provision, because the server must load every component. Parts of the server that are not needed, can be switched of by configuring the server's server.conf.yaml file as described in Configuring an integration server by modifying the server.conf.yaml file. However, while the product does not run the functions that are switched off, the server still loads or creates the underlying resources that would be required if the functions were switched on. Loading the resources adds to the startup time. Server startup time can be reduced by optimizing the server. This process analyses the content of a server's work directory and determines which parts of an integration server are needed at startup. The purpose of optimization is to speed up server startup in cloud container environments. Optimization applies to independent integration servers only and is not suitable for integration servers that are managed by an integration node. Server optimization is suitable when:
  • Server content is fixed for each IntegrationServer run.
  • Server startup times are important and need to be as short as possible.
To use server optimization successfully, you must have the following:
  • IBM App Connect Enterprise and all of its dependencies installed.
  • An independent server work directory (For example, created by running the mqsicreateworkdir command, as described in mqsicreateworkdir command).
  • Server configuration is set (in server.conf.yaml).
  • The server work directory is populated with runtime content such as BAR files (source to server)
  • Any compilable resources such as maps are compiled (by using the --compile-maps-and-schemas parameter with either the ibmint package command or the ibmint deploy command.) For more information, see ibmint package command and ibmint deploy command.
  • Any desired overrides are applied to the applications flows (by running the ibmint apply overrides command).

You can now optimize the integration server. Optimization analyses the content of a server's work directory and determines which parts of an integration server are needed at startup.

Procedure

Optimize the startup time for an integration server by running the ibmint optimize server command, as described in ibmint optimize server command.
  • The work directory passed in needs to be a valid server work directory. If it is not, then the command reports an error.
  • A server.conf.yaml file must be present.
  • The content of the server's work directory must be valid with no missing dependencies.
  • If any content is missing, the command fails and a log is displayed showing what is missing.
  • If the server's work directory contains independent resources (server level resources in IIB V10 and earlier), then an attempt is made to use the defaultApplication that is specified in the server.conf.yaml file. If this file is not present, then the --default-application parameter can be used.
  • If a default application name is required but not supplied, then the command fails.
  • Any BAR files that are present in an integration server work directory are unpacked.
  • If the BAR file contains server level resources, they are moved into the default application.
  • The deployed resources that were in the BAR file are evaluated in the same way as any other deployed resource. This step gives an optimization such that one server startup BAR files are not unpacked.
  • The work directory must contain at least one message flow for the analysis to be successful.
  • If no valid message flows are found, then a warning is reported.
  • If work directory content is such that an integration server requires every component, then a server.components.yaml is not created and the user is informed. (This scenario is highly unlikely.)
  • All work directory content is analyzed irrespective of whether it is used at run time or not. For example:
    • All policies are considered whether they are used by message flows or not.
    • All shared library content is analyzed whether they are referenced by applications or not.
    • All applications and message flows are analyzed whether they are going to run or not.
    • Subflows are not scanned individually but their content is considered when they are instantiated in the message flows that use them.
  • The --verbose parameter can be used to show what steps were taken during the analysis and why components were enabled (an example is shown at the end of this document).
  • It is possible to override the analysis results by using the -enable and --disable parameters. These parameters name components as they appear in the server.components.yaml file. For a component that falls under a subsection in the server.components.yaml, the component can be specified as <subsection>/<component>, or by <component> . For example, --enable ResourceManagers/GlobalCache or --enable GlobalCache. The longer form might be preferred for clarity. Overrides are applied after the analysis is complete and so they do not appear in the --verbose log output. If you want to enable or disable multiple components, you can specify overrides multiple times.
    • Expect to use the --enable and --disable parameters only in rare circumstances. In most cases, the analysis provides a satisfactory optimization without further user intervention.
    • The --enable and --disable parameters are simpler to use than hand-editing or script-editing the server.components.yaml file after analysis, but you must ensure they result in a consistently optimized integration server.
  • The --trace parameter produces service trace as with all other ibmint commands.
  • You can run the ibmint optimize server multiple times on the same work directory. If the command is successful, any existing server.components.yaml file is removed, and a new server.components.yaml file replaces it. No settings are preserved from previous invocations.

Output and results

When you run this command, the following items in the work directory are analyzed:
  • The server.conf.yaml file.
  • The content of any overrides/server.conf.yaml file is factored in.
  • Deployed resources in the server's run directory.
  • Policy projects in the server's run directory.

If the command completes successfully, then a server.components.yaml file is written in the server's work directory. A message similar to the following example is output to the console:

BIP15237I: The components file 'C:\Work\Development\V12\FileNodes\Server1\server.components.yaml' was created. It can be further edited if required.

The server.components.yaml file is different from to the server.conf.yaml file as it indicates only which of the main components need be created or loaded when the server starts-up. An example server.components.yaml file is shipped with the product in <product_install_directory>/server/sample/configuration/server.components.yaml. This sample file is supplied to show which components exist that can be enabled or disabled and it is not used directly at run time. No components are disabled in the sample, as all are commented out. The sample file has the following entries for the main embedded run-times:

  • JVM: true/false
  • NodeJS: true/false
  • CLR: true/false
These entries indicate whether the server creates a JVM, NodeJS and/or .NET CLR run-times at server startup. .NET CLR is available on Windows only and is needed to run the .NET function.
The next entries are subsections of component switches:
  • Nodes Message flow node functional areas that need the native run-time and for which product LILs are loaded from the <product_install_directory>/server/bin directory.
  • JavaNodes Message flow node functional areas that need the embedded JVM run-time and for which JAR/PAR files are loaded from the <product_install_directory>/server/bin directory. If JVM is set to false, then all of the entries in the JavaNodes section are ignored.
  • JavaScriptNodes Message flow node functional areas that need the embedded NodeJS run-time. If NodeJS is set to false, then all of the entries in the JavaScriptNodes section are ignored.
  • ResourceManagers A list of resource managers that can be chosen not to be created at startup.
  • FlowSecurityProviders A list of message flow security providers that might or might not be needed at server startup. The list is made up from LSL files that are loaded from the <product_install_directory>/server/bin directory.
When a server.components.yaml file is created by the ibmint optimize server command, it also contains the following three fields:
  • optimizeComponents
  • optimizeVersion
  • serverComponentsVersion
These fields contain the optimization version information and you must not change these fields.

Sample server.components.yaml file:


# ACE Integration Server components file
#
# Created at 12.0.4.0 - ea79f52d3cb39a25157204509f9e022f-3aea14dc5eb1f6d1bb87dd2db20873e9
#
# General notes :
#  - Integration Server will load server.components.yaml from directory set via --work-dir
#  - To ensure valid YAML avoid any use of TAB characters
#  - File paths may be taken as absolute, or relative to the integration server's work directory

serverComponentsVersion: 1
# reportExcludedComponents: true # Controls whether messages are written on integration server startup indicating components are excluded. Default is true.

# Main embedded run-times
# JVM: false      # Do not create a JVM within the Integration Server
# NodeJS: false   # Do not create NodeJS run-time within the Integration Server
# CLR: false      # Do not create a .NET CLR run-time within the Integration Server

Nodes:
#  CallableFlow: false  # Controls whether the callable flow nodes are loaded.
#  Aggregation: false   # Controls whether the aggregation nodes are loaded.
#  Basic: false         # Controls whether the basic nodes are loaded.
#  Connector: false     # Controls whether the connector node framework is loaded.
#  .NET: false          # Controls whether the .NET nodes are loaded.
#  JMSTransform: false  # Controls whether the MQJMSTransform and JMSMQTransform nodes are loaded.
#  MQ: false            # Controls whether the IBM MQ nodes are loaded.
#  ESQL: false          # Controls if the Compute, Filter, Database, DatabaseInput and Traces nodes are loaded.
#  SecurityPEP: false   # Controls whether the SecurityPEP node is loaded.
#  Group: false         # Controls whether the GroupScatter, GroupGather and GroupComplete nodes are loaded.
#  REST: false          # Controls whether the RESTRequest, RESTAsyncRequest and RESTAsyncResponse nodes are loaded.
#  Timer: false         # Controls whether the TimeoutControl and TimeoutNotification nodes are loaded.

JavaNodes:              # If JVM is set to false then the JavaNodes section will not apply
#  Adapters: false      # Controls whether the SAP, Oracle JD Edwards, Oracle Peoplesoft and Oracle Siebel nodes are loaded.
#  CICS: false          # Controls whether the CICSRequest node is loaded.
#  Collection: false    # Controls whether the Collector, Sequence and Resequence nodes are loaded.
#  Email: false         # Controls whether the EmailInput and EmailOutput nodes are loaded.
#  File: false          # Controls whether the File, IBM Sterling C:D and FTE nodes are loaded.
#  CORBA: false         # Controls whether the CORBARequest node is loaded.
#  IMS: false           # Controls whether the IMSRequest node is loaded.
#  Mapping: false       # Controls whether the Mapping node is loaded.
#  ODM: false           # Controls whether the ODMRules node is loaded.
#  TCPIP: false         # Controls whether the TCPIP nodes are loaded.
#  XSLT: false          # Controls whether the XSLTransform nodes are loaded.
#  WSRR: false          # Controls whether the WSRR EndpointLookup and RegistryLookup nodes are loaded (if this component is installed).
#  MQTT: false          # Controls whether the MQTT connector is loaded.
#  Kafka: false         # Controls whether the KAFKA connector is loaded.
#  JavaCompute: false   # Controls whether the JavaCompute node is loaded.
#  JMS: false           # Controls whether the JMS nodes are loaded.
#  JDBC: false          # Controls whether the DatabaseRetrieve and DatabaseRoute nodes are loaded.

JavaScriptNodes:        # If NodeJS is set to false then the JavaScriptNodes section will not apply
#  Salesforce: false    # Controls if the Salesforce connector is loaded.
#  LoopBack:   false    # Controls if the LoopBack connector is loaded.

ResourceManagers:
#  GlobalCache: false   # Controls whether the GlobalCache resources are loaded.

FlowSecurityProviders:  
#  LDAP: false          # Controls whether the LDAP Security provider is loaded for message flow security.
#  TFIM: false          # Controls whether the TFIM Security provider is loaded for message flow security.
#  WS-Trust: false      # Controls whether the WS-Trust v1.3 STS Security provider is loaded for message flow security.
#  Local: false         # Controls whether the Local Security provider is loaded for message flow security.
#  ENV: false           # Controls whether the Environment variable Security provider is loaded for message flow security.

The --verbose output:

The analysis of the work directory content can be displayed by using the --verbose parameter. The messages that are used in this output can be split into two main categories:
A) Messages that show what resources are being navigated to and analyzed:
  • BIP15243I: Analyzing server configuration.
  • BIP15244I: Analyzing application '&1'.
  • BIP15245I: Analyzing service '&1'.
  • BIP15246I: Analyzing REST API '&1'.
  • BIP15247I: Analyzing library '&1'.
  • BIP15248I: Analyzing shared library '&1'.
  • BIP15249I: Analyzing policy project '&1'.
  • BIP15250I: Analyzing message flow '&1'.
  • BIP15251I: Analyzing message flow node '&1'.
  • BIP15252I: Analyzing subflow instance '&1'.
  • BIP15253I: Analyzing policy '&1'.
B) Messages that indicate when a component is required based on their content:
  • BIP15254I: The component '&1' is required for message flow node '&2' of type '&3'.
  • BIP15255I: The component '&1' is required for deployed resource '&2'.
  • BIP15256I: The component '&1' is required for policy '&2' of type '&3'.
  • BIP15257I: The component '&1' is required for YAML configuration setting '&2' with value '&3.

What to do next

  • After the server.components.yaml is created, you can edit it directly to enable or disable components by setting them to true or false.
  • If you no longer want the server to be optimized, then you can delete the server.components.yaml file from the work directory. The server then starts up exactly as it did before you optimized it. Unlike server.conf.yaml file, the server.components.yaml file does not support a copy in the overrides directory.