IBM Integration Bus considerations for GDPR readiness

Data lifecycle

IBM Integration Bus is a general-purpose integration engine which enables users to route and transform data as it is passed between third-party applications. IBM Integration Bus supports a large range of protocols and data formats for the purpose of connecting to bespoke applications, and provides prebuilt components that are capable of communicating with popular packaged applications. As such, IBM Integration Bus touches many forms of data, some of which could potentially be subject to GDPR. Most frequently, data passes through the IBM Integration Bus architecture in real time, with IBM Integration Bus making synchronous connections to online endpoints. However, IBM Integration Bus also interacts with persistent forms of data such as messaging systems (both traditional on-premises message systems such as IBM MQ and Kafka, and cloud messaging providers such as IBM Event Streams), databases (relational databases and NoSQL databases), data held on local or remote file systems, email systems, and other CRM and ERP systems.

There are several third-party products with which IBM Integration Bus might exchange data. Some of these are IBM-owned, but many others are provided by other technology suppliers. IBM provides links to Software Product Compatibility Reports (SPCR) that list the associated software; for example, the SPCR page for IIB v10 on Windows. For organizations considering a third-party product to support their GDPR readiness, consult that product's documentation.

IBM Integration Bus users control the way in which it interacts with data passing through it, by the definition of message flows. A message flow is commonly constructed by a user acting in the role of IBM Integration Bus developer, working with the IBM Integration Toolkit. A message flow is composed from a set of discrete building blocks (known as message flow nodes) that are wired together in an ordered fashion by the IBM Integration Bus developer. Message flow nodes are configured graphically, and in some cases also allow the IBM Integration Bus developer to drop into "code" (Java™, ESQL, .NET).

What types of data flow through IBM Integration Bus?

As a general-purpose integration engine, there is no one definitive answer to this question because use cases vary from user to user. However, it is entirely possible that customers of IBM Integration Bus use it to interact with data that relates to the following categories:

• Employees of the customer (for example, IBM Integration Bus might be used to connect the customer's payroll or HR systems)
• The customer's own clients' personal data (for example, IBM Integration Bus might be used by a customer to transform data related to their clients, such as taking sales leads and storing data inside their CRM system)
• The customer's own clients' sensitive personal data (for example, IBM Integration Bus might be used within industry contexts that require personal data to be transmitted through IBM Integration Bus, such as HL7-based healthcare records when integrating clinical applications.

Personal data used for online contact with IBM

IBM Integration Bus clients can submit online comments/feedback/requests to contact IBM about IBM Integration Bus subjects in a variety of ways, primarily:

• Public comments area on pages in the IBM Integration community
• Public comments area on pages of IBM Integration Bus online product documentation
• Feedback forms in the IBM Integration community

Typically, only the client name and email address are used, to enable personal replies for the subject of the contact, and the use of personal data conforms to the IBM Online Privacy Statement.

Data collection

IBM Integration Bus can be used to collect personal data. When assessing your use of IBM Integration Bus and your need to meet with GDPR requirements, you should consider the types of personal data which in your circumstances are passing through IBM Integration Bus. You may wish to consider aspects such as:

• How does data arrive into your IBM Integration Bus message flows (synchronously/asynchronously? Across which protocols? Is the data encrypted? Is the data signed?)?

• How is data sent out from your IBM Integration Bus message flows (synchronously/asynchronously? Across which protocols? Is the data encrypted? Is the data signed?)?

• How is data stored as it passes through your IBM Integration Bus message flows (Do you use any of the product features highlighted in the Data lifecycle section of this document? If so, are you aware of how those features could potentially expose aspects of the data passing through the product?)?

• How are credentials collected and stored where needed by IBM Integration Bus to access third-party applications ?

  IBM Integration Bus typically needs to communicate with third-party applications and systems, such as DB2® and SAP, and many such systems require IBM Integration Bus to authenticate. Where needed, authentication data (user IDs, passwords, and API keys) is collected and stored by IBM Integration Bus for its use in such communication.

  Wherever possible, you should avoid using personal credentials for IBM Integration Bus authentication, to limit the chances of your personal credentials being used by message flows which were not your original intention. In any case, consider the protection of the storage used for authentication data. (See Data Storage below.)

• What test data do you want to collect during development?

  IBM Integration Bus provides a Flow Exerciser feature, which is used during unit testing to record on disk the logical tree structure that represents data structures that are normally held in memory of the product as the data is flowing through IBM Integration Bus.

  For more information, see Testing and troubleshooting message flows.

• When using the IBM Integration Toolkit to discover information about third-party interfaces, is sensitive data found and persisted?

  When developing new message flows, the product provides discovery capabilities that allow it to connect to third-party systems so that it can retrieve metadata about interfaces to those systems, which then enable IBM Integration Bus to construct data in the correct formats ready for exchange with those systems. For example, IBM Integration Bus provides discovery capabilities for SAP (to discover BAPI and IDoc structures), REST APIs, and WSDL-based web services. It is possible that during this build-time discovery phase, sensitive data could be discovered and persisted as part of these operations.

• When using IBM Integration Bus, how can you avoid sharing personal or sensitive information within build-time message flow artifacts?

  When configuring particular message flow nodes, a developer using IBM Integration Bus might want to pass a property that they consider to be personal or sensitive information. The IBM Integration Bus product has catered for the most obvious examples of this kind of data by providing some nodes with properties to express a security identity. The security identity provides a method of abstracting a reference to a user ID/password combination that is then defined to a runtime IBM Integration Bus system using the mqsisetdbparms command. This approach avoids a developer having to expose this kind of sensitive information in version control systems and in BAR files that might be considered unnecessarily exposing security risks to a larger population of employees within the client's context. For other cases, IBM Integration Bus provides the ability to override message flow node properties on a message by message basis at runtime, using the LocalEnvironment tree structure. IBM Integration Bus also provides the option of using User Defined Properties which can be passed in to message flows and avoid personal data being carried on the build-time flow artifact. Similarly, user-defined configurable services might be of interest for the same reason.

Data storage

When data travels through IBM Integration Bus message flows as part of normal operation, IBM Integration Bus does not mandatorily persist that data directly to stateful media. However, users of IBM Integration Bus are given entitlement to the IBM MQ messaging product, which can be used to define persistent queues that are designed to hold messages for exchange in an asynchronous manner between applications. So, although IBM Integration Bus does not directly persist data to stateful stores, by association IBM Integration Bus users may want to consider securing data at rest that is written to input or output queues.

The following items highlight areas where IBM Integration Bus does require data storage mechanisms, which users may wish to consider when planning for their GDPR readiness.

• Aggregation and Collector nodes

  IBM Integration Bus provides Aggregation and Collector nodes that are designed to deal with scenarios where multiple third-party systems might send data in to IBM Integration Bus to be combined into a new format. Aggregation typically involves a fan-out flow and a fan-in flow. These flows exchange metadata so that IBM Integration Bus knows how many messages have been fanned out to external systems, so that when response messages come back into IBM Integration Bus it knows when all replies have been received, or when to stop waiting for replies. In the interim before all replies have been received, IBM Integration Bus needs to store the messages that have returned. This storage of messages is so that in the unlikely event that IBM Integration Bus is stopped (either deliberately or due to hardware failure of some form), when it is restarted the messages can be recovered from the storage location. IBM Integration Bus uses an IBM MQ queue manager to store this data in queues.

• Record and Replay

  If you need an audit record of data passing through the flows deployed to an IBM Integration Bus integration node, you can use the record and replay feature to record messages to a database, and then view them and replay them if needed. You might want to record messages for audit and security purposes, or if you want to keep a history of the data for development and test purposes, or to help with problem determination. You can configure bespoke settings in a message flow to specify which data should be recorded (including which messages, which fields from within them, and using XPath expressions you can select messages which meet particular criteria). The data is stored in a database, which can be an Oracle, Microsoft SQL Server, or DB2 database. After you have recorded data, you can view it using the IBM Integration Bus web user interface, including seeing a list of recorded messages or the details of a specific message. You can also view recorded data by using the IBM Integration API or the IBM Integration Bus Representational State Transfer (REST) API. You can replay a message to an MQ queue by using the web interface or IBM Integration API. You can also replay the message to another message flow for testing or problem determination.

  For more information, see Recording, viewing, and replaying data.

• Business transaction monitoring

  You can use business transaction monitoring to monitor a message across multiple message flows, so you can track and report the lifecycle of a payload message through an end-to-end enterprise transaction. The IBM Integration Bus web user interface is used to create a business transaction definition to identify the message flows that form the business transaction, and define how they interact with each other. You can view the monitoring events that are defined for the message flows that make up the business transaction. Some of these events are significant to the business transaction. The business transaction definition defines how these events apply to the business transaction. Existing monitoring events (the same monitoring events that drive the record and replay feature) can be selected as start, end, and failure events in the business transaction to show how the transaction is progressing. You can be selective - not all monitoring events need to be flagged as business transaction events.

• Data storage in databases

  Many IBM Integration Bus users might choose to have the product persistently store data inside relational databases, or NoSQL databases. IBM Integration Bus provides JDBC and ODBC connectivity to the relational databases, and a LoopBackRequest node which assists with NoSQL databases.

  For more information, see Working with databases.

• User and service trace

  IBM Integration Bus provides user and service trace features, which record the internal IBM Integration Bus code paths through which the data flows. Users can decide to take snapshots of what the data looks like passing through IBM Integration Bus. IBM Integration Bus also provides real-time interfaces such as a graphical debugger view in the IBM Integration Toolkit, which presents this snapshot data to theIBM Integration Bus developer.

  User trace and service trace can include entries made by parsers and protocols including data from customer messages passing through IBM Integration Bus. This information is only discovered when trace is turned on.

  To protect access to user trace and service trace, consider the following actions:

  • Use file-level or volume-level encryption to protect the contents of the directory that is used to store trace logs.
  • Remove personal data from trace information. When IBM Integration Bus generates trace logs, a user runs the mqsireadlog command to capture the information in an XML format and then runs the mqsiformatlog command to render it in human-readable format. Users may wish to remove personal data from one or both of these files.
  • After uploading service trace to IBM, you can delete your service trace files if you are concerned about the contents potentially containing personal data.

  For more information, see Trace, overview and Using trace, tasks.

• Authentication data needed for IBM Integration Bus to connect to third-party applications and systems

  IBM Integration Bus typically needs to communicate with third-party applications and systems, such as DB2 and SAP, and many such systems require IBM Integration Bus to authenticate. Where needed, authentication data (user IDs, passwords, and API keys) is collected and stored by IBM Integration Bus for its use in such communication.

  An IBM Integration Bus administrator configures IBM Integration Bus with authentication data, which is then stored in the product configuration directory (for example, C:\ProgramData\IBM\MQSI on Windows).

  When IBM Integration Bus is installed, the production configuration directory is set up with group-based access control such that IBM Integration Bus can read the configuration files and use the passwords to connect to these systems. IBM Integration Bus administrators are also members of this group so have read access to the files. The files are obfuscated using a private algorithm but they are not encrypted. For this reason, to fully protect access to passwords, you should consider the following actions:

  • Use file-level or volume-level encryption to protect the contents of the configuration directory
  • Ensure that only administrators are members of the group that has read access to the product configuration directory and that no other access is allowed.
  • Encrypt backups of the production configuration directory and store them with appropriate access controls.

  For more information, see Administration security overview.

Data access

IBM Integration Bus data can be accessed through the following defined set of product interfaces, some of which are designed for access through a remote connection, and others for access through a local connection.

• Web user interface (remote only)
• RESTful API (remote only)
• Java Integration API (local and remote)
• IBM Integration Toolkit (local and remote)
• IBM Integration Bus commands (local, and, for some commands, remote)

These interfaces are designed to allow users to make administrative changes to your IBM Integration Bus installation. Administration can be secured, such that there are three logical, ordered stages involved when a request is made - authentication, role mapping, and authorization.

Authentication

If the administrative action was requested from a local connection, then the source of this connection is a running process on the same system. The user running this process had to have been authenticated by the operating system at some point. Therefore, for local connections, the user name of the owner that runs the process from which the connection was made is taken to be a pre-authenticated user. IBM Integration Bus does not perform any additional authentication steps in this case. For example, this could be the name of the user running the shell from which an IBM Integration Bus command has been issued.

If the administrative action was requested from a remote connection, then communications with IBM Integration Bus are made through an HTTPS interface (whether it originates from the IBM Integration Bus Web Admin UI, the IBM Integration Toolkit, or other applications using the IBM Integration Bus REST API). The user name and password which are provided by the user are checked against settings held in the local registry for IBM Integration Bus. It is also possible to configure IBM Integration Bus to pass the user name and password to an LDAP registry for authentication.

Role mapping

In the role mapping stage, the user name that was provided in the authentication stage is mapped to a system ID. This system ID is then used when authorizing which administrative activities are allowed to be carried out by the authenticated user.

Authorization

IBM Integration Bus Version 10.0 removed the mandatory product prerequisite of IBM MQ. In releases before Version 10.0, authorizations were configured using MQ queues. In Version 10.0, the option to define authorizations using MQ is maintained in addition to the provision of a new file-based alternative that was introduced.

IBM Integration Bus defines a list of administrative actions that a user can perform. To provide a user with authorization for each action, specific read/write/execute permissions are required on documented MQ SYSTEM queues (or files if you have opted to use the file-based alternative). These permissions must be set to allow the user to complete these administrative tasks when administration security is enabled. For a list of the administrative actions and the permissions that are needed, see Tasks and authorizations for administration security

Logging administration activity

Some users of IBM Integration Bus would like to create an audit record of administrative accesses to the product. Examples of desirable audit logs might include configuration changes that contain information about the change in addition to who requested it.

The following sources of information are available out of the box in the context of this requirement:

1. IBM Integration Bus writes messages into syslog for configuration changes (BIP2152/2153/2154/2155).
2. IBM Integration Bus publishes notification messages on the following topics:
   • Configuration changes ($SYS/Broker/IntegrationNodeName/Configuration)
   • Operational state changes ($SYS/Broker/IntegrationNodeName/Status)
3. IBM Integration Bus provides an API to retrieve the administration log, which can also be seen the web admin user interface.

Items (1) and (2) do not include the name of the user that requested the change, but (3) does. When considering these kind of solutions, users might want to give consideration to the following points:

• The administration log information is not persisted anywhere so that, when the integration node restarts, the information is lost. Any implementation using the Integration API should therefore poll the log regularly and transfer the entries to more persistent storage.
• The application extracting the information from the log should run under a different user than the system ID for IBM Integration Bus administrators. Otherwise an administrator would be able to deactivate the program (intentionally or by accident), which would lead to administrative actions not being captured.
• IBM Integration Bus Administrators have sufficient privileges to clear the administration log.

Data processing

Users of IBM Integration Bus can control the way in which the product processes personal data, through the definition and configuration of the message flows that are deployed to the IBM Integration Bus runtime. Message flows begin processing when input data arrives into IBM Integration Bus through an input node, and they complete when data is sent out from an IBM Integration Bus output node or request node. A large range of protocols are supported, some of which include provision for the data to be encrypted when it is passed into and sent out from IBM Integration Bus. Encryption provides a method by which the data is converted from a readable form to an encoded version that can only be decoded by another program if it has access to a decryption key.

Encryption using a Public Key Infrastructure (PKI)

You can secure message flows by configuring nodes within the message flows to use SSL and certificates. After you establish a PKI configuration for your whole integration node or for some of its integration servers, you can use the configuration within your message flows. For information about establishing a public key infrastructure, see Setting up a public key infrastructure. When you configure message flow nodes that make or accept TCP connections, you can set SSL encryption and certificates on those connections.

For more information about using SSL and certificates to secure your message flows, refer to the following topics:

• Configuring the integration node to use SSL with JMS nodes
• Configuring SOAPInput and SOAPReply nodes to use SSL (HTTPS)
• Configuring SOAPRequest and SOAPAsyncRequest nodes to use SSL (HTTPS)
• Configuring HTTPInput and HTTPReply nodes to use SSL (HTTPS)
• Configuring the HTTPRequest and HTTPAsyncRequest nodes to use SSL (HTTPS)
• Securing the connection to CICS® Transaction Server for z/OS by using SSL
• Securing the connection to IMS by using SSL
• SSL and the TCP/IP nodes

Using the PKI security facilities that are provided by transport mechanisms is the first step towards securing data processing with IBM Integration Bus. However, without enabling furtherIBM Integration Bus security features, the behavior of the IBM Integration Bus runtime is to process all messages that are delivered to it, using the integration node service identity as a proxy identity for all message instances. In these circumstances, any identity that is present in the incoming message is ignored.

Security profiles and the IBM Integration Bus security manager

The next step to securing IBM Integration Bus data processing is to use an IBM Integration Bus security profile. You can use a security profile to configure an integration node for processing end-to-end an identity that is carried in a message through a message flow. By creating a security profile, you can configure security for a message flow to control access based on the identity that is associated with the message, and this provides a security mechanism that is independent of both the transport type and message format.

Only a subset of the connectors available in IBM Integration Bus use security profiles to control and vary the identity that is used when the connector interacts with an external system. For other connectors, a fixed identity can be specified, which is used to authorize access to the external system. For those connectors, the integration node has its own repository of identities, which can be updated with the mqsisetdbparms command. This command is covered elsewhere in this topic.

Security profiles are used by the IBM Integration Bus security manager component which enables the integration node to perform the following activities:

• Extract the identity from an inbound message
• Authenticate the identity (by using an external security provider)
• Map the identity to an alternative identity (by using an external security provider)
• Check that either the alternative identity or the original identity is authorized to access the message flow (by using an external security provider)
• Propagate either the alternative identity or the original identity with an outbound message.

Security profiles are created by the integration node administrator and accessed by the security manager at runtime. The following external security providers (also known as Policy Decision Points or PDPs) are supported:

• Tivoli® Federated Identity Manager (TFIM) V6.1 for authentication, mapping, and authorization
• WS-Trust V1.3 compliant Security Token Service (including TFIM V6.2) for authentication, mapping, and authorization
• Lightweight Directory Access Protocol (LDAP) for authentication and authorization
• Windows Domain Controllers for authentication
• Kerberos KDCs for authentication

You can invoke message flow security by configuring either a security enabled input node or a SecurityPEP node. You can use the SecurityPEP node to invoke the message flow security manager at any point in the message flow between an input node and an output (or request) node. For an overview of the sequence of events that occur when the message flow security manager is invoked by using either a security enabled input node or a SecurityPEP node, see the following topics:

• Invoking message flow security using a security enabled input node
• Invoking message flow security using a SecurityPEP node

The following input nodes support message flow security:

• MQInput
• HTTPInput
• SCAInput
• SCAAsyncResponse
• SOAPInput

WS-Security and IBM Integration Bus policy sets

The WS-Security specification provides mechanisms for securing web services at the message level including identity, authentication, authorization, integrity, confidentiality, nonrepudiation, and basic message exchange. Both traditional web security and message-level security share many of the same mechanisms for handling security, including digital certificates, encryption, and digital signatures. These details can be provided to IBM Integration Bus by using policy sets and bindings. Policy sets and bindings define how to configure your WS-Security (and WS-RM) requirements for several IBM Integration Bus nodes including the SOAPInput, SOAPReply, SOAPRequest, SOAPAsyncRequest, and SCAAsyncResponse nodes. A policy set is a container for the WS-Security and WS-RM policy types. A policy set binding is associated with a policy set and contains information that is specific to the environment and platform, such as information about keys. You can use policy sets and bindings to define the following items for both request and response SOAP messages:

• Authentication for the following tokens:
  • User name tokens (requires a security profile to specify the external security provider)
  • X.509 certificates (requires the integration node keystore and truststore, or a security profile to specify the external security provider)
  • SAML assertions, using SAML 1.1 or 2.0 pass-through (requires a security profile to specify the external security provider)
  • LTPA tokens, using LTPA pass-through (requires a security profile to specify the external security provider)
• Asymmetric encryption (confidentiality) using X.509 certificates (requires the integration node keystore and truststore)
• Symmetric encryption (confidentiality) using Kerberos tokens (requires the host to be configured for Kerberos)
• Asymmetric signature (integrity) (requires the integration node keystore and truststore)

Either the whole SOAP message body, or specific parts of the SOAP message header and body can be encrypted and signed.

Data deletion

IBM Integration Bus provides commands and user interface actions to delete data which has been provided to the product. This enables users with facilities to delete data which relates to particular individuals, should this be required.

• Areas ofIBM Integration Bus behavior to consider for GDPR client data deletion:

  • You can delete data stored by the Aggregation nodes by removing data from the MQ queues named SYSTEM.BROKER.AGGR.*
  • You can delete data stored by the Collector and Resequence nodes by removing data from the MQ queues named SYSTEM.BROKER.EDA.*
  • You can delete data stored by the record and replay function by dropping it from your database. The DataCaptureSchema.sql file provided in the IBM Integration Bus installation directory (install_dir\server\ddl\db2) names the specific tables involved.
  • You can delete data stored by the Business Transaction Monitoring function by dropping it from your database. The DataCapture policy contains the database details.
  • You can delete data stored by the User and Service trace commands by deleting the files in your common log directory (for example by default on Windows this is at C:\ProgramData\IBM\MQSI\common\log)

• Areas of IBM Integration Bus behavior to consider for GDPR account data deletion:

  • You can delete account data stored by IBM Integration Bus for connecting to third-party applications using the mqsisetdbparms command command
  • You can delete account data stored by IBM Integration Bus for authorizing IIB administration using themqsiwebuseradmin command command

Data monitoring

• IBM Integration Bus provides a monitoring feature that users can exploit to publish copies of the data passing through a flow over MQ or MQTT transports.

  For more information, see Monitoring.

• As part of standard operation, IBM Integration Bus records information in the system log of the platform where it is installed (for example, Windows Event Viewer) to describe informational, warning and error information. It is possible in some circumstances that the diagnostic information provided in these entries will include extracted portions of data from individual messages that have passed through IBM Integration Bus. For example, if an input message is passed into IBM Integration Bus and a parsing exception is thrown, IBM Integration Bus would inform the user where the parsing failed, with a byte location and an extract of the data which was found immediately prior to the parsing failure. This is designed to help users interpret and correct the problem (either by correcting the data itself, or the message model used to parse it)

• Activity logs provide immediate, basic information about what is happening in your message flows, and how they are interacting with external resources. The logs are also useful for diagnosing problems that would not be easily resolved by using lower-level trace. For example, they can help you to understand why your message flow is not processing any messages from a remote resource. Activity logging is enabled by default. To view activity log you can use the web user interface, or if wanting to see information over a longer period you can configure IBM Integration Bus to write the same data to a file. Some users may prefer to control access to this information by applying authorization rules for users logging in to the IBM Integration Bus web user interface.

  For more information, see Using activity logs.

Capability for restricting use of personal data

Using the facilities summarized in this document, IBM Integration Bus enables a user to restrict usage of their personal data.

Under GDPR, users have rights to Access, Modify and Restrict Processing. Refer to other sections of this document to control the following:

• Right to Access
  • IBM Integration Bus administrators can use IBM Integration Bus features to provide individuals access to their data.
  • IBM Integration Bus administrators can use IBM Integration Bus features to provide individuals information about what data IBM Integration Bus holds about the individual.
• Right to Modify
  • IBM Integration Bus administrators can use IBM Integration Bus features to allow an individual to modify or correct their data.
  • IBM Integration Bus administrators can use IBM Integration Bus features to correct an individual's data for them.
• Right to Restrict Processing
  • IBM Integration Bus administrators can use IBM Integration Bus features to stop processing an individual's data.