Respond to IBM Z application events pattern

← Back to Application modernization patterns

This pattern is a specialization of the broader event sourcing pattern for the integration of two or more silos of applications through event-based loose coupling. In event-based loose coupling, an event is raised by an application and delivered by an event stream infrastructure to other interested applications to complete follow-up actions. In this specialized pattern for IBM Z application modernization, the source of an event is a core application that is based on IBM Z. The event is notified to one or more down-stream applications that run on IBM Z or on another platform, such as a public cloud.

Many organizations rely on core applications on IBM Z to run their business-as-usual operations. The system-of-record (SOR) data that those core applications maintain is shared across many other applications that support different lines of business (LOBs). Organizations face many challenges in quickly modifying core applications to respond to new marketplace needs, and key among them is avoiding business risks. Any changes to core SOR applications are done carefully, resulting in lengthy design, development, testing, and validation of changes, including all aspects of key service level agreements (SLAs), such as performance, security, availability, and data integrity. Small incremental changes are preferred over a risky and time-consuming rewrite.

However, organizations must also respond to marketplace needs with agile solutions. Many organizations need to develop new solutions with real-time interactions with customers. For example, banks need to notify customers through mobile apps when an account total is below a pre-specified level. Organizations also seek to automate many post-processing operations that follow on to an SOR transaction, such as the opening of a new account. New services are to be developed not only as cloud-native applications, but also integrated with ease by using modern capabilities such as APIs, and in a non-disruptive manner.

IBM Z provides several ways to extend the functions of core business applications and integrate in a scalable way with cloud-native applications. In addition to support for the integration of new or enhanced functions with a tight coupling approach, IBM Z provides capabilities to integrate new services through loose coupling by using the event sourcing pattern. In that approach, an application on IBM Z can raise an event, which can be then distributed with an event-streaming backbone to other interested applications, whether they run on platform, distributed, or public cloud. The event-based loose coupling is enabled by the following capabilities:

• An application that runs on IBM Z to analyze changes in data as part of the application logic that pertains to an event. The application on IBM Z interfaces with an event distribution system through a message or an outbound API.
• A Kafka connector or adapter that transforms the message or API request and publishes an event. An event backbone can run on, off, or partially across IBM Z to distribute this event.
• One or more downstream applications, which are developed to handle new response logic in a flexible way without introducing risks in the core IBM Z application. The new response handling applications can be deployed anywhere, either on IBM Z or off-platform.

For example, a bank notifies account holders with a low-balance alert by using messaging-based integration with the event distribution system. In this example, the customer account balance data is stored in IBM® DB2® and CICS® handles the transaction logic. Assume that the process that changes the balance to one that is below the customer-specified set limit already occurred. The reconciliation of the new balance can happen either as a transactional application or as a batch job.

The sequence of actions to start the events are as follows:

The key business benefits of using this pattern in developing new solutions are as follows:

• Avoiding any changes to the core application and avoiding any business risks of disruptions.
• Developing agile solutions with loose coupling to quickly respond to changing marketplace needs.
• Modernizing solutions with newly developed logic that run as cloud-native applications, and gaining all the benefits of cloud-based development and deployment of new applications.
• Using application integration mechanisms with ease-of-development and a robust solution.

When you choose the deployment topology for the event distribution architecture and integration with core applications on IBM Z, you need to make two sets of considerations.

Deployment of event distribution system (brokers)

A key part of event driven architectures is the event backbone. This backbone deployment configuration impacts performance, uptime and resilience guarantees, SLAs, and more. Kafka-based back ends are typically on distributed platforms on premises, such as Confluent Platform and IBM Event Streams, or in the cloud such as Confluent Cloud. However, the trend is to move the brokers closer to the producers and consumers to reduce latency. For more information, see the Collocate applications pattern. Event back ends (Kafka brokers) can run on IBM Z or IBM® LinuxONE to achieve this goal. In that case, the Kafka brokers run through an enterprise grade platform (Confluent Platform or IBM Event Streams) on Linux® on IFL engines (IBM Z or LinuxONE). IFL engines, such as zIIPs, do not have consumption-based pricing, so the compute resources can be used as much as required without incurring MIPS increase. Brokers that are colocated with connectors can also benefit from latency reduction due to in-memory communication. In addition to reduced latency, IBM Z and LinuxONE offer improvements in throughput that range from a modest double-digit percent up to a few times higher than broker performance on distributed systems.

Messaging versus API-based integration

When you choose the approach to integration with core applications on IBM Z, you need to consider ease of integration, message delivery semantics, latency, and persistence.

IBM z/OS Connect EE provides a way to extend an IBM Z application for calling an external API, and in this case, to the event distribution system whether it is deployed on IBM Z or off-platform. It provides a graphical tool to generate artifacts for calling an API and supports a scalable and highly available runtime deployment. Many organizations have already adopted the OpenAPI standards-based integration of IBM Z applications. Also, from an application integration perspective, API provides higher-level semantics that are easier to maintain. The call by the application is synchronous, and it relies on the event distribution system to ensure delivery and persistence of such requests.

However, some applications might still rely on an older messaging-based integration and might develop or deploy messages through the messaging system. In such scenarios, you can continue with the messaging-based integration or evolve to a modern API-based integration.

This messaging method is popular given the ubiquitous nature of the adoption of IBM MQ in enterprise architecture. Architectures that are based on IBM MQ can incur more latency from the addition of a queue and more cost from compute consumption on IBM Z, but they offer the highest levels of delivery guarantees. Messaging-based mechanisms are also desirable when “exactly once” semantics are required, although for loosely coupled idempotent microservices, these guarantees are rarely required.

• Review these patterns:
  • Extend with cloud native
  • Collocate applications
  • Respond to external events
  • Optimize CQRS
  • Enterprise DevOps
• Read broader  Event-driven patterns on the IBM Cloud Architecture site.
• Read IBM’s point-of-view on  Event-driven architecture.
• Read these resources:
  • Martin Fowler’s article on event sourcing
  • Chris Richardson’s article on the event sourcing pattern
• Watch  Greg Young on event sourcing at the GOTO Conference.
• Review an implementation of event sourcing: All the microservices in the  Reefer container shipment solution are using event sourcing. The solution uses Kafka with long persistence. The order management service uses CQRS with event sourcing. An integration test validates the pattern.
• Read IBM MQ integration with Kafka:  IBM MQ in the context of EDA.