Respond to external events pattern

Share events that are generated by applications that are external to IBM Z® to drive the invocation of IBM Z application logic. Develop flexible logic without introducing risks in core applications.

Overview

← Back to Application modernization patterns

The “Respond to external events” pattern is a specialization of the broader pattern for the integration of two or more silos of applications through event-based loose coupling. In event-based loose coupling, an event is raised by an application and delivered through an event stream infrastructure to other interested applications to complete follow-up actions.

In this pattern, the event source is an external application and responding to this event logic uses one or more core applications on IBM Z. This pattern is almost an inverse of the “Respond to IBM Z application events” pattern, where an IBM Z application is the source of an event.

Many organizations rely on core applications on IBM Z to run their business-as-usual operations. The system-of-record (SOR) data that those core applications maintain is shared across many other applications that support different lines of business (LOBs). Organizations face many challenges in quickly modifying core applications to respond to new marketplace needs, and key among them is avoiding business risks. Any changes to core SOR applications are done carefully, resulting in lengthy design, development, testing, and validation of changes, including all aspects of key service level agreements (SLAs), such as performance, security, availability, and data integrity. Small incremental changes are preferred over a risky and time-consuming rewrite.

However, organizations must also respond to marketplace needs with agile solutions. Many organizations need to develop new solutions with real-time interactions with customers. For example, an airline needs to provide a way for customers to reschedule flights due to weather constraints or personal decisions. The airline customers can submit a reschedule request to the airline, which the new applications respond to. New services are to be developed not only as cloud-native applications, but also integrated with ease by using modern capabilities such as APIs, and in a non-disruptive manner.

Solution and pattern for IBM Z®

IBM Z provides several ways to extend the functions of core business applications and integrate in a scalable way with cloud-native applications. In addition to support for the integration of new or enhanced functions with a tight coupling approach, IBM Z provides capabilities to integrate new services through loose coupling, where new intelligent application logic can handle the “fire and forget” style of asynchronous external requests.

Figure 1. Tight coupling: Integrating applications by using APIs with new cloud or distributed applications

In contrast, in the loose coupling approach, an external application that is distributed or public-cloud based can raise an event that can be distributed with an event-streaming backbone to other applications, including to applications that run on IBM Z. Typically, new application logic is developed to interpret such a high-level request and respond to the event. The new application logic in turn uses the core applications on IBM Z as part of its response logic by using APIs that are enabled by IBM® z/OS® Connect EE.

 

Figure 2. Loose coupling: Including event-driven architecture with the application architecture

According to this pattern, event-based loose coupling is enabled by these capabilities:

  • An external application that runs on a distributed platform or on a public cloud analyzes changes in data as part of the application logic that pertains to an event.
  • The application publishes an event to the event distribution system by using an API or a connector. Those details aren’t shown in the diagram. This request is a “fire and forget” style request.
  • An event backbone can run on, off, or partially across IBM Z to distribute this event.
  • One or more downstream applications handle new response logic in a flexible way without introducing any risks to the core IBM Z applications. The new response-handling business logic can be deployed anywhere, preferably on IBM Z to colocate with core applications that the response logic uses.
  • The new business logic uses APIs to access core applications on IBM Z.

Figure 3. Flight application example

In the flight application example, a solution allows people to opt in to a feature that informs them of a cheaper flight. This action is typically done when a passenger books a flight but is flexible in the time scheduling and wants a lower price, if available. Because no hard requirements exist around response times, this use case is ideal for loosely coupled additions of new business logic to enhance solutions without incurring risks. In this example, flight data is stored in IBM® Db2® and a CICS® application handles the flight pricing logic. Other databases and business logic components are needed, but this example is a scaled-down representation to illustrate this pattern. Figure 3 shows how a sample flight booking application might look without an event-based architecture and without the new capability that was mentioned earlier in this example.

More services are introduced with an event-based loose coupling approach. This sequence of actions, which is shown in Figure 4, is as follows:

  1. The user selects a date and time range and price limit from the booking front end. The use might use a web-based or mobile app or even an airport kiosk. In event-driven architecture, this component is the equivalent of a source, which is called a producer in Kafka.

  2. This request is sent to the backend through the event backbone. The event backbone is typically an Apache Kafka based platform such as IBM® Event StreamsConfluent Platform, or Red Hat® AMQ Streams. The event data can include customer information and preferences.

  3. New flight scheduling business logic is written in a modern language such as Java®, JavaScript, or Go. This new loosely coupled logic can coexist with COBOL or CICS logic and doesn’t disrupt the existing environment, which is essential for a flight reservation system. The colocation pattern describes the optimal deployment for a new workload that optimizes for SLAs, total cost of ownership (TCO), and so forth.

  4. A Kafka connector such as a Confluent REST Connector or REST proxy can accept the events from the backbone and forward them to the Java business logic. The Java business logic in turn communicates with the COBOL flight pricing logic by using z/OS Connect EE. Both the Kafka Connector and z/OS Connect EE are zIIP eligible, which enables cost-efficient, loosely coupled integration with Kafka. While applications can consume Kafka directly by using the Kafka client libraries, in general, the use of a connector provides a consistent and scalable access point into the event backbone.

  5. Communication in this example happens with the JSON data format end-to-end and can be optimized or configured and governed by the Schema Registry.

Figure 4. Event-driven implementation for the new service

Advantages

The key business benefits of using this pattern to develop solutions are as follows:

  • Avoid any changes to the core application and avoid any business risks of disruptions

  • Develop agile solutions with loose coupling to quickly respond to changing marketplace needs and use core applications

  • Modernize solutions with new logic that runs as cloud-native applications and gain all the benefits of cloud-based development and deployment of new applications

  • Use OpenAPI that is based on z/OS Connect EE to invoke core applications on IBM Z

Considerations

When you choose the right deployment topology for an event-distribution architecture and for integration with core applications on IBM Z, you need to consider several factors.

A key part of event-driven architectures is the event backbone itself. The backbone deployment configuration impacts performance, uptime and resilience guarantees, SLAs, and more. While Kafka-based backends typically are on distributed platforms on premises (Confluent PlatformIBM Event Streams, or Red Hat AMQ Streams) or in the cloud (Confluent Cloud), the trend is to move the brokers closer to the producers and consumers to reduce latency. To achieve that goal, event backends (Kafka brokers) can run on IBM Z or IBM® LinuxONE. In that case, the Kafka brokers run by using an enterprise-grade platform (Confluent Platform or IBM Event Streams) on Linux® on IFL engines (IBM Z or LinuxONE). IFL engines such as zIIPs don’t have consumption-based pricing, so you can use the compute resources as much as you need without incurring a MIPS increase. Brokers that are colocated with connectors can benefit from latency reduction due to in-memory communication. In addition to reduced latency, IBM Z and LinuxONE offer improvements in throughput that range from a modest double-digit percent up to a few times higher than broker performance on distributed systems.

Connectors offer a standard mechanism for sources or sinks in event-driven architecture. They abstract away middleware-specific differences, such as message queues versus databases, and provide a standard mechanism for integration with event backbones. While application developers can use native event backbone client access libraries, the use of connectors ensures consistency. Connectors include concepts such as proxies and forwarders, and they might not be a separate tool depending on your event backbone vendor. Vendors typically provide service and support for connectors, which might not be provided for open-source client libraries alone.

What's next

Contributors

Elton DeSouza
IBM Systems IBM

Asit Dan
z Services API Management, Chief Architect, IBM Master Inventor, IBM Systems IBM