Delivering at scale

This series of articles was contributed and developed along with Alyssa Small and Brian Adams

This series of posts details how IBM iX designed, developed, and delivered the Facebook Messenger Bot available at the The Championships, Wimbledon 2018.  The series is broken into three pieces:

• Part I: This article will focus on the purpose of the Bot, our high-level approach, and how we developed the integration with the Facebook Messenger Platform.

• Part II: This piece will focus more on the broadcast integration within Facebook and how we persisted user preferences using IBM Cloudant and Compose for Redis.

• Part III: Finally, we will review our integrations with on-site systems at the All England Club and how we used Multi-Region within IBM Cloud to ensure scale and availability.

In Part I and Part II of this series, we detailed how we set up our liberty runtime to be able to communicate with the Facebook platform and how we used Cloudant to store user subscriptions. We also discussed how we used Watson Assistant to identify user intents. In the final installation of this series, we are going to discuss how we sent alerts from our on-site systems to Facebook and how we architected the solution for delivery at scale with resiliency.

Integration with on-site systems: Sending player scoring alerts

Data collection at the Wimbledon Championships is performed by highly trained data collectors on every court. This data is then distributed to multiple TV broadcasters and international consumers as well as the IBM iX scoring system. This system ingests the data and produces the various outputs needed for all the digital platforms in support of Wimbledon.

One of the features we have offered is notifications on the viewers’ mobile devices through our mobile apps. Wimbledon fans can favorite players and receive scoring notifications for them throughout The Championships. As players walk on to the court, warm up, and complete sets and the match, we send notifications from our on-site scoring system to the viewers’ devices, allowing them to stay completely up to date on the action.

We wanted to utilize this functionality on our Facebook chat bot activation as well.

How we send alerts

Our alerting system is agnostic to the source of the data it sends and how the data actually is transmitted to the end device. At Wimbledon, we make use of IBM Watson Campaign Automation to send push notifications to both iOS and Android devices.

The Event Notification System makes use of the Command Pattern. We have a standard interface which details that a command should implement an “execute” method, and it’s up to the implementing classes to decide how they integrate with the appropriate target end point. The system uses a configuration document to detail which classes should be invoked for each operation.

For Facebook, we created a FacebookBroadcast command and added that into our configuration. This would then POST to our Liberty runtime (along with an access token) and request for a given scoring notification to be sent. That would then run through the Cloudant view, identify the right label id, and create and send a broadcast to the users.

Delivering at scale with resiliency

With only a fortnight to attract and maintain our audience, we have to ensure our delivery is available and can handle any traffic volumes we could expect. We approach our solutions as disaster avoidance rather than disaster recovery. The core difference for us is maintaining service through an unintended outage and designing our systems to have resiliency built in from the outset.

When we develop solutions on the IBM Cloud, we approach this by using every region as its own isolated silo that can handle the workload standalone. We then use replication at the data layer to provide data consistency, if and when it’s required. For this solution, our sample stack looked like the following:

Each region had a liberty runtime which contained our application code. We then provisioned an instance of IBM Watson Assistant in the region along with a IBM Cloudant instance. We then used VCAP services to ensure that our code base had no hard-coded reference to the services connected to the runtime.  This meant we could use Cloud Foundry to deploy the same application code to any region and then use VCAP services to connect the runtime to the appropriate local services.

We used Cloudant replication to replicate our datastores to ensure that regardless of which region you hit, the data was available to you.

We then fronted the regions with a Global Traffic Manager (GTM) service from Akamai; this allowed us to direct users to the IBM Cloud region closest to them. In the unlikely event where the GTM solution couldn’t connect to the local IBM Cloud region, it would instead direct them to one of the other regions.

This approach means our solution would be available and up throughout any service disruptions to a certain region.

Conclusion

Throughout this series we have aimed to explain how we delivered the Facebook Messenger service during The Championships. We hope the lessons and approaches we have detailed here will help your utilization and success on the IBM Cloud platform.