At the Masters®, storied tradition meets state-of-the-art technology. Through a partnership spanning more than 25 years, IBM has helped the Augusta National Golf Club capture, analyze, distribute and use data to bring fans closer to the action, culminating in the AI-powered Masters digital experience and mobile app. Now, whether they’re lining the fairways or watching from home, fans can more fully appreciate the performance of the world’s best golfers at the sport’s most prestigious tournament.

In a continuous design thinking process, teams from IBM Consulting and the club collaborate to improve the fan experience year after year. New features in 2024 include Hole Insights, stats and projections about every shot, from every player on every hole; and expanded AI-generated narration (including Spanish language) on more than 20,000 highlight clips.

The Masters has long relied on IBM to manage its data, applications and workloads across on-premises servers and multiple clouds, but this year marks an important evolution: the entire AI lifecycle is being managed on the AI and data platform IBM® watsonx™.

Collecting data

The IBM watsonx platform includes watsonx.data, a fit-for-purpose data store built on an open lakehouse architecture. This allows the Masters to scale analytics and AI wherever their data resides, through open formats and integration with existing databases and tools.

“The data lake at the Masters draws on eight years of data that reflects how the course has changed over time, while using only the shot data captured with our current ball-tracking technology,” says Aaron Baughman, IBM Fellow and AI and Hybrid Cloud Lead at IBM. “Hole distances and pin positions vary from round to round and year to year; these factors are important as we stage the data.”

The historical sources watsonx.data accesses comprise relational, object and document databases, including IBM® Db2®, IBM® Cloudant, IBM Cloud® Object Storage and PostgreSQL.

Lastly, watsonx.data pulls from live feeds. “We’ll hit a variety of feeds from the system, including scoring, ball tracking, pin location, player pairings and scheduling,” says Baughman. “We also pull in video, which is where we add the commentary and embed it into the clips.”

Watsonx.data lets organizations optimize workloads for different uses. For the Masters, “Consumer-facing data access is fronted by a CDN that caches resources so the traffic doesn’t hit our origin servers, whereas our AI workflow calls on data directly from the origin to ensure it’s as up to date as possible,” says Baughman.

Preparing and annotating data

IBM watsonx.data helps organizations put their data to work, curating and preparing data for use in AI models and applications. The Masters uses watsonx.data to organize and structure data relating to the tournament—course, round and holes—which can then be populated with live data as the tournament progresses. “We also have player elements, ball tracking information and scoring,” says Baughman. “Being able to organize the data around that structure helps us to efficiently query, retrieve and use the information downstream, for example for AI narration.”

Watsonx.data uses machine learning (ML) applications to simulate data that represents ball positioning projections. “With the data we’ve prepared we can then calculate the odds of a birdie or an eagle from a particular sector; we can also look across to the opposite side of the fairway for contrastive statistics,” says Baughman.

Developing and evaluating AI models

The IBM® watsonx.ai™ component of watsonx lets enterprise users build AI applications faster and with less data, whether they’re using generative AI or traditional ML.

“For the Masters we use 290 traditional AI models to project where golf balls will land,” says Baughman. “When a ball passes one of the predefined distance thresholds for a hole, it shifts to the next model, eventually ending up on the green. In addition, there are four possible pin locations—front left, front right, back left or back right—for a total of about 16 models per hole. It would be a huge challenge for a human to manage these models, so we use the autoAI feature of watsonx to help us build the right model and pick the best projection.”

Watsonx.ai also helped the digital team build a generative AI model for text creation, as the basis for spoken commentary. This makes it possible to then use watsonx.governance to evaluate the quality of the output, using metrics such as ROUGE, METEOR and perplexity scores while using HAP guardrails to eliminate any hate, abuse or profanity content.

“The tools in watsonx.governance really help,” says Baughman. “We can keep track of the model version we use, promote it to validation, and eventually deploy it to production once we feel confident that all the metrics are passing our quality estimates. We also measure response time since this is a near real-time system. Watsonx.governance makes it easy to manage and deploy all these models effectively.”

Training and testing models

The Masters digital team used watsonx.ai to automate the creation of ML models used in Hole Insights, based on 8 years of data. For AI narration, they used a pretrained large language model (LLM) with billions of parameters.

“We used few-shot learning to help guide the models,” says Baughman. “Rather than fine tuning the models through the tournament, we fine modify the input statistics that go into the models. It’s a compromise that delivers the results we need while minimizing risk.”

Watsonx.governance also provides multiple LLMs used to validate the data of the main model, for example to eliminate HAP content. “We have a lot of guardrails, right down to regular expressions,” says Baughman. “Watsonx gave us confidence that we could identify and mitigate HAP content in real time, before it gets published.”

Deploying and managing models

After tuning and testing ML or generative AI models, watsonx.ai provides a variety of ways to deploy them to production and evaluate models within the deployment space. Models can also be evaluated for fairness, quality and drift.

“We used Python scripts in watsonx to deploy the ML models on Watson Machine Learning [a set of Machine Learning REST APIs running on IBM Cloud],” says Baughman. “We also run the models locally, since we have containers that load the models in memory, so there’s no network latency at all. We have both strategies—we typically run the ones in memory first, then if anything goes wrong, we use the models deployed on Watson Machine Learning.”

The team took a different approach to deploy the LLM used for AI narration, by using a deployed model within watsonx.ai (where its generative parameters can be managed) and secondly, using a model that was deployed to Watson Machine Learning through watsonx.governance.

Governing and maintaining models

Watsonx.governance provides automated monitoring of deployed ML and generative AI models and facilitates transparent, explainable results. Users can establish risk tolerances and set alerts around a wide variety of metrics.

“Watsonx.governance alerts us if the models fail on any dimension, and allows us to easily fix them,” says Baughman. “We can also run experiments on demand, create AI use cases and ensure they work as expected.” One such experiment: after a round ends, the teams have some ground truth for that round that can be added into the model and revalidated, enabling continual improvement and improved results.

The 88^th Masters Tournament will be played from April 11 to 14 at Augusta National Golf Club in Augusta, GA. To see IBM technology in action, visit Masters.com or the Masters app on your mobile device, available on the Apple App Store and Google Play Store.