An IBM Data Engineer Transforming the Drive-Thru Customer Experience

We recently spoke with Jose, an IBM Data Engineer on the speech and machine-learning team, who joined IBM through the McDonald’s TechLabs acquisition.

In December 2021, IBM acquired the McDonald’s TechLabs team and their Conversational AI software, now called IBM Watson Orders.

The Silicon Valley based IBM Watson Orders product team is currently focused on changing the Quick-Service Restaurant (QSR) ordering experience. Through world-class, conversational artificial intelligence, they are improving the quality of the work environment in the kitchen, while also ensuring that Drive-Thru customers have a consistent, delightful experience.  Starting with the QSR Drive-Thru use-case, the team’s long-term mission is to deliver advanced technology solutions to address real-world, data-driven needs in many different customer-facing environments.  They are developing state-of-the-art machine learning, artificial intelligence, and other similar technologies to transform the customer experience.

Tell us about some of the work you do as a Data Engineer on the Watson Orders team?

One of the most interesting projects I’ve worked on is leveraging our machine learning models to collect Drive-Thru audio of Spanish-speaking customers for further transcription by a third party. The speech recognition models we deploy in the field are only as good as the data they’re trained on.

Nationwide deployment of our automated order taker will require that we’re able to serve customers across multiple accents, dialects, and languages. Spoken Spanish within the United States often involves code-switching, meaning the same phrase can contain a mix of both Spanish and English. This adds to the complexity our models must be trained to handle. To make matters more interesting, there are dozens of distinct, Spanish-speaking communities in the U.S., each with its own vastly different accent and colloquialism that must be accounted for.

What kind of impact does this project have on you?

This project is personally exciting across multiple dimensions. As an engineering challenge, it’s rare that you have access to such a diverse and massive spoken-word dataset on the order of hundreds of gigabytes of Drive-Thru data per day. Making sure our systems collect the most representative data, and handle scale, is a big part of this challenge.

On a personal level, it’s satisfying to know that I’m helping build a multilingual system that my parents, who are immigrants from Mexico, will be able to use at their local Drive-Thru in the way that’s most natural to them.

What impact does this project have for our clients and IBM?

Before becoming Watson Orders, our automatic order taker already performed better in the Drive-Thru setting than similar speech recognition systems developed by leaders in the field. With the support of IBM, Watson Orders is poised to become the industry leader in automated order handling systems, which is applicable in many contexts, even outside of the Drive-Thru business. Our clients can look forward to a more seamless, accurate Drive-Thru experience, while our partners benefit from the stability of a system that is always ready to service even the most complicated customer requests.

Do you see this as a strategic growth offering for IBM? What opportunities for development exist now that Watson Orders is part of IBM?

IBM is highly committed to investing in Watson Orders; we’re currently in the process of deploying the tech in McDonald’s stores around the U.S. This will be a resource intensive process that requires collaboration across all levels. Leadership has been supportive by ensuring our development and product teams have been unblocked to reach this ambitious goal.

Continuing to work with the broader IBM community will be invaluable to ensure we can meet these performance demands with regard to scalability, stability, and marketing. Through the support of IBM, Watson Orders recently had the opportunity to have thousands of ambitious franchisee owners experience the technology for themselves at the 2022 McD Worldwide Convention. With IBM’s resources, there are countless more opportunities to get Watson Orders out into the public and into every Drive-Thru in every neighborhood.

On a personal level of development, it has been a rare opportunity to see a project evolve from the working prototype stage all the way through a fully managed solution that’s deployed across tens of thousands of physical locations with multiple partners involved.

Do you see technology innovation that gives you confidence IBM is among those on the leading edge?

Since joining IBM, Watson Orders is working closely with the IBM Research team to explore cutting edge machine learning techniques, which can further improve our system.  IBM Research is an impressive collection of distinguished engineers who have the expertise to explore alternate solutions to difficult problems, which has been a force multiplier in helping Watson Orders find new, innovative solutions. One promising example of this collaboration is an unsupervised approach to anomaly detection, which would allow us to detect and react to transient disturbances that could affect the order, like loud train whistles near stores. This kind of innovation and collaboration gives me confidence that IBM is on the leading edge!

Be a part of the IBM Watson Orders team

The IBM Watson Orders team is at the forefront of advanced natural language, AI technologies. They are delivering optimized, customized, and managed solutions for interactive voice automated use-cases through edge-managed and private cloud services.

If the work that Jose is doing sounds exciting to you, and you want to learn more about the opportunities that exist within the Watson Orders team, you can view our open roles through our careers website.