A recent report published by Goldman Sachs has fueled a new debate around generative AI’s business value. Titled “Gen AI: Too much spend, too little benefit,” the report presents a contrasting view on what the technology currently delivers, approximately two years after its initial boom. The report echoes recent news stories that question whether gen AI is living up to its hype. However, exploring how larger businesses have been able to implement and yield real-world value from the tech in the present day offers a more nuanced portrait. We took a closer look.

Uncertainty surrounding gen AI

Daron Acemoglu, Institute Professor at MIT, is extensively interviewed throughout the Goldman report and is a clear skeptic. Acemoglu doesn’t believe that AI will have a big impact on the economy; he forecasts that the technology will impact less than 5% of all tasks over the next 10 years. He predicts that AI will increase US productivity by 0.5% and GDP growth by a mere 0.9% in the coming decade.

“Every human invention should be celebrated, and generative AI is a true human invention. But too much optimism and hype may lead to the premature use of technologies that are not yet ready for prime time,” says Acemoglu.

Goldman Sachs’ own Jim Covello says that today’s AI technology isn’t designed to solve complex problems that will justify its high costs. He’s also skeptical about the cost reductions that could come with automating certain tasks.

AI investment still soars

This kind of skepticism hasn’t slowed down funding for AI technology, in the US or worldwide. According to KPMG, a full 20% of global VC funding was captured by AI companies during the second trimester of 2024. And in the US, AI investments drove a 47% increase in VC funding last quarter, according to Reuters.

Tech entrepreneur Gilles Raymond believes that while increased investment in AI is expected, it’s the tech behind AI that stands out. “Of course, AI attracts a lot of VC funding. It’s logical that down the line people will expect to see additional revenues and not only cost savings. It is, however, interesting to see that the real winners of AI are those who don’t do AI but sell servers, microchips, etc. They truly are the big winners in this race.”

Cyril Maury, partner at international consulting firm Stripe Partners, observes that many of his clients in B2B and B2C are experimenting with gen AI. Much of this experimentation is driven by a fear of missing the boat, but it rarely moves on into production.

“We do have two types of clients: those who are in tech and want to develop AI and integrate it in their workflows. Others are ‘legacy’ businesses, that fear disruption. They see AI as a very powerful tool, but they don’t see the benefits,” Maury explains. “We are still in the experimentation phase. Everyone wants to try and experiment with AI, but beyond the sandboxes, we only see marginal change so far.”

Gen AI delivers real value

After all the AI activity of 2023, the question of returns remains central. A recent study conducted by IBM’s research team and based on a survey of more than 5,000 executives on their use of gen AI brings a more nuanced portrait to light. IBM’s report shows that AI delivered a higher average return on investment (ROI) in 2023 than it had in 2022.

“One of the interpretations is that there is a wider variety of ROIs for projects when businesses are experimenting with different and smaller projects,” explains Brian Goehring, Associate Partner and AI Research Lead at IBM’s Institute for Business Value (IBV).

The study indicates some analysts are skeptical. They anticipate that this hype-driven adoption spike will be followed by a “trough of disillusionment,” where one-third of the organizations pause their generative AI use cases in core business functions. However, two-thirds of the experimental projects will continue with AI after their pilot phases.

“There is a segment of companies that will probably decide that gen AI is not as attractive for them based on their experimentation. But we fully expect, based on the data that we continue to see, that there is a large, and perhaps larger segment of the population that we’re looking at that will continue and maybe even accelerate their investments,” adds Goehring.

These ongoing users will most likely be large, incumbent companies with significant data wealth (and have permission from their employees and customers to use it) and have AI governance guardrails in place.

“The economics of AI slant toward large enterprises and cross-organization platforms, at least for now,” says Goehring, citing as evidence some of the work the IBV has done with the MIT-IBM Watson AI Lab.

Many larger businesses and local governments have already successfully adopted gen AI to answer some of their challenges, whether to facilitate the analysis of customer data, enhance customer care, or improve knowledge modeling efficiency.

Gen AI governance considerations

Governance and trust are becoming central questions for businesses that are not only adopting but scaling AI. A global study from IBV on the realities executives must face in the gen AI era found that 75% of CEOs say trusted AI is impossible without effective AI governance in their organization—but only 39% have good gen AI governance in place today.

“How can we operationalize AI if you’re already behind on trust?” asks Hans-Petter Dalen, Business Leader EMEA, IBM watsonx and embeddable AI. “To me, the fundamental answer to that is governance. There is not just regulatory risk or reputational risk. The biggest risk for companies now is operational risk, where you’re not able to actually do AI innovation at scale. The enabler for that is going to be AI governance.”

Ultimately, while doubts about gen AI’s ROI may persist, it’s clear that companies today are extracting real-world value from the technology. Company size, industry and governance all play a role in just how big those gains can be, and the ability to move from pilot to production will be contingent on organizations taking a disciplined approach to deploying the technology.