What are AI hallucinations?

Generally, if a user makes a request of a generative AI tool, they desire an output that appropriately addresses the prompt (that is, a correct answer to a question). However, sometimes AI algorithms produce outputs that are not based on training data, are incorrectly decoded by the transformer or do not follow any identifiable pattern. In other words, it “hallucinates” the response.

The term may seem paradoxical, given that hallucinations are typically associated with human or animal brains, not machines. But from a metaphorical standpoint, hallucination accurately describes these outputs, especially in the case of image and pattern recognition (where outputs can be truly surreal in appearance).

AI hallucinations are similar to how humans sometimes see figures in the clouds or faces on the moon. In the case of AI, these misinterpretations occur due to various factors, including overfitting, training data bias/inaccuracy and high model complexity.

Preventing issues with generative, open-source technologies can prove challenging. Some notable examples of AI hallucination include:

• Google’s Bard chatbot incorrectly claiming that the James Webb Space Telescope had captured the world’s first images of a planet outside our solar system.¹

• Microsoft’s chat AI, Sydney, admitting to falling in love with users and spying on Bing employees.²

• Meta pulling its Galactica LLM demo in 2022, after it provided users inaccurate information, sometimes rooted in prejudice.³

While many of these issues have since been addressed and resolved, it’s easy to see how, even in the best of circumstances, the use of AI tools can have unforeseen and undesirable consequences.

AI hallucination can have significant consequences for real-world applications. For example, a healthcare AI model might incorrectly identify a benign skin lesion as malignant, leading to unnecessary medical interventions. AI hallucination problems can also contribute to the spread of misinformation. If, for instance, hallucinating news bots respond to queries about a developing emergency with information that hasn’t been fact-checked, it can quickly spread falsehoods that undermine mitigation efforts. One significant source of hallucination in machine learning algorithms is input bias. If an AI model is trained on a dataset comprising biased or unrepresentative data, it may hallucinate patterns or features that reflect these biases.

AI models can also be vulnerable to adversarial attack, wherein bad actors manipulate the output of an AI model by subtly tweaking the input data. In image recognition tasks, for example, an adversarial attack might involve adding a small amount of specially-crafted noise to an image, causing the AI to misclassify it. This can become a significant security concern, especially in sensitive areas such as cybersecurity and autonomous vehicle technologies. AI researchers are constantly developing guardrails to protect AI tools against adversarial attacks. Techniques such as adversarial training—where the model is trained on a mixture of normal and adversarial examples—are shoring up security issues. But in the meantime, vigilance in the training and fact-checking phases is paramount.

The best way to mitigate the impact of AI hallucinations is to stop them before they happen. Here are some steps you can take to keep your AI models functioning optimally:

Generative AI models rely on input data to complete tasks, so the quality and relevance of training datasets will dictate the model’s behavior and the quality of its outputs. In order to prevent hallucinations, ensure that AI models are trained on diverse, balanced and well-structured data. This will help your model minimize output bias, better understand its tasks and yield more effective outputs.

Spelling out how you will use the AI model—as well as any limitations on the use of the model—will help reduce hallucinations. Your team or organization should establish the chosen AI system’s responsibilities and limitations; this will help the system complete tasks more effectively and minimize irrelevant, “hallucinatory” results.

Data templates provide teams a predefined format, increasing the likelihood that an AI model will generate outputs that align with prescribed guidelines. Relying on data templates ensures output consistency and reduces the likelihood that the model will produce faulty results.

AI models often hallucinate because they lack constraints that limit possible outcomes. To prevent this issue and improve the overall consistency and accuracy of results, define boundaries for AI models using filtering tools and/or clear probabilistic thresholds.

Testing your AI model rigorously before use is vital to preventing hallucinations, as is evaluating the model on an ongoing basis. These processes improve the system’s overall performance and enable users to adjust and/or retrain the model as data ages and evolves.

Making sure a human being is validating and reviewing AI outputs is a final backstop measure to prevent hallucination. Involving human oversight ensures that, if the AI hallucinates, a human will be available to filter and correct it. A human reviewer can also offer subject matter expertise that enhances their ability to evaluate AI content for accuracy and relevance to the task.

While AI hallucination is certainly an unwanted outcome in most cases, it also presents a range of intriguing use cases that can help organizations leverage its creative potential in positive ways. Examples include: