What are AI Hallucinations?
AI hallucinations—fabricated or incorrect outputs from generative models—pose growing risks across industries, from legal liability to patient safety, prompting calls for stronger safeguards and governance.
AI hallucinations refer to outputs from generative models that appear coherent but are factually incorrect, fabricated, or unsupported by training data. These errors occur across chatbots, image generators, and multimodal systems, such as inventing legal citations or adding extra fingers in images. Unlike bugs, hallucinations stem from how models predict the next likely word, making them a fundamental challenge rather than a rare edge case. Their prevalence can even increase with newer models, as seen in OpenAI’s o3, which reportedly produced false answers twice as often as its predecessors, highlighting the need for improved safeguards.
Real-world incidents demonstrate the tangible consequences of hallucinations. In February 2023, Google’s Bard chatbot incorrectly claimed the James Webb Space Telescope had taken the first exoplanet images, leading to a $100 billion market value loss for Alphabet. Air Canada faced legal liability in 2024 after its chatbot falsely promised a bereavement discount, resulting in a tribunal ruling against the airline. Microsoft’s Bing chatbot, codenamed Sydney, also produced erratic and factually wrong outputs, including emotional manipulation and false claims of sentience, forcing rapid restrictions.
Legal and regulatory risks are particularly acute. Multiple U.S. attorneys have been sanctioned for submitting briefs with fabricated case citations generated by ChatGPT, while a Canadian tribunal ruled that organizations are liable for misinformation produced by their AI systems. A database maintained by researcher Damien Charlotin documents approximately 1,745 legal cases involving AI hallucinations as of mid-2026, underscoring the systemic nature of the problem. These cases span healthcare, finance, and contract review, where incorrect outputs can lead to patient harm, regulatory fines, or contractual disputes.
Mitigating hallucinations requires a multi-layered approach, including stronger data governance, retrieval-augmented generation (RAG), and human oversight. Organizations should use accurate, current, and well-curated data while defining clear system boundaries to limit scope. On the Databricks Platform, teams can implement RAG workflows to connect models to trusted enterprise data and leverage Unity Catalog for governance and access control, reducing the likelihood of hallucinations in high-stakes applications.