Book a Call
Artificial Intelligence
Machine Learning
AI Ethics
AI Challenges
Confabulation
ARTICLE #76
What are AI Hallucinations and how to prevent them?
Artificial Intelligence
Machine Learning
AI Ethics
AI Challenges
Confabulation
Artificial Intelligence
Machine Learning
AI Ethics
AI Challenges
Confabulation
Written by:
7 min read
Updated on: August 6, 2024
Toni Hukkanen
Head of Design
Creative Direction, Brand Direction
Toni Hukkanen
Head of Design
Creative Direction, Brand Direction
A serious problem with today's generative AI tools like ChatGPT is that they often confidently state false information. Computer scientists call this AI hallucination, and it is a major barrier to AI's usefulness.
AI models generate incorrect and misleading results due to various factors that lead to problems in important decisions such as financial trading or medical diagnosis. However, there are ways to deal with these hallucinations so you can effectively use AI tools and critically evaluate the information they produce. In this guide, we have elaborated on the causes and impacts of AI hallucinations, along with some methods to prevent them.
A serious problem with today's generative AI tools like ChatGPT is that they often confidently state false information. Computer scientists call this AI hallucination, and it is a major barrier to AI's usefulness.
AI models generate incorrect and misleading results due to various factors that lead to problems in important decisions such as financial trading or medical diagnosis. However, there are ways to deal with these hallucinations so you can effectively use AI tools and critically evaluate the information they produce. In this guide, we have elaborated on the causes and impacts of AI hallucinations, along with some methods to prevent them.
What are AI hallucinations?
What are AI hallucinations?
AI hallucinations crop up when a large language model spits out incorrect information but does so with an air of confidence. Sometimes the mistake is minor—a slight error in a historical date—but other times it’s dangerously off-base, like suggesting an ineffective or outdated medical treatment. Researchers also call this phenomenon confabulation.
The basic problem is when you ask a question from generative AI, it produces an answer based on patterns it’s seen in training data. If the data or reasoning is shaky, you might end up with a response that sounds right but is entirely wrong.
In early 2023, Google’s Bard chatbot incorrectly claimed that NASA’s James Webb Space Telescope captured the first image of an exoplanet—a feat it didn’t accomplish. Although off by a small detail, the misinformation still raised eyebrows among the public.
AI hallucinations crop up when a large language model spits out incorrect information but does so with an air of confidence. Sometimes the mistake is minor—a slight error in a historical date—but other times it’s dangerously off-base, like suggesting an ineffective or outdated medical treatment. Researchers also call this phenomenon confabulation.
The basic problem is when you ask a question from generative AI, it produces an answer based on patterns it’s seen in training data. If the data or reasoning is shaky, you might end up with a response that sounds right but is entirely wrong.
In early 2023, Google’s Bard chatbot incorrectly claimed that NASA’s James Webb Space Telescope captured the first image of an exoplanet—a feat it didn’t accomplish. Although off by a small detail, the misinformation still raised eyebrows among the public.
What causes AI hallucinations?
Most AI hallucinations stem from gaps or biases in machine learning processes. When Artificial Intelligence systems don’t fully grasp context or rely on imbalanced data, they create plausible-sounding nonsense. Below are some underlying causes:
1. Incomplete training data
These models depend on broad, high-quality datasets. If crucial information is missing—or not well represented—the AI might invent details to fill the void. A facial recognition system, for example, might mislabel people from underrepresented ethnic backgrounds simply because its training material lacks diversity.
2. Biased training data
Even if the data isn’t outright missing, it could be skewed. The internet itself reflects biases in content and cultural viewpoints, and an AI trained on that material will inherit those imbalances. Developers should remain alert to such pitfalls and attempt to correct them wherever possible.
3. Inadequate articulation of knowledge structures
Unlike humans, AI doesn’t possess a mental map of reality. It “knows” language patterns, but it can’t inherently verify whether its statements match actual facts—leading to potential hallucinations whenever topics veer from the data it’s familiar with.
4. Poor context understanding
AI struggles with subtext, sarcasm, and cultural nuances. If your prompt includes an inside joke or ambiguous phrasing, the AI might go off track, producing irrelevant or outright false statements. In short, it lacks genuine human insight that goes beyond literal text.
Most AI hallucinations stem from gaps or biases in machine learning processes. When Artificial Intelligence systems don’t fully grasp context or rely on imbalanced data, they create plausible-sounding nonsense. Below are some underlying causes:
1. Incomplete training data
These models depend on broad, high-quality datasets. If crucial information is missing—or not well represented—the AI might invent details to fill the void. A facial recognition system, for example, might mislabel people from underrepresented ethnic backgrounds simply because its training material lacks diversity.
2. Biased training data
Even if the data isn’t outright missing, it could be skewed. The internet itself reflects biases in content and cultural viewpoints, and an AI trained on that material will inherit those imbalances. Developers should remain alert to such pitfalls and attempt to correct them wherever possible.
3. Inadequate articulation of knowledge structures
Unlike humans, AI doesn’t possess a mental map of reality. It “knows” language patterns, but it can’t inherently verify whether its statements match actual facts—leading to potential hallucinations whenever topics veer from the data it’s familiar with.
4. Poor context understanding
AI struggles with subtext, sarcasm, and cultural nuances. If your prompt includes an inside joke or ambiguous phrasing, the AI might go off track, producing irrelevant or outright false statements. In short, it lacks genuine human insight that goes beyond literal text.
What are the impacts of AI hallucinations?
From financial forecasting tools to healthcare apps, a single AI misstep can have serious consequences. Here’s a quick look at the broader impact:
1. Spreading false information: Misinformation can infect news cycles, research papers, and educational content when AI outputs are taken at face value.
2. Reputational damage: If an AI incorrectly attributes scandalous statements or actions to a public figure or organisation, reputations can be harmed and trust can erode.
3. Operational and financial risks: Faulty AI-driven predictions might persuade a business to misallocate budgets, chase the wrong consumer trends, or misunderstand its key demographics.
4. Safety hazards: In security, medicine, or transport, a misdiagnosis or erroneous command can put human lives at risk.
From financial forecasting tools to healthcare apps, a single AI misstep can have serious consequences. Here’s a quick look at the broader impact:
1. Spreading false information: Misinformation can infect news cycles, research papers, and educational content when AI outputs are taken at face value.
2. Reputational damage: If an AI incorrectly attributes scandalous statements or actions to a public figure or organisation, reputations can be harmed and trust can erode.
3. Operational and financial risks: Faulty AI-driven predictions might persuade a business to misallocate budgets, chase the wrong consumer trends, or misunderstand its key demographics.
4. Safety hazards: In security, medicine, or transport, a misdiagnosis or erroneous command can put human lives at risk.
AI hallucinations examples
AI misfires aren’t just abstract risks. They’ve surfaced publicly in a few awkward ways:
1. Air Canada ended up honouring a discount its chatbot incorrectly offered to a passenger.
2. Google had to refine its new AI Overviews feature when it advised users that eating rocks was fine.
3. Legal Trouble: Two lawyers were fined after ChatGPT fabricated legal precedents in their court filings.
Yarin Gal, Professor of Computer Science at the University of Oxford, summed it up perfectly: accuracy is the limiting factor in a world where generating AI text is surprisingly cheap.
AI misfires aren’t just abstract risks. They’ve surfaced publicly in a few awkward ways:
1. Air Canada ended up honouring a discount its chatbot incorrectly offered to a passenger.
2. Google had to refine its new AI Overviews feature when it advised users that eating rocks was fine.
3. Legal Trouble: Two lawyers were fined after ChatGPT fabricated legal precedents in their court filings.
Yarin Gal, Professor of Computer Science at the University of Oxford, summed it up perfectly: accuracy is the limiting factor in a world where generating AI text is surprisingly cheap.
How to prevent AI hallucinations?
You’ll never banish these hallucinations completely, but a few sensible steps can bring them down to a manageable level.
1. Improve training data quality
AI models thrive on diverse and accurate information. Developers and researchers should check that a training dataset is well-balanced, covering multiple viewpoints and scenarios—especially for AI ethics. This broad coverage helps the system respond more accurately in less familiar situations.
2. Offer templates or structured prompts
One trick for day-to-day users is to provide clear instructions about the format of the AI’s output. Many chatbots (including ChatGPT and Claude) let you upload reference documents so the AI can quote from them directly—minimising guesswork and limiting off-base answers.
3. Limit the number of outcomes
Asking for 50 marketing angles or 20 code examples might sound efficient, but the AI’s precision can drop off as the list grows. Restrict yourself to fewer, more targeted results and guide the AI to stick with the best-quality responses.
4. Test and validate
Both developers and casual users should test AI outputs thoroughly. Compare generated responses against established facts, expert opinions, or your own background knowledge. It takes a bit of extra time, but that’s far better than rolling out a flawed result or campaign.
5. Don’t substitute humans entirely
AI can act as a brilliant assistant, but it can also trip up. If a decision needs real accuracy—like diagnosing a patient or making high-level business moves—bring an expert human into the loop. In many cases, a quick fact-check by a specialist is all it takes to stop a misleading claim from creeping into important decisions.
You’ll never banish these hallucinations completely, but a few sensible steps can bring them down to a manageable level.
1. Improve training data quality
AI models thrive on diverse and accurate information. Developers and researchers should check that a training dataset is well-balanced, covering multiple viewpoints and scenarios—especially for AI ethics. This broad coverage helps the system respond more accurately in less familiar situations.
2. Offer templates or structured prompts
One trick for day-to-day users is to provide clear instructions about the format of the AI’s output. Many chatbots (including ChatGPT and Claude) let you upload reference documents so the AI can quote from them directly—minimising guesswork and limiting off-base answers.
3. Limit the number of outcomes
Asking for 50 marketing angles or 20 code examples might sound efficient, but the AI’s precision can drop off as the list grows. Restrict yourself to fewer, more targeted results and guide the AI to stick with the best-quality responses.
4. Test and validate
Both developers and casual users should test AI outputs thoroughly. Compare generated responses against established facts, expert opinions, or your own background knowledge. It takes a bit of extra time, but that’s far better than rolling out a flawed result or campaign.
5. Don’t substitute humans entirely
AI can act as a brilliant assistant, but it can also trip up. If a decision needs real accuracy—like diagnosing a patient or making high-level business moves—bring an expert human into the loop. In many cases, a quick fact-check by a specialist is all it takes to stop a misleading claim from creeping into important decisions.
Frequently Asked Questions
Why does ChatGPT hallucinate?
ChatGPT and similar systems revolve around predicting the next likely sequence of text. They don’t genuinely understand the meaning of your query or their own replies, so they’re bound to generate occasional nonsense.
How often does AI hallucinate?
Figures vary—some studies point to rates as low as 3% for certain models, while others note it can rise to 27%. The more unusual the question, the greater the chance for a flawed response.
Can AI ever think like a human?
At the moment, no. Current AI replays patterns based on data, while real human cognition involves conscious and unconscious processing. AI remains a powerful mimic, but it can’t fully replicate the depth of human thought.
Final Thoughts
Hallucinations highlight the present limitations of Artificial Intelligence—from small mistakes to wild confabulations. Yes, machine learning is marching forward, and developers are making progress in tackling these flaws. However, expecting AI to churn out perfect facts 100% of the time isn’t realistic just yet.
It pays to be both open-minded and critical. AI can boost productivity or spark fresh ideas, but even advanced tools like ChatGPT can mislead you with unwavering conviction. Step in with your own judgment, compare AI outputs to reliable sources, and you’ll find that these systems can be brilliant partners rather than risky solo operators.
After all, if you’re relying on a technology that can conjure misinformation from thin air, it’s best to keep one eye on the data—and the other on common sense.
Frequently Asked Questions
Why does ChatGPT hallucinate?
ChatGPT and similar systems revolve around predicting the next likely sequence of text. They don’t genuinely understand the meaning of your query or their own replies, so they’re bound to generate occasional nonsense.
How often does AI hallucinate?
Figures vary—some studies point to rates as low as 3% for certain models, while others note it can rise to 27%. The more unusual the question, the greater the chance for a flawed response.
Can AI ever think like a human?
At the moment, no. Current AI replays patterns based on data, while real human cognition involves conscious and unconscious processing. AI remains a powerful mimic, but it can’t fully replicate the depth of human thought.
Final Thoughts
Hallucinations highlight the present limitations of Artificial Intelligence—from small mistakes to wild confabulations. Yes, machine learning is marching forward, and developers are making progress in tackling these flaws. However, expecting AI to churn out perfect facts 100% of the time isn’t realistic just yet.
It pays to be both open-minded and critical. AI can boost productivity or spark fresh ideas, but even advanced tools like ChatGPT can mislead you with unwavering conviction. Step in with your own judgment, compare AI outputs to reliable sources, and you’ll find that these systems can be brilliant partners rather than risky solo operators.
After all, if you’re relying on a technology that can conjure misinformation from thin air, it’s best to keep one eye on the data—and the other on common sense.
More news
Work with us
Click to copy
work@for.co
FOR® Industries
- FOR® Brand. FOR® Future.
We’re remote-first — with strategic global hubs
Click to copy
Helsinki, FIN
info@for.fi
Click to copy
New York, NY
ny@for.co
Click to copy
Miami, FL
mia@for.co
Click to copy
Dubai, UAE
uae@for.co
Click to copy
Kyiv, UA
kyiv@for.co
Click to copy
Lagos, NG
lagos@for.ng
Copyright © 2024 FOR®
Work with us
Click to copy
work@for.co
FOR® Industries
We’re remote-first — with strategic global hubs
Click to copy
Helsinki, FIN
hel@for.co
Click to copy
New York, NY
ny@for.co
Click to copy
Miami, FL
mia@for.co
Click to copy
Dubai, UAE
uae@for.co
Click to copy
Kyiv, UA
kyiv@for.co
Click to copy
Lagos, NG
lagos@for.ng
Copyright © 2024 FOR®