A hallucination is a confident guess from a new hire who would rather not say "I do not know."
Why does AI make things up?
By Palace City ·
Sometimes an AI tells you something that sounds right and is wrong. People call this a hallucination. A better name might be a confident guess.
Why it happens
An AI model is built to produce a helpful-sounding answer. Think of a new hire who badly wants to impress you. Ask them something they are unsure of, and some will guess instead of admitting it.
The model does not feel doubt the way a person does. It produces the most likely-sounding answer. When the real answer is missing, a smooth and wrong one can come out in the same confident voice as a correct one.
Where it shows up most
- Specific facts. Names, dates, numbers, and quotes.
- Sources. An AI can invent a reference that looks real.
- Your own business. If it was never told your prices or policies, it may invent plausible ones.
How to protect yourself
You cannot remove the risk entirely. You can reduce it:
- Give it your real information. An AI that has your price list in front of it guesses less.
- Ask for sources, then check them. If it cannot point to where an answer came from, treat the answer as a draft.
- Keep a person in the loop. For quotes, contracts, and anything customer-facing, someone reads it before it goes out.
- Test on known answers. Run past jobs through a tool and compare to what you actually did.
The takeaway
AI is a fast, tireless first-draft machine. It is not a fact-checker. Use it for the first draft, and keep your people for the final say.