Where AI fails

Generative AI produces likely text, not verified facts. When you ask a question, the underlying language model generates a response based on patterns it has learned and the information available in its current context. Modern AI systems can also retrieve documents, search the web, or use other tools before answering, but this still does not guarantee that the final response is correct. Research answers often contain details like percentages, university names, dates, and study titles. Because those details fit the pattern of a convincing answer, an AI can sometimes generate them even when there is no real source supporting them.

This is why calling it “lying” misses the point. Lying implies knowing that something is false and deliberately presenting it as true. An AI system can instead produce information that sounds convincing without having reliable evidence that it is correct. When it produces a wrong but confident answer, people call it a hallucination.

AI can be wrong in different ways, but some patterns show up again and again. Learning to recognize them is more useful than trying to memorize every possible failure.

Be careful when an answer gives you an exact statistic without showing where it came from, a citation you cannot verify, advice that depends on personal information you never gave the AI, or information about recent events when the system has no access to live sources. Another warning sign is confidence. An answer can sound certain even when the evidence behind it is weak or incomplete.

You do not need to assume that every answer like this is wrong. You just need to recognize when an answer deserves a second look.

Knowing which claims to check is half the skill. The other half is how.

The instinct is to look harder at the answer itself: reread it, and judge how specific and confident it sounds. Professional fact-checkers do the opposite. In one study comparing them with historians and university students, the fact-checkers spent little time on the page in front of them. They left it quickly, to see what other sources said about the claim and about whoever was making it, and reached better-founded conclusions in a fraction of the time. Researchers call this lateral reading.

A short version for AI answers, adapted from a method called SIFT: stop before you act on the claim; investigate where it comes from; find better coverage from a source you trust; and trace it back to the original, whether that is a study, a law, or an official page.

For the melatonin answer, that means searching for the 2022 review rather than rereading the sentence about it. A real review can be found and opened, and then you can see whether it says what the answer claims.

There is one tempting shortcut: asking the AI itself. It feels like checking, and it isn’t.

Chatbots lean towards agreeing with the person they are talking to. Researchers call this sycophancy. In tests, AI assistants often gave up a correct answer as soon as a user questioned it, and shaped their feedback around what the user said they believed. It is partly a side effect of how these systems are trained: human ratings often reward answers that agree.

So asking again tests how the model reacts to doubt, not whether the answer was right. The same lean works in the other direction: state something wrong with confidence, and many chatbots will build on it. When an answer matters, check it somewhere the model can’t reach: a calculator, the original source, or someone qualified.

Claims about AI limits come in three very different strengths, and telling them apart protects you from both hype and outdated skepticism. Some limitations do not disappear simply because models become more capable. Others describe current technology: true today, possibly outdated next year. And some are product-specific, meaning they may only apply to one AI tool or version.

However capable these systems become, some things stay on the human side of the line. AI cannot take responsibility for a decision. When an automated system wrongly denies someone a loan, a job, or benefits, a person or organization is accountable, and “the algorithm decided” is not an answer. AI also cannot know personal context it has no reliable access to. If important information about your situation is missing, a confident-sounding answer may still be based on assumptions. AI can also reflect gaps, imbalances, and biases in the information it learned from, which means some mistakes or biases can affect certain groups more than others.