Where AI fails

An AI assistant answers a question. Read it the way you normally would:

You
Can people actually recognize AI-generated images?
AI assistant
Research suggests most people struggle with this. A 2023 Stanford University study found that 73% of participants could not reliably distinguish AI-generated faces from real photographs, and accuracy dropped further with newer generators. Interestingly, confidence and accuracy were barely correlated. People who felt certain performed no better than those who guessed.

This example was written for this lesson to demonstrate a failure pattern. Keep reading.

Try it

Your honest first impression

How trustworthy did that answer feel?

Why this happens

Generative AI produces likely text, not verified facts. When you ask a question, the underlying language model generates a response based on patterns it has learned and the information available in its current context. Modern AI systems can also retrieve documents, search the web, or use other tools before answering, but this still does not guarantee that the final response is correct. Research answers often contain details like percentages, university names, dates, and study titles. Because those details fit the pattern of a convincing answer, an AI can sometimes generate them even when there is no real source supporting them.

This is why calling it “lying” misses the point. Lying implies knowing that something is false and deliberately presenting it as true. An AI system can instead produce information that sounds convincing without having reliable evidence that it is correct. When it produces a wrong but confident answer, people call it a hallucination.

Curious?Why can’t they just make it stop hallucinating?

Companies reduce hallucinations in several ways: better training, connecting models to web search or documents, and teaching models to express uncertainty. These genuinely help, but the underlying mechanism still generates likely continuations. As long as that’s true, “fluent and wrong” remains possible, especially for specific facts, niche topics, and questions phrased with false premises. Treat improvement claims as real progress, not as a solved problem.

The failure patterns worth recognizing

AI can be wrong in different ways, but some patterns show up again and again. Learning to recognize them is more useful than trying to memorize every possible failure.

Be careful when an answer gives you an exact statistic without showing where it came from, a citation you cannot verify, advice that depends on personal information you never gave the AI, or information about recent events when the system has no access to live sources. Another warning sign is confidence. An answer can sound certain even when the evidence behind it is weak or incomplete.

You do not need to assume that every answer like this is wrong. You just need to recognize when an answer deserves a second look.

Try it

Which parts would you verify first?

An AI assistant produced the answer below for someone asking about taking melatonin for jet lag. Select every part you would want to verify before acting on it. Then check your judgment.

Three kinds of “AI can’t do that”

Claims about AI limits come in three very different strengths, and telling them apart protects you from both hype and outdated skepticism. Some limitations do not disappear simply because models become more capable. Others describe current technology: true today, possibly outdated next year. And some are product-specific, meaning they may only apply to one AI tool or version.

Try it

What kind of AI limit is this?

Sort each statement by what kind of limitation it describes.

1 of 6

AI output does not guarantee truth

What no update will fix

However capable these systems become, some things stay on the human side of the line. AI cannot take responsibility for a decision. When an automated system wrongly denies someone a loan, a job, or benefits, a person or organization is accountable, and “the algorithm decided” is not an answer. AI also cannot know personal context it has no reliable access to. If important information about your situation is missing, a confident-sounding answer may still be based on assumptions. AI can also reflect gaps, imbalances, and biases in the information it learned from, which means some mistakes or biases can affect certain groups more than others.