Images, voices & deepfakes

Your phone rings. It is your son’s voice. He sounds stressed and says his phone was stolen. He needs you to transfer money quickly.

The number is unfamiliar, but the voice sounds exactly like him.

Generative AI does not only create text. Models can also generate or alter images, voices, music, and video.

A generated face may belong to nobody. A voice can be made to sound like a real person. A video can make someone appear to say or do something they never did.

The term deepfake is commonly used for convincing AI-generated or AI-manipulated media that imitates a real person. Deepfakes can be used to deceive people, but deception is not required for the technology itself to exist.

The important question is not simply whether something was generated with AI. It is what it is being used for, whether the people involved consented, and whether someone is being misled.

You might spot some bad fakes. Strange hands, unnatural movement, odd lighting, or a voice that sounds slightly wrong can still give something away.

The problem is that you cannot rely on those signs.

In one study, people could not reliably distinguish a set of AI-generated faces from photographs of real people, and the generated faces were rated slightly more trustworthy. Other experiments have found that people struggle to reliably identify deepfake videos, even when they know deepfakes may be present.

A 2024 review combined results from 56 studies. On average, people judged correctly about 55% of the time, where guessing would get 50%, and that gap was too small to show they were reliably doing better than chance. Performance also depended on the kind and quality of the fake and the conditions of the experiment.

So the lesson is not “you can never spot a fake.” It is that looking real is no longer enough evidence that something is real.

Synthetic media creates another problem. If fake evidence can look real, real evidence can also be dismissed as fake.

Someone confronted with a genuine recording can now respond: “that was AI generated.” Researchers have called this the liar’s dividend: the existence of convincing fakes can give dishonest people a new way to deny real evidence.

So there are two mistakes to avoid: believing something because it looks real, and rejecting something just because it could have been faked. The answer to both is the same: look for evidence beyond the media itself.

Another way to judge media is to look at its history.

Content Credentials are an open standard designed to attach verifiable information about the origin and editing history of digital content. A credential can record information such as which tool created or edited a file and whether AI was involved. Cryptography helps make changes to that recorded history detectable. Think of it as a kind of history label for digital media.

But there are two important limits. A valid credential can tell you information about where a file came from and what happened to it. It does not prove that everything shown or claimed in the content is true. And if a file has no Content Credential, that does not mean it is fake: not every camera, platform, or editing tool uses them, and credentials can also be removed.

Provenance can give you useful evidence. It is not a truth button.

For urgent calls involving money, passwords, or personal information, agree on one rule with the people close to you: verify the request through a contact method you already know.

If someone calls claiming to be a family member and urgently needs money, hang up and call their normal number yourself.

The urgency is exactly why the extra check matters. Consumer protection agencies recommend this approach for scams involving cloned voices.

You can add a second check: a code word or phrase that only your family knows, agreed in person and never posted anywhere. The FBI recommends exactly this. A cloned voice can copy how someone sounds, but it can’t know a word that was never said online.