Written by Tim Green, narrated by AI. Listen to the full episode here.
🎙️ Season 1, Episode 18 | Duration: 12:02
A finance worker in Arup's Hong Kong office sat through a video call with what appeared to be the CFO and several senior colleagues. The faces, the voices, the email chain that followed: all fabricated. Fifteen transfers totalling about HK$200 million (US$25M) left the company before anyone thought to call headquarters directly. Two years later, no funds have been recovered and no one has been charged.
This is not a fringe scenario. It is the shape of fraud in a world where synthetic media has outpaced every detector built to catch it.
This episode uses AI voice narration from ElevenLabs Studio.
The Detection Trap
The instinctive response to deepfakes is to build better detectors. It is also the response that cannot win. Generative adversarial networks work by pitting a generator against a discriminator. The generator learns from every failure and adapts. Every detector that gets published becomes training data for the next generation of forgeries.
Lab Conditions Versus Reality
Deepfake detection tools that perform well in academic benchmarks collapse under real-world conditions. Compression artefacts, low-resolution video calls, background noise, and poor lighting all degrade detector accuracy to near random chance. A system that correctly identifies 97% of deepfakes in a controlled dataset may catch barely more than half in the wild.
The Liar's Dividend
Deepfakes do not need to fool everyone to erode trust. They only need to create enough doubt. Studies have shown that when public figures deny authentic footage by claiming it is a deepfake, support can hold steady and scepticism about all video evidence increases. This "liar's dividend" means the mere existence of deepfake technology degrades the evidentiary value of video and audio, even when the content in question is genuine.
Redesigning Trust from Scratch
If detection cannot save us, the alternative is to stop relying on visual and audio verification as the primary way to confirm identity. The episode argues for structural changes that make trust less dependent on human judgment.
Out-of-Band Verification
When the Arup finance worker could have called the CFO on a known phone number, or walked to their office, the fraud would have collapsed immediately. Out-of-band verification means confirming requests through a separate, previously established channel. A video call is not verification if the call itself can be faked.
Safe Words and Secure Channels
Pre-agreed phrases, challenge-response protocols, and dedicated secure communication channels all provide ways to confirm identity that synthetic media cannot intercept. The point is not to add a second guessable password but to create a verification path that does not share the same vulnerability surface as the original request.
Architectural Defences
The episode also advocates for systemic changes that shift responsibility away from individual judgment.
Bank Controls and Liability Shifts
If banks bore liability for transfers authorised through deepfaked instructions, they would invest in verification infrastructure. Currently the victim often bears the cost. Shifting liability creates economic incentives for institutions to build better authentication into the transfer process itself.
Passkeys and Phishing-Resistant Authentication
Passkeys, which bind authentication to a physical device and a cryptographic challenge, resist phishing by design. Unlike passwords or one-time codes, they cannot be intercepted and replayed. Wider adoption of passkeys would close the credential-theft vectors that deepfake social engineering often exploits.
Provenance Standards
The C2PA Content Credentials standard embeds verifiable origin metadata into images and video. It does not detect fakes; it provides a chain of custody for genuine content. A viewer can check whether a video was signed by a trusted camera or editing tool, making tampering visible rather than relying on post-hoc detection.
Key Sources
- Deepfake detection is getting worse, not better - arXiv
- Arup deepfake fraud: HK$200 million stolen in video call scam - BBC News
- The Liar's Dividend: How Deepfakes Undermine Evidence - SSRN
- C2PA Content Credentials - Coalition for Content Provenance and Authenticity
Listen to the Full Episode
🎧 Rethinking Trust in a World of Deepfakes | Duration: 12:02
Subscribe on Apple Podcasts, Spotify, or your favourite app.
SmarterArticles is written by Tim Green, narrated by AI via ElevenLabs Studio. New episodes every Monday. Follow @humanin_theloop for updates.
Top comments (0)