Deepfake Attacks: Why Phishing Training Isn’t Enough
Date
Authors
Chase Johnston
Follow Us
A deepfake video call cost one engineering firm $25 million. Against fakes this convincing, the control that holds is disciplined verification: confirm any high-risk request only through a channel you source yourself, and rehearse that reflex before an attacker tests it.
In January 2024, a finance employee at the Hong Kong office of engineering firm Arup received an email from the company’s UK-based CFO asking him to handle a confidential transaction. He thought it looked like phishing. He was right, and catching it is exactly what a decent awareness program drills into people. What he did next is where it went wrong. Rather than leaving the suspicious thread alone and confirming through a channel he controlled, he let the matter move onto a video call with the “CFO” and several colleagues.
Everyone on that call was fake. The CFO, the colleagues he recognized, every face and voice was AI-generated from footage the attackers had scraped off the internet. Reassured by the people he saw and heard, he authorized 15 transfers totaling about $25.6 million to five Hong Kong accounts. The fraud surfaced only when he checked with headquarters afterward. Arup’s global CIO later confirmed the fake voices and images and said this kind of thing happens more often than people realize. [1]
Source
[1] CNN, “Finance worker pays out $25 million after video call with deepfake ‘chief financial officer,'” February 2024.
https://www.cnn.com/2024/05/16/tech/arup-deepfake-scam-loss-hong-kong-intl-hnk
Key Takeaways
- A deepfake video call convinced a finance employee at Arup to wire $25.6 million — every face and voice on that call was AI-generated.
- In controlled testing, people specifically primed to look for deepfakes still caught only one in a thousand synthetic clips.
- 62% of organizations reported facing a deepfake attack in the past year (Gartner, 2025).
- Detection-based training doesn’t work anymore. The control that holds is independent verification through a channel you source yourself, regardless of who appears to be asking.
- Tabletop exercises using a real, working deepfake build the verification reflex in a way a hypothetical script never will.
Why Security Awareness Training Misses Deepfake Attacks
Two things are true of most awareness programs, and both are a problem. Deepfakes are barely in them. Training is still built around the inbox, and it shows at the employee level: a National Cybersecurity Alliance survey found that 58% of people who use AI tools had received no training on the security or privacy risks that come with them. [2] Where deepfakes do make it into a program, they usually arrive as a detection lesson, a checklist of tells to watch for. That is the half that does not work.
The whole approach leans on a premise that has quietly expired, that a fake has seams a careful person can catch and that a live face or voice is something you can trust on sight. When iProov primed two thousand people to watch for deepfakes and then tested them, one in a thousand caught every synthetic clip. [3] They were looking as hard as they could. A convincing voice clone now takes a few seconds of source audio, and the source is sitting in public: earnings calls, conference talks, a LinkedIn video, a voicemail greeting. The seams we tell employees to look for are the ones this year’s tools have learned to hide.
No matter how good a deepfake is, the attacker still has to ask for something suspicious, and the ask is where the pattern shows: money moved in several transfers, pressure for secrecy and speed, a reason you have to skip the normal approval, a push onto some new or “temporary” video platform, a meeting link from a domain that isn’t your company’s. Those survive a flawless render, because they describe what the attacker wants rather than how good the fake looks. Treat any of them as the cue to stop and run your verification, not as a test you can pass by staring harder. The lip-sync lag and flat, emotionless delivery that give away cheaper fakes are a bonus if you happen to catch them, but the Arup fake showed none of it, so nothing in your defense should rest on eyes and ears the technology has already beaten.
None of this is a fringe scenario. A 2025 Gartner survey of 302 security leaders found 62% of organizations had faced a deepfake attack in the prior year. [4] Deloitte’s Center for Financial Services projects generative-AI-enabled fraud losses in the US rising from $12.3 billion in 2023 to $40 billion by 2027. [5] And the FBI’s 2025 internet crime report counted more than 22,000 AI-related complaints, with reported losses above $890 million. [6] The trend line is not subtle.
Sources
[2] National Cybersecurity Alliance and CybSafe, “Oh Behave! The Annual Cybersecurity Attitudes and Behaviors Report 2025-2026,” September 30, 2025.
https://www.staysafeonline.org/press/study-65-now-use-ai-but-majority-remain-untrained-on-risks
[3] iProov, “iProov Study Reveals Deepfake Blindspot: Only 0.1% of People Can Accurately Detect AI-Generated Deepfakes,” February 12, 2025.
https://www.iproov.com/press/study-reveals-deepfake-blindspot-detect-ai-generated-content
[4] Gartner, “Gartner Survey Reveals Generative AI Attacks Are on the Rise,” September 22, 2025.
https://www.gartner.com/en/newsroom/press-releases/2025-09-22-gartner-survey-reveals-generative-artificial-intelligence-attacks-are-on-the-rise
[5] Deloitte Center for Financial Services, “Generative AI is expected to magnify the risk of deepfakes and other fraud in banking,” 2024.
https://www.deloitte.com/us/en/insights/industry/financial-services/deepfake-banking-fraud-risk-on-the-rise.html
[6] FBI Internet Crime Complaint Center, 2025 Internet Crime Report; see also FBI, “Cryptocurrency and AI Scams Bilk Americans of Billions,” April 9, 2026.
https://www.fbi.gov/news/press-releases/cryptocurrency-and-ai-scams-bilk-americans-of-billions
How a Deepfake Attack Works: The 4-Stage Attack Chain
Stage 1: Reconnaissance — Building the Fake from Public Footage
The raw material for a convincing fake is already public. A usable voice clone needs only a few seconds of clean audio, and a face model needs a handful of images or a short clip, both of which sit in earnings calls, conference keynotes, podcast interviews, webinar recordings, and the video an executive posted to LinkedIn last quarter. Alongside the media, the attacker builds a map: who reports to whom, who can authorize a payment and up to what limit, how a wire actually gets approved, which vendors are already on file. Most of that comes from LinkedIn, the company website, SEC filings, and old press releases. The richer an executive’s public profile, the better the training data, which is why the CFO and CEO are the usual models. Attackers also watch for timing, a conference week or a publicized trip that puts the real executive on a plane and out of easy reach, so a quick sanity check is harder to run.
Stage 2: Authority — Impersonating an Executive Employees Trust
With the models built, the attacker makes contact as the executive and frames the request as a decision that has already been made. A spoofed or lookalike email often opens it, then the pressure moves to a live channel where the fake does its work: a video call on a platform the attacker proposes, a WhatsApp thread, a phone call. Two things happen at once. The voice and face the target recognizes lower the reflex to question, and the seniority of the apparent sender makes questioning feel like insubordination. A junior treasury analyst is not inclined to tell the CFO that the request looks off. The counter to this stage is cultural rather than technical: leadership has to say out loud, and often, that verifying a request attributed to them is expected and will never be held against anyone.
Stage 3: Procedural Exploit — Where the Payment Actually Moves
This is the move that actually moves the money, and it is the one every organization controls. The fraud works only when a single person can push a payment through on the strength of one conversation, with nothing in the workflow forcing an independent check. The gaps are specific: no mandatory callback to a known number before a wire goes out, no second approver for payments over a threshold, no hold on transfers to a newly added account, no separation between the person who requests a payment and the person who releases it. Each of those is a control you can write down, enforce in the payment system, and test. When they are missing, a good deepfake is enough. When they are present and actually enforced, the fake hits a wall it cannot talk its way past.
Stage 4: Pressure — Manufacturing Urgency and Secrecy
The last move exists to keep the first three from being interrupted. The request is always urgent, a deal that closes today, a regulator waiting, a payment that has to clear before the market does, and it is always confidential, so sensitive that the target is told not to loop anyone in. Those two pressures are aimed at the exact things that would save the company: the time to run a verification, and the colleague who would say this feels wrong. When a message insists you skip the normal process because this case is special, the insistence itself is the red flag. A legitimate executive request will survive a verification step; a fraudulent one depends on your skipping it.
Why Deepfake Tabletop Exercises Build the Right Reflex
A person learns to hold off on an urgent request that carries the boss’s authority, until it clears an independent check, by having sat through that exact moment before, when it was safe to get it wrong. That is what tabletops are for. The trouble is that most tabletops run on imagination, and most phishing training doesn’t incorporate deepfakes. A facilitator reads out “the CFO calls asking for an urgent wire,” and around the table everyone gives the textbook answer. Nobody’s pulse moves. The real event feels nothing like that, and the space between what people say they would do and what they actually do, once a familiar face is on a live video call, is exactly where this fraud lives. Putting a working deepfake in the room closes that space.
When the scenario is a synthetic version of an executive the staff actually recognize, the exercise stops being hypothetical. The recognition kicks in the moment they see a face they know, and the authority and urgency do the rest, so people make the call under something close to real pressure. That is the point. The first time someone faces a flawless fake should be in a room where a wrong answer costs nothing and a facilitator can walk them back through it, not on the day an attacker has the wire instructions ready. Someone who has been fooled once, safely, and then shown exactly what the verification step would have caught, carries that pattern into the real moment. Rehearsal against a convincing fake is how the response turns into a reflex rather than a paragraph in a policy nobody rereads.
Built this way, the exercise seats the people who move money and grant access: treasury and accounts payable, the executive assistants who field urgent asks from leadership, the help-desk staff who reset credentials. Then it runs live injects. A deepfaked CFO on a video call directs a same-day wire. An assistant gets a voice note from the “CEO” asking her to buy gift cards for a closing dinner. The help desk takes a call from a panicked “VP” locked out ahead of a board meeting. At each beat the group has to act rather than narrate, and the facilitator marks the moment a decision skips the one control that matters: independent confirmation on a channel the team sources itself, never the one the request arrived on. The output is a short list of the precise points where your approval chain bends, each mapped to an owner.
Deepfake Tabletop Exercises and Red Team Testing
Our Builders team conducts Tabletop Exercises which can include deepfakes. With your sign-off, we collect video and audio of a chosen executive and build a working deepfake of them, then bring it into the session as a live asset. Rather than asking your people to imagine a call from the executive, we put a synthetic version of someone they know in front of them and make them work through the decision in the room. We facilitate the exercise, feeding in injects that escalate the pressure and move the scene along so the group has to decide in the moment rather than in hindsight. As it plays out, we track decisioning and afterwards analyze the exercise and report those findings back with recommendations to strengthen your program.
In addition, our Breakers team can augment your phishing testing with deepfakes, even incorporating deepfakes into red team exercises, enhancing the breadth and depth of the testing and ensuring that your organization is prepared for a future dominated by AI.
Frequently Asked Questions
What is a deepfake attack in cybersecurity?
A deepfake attack uses AI-generated audio or video of a real person, usually an executive, to impersonate them convincingly enough to authorize a fraudulent request such as a wire transfer. Unlike email phishing, it exploits trust in a familiar face or voice rather than a suspicious link.
Can employees be trained to spot a deepfake on a video call?
Not reliably. In one study, people specifically primed to watch for deepfakes still only caught one in a thousand synthetic clips. Convincing fakes no longer have visible seams, so detection-based training misses the point. Verification through an independent channel is the control that actually holds.
How did the Arup deepfake scam work?
A finance employee at Arup’s Hong Kong office joined a video call where every participant, including the “CFO,” was an AI-generated deepfake built from public footage. Reassured by faces and voices he recognized, he authorized 15 transfers totaling $25.6 million before the fraud was discovered.
What controls stop deepfake fraud even if the video looks real?
The controls that work don’t depend on detecting the fake: a mandatory callback to a known number, a second approver on payments over a set threshold, a hold on transfers to newly added accounts, and separation between who requests a payment and who releases it.
What is a deepfake tabletop exercise?
A deepfake tabletop exercise puts employees through a simulated attack using a real, working deepfake of an executive they recognize, rather than a hypothetical script. It recreates the same recognition and urgency as a real attack so the verification reflex forms before an attacker tests it for real.
