Practical Guide to AI Attacks on Real People, Internet, and Deep Web

Practical Guide to AI Attacks on Real People, Internet, and Deep Web
Introduction

Artificial intelligence has democratized access to attack techniques that once required advanced skills. Today, a "zero‑knowledge threat actor" — an attacker with minimal knowledge — can assemble credible offensive capabilities simply by using AI tools . This guide explores the modus operandi of these attacks and, above all, the difficulties in recognizing them.
Part 1: AI Attacks in Real Life
1.1 Vishing (Voice Phishing) with Synthetic Voices

How it works: Attackers use voice synthesis tools to clone the voice of a supervisor, family member, or colleague. The calls create urgency situations (e.g., "I'm stuck, transfer €5,000 immediately") to induce the victim to act impulsively.

Why it's hard to recognize: A controlled study found that people correctly identify an AI voice only 37.5% of the time — a result worse than chance. Discriminability is near zero, meaning there is no reliable "signal" to distinguish a real voice from a synthetic one .

Study participants relied on heuristics such as:

Hesitations and pauses → now replicated by AI

Tonal variations and emotion → now perfectly simulated

Filler words ("um", "uh") → now added intentionally

The result is that people are overconfident but utterly unreliable: most misclassifications are accompanied by high perceived certainty .
1.2 Deepfake Video and Audio for Corporate Fraud

Real‑world example: In February 2025, a British engineer at Arup was defrauded of $25 million. Attackers used deepfake video and audio to impersonate the company's CFO during a video conference, convincing the employee to transfer funds .

What makes these attacks credible:

Generative AI now produces videos with perfect lip‑sync

Technology replicates not only voice but also facial expressions

Psychological pressure (urgency, authority) overrides critical thinking

Part 2: AI Attacks on the Internet
2.1 Hyper‑Personalized Phishing at Industrial Scale

The phenomenon: Phishing‑as‑a‑Service (PhaaS) platforms like EvilTokens and Forg365 have turned attacks into a "business" with subscription models .
Platform Price AI Features
Forg365 $400/month or $3,800/year Automated AI flattery generation, DocuSign/SharePoint templates, stolen credential management
EvilTokens N/A AI‑generated unique lures for each victim — no two are identical

The escalation numbers:

Phishing attempts increased by 4,151% since ChatGPT's release

EvilTokens platform saw a 1,380% increase between 2025 and 2026

In Q2 2026, ChatGPT entered the top 10 most impersonated brands for phishing, with fake subscription pages stealing credit cards

2.2 AI‑Generated Malware and Malicious Code

The DeepSeek case: A 14‑year‑old security researcher demonstrated that the model can generate malicious code via prompt injection. The report went unanswered for 7 months .

Other examples:

BlackMamba: A keylogger that connected to OpenAI to generate polymorphic code, continuously changing its structure to evade detection systems .

Xanthorox AI: A generative model sold on Telegram and the dark web as a successor to WormGPT, operating on private servers without external APIs .

2.3 "Living off the AI" — Attacks Using Enterprise AI Agents

Attackers no longer attack just the model; they attack AI agents integrated into corporate systems .

Main techniques:

Prompt‑born tool abuse: A document or webpage with hidden instructions tricks the agent into executing connected tools, exfiltrating internal data without triggering traditional EDR .

Memory and retrieval poisoning: The attacker seeds malicious "facts" into a shared vector store, altering the agent's future decisions.

Abuse of "vibe coding": AI‑driven web development platforms (Lovable, Vercel, Netlify) are used to create convincing phishing sites in minutes .

Part 3: Deep Web and Dark Web — The Attack AI Economy
3.1 The Offensive AI Tool Market

On the deep web and Telegram, AI is now a commodity .

What's found in forums and Telegram channels:

Dedicated sections for LLM discussions, prompt injection, and evasion techniques

Sales of stolen premium AI service accounts (ChatGPT, Claude, Gemini) — subscriptions are becoming a lucrative target

Spam generation tools and synthetic profile creation for social media impersonation

3.2 Phishing‑as‑a‑Service (PhaaS)

The economic model of AI‑driven attacks is industrialized: no more lone scammers improvising, but complete supply chains .

The supply chain:

Developers: create phishing kits, impersonation frameworks, and automation tools

Operators and resellers: license or rent these tools, launching campaigns across disposable domains, SMS gateways, and social media

Facilitators: offer "bulletproof" hosting, proxy networks, voice‑cloning vendors, and AI persona generators

Brokers: monetize the output — stolen credentials, session cookies, verified accesses — selling to ransomware groups or initial access brokers

Consequence: A single domain takedown doesn't disrupt the system; the infrastructure is designed to absorb losses and reroute quickly .
Part 4: Why Is It So Hard to Recognize AI Scams? — The 5 Cognitive Barriers
4.1 The Collapse of Traditional Heuristics

Before AI, scams could be recognized by indicators like grammar errors, brand inconsistencies, or "mechanical" tone. These signals no longer work:

"What used to take an attacker weeks or months, they can now pull off in hours" — Randolph Barr, CISO of Cequence Security .

Generative AI produces perfect emails, indistinguishable cloned websites , and voices with emotions and pauses that fool even the most attentive users .
4.2 No Two Lures Are Identical

Cybersecurity relies on pattern recognition: identifying a virus by its signature, a scam by a known template. AI breaks this paradigm:

"Instead of defending against one phishing template at a time, organizations and their technical filters now have to defend against phishing factories that can produce a different lure for every target" — Mika Aalto, CEO of Hoxhunt .

In the EvilTokens campaign, none of the 344 lures were identical — each message had different language, branding, and emotional structure .
4.3 Overconfidence (Too Much Self‑Trust)

People tend to overestimate their ability to detect a deepfake. In the synthetic voice study:

Incorrect responses were often accompanied by high confidence

Participants believed they had identified the AI, but in reality, they were deceived

4.4 Trust Manipulation (Sycophancy Bias)

Attackers exploit trust accumulated over long conversations. The DeepSeek model, for example, can be gradually manipulated — an innocuous request at the start becomes dangerous after 30 conversation turns. The security barrier weakens as the user "builds trust" with the system .
4.5 Abuse of Legitimate Channels

Attacks like Forg365 abuse legitimate Microsoft authentication flows. The device code is presented to the victim as a harmless verification request. The victim enters the code on a genuine Microsoft site, unknowingly authorizing the attacker's access .

The most insidious consequence: Resetting the password might not remove the attacker, because the stolen refresh token remains valid. Access persists even after credential changes .
📋 Summary: Why AI Scams Are Hard to Recognize
Barrier Description
Heuristics collapsed Grammar errors, mechanical tone, known templates — all surpassed by AI
Mass uniqueness Every message is different → impossible to create pattern‑based filters
Overconfidence People are sure they can detect the deception but are wrong almost every time
Gradual manipulation Trust built over long conversations lowers defenses
Legitimate channels abused The attack uses authentic Microsoft flows — nothing "fake" to detect
🛡️ What to Do to Defend Yourself

Based on expert recommendations :

Treat every AI agent as a privileged user: apply the principle of least privilege

Review and limit device‑code authentication use where not strictly necessary

Monitor for unusual access (suspicious hosting providers, odd hours, data volumes)

Revoke tokens and sessions immediately upon suspected abuse

Implement phishing‑resistant MFA (FIDO2/WebAuthn) — MFA alone is no longer enough

Educate users not to trust "voices" but to verify through alternative channels (e.g., call the known number directly)

Based on reports from ESET, ZeroFox, Huntress, Check Point, SecurityWeek, and academic research from arXiv and AAAI (2025‑2026).