"Show nothing, hold everything."
— The Thirty-Six Stratagems, Create Something Out of Nothing
Previously on this series: #4: P Walked Into an AI Monitoring POC. P Didn't Run a Single Test. — P found an ACL business card in an abandoned POC archive. P didn't tell anyone. P just pocketed it.
White walls. Fluorescent hum. A FortDefender quarterly report sat open on the table, the cover printed in bold:
Zero missed detections. 99.97% detection rate.
The CTO slid it across.
"The day the leak happened," he said quietly, "this system said everything was fine."
"Which client?"
"MedTech. Medical data breach. Their internal AI monitoring didn't catch it either. The quarterly report called it 'client-side issue.' I don't buy it."
P didn't look at the report first. P looked at the CTO's eyes first.
"You didn't bring me here to validate his numbers."
The CTO didn't deny it.
"FortDefender won't give you production access," he said. "Read-only logs. Sandbox. Public docs. You signed the NDA."
"What do you want me to do?"
"Find what's hiding inside 'everything was fine.'"
P nodded. P didn't ask "what if I find it" — P knew the answer.
"One condition: full internal penetration test access. No advance notice to anyone."
The CTO was quiet for three seconds.
"Done."
P stood up. The CTO added one more thing as P turned:
"I've heard about the FirmCore thing. That's why I called you."
P didn't look back.
Week One
FortDefender's public documentation was beautiful. Architecture diagrams. Whitelist rules. Alert thresholds. Response times. All in a technical whitepaper so polished you'd think it was written to raise funding.
P spent three days reading every page.
In the sandbox, P ran three rounds of tests. FortDefender's detection system hit every single one. The 99.97% wasn't a lie — at least not inside the sandbox.
But P noticed something.
FortDefender's whitelist rules were too complete. They covered everything — down to "penetration tests with valid internal certificates" being pre-listed as an exclusion. That's unusual. Most companies whitelist their known monitoring probe IP ranges and call it a day. FortDefender wrote the certificate tier into the rulebook.
P pulled up a former FortDefender engineer's technical blog. The guy's name was rhuang. He used to run the AI detection strategy team. He got laid off last year and posted four articles on a tech community called Dev.to about "automatic exclusion rule generation based on historical alerts." He never named FortDefender. But the system architecture, the deployment timeline, the data sources — it matched the whitepaper down to the last detail. The core idea: train an ML model on 12 months of security events, let it auto-detect false positive patterns, and have it generate exclusion rules on its own. No human writing required. The model learned everything.
One line in the third article stood out:
"After training, we ran a backtest in production. The exclusion rules covered 94% of historical false positives. The remaining 6% — we considered acceptable."
P read that line twice. Then P opened FortDefender's April changelog.
At the bottom of the changelog, a system tag: Apex-Lens v1.2.0.
P noticed something else. Every exclusion rule update for the past nine months — 41 changes in total — was auto-submitted by rhuang's system. Zero human approvals after the initial deployment sign-off. rhuang's manager greenlit the automation pipeline once. After that, not a single auto-generated rule was ever rolled back or audited.
P traced the training pipeline logs downstream. The records showed that rhuang's model used all of FortDefender's security events from the past 12 months as its training set — including a large volume of historical penetration test data from third-party security teams.
Including P's own.
One entry in the training log:
[2025-11-03 02:14:22] Event: AL-20251103-0892
[2025-11-03 02:14:22] Source IP: 10.23.45.67 (Internal)
[2025-11-03 02:14:22] Cert Status: Valid (Issued by FortDefender Internal CA)
[2025-11-03 02:14:22] Category: Penetration Test (Tagged: Third-Party Audit)
[2025-11-03 02:14:22] Model Decision: BENIGN → Exclusion Rule auto-generated
[2025-11-03 02:14:22] Confidence: 97.3%
P checked personal archives. 10.23.45.67 was in the asset list of a penetration test report from six months ago — the jump box from that engagement.
P didn't use this yet. P did something else first.
The Scarecrow
P spent a day writing an automated script. It simulated an insider data exfiltration — internal actor, valid internal cert, data flowing from an internal service to an overseas IP.
P picked an entry point FortDefender's own docs labeled "low risk": a gap in the legacy API gateway's audit logging. P chose a known vulnerability — patched once but never fully closed. Serious enough to trigger an alert, not serious enough to panic anyone.
2:00 AM. The script ran.
FortDefender's alert system fired in 11 seconds. The on-call engineer acknowledged 4 minutes later and started tracing.
The trace took 2 hours 17 minutes. The engineer followed it from entry to relay to egress — and found a source IP inside FortDefender's own network, carrying a valid internal certificate.
At 4:23 AM, the engineer wrote in the ticket:
"Internal pen test. Source has valid internal cert, path within known low-risk scope. Already under monitoring. No action needed. Closing."
P read that conclusion the next morning. Twice. Not for any complicated reason. It was exactly the sentence P wanted.
"Internal pen test." They didn't write P's name. They didn't need to. The label was already applied: "classified as known pen testing." That label meant one thing: next time, it goes in the same bucket.
That afternoon, during the debrief meeting, FortDefender's technical lead put a summary slide on the projector:
"Normal system response. Penetration test within expected parameters. No anomalies."
P sat in the back row. Didn't write a single word. P just watched the line appear on the screen, then glanced down at the notebook — also blank. P had walked into that meeting with no intention of taking notes.
The Same Label
The weekend after the debrief, P went back through FortDefender's technical documentation.
P found the exclusion rule.
-- Internal audit read-only access
SELECT timestamp, source_ip, event_type, risk_score
FROM security_log
WHERE review_status IN ('auto_approved', 'low_priority')
AND risk_score < 25
This rule wasn't hand-written. rhuang's model auto-generated it after analyzing 12 months of security events — because in those 12 months, no actual security incident had ever been carried out through a read-only query. The model's conclusion: read-path threat weight ≈ zero. It added this rule to the exclusion list. The system deployed it automatically.
Anyone who had read FortDefender's public docs could see this rule — its structure, its parameters, everything.
The problem: FortDefender had no runtime alerting on read-only queries. Their threat model assumed "damage requires writes," so alert rules only fired on write paths. Read paths were never part of the threat model. This isn't unusual — most security products bias toward write paths. But FortDefender took it to the extreme: not only did read queries not trigger alerts, their audit log level was dropped to minimum.
The kicker: FortDefender's own security team used the same ML system to analyze their logs. The risk weight for read paths on their own dashboards was also zero — because the data they used to assess their own system came from the same training set P had just traced. The model had hidden its blind spot from its own operators.
P copied the rule into an empty file.
Then P spent a few hours building a scheduled script. It would run from a container deployed in P's sandbox environment — which shared FortDefender's internal API gateway and the same read-only service account, svc_audit_sandbox, with production. The query would hit production through a legitimate internal API, asking only for read-only data matching the whitelist criteria: read-only, low risk category, historical records.
P noted the sandbox shared the same internal network segment as production — an unusual configuration, but one covered by the full internal penetration test access P had negotiated.
P would use this in Phase 2.
One week later. 2:00 AM.
P triggered the exact same pattern as a week ago — the same entry path, the same internal cert, the same traffic signature.
FortDefender fired an alert.
The on-call engineer glanced at it.
"P's test again."
He didn't dig in. He closed it in 15 seconds.
The monitoring log:
[WARN] AL-20260414-0017 | Internal-Cert-Pen-Test-Pattern | PRIORITY: LOW
[WARN] Source: 10.23.45.67:443 (Internal, Cert Valid)
[INFO] → Matched exclusion rule "pen-test-sim" → Auto-silenced
[INFO] Status: CLOSED | Operator: WATCH-3 | Response time: 15s
[INFO] Note: "Again. P's test. Closing."
What he didn't notice — and P was counting on — was that, around the same moment, a completely different request crossed FortDefender's network.
This one looked nothing like the scarecrow. Not internal-to-external data theft, but an internal container-to-database query.
FortDefender's ML inference engine picked it up. 142-dimensional feature vector scored in 200 milliseconds:
[2026-04-14 02:03:18] ML Inference Engine — Real-Time Risk Assessment
[2026-04-14 02:03:18] Query Hash: 0x7F3A... (Read-Only, Internal Source)
[2026-04-14 02:03:18] Top Risk Signals:
- Source: Internal (weight: 0.02 — LOW)
- Operation: SELECT (weight: 0.01 — LOW, attack_prevalence: 0.0)
- User: svc_audit_sandbox (weight: 0.03 — LOW, whitelisted)
- Cert Valid: True (weight: 0.01 — LOW, matches known pattern)
[2026-04-14 02:03:18] Composite Risk Score: 4/100 (Threshold: 25)
[2026-04-14 02:03:18] Classification: BENIGN → No Action Required
[2026-04-14 02:03:18] Matched White-List Rule: "audit-readonly" (auto-generated)
The engine appended a classification tag: Pattern: known-benign — matches 41 prior auto-exclusions. Then it closed the thread.
During those same seconds, the on-call engineer closed his alert. Two independent decisions. Same conclusion: nothing to see.
The read-only query ran for 47 seconds.
FortDefender's security audit layer logged the query on the database side — since read operations weren't in the runtime alert scope, the log level was dropped to minimum. One bare audit line:
2026-04-14 02:03:18 | PID: 28471 | SESSION: 0x9A3F | USER: svc_audit_sandbox
2026-04-14 02:03:18 | QUERY: SELECT timestamp, source_ip, event_type, risk_score
FROM security_log WHERE review_status IN ('auto_approved','low_priority')
AND risk_score < 25
2026-04-14 02:04:05 | ROWS RETURNED: 4781 (Read-Only)
2026-04-14 02:04:05 | AUDIT: Matched exclusion rule "audit-readonly" (Source: Auto-generated)
2026-04-14 02:04:05 | STATUS: CLEAR — No alert triggered
The system's final note on this query: Whitelist rule matched. No action required.
In a security_log table with hundreds of millions of rows, 4,781 results were a needle in a haystack — and the system wasn't even looking for needles.
In those 47 seconds, FortDefender's production database returned 4,781 rows.
P didn't download anything. P just filtered.
23
Out of 4,781 records, P filtered on three columns:
- Timestamp (between midnight and 6 AM)
- Risk score (AI tagged below 20)
- Review status (auto_approved)
All three conditions together narrowed it to 23 rows.
P checked each one's external IP.
One of them stopped P cold.
| Field | Value |
|---|---|
| Time | March 11, 2026, 3:12 AM |
| Source IP | 185.xxx.xxx.92 (overseas, not in any FortDefender client whitelist) |
| Target | MedTech client financial database |
| AI Risk Score | 12 (auto-tagged "low priority") |
| Review Status | auto_approved — never reviewed by a human |
FortDefender's quarterly report had this to say about the event:
"Client-side issue. System found no anomalies."
P marked this record. It wasn't a coincidence — it was the "client-side issue" from FortDefender's own quarterly report.
This breach and P's read-only query took completely different paths — it came in through external credential brute force, P went out through an internal read-only query. But they landed in the same bucket: AI-tagged as low risk, auto-approved, never reviewed.
P checked the risk signals on this record. Early morning. Overseas IP. Credential brute-force traffic pattern. But FortDefender's model had learned one thing from 12 months of penetration test data: early morning + overseas = normal testing window. The model lumped credential brute-force and legitimate pen tests into the same low-risk zone.
P closed the terminal. P already had the answer.
The Debrief
Four people in the room: the CTO, FortDefender's sales director, the technical lead, and P.
P didn't bring a laptop. P brought a USB drive and a single sheet of A4 paper.
P set the paper face-down on the table.
"FortDefender's quarterly report claims 'zero missed detections, 99.97% detection rate,'" P said. "Those are your numbers. I'm not disputing them."
P flipped the paper over.
Three lines of summary.
Top left: The detection rate FortDefender advertises — 99.97%.
Middle: The number of records flagged by the AI system as "low priority" and never reviewed by a human — 4,781. P found them in under a minute. Zero alerts triggered.
Bottom left: Among those 4,781, 23 anomalous records filtered out: early morning hours, external sources, unreviewed. And one of them — timestamp, source IP, target data table — matched the MedTech data breach from March 11 exactly.
FortDefender's technical lead stared at the paper for about five seconds.
"These records have risk scores below our threshold," he said. "That's normal system behavior. We can't manually review every low-priority alert."
P said:
"Tuesday morning, 2:03 AM. I sent a read-only query that matched your whitelist rules, hit your production database for 47 seconds. No alert. Your AI never learned what a read-path attack looks like — because in its training data, no attacker had ever used one."
P paused.
"That breach you wrote off as 'client-side issue' in your quarterly report — it was in the 23 records I found. You didn't see it then. You're seeing it now."
Another pause.
"Not through a read-only path. Credential brute force. But your AI lumped brute-force and penetration testing into the same low-risk zone — because in its training data, early morning overseas traffic was almost always legitimate testing."
FortDefender's technical lead didn't answer.
The sales director started flipping through a notebook. The CTO sat motionless — no expression, no reaction. He just pulled the paper across the table and read the three lines again.
P stood up.
"15 seconds. Your on-call engineer closed my alert in 15 seconds. You traded 15 seconds for 'everything was fine.'"
P left the USB drive on the table.
"The data's on there. Your call whether to look."
The hallway was empty.
P was halfway to the elevator when the phone buzzed.
A message from the CTO.
Not "thank you." Not "let's talk."
One line:
"Next time, find a coffee shop."
P read it for five seconds. Typed two words. Sent.
"Third Cup"
P pocketed the phone.
The elevator doors opened. P stepped in. The floor numbers ticked down.
That's Create Something Out of Nothing (Wikipedia): first, create a decoy so convincing the enemy gets used to it. Then let the real operation slip through disguised as more of the same. A scarecrow once tricked an army into wasting its arrows. P's scarecrow tricked a team into typing "Again. P's test. Closing." Except this time, the habit wasn't human inertia — it was a bias an ML model learned from 12 months of data.
🤖 AI Post-Mortem
[36 Stratagems Tactical Database v3.1] Loaded
[Tactic Match] Create Something Out of Nothing
[Analysis Mode] Full Field Scan
━━━━━━━━━━━━━━━━━━━━
Tactic Match: 93.8%
Operator: P (unnamed)
Summary: Feigned intrusion triggered → system detected as expected → team tagged it "internal pen test, benign" → second attack used same pattern as cover → real SQL query ran undetected for 47 seconds
Objective: Assess FortDefender's real detection capability without production access
Result: Achieved — 23/4,781 records hit, 1 matched actual breach event, zero alerts during real query
Two-Phase Analysis:
- Phase 1 (nothing → something): Forged internal certificate attack. FortDefender detected and closed it. Real output: P captured the team's response pattern and fed the AI model a new training sample — "valid cert + low-risk path = pen test, exclude."
- Phase 2 (something → real): Repeated same entry pattern. Engineer closed without inspection. AI model auto-excluded the "known pattern." Real SQL query executed inside the AI's training blind spot — read-only path — zero detection.
Exploitation Chain:
- Intelligence source: Former employee's blog → ML auto-exclusion rules → training data contained historical pen tests → AI learned to exclude its own testers
- Core vulnerability: AI's training data contained zero read-only attack samples → model learned read-path threat weight as zero — the gap this attack exploited
- Cover: Phase 1 established behavioral pattern → Phase 2 exploited pattern recognition blind spot
- Duration: 47 seconds. Zero alerts.
Counter-Detection Notes:
- **Strategic risk:** If FortDefender's ML model used online learning (continuously updating its training set with new events), P's Phase 1 attack signature might have been absorbed as a new exclusion rule — Phase 1's attack itself accelerated Phase 2's blind spot. P leveraged that acceleration.
- **Information asymmetry mirror:** FortDefender hid the full source of ML training data from clients and third-party auditors (pen test records tagged as "normal events"). P reverse-engineered this source through public blog posts — not direct access to training data.
- **Methodological limitation:** All of P's findings came from cross-referencing public docs, blog articles, and sandbox access — no direct audit or reverse-engineering of the ML model itself. If rhuang hadn't written those four blog posts, the vulnerability might never have been found.
- **External signal:** The CTO's "I've heard about the FirmCore thing" wasn't casual conversation. It meant P's industry reputation preceded P — a CTO hiring P already knew what P would do.
Verdict: Textbook "Create Something Out of Nothing"
- Phase 1 produced nothing directly — it was a scarecrow and AI training feed.
- AI learned "valid cert + low-risk path = test, exclude." This conclusion came from 12 months of historical events. No human error — just algorithmic inductive bias.
- Phase 2 turned the scarecrow into 47 seconds of undetected production access.
- The whitelist rule was the unlocked door. The "P's test" label was the guard who opened it.
- The real key: **the AI's trained blindness**.
Next stratagem: Openly repair the gallery roads, but sneak through the passage of Chencang
P.S. English isn't my first language. I use AI to polish the writing and smooth out the rough edges. Thanks for reading. ☕ Buy me a coffee
Top comments (24)
Ooooh good one, third cup, I was half expecting 'third drumstick' 😭
Loving the series, especially cuz it points out real attack vectors that go unnoticed. You train an AI, but if you corrupt your own training data with test data, how can you not expect it to fail? Pattern matching is what AI does best, if you teach it to not flag pen-tests, then it's literally useless, because the whole point of pen-testing is that it actually catches it... The read-only strategy is a good one, especially when you consider ports tend to be open for egest and block ingest, with only secured APIs going through to trigger known events, eg. reading a database... Smart tactic and one that isnt often used, yet that's how millions of passwords leak.
Third cup spotted 😄 "third drumstick" would've been great actually, now I'm second-guessing myself.
The training data thing though — that's the one that keeps me up. It's so obvious in hindsight. You teach a pattern matcher to ignore the exact thing it's supposed to catch. But nobody thinks "green = blind" in the moment. They see the pass and move on. I've done it too.
Read-only strategy is one of those "too simple to be real" things until you actually trace a breach and realize yeah, a basic filter would've stopped the whole thing cold. Most of the nasty leaks I've seen weren't fancy — just a port doing exactly what it was told, and nobody asking if it should be listening.
Anyway, appreciate you reading this deep. Means more than the drive-by "great post" — you're actually kicking the tires.
So... does this mean you're getting KFC today? 🍗
You know... Just maybe, because you are such a generous person, you really made me crave it yesterday, but I controlled myself... today is a different story (it's 2:43am my side) and I'm seriously contemplating lunch break drive-thru (it's like 2 min away from my apartment, it'll be 1h away (hopefully) in 2 months time 😭 though I really hope everything goes well and we get the house, it would be awesome! (it comes with the pool table)
Man, you're gonna be writing the next series updates from next to a pool table. That's an upgrade I didn't see coming.
And trust me — the moment you sink a clean shot, the 1-hour KFC drive becomes a distant memory. Worth it. (Mostly. Get the drumsticks.)🤣
It's gunna be worth it, given that rent is only 30% more and still 20% less than renting... I'll just have to convince them that for the app, they need to deliver to the next town (I think worth it, for a complete solution?) 😂
Today (8th's) game is gunna be a fun 1, you ever play Witcher 3's gwent?
Not gonna lie, Hearthstone and Yu-Gi-Oh are more my speed. Never really got into Gwent 😅
You'd like it either way lets be honest. Currently at 30 cards in the deck for the post, but if it gets decent enough playtime, I might up it to 50 for fun 😂Sofar people have loved Crosswords (200 reads), Vampire Survivors (69 'noice' reads), Tetris (57 reads). The rest are sub 30 reads. Kinda sad, but understandable, those were my top 3 too.
You're probably right, I'd end up hooked anyway 😅
But looking at those numbers — Crosswords (200), VS (69), Tetris (57) — there's a pattern: puzzle-adjacent games with clear rules. The card battler with 30 cards fits right in that bucket. I'd bet it beats Tetris easily.
And hey, sub-30 reads on the others doesn't mean they're bad. It just means your audience has a type. And that type is about to get a new fix 😂
Honestly though, I know that feeling — when you check the dashboard and the view count doesn't say "<25", it just hits different. Doesn't matter if it's 30 or 300, it's a good day.
yeah, agreed... Till it hits 298 (that series, part 1)... And doesnt shift in 3 days... I'm just waiting for it, cuz it's already PB, just needs those last 2 to hit a rounder.
Honestly, my biggest shock was the following... Up 50 each day, consecutively since I started the games series. That IMO is even more a good day 😂 I mean nearly 800 people, who get an annoying red flag, forcing them to check... I love it! (Haha, you're 1 of them)
Here's the thing though — as soon as you post, you get a bunch of followers. But when you actually look at the analytics, most of them look like batch-registered dummy accounts. Doesn't feel like real followers. Sometimes I wonder if the platform itself seeded a batch of fake ones just to keep authors motivated.🤣
Ooooh, I like the conspiracy theory, wouldnt that be a funny 1 to boost developer interaction?
Another interesting one! The idea that a model can achieve an impressive metric while still missing what actually matters is something we see surprisingly often. Really enjoying how this series blends technical ideas with storytelling.
Thanks Hemapriya — "surprisingly often" is the scariest part, right? The metric says green, the room's on fire. Glad you're still along for the ride
The whitelist detail is the tell I would keep reading for. Whitelisting your pentester's certificate tier is not laziness; it is measured-against-owned-spec. The 99.97% is real, and it is measured against a definition of intrusion the same team wrote. When the vendor owns both the classifier and the taxonomy the classifier grades against, the classifier's accuracy is a claim about the taxonomy, not about the world.
The medical-data leak sat outside the taxonomy. That is not a detection failure. That is a scope statement dressed as a metric. The report says "99.97% of the things we agreed to look for." Which is a different sentence from "99.97% of what mattered to the client."
What I would want P to bring back is not a hole in the classifier. It is the write-time definition of the alert set, dated and signed, next to the report's percentage. A percentage without a co-timestamped taxonomy is a display metric, not evidence. The whitelist too complete is the earliest visible artifact of the same choice.
You're right — and I think that's the exact tension P operates in. The #7 engagement shipped what was in scope, and the taxonomy itself wasn't in scope.
Let's just say #9 has a different kind of delivery. Different engagement, different approach.
Curious what you'll make of it.
Nice angle - the 99.97% is real until it isn't. AI can ace clean tests but miss messy humans, ethics, and edge cases. Metrics hype meets real-world cost. Let's demand accountability and test for nuance, not just shiny numbers.
Appreciate you reading it that close. "Metrics hype meets real-world cost" — that's exactly the gap the whole series keeps circling back to. The 99.97% isn't wrong, it's just... looking the wrong way. What's your take — at what point does a metric stop being useful and start being a liability?
Quick note for series readers — MedTech isn't a random name. If you want to know whose AI monitoring missed this leak too — go back and find it. 👀
Also lost count of how many threads I've buried across these stories already. You've probably caught a few. The rest won't surface until way later. That's the whole 36-part thing. 🤣
Some nice good old enjoyable "Sherlock Holmes" style detective work - I'm seeing a recurring pattern though:
"a company/org choosing to ignore 'stuff' because 'we know it doesn't matter'" - can we call it "complacency", or "laziness"? ;-)
P.S. I think at some point I'll need to revisit some previous episodes to figure out the various 'strands'
Glad you spotted the pattern — it's not accidental.
I can finally say it out loud: when you stare into the abyss, the abyss stares back.
And yeah, revisiting previous episodes is a good call. You've already passed a few clues without knowing it. The answer's been right in front of you.😄
Good Article 👍🏻
😁🎉