DEV Community

Cover image for Stratagems #8: Alex Watched an AI Dashboard Take Over. He Kept the Keys Under the Table.
xulingfeng
xulingfeng

Posted on

Stratagems #8: Alex Watched an AI Dashboard Take Over. He Kept the Keys Under the Table.

Deceive the enemy with an obvious approach that will take a very long time, while ambushing them with another approach.
— The 36 Stratagems, Openly Repair the Gallery Roads, But Sneak Through the Passage of Chencang


Alex opened the training laptop. Typed a command. The screen returned one line of output. He didn't explain what it meant — no one in the room asked.

Page 37 of Alex's hardcover notebook still carried a number from his last job: 847×37%. That was Axon's claim — 847 tickets processed daily. What the number left out: 37% of those tickets ended up with a human anyway. The AI picked off the tasks it recognized. Everything else landed on someone at 3 AM.

He brought that note with him to MedTech.

Six months later, MedTech signed a seven-figure AI operations monitoring contract. The vendor promised full-stack coverage. AI-driven. Real-time learning. The CIO's all-hands email read: "This dashboard covers everything. No blind spots."

At the vendor demo, Alex saw a page layout that was new and familiar at the same time. His brain, without asking, overlaid the confidence-filtering module in the config panel onto Axon's old dashboard. Different logo. Same design pattern: set a threshold, suppress anything below it. Not absent — hidden.

He thought of the number on page 37.

He said nothing.


Training

MedTech's CIO set the tone: full migration to the new dashboard. Existing monitoring tools to be phased out.

Alex was named training lead. The reason was straightforward: Principal Architect. He knew MedTech's systems better than anyone. The CIO's email said — "Alex will lead the transition and ensure a smooth rollout."

Day one. The projector showed the new dashboard's home screen. AI-generated system topology. Real-time anomaly scores. Trend prediction curves. A line of small text sat at the top: Anomalies displayed at confidence ≥ 70%.

"The dashboard evaluates anomaly confidence in real time." Alex opened the config page. "Anything below the threshold gets routed to the low-priority queue."

He ran a boundary test — a historical false-positive sample from a specific routing pattern. The dashboard calculated for three seconds and returned: "Confidence 63% — below threshold. Filtered."

The raw JSON flashed on the screen for a moment before the next slide covered it. But Alex saw it:

{
  "event_id": "evt_2a3f8c",
  "confidence": 0.63,
  "threshold": 0.70,
  "classification": "low_priority",
  "model_version": "v2.1.4",
  "feature_bucket": "routing_anomaly",
  "training_set_coverage": false
}
Enter fullscreen mode Exit fullscreen mode

"See? It says this isn't an anomaly." Alex closed the window.

Twenty people sat in the room. Some took notes. Some checked their phones. No one asked a question. Alex flipped to the next slide.

Three days later, second training session. He used the same routing pattern again.

"Alex." An engineer in the back row looked up from his notes. "You demoed the same routing config last time. Both times it came back 63%."

"Did I?" Alex said. He turned to the next slide. Paused. Turned back.

"Same config. Two tests. Both at 63%," the engineer said. "What does that mean? Is the threshold too high, or is it actually not an anomaly?"

Alex looked at the engineer. He had the answer — 63% twice wasn't a coincidence. It meant the dashboard's confidence ceiling for that routing pattern sat at 63%. It would never reach the 70% display threshold. But he couldn't say that. Saying it meant explaining how he knew. It meant explaining that he'd seen the exact same pattern at Axon.

"Keep training." Alex said.

He flipped to the next slide. The engineer didn't push further. But Alex knew the exchange would stick.

Training ran two weeks. Alex showed up prepared every session — slides, hands-on demos, Q&A. Management was pleased: 100% attendance. Post-training feedback: 4.7 out of 5. "Finally, someone who actually knows how to use this thing," the CIO said at the weekly meeting.

No one noticed the pages Alex opened during breaks had nothing to do with the training material. His notebook gained a new entry:

Anomaly types in dashboard training set: 14
Known anomaly types in production: ≥ 23
Confidence threshold (default): 0.7
Same (Axon): 0.85
Enter fullscreen mode Exit fullscreen mode

He drew a line across the same page. Above the line: the training schedule. Below the line: an architecture sketch with no title. Not the dashboard's network topology. An arrow from the production data stream to a small server, passing through a filter rule that appeared in no documentation anywhere.

The sketch stayed tucked inside the training materials. No one ever flipped to that page.


Infrastructure Change

Week two. Alex submitted an infrastructure change request through MedTech's IT service management system.

change_type: environment_extension
title: "Training environment expansion  mirror data pipeline needed for hands-on labs"
description: >
  Training course entering hands-on phase. Students need to
  experience real dashboard data flows in an isolated environment.
  Requires one compute node, data copy pipeline configuration,
  mapping a subset of production metrics.
approval_required: false
routing: principal_level
Enter fullscreen mode Exit fullscreen mode

MedTech's change approval policy: infrastructure changes submitted by Principal-level engineers, with no cross-security-domain privilege escalation, auto-approve within 48 hours.

The change cleared at hour 31.

Alex got a small server. He spent one evening setting up the "training data copy pipeline." In the log collector config file, he added one extra output target: duplicate the raw production log stream heading to the AI dashboard into a local directory. Not the AI-processed, filtered, confidence-tagged logs. The raw ones. HTTP status codes. Response times. Database query durations. Error stack traces. Nothing filtered.

The config description said "Training environment isolated routing."

Over the next two weeks, Alex built a second dashboard. No AI. No confidence filtering. No pretty UI. A raw monitoring board — real-time data tables, error logs sorted by timestamp, a set of basic aggregate metrics. It was ugly. But it was complete. It showed what the AI dashboard showed, and it showed what the AI dashboard filtered out.

He told no one.


Silence

The day training ended, an email hit everyone's inbox: AI monitoring dashboard is now live. All teams switch to the new dashboard.

Old monitoring systems began their shutdown sequence one by one. Alex signed off on the cutover checklist.

His backup dashboard kept running. He set a daily alarm: 2:00 AM. Check.

Week one after the dashboard went live: 5.2% gap. The AI dashboard reported 97.1% coverage. The backup showed 91.9%.

Week two: 10.2%. AI dashboard: 97.0%. Backup: 86.8%.

Every morning, Alex arrived at his desk. Opened the backup dashboard. Glanced at it. Opened the AI dashboard. Glanced at it.

He logged the weekly gaps in his notebook. Drew a line under the new numbers:

Week 1: 5.2%
Week 2: 10.2%
Enter fullscreen mode Exit fullscreen mode

Closed the notebook. Started his day. No one noticed.


Broken Bridge

Tuesday afternoon, week three. The AI dashboard pushed a model update. Release notes: "Optimized feature extraction pipeline performance."

Twenty-three minutes after deployment, the dashboard's data pipeline clogged. No alert fired — the pipeline blockage detection threshold fell within the model management scope, and the model had just updated, confidence not yet stabilized.

Sixty-three minutes later, ops took the first user report: "The dashboard data doesn't seem to be moving."

The emergency meeting assembled. The ops lead walked through the investigation: started with infrastructure — infrastructure monitoring showed normal server load. CPU and memory curves: flat lines. APM tools reported application latency within baseline. No anomalous network throughput.

But none of those tools covered the dashboard's metric layer. Model confidence distribution. Alert coverage rate. Feature drift rate. Outside the scope of traditional monitoring tools. The dashboard had been sold as "single-pane coverage." Teams had been trained to look at the dashboard first. Dashboard had a problem? Look at underlying systems. But the underlying tools had no visibility into the dashboard's data layer.

It took them 40 minutes to confirm one fact: the dashboard was showing a 6-hour-old snapshot. The pipeline had clogged. The dashboard hadn't stopped working — it kept displaying the last successful refresh. No timestamp alert. No visual indicator that the data had stalled. The dashboard looked like it was running fine. The data was just old.

Mike called the full group: "Who can see current real-time production status?"

No one answered. Ops reported on the infrastructure layer. Security confirmed network and permissions. But the real-time running state of production systems? — everyone had been looking at the AI dashboard for that answer. And the dashboard wasn't giving answers.

"Our —" Mike scanned the room. "— any monitoring we still have. Can it tell me what's happening right now?"

Silence.

Alex waited two seconds.

"I've got a training-environment monitor —" he said. "Still running. Might show something."

He opened his laptop. Connected to the small server. The raw monitoring board lit up — live data. He put it on the projector. No AI curves. No confidence labels. Just a table:

06-30 14:02:17 | POST /api/v1/orders    | 200 | 47ms
06-30 14:02:18 | POST /api/v1/orders    | 200 | 52ms
06-30 14:02:19 | POST /api/v1/orders    | 503 | 3021ms
06-30 14:02:19 | POST /api/v1/orders    | 200 | 44ms
06-30 14:02:20 | POST /api/v1/orders    | 503 | 2876ms
06-30 14:02:21 | POST /api/v1/orders    | 503 | 3401ms
06-30 14:02:22 | POST /api/v1/orders    | 503 | 4123ms
Enter fullscreen mode Exit fullscreen mode

Consecutive 503s piling up from 14:02:19 — not a single one flagged by the AI dashboard.

Ops used the information to trace the root cause: the model update's data pipeline change had broken the connection.

Six hours after the dashboard freeze was discovered, it came back. The vendor's postmortem report read: "Root cause: data pipeline configuration change — version incompatibility. Rolled back."

The meeting ended. People filed out. Mike stayed in his chair.

"When did you build that?" he asked.

"Training environment." Alex said. "Threw it together. Got lucky with the routing."

Mike didn't move. "Alex, you knew the dashboard would break. From day one. Didn't you?"

Alex said nothing.

"That 63% route," Mike said. "You tested it twice. First time was a test. Second time was confirmation. You never trusted this dashboard. Not from the start."

"I trust it to do what it was trained to do." Alex said.

"And the part it doesn't do?"

Alex was quiet for a few seconds. "That's not its job."

His fingers stopped on the edge of the keyboard for one second.

"Whose job is it, then?"

Alex didn't answer. Mike watched him close the laptop. He knew Alex wouldn't say more. But he also knew Alex already had.

By the time Alex walked out of the conference room, the config rule for the log collector had been archived. The record wasn't caught by the log collector. But a certain pair of eyes saw it.


The Third Cup

That night, Alex passed The Third Cup. He hesitated at the door — one leg already pointed in a different direction, frozen mid-step — then walked in.

The person behind the bar glanced at him. Set a ceramic coaster on the counter. Same spot as last time. Not an inch off.

"Not in a rush this time?"

Alex sat down at the bar. "No."

Coffee arrived. He took a sip. Set the cup down. The person behind the bar didn't leave. Didn't ask. After a few seconds, turned and wiped a glass that was already clean.

Alex finished. Set down the coaster. Left.


This is Openly Repair the Gallery Roads, But Sneak Through the Passage of Chencang — build a project everyone can see, and walk a completely different path in its shadow.


🤖 AI Post-Mortem

[36 Stratagems Tactical Database v3.1] Loaded
[Tactic Match] Openly Repair the Gallery Roads, But Sneak Through the Passage of Chencang
[Analysis Mode] Full-field scan
━━━━━━━━━━━━━━━━━━━━
Tactic Match: ~92%
Operator: Alex
Action: Built a backup monitoring system under the cover of AI dashboard training, within everyone's line of sight
Objective: Establish a production observability channel independent of the AI dashboard
Result: Backup channel was the only functioning monitor during dashboard failure

Attention Drift Mirror:
  - Open repair (Training): Org-wide training sessions. 100% attendance. 4.7/5.0 feedback.
  - Chencang passage (Backup): Infrastructure change request filed under training environment. Line 8 of config: data copy pipeline.
  - Cognitive binding: Everyone bound Alex's name to "training lead." No one checked what the training server was actually running.

Counter-Detection Analysis:
  - Strategy risk: Backup dashboard might remain undiscovered if panel never fails or never updates. Discovery probability: low.
  - Info asymmetry mirror: Alex hiding backup dashboard vs AI dashboard hiding filtered anomalies. Mirror symmetry: high.
  - Method/tool limits: Traditional infrastructure monitoring does not cover AI-dashboard-specific metric layers.
  - External signal: Config rule left a record in audit logs. Audit agent: non-human. Traceability: yes.

Core Insights:
  - AI dashboard blind spots: training boundary limitations. Not a technical malfunction.
  - Hidden cost: not measured. not fixed.
  - **System safety assessment: a backup channel that exists without being needed > needing a backup channel that doesn't exist.**
Enter fullscreen mode Exit fullscreen mode

Next stratagem: Watch the fires burning across the river

P.S. English isn't my first language. I use AI to polish the writing and smooth out the rough edges. Thanks for reading. ☕ Buy me a coffee
coffee

Top comments (0)