DEV Community: xulingfeng

I Wrote 10 AI Stories in 10 Days. My Keyboard Started Smoking on Day 4.

xulingfeng — Tue, 09 Jun 2026 08:29:19 +0000

Biggest thing I learned writing the AI, Ego & Regret series: I argue with myself way more than I thought.

Every post goes through the same loop:

10 PM: "This story's fire. Gonna blow up tomorrow."

1 AM: "Wait — did I make it clear that 450ms wasn't just a random number?" → Scrolls back to check. Yes. OK. Move on.

Next morning: "What was I thinking? Scrap it. Rewrite from scratch."

The cover images were the worst part. One article went through 6 different backgrounds before circling back to the first one. 45 minutes I'll never get back.

Then there's that one line: "It was right about yesterday — and yesterday wasn't running anymore." Rewrote it 11 times. My wife walked by and said, "I thought you were writing code, not poetry."

Ben Halpern hit me with a 5-reaction combo while I was eating instant noodles. Almost choked.

Waking up at 3 AM. First instinct: check comments. Nothing. Go back to sleep. 5 minutes later: check again. Still nothing.

Writing code? I'm normal. Writing stories? I'm the guy verifying his own made-up RabbitMQ number at 1 AM.

Would I do it again? Yeah. Probably. But I'd get a better keyboard this time.

This coffee's about to run out — and I'm not done typing yet. If these stories made you smile, chuckle, or roll your eyes, buy me a coffee and keep the keys smoking ☕🔥

Also — if you've got a story that's been sitting in your head, something that made you laugh, cringe, or question every life decision that led to that moment — send it over. I'll turn it into a story. Yours could be the next one.

No pressure. Just a keyboard that's already warm.

My company packaged 12 years of my experience into an AI Skill, then laid me off. When it crashed, the CTO called at 5x my salary.

xulingfeng — Mon, 08 Jun 2026 14:00:00 +0000

A story about knowledge extraction, Kafka consumer rebalance, and what happens when a company discovers their AI Skill knows the past — but not the present.
Based on a submission from a community member. If you have a similar story or something you need to get off your chest — reach out. The next one could be yours.

Has your company ever extracted your experience into a system — then decided the system was good enough without you?

Have you watched an AI Skill handle 312 scenarios correctly, and wondered whether you'd be around when number 313 finally showed up?

This is that story.

Act 1 · Knowledge Extraction

The conference room had no windows. Three months.

Every Monday through Thursday I sat in that plastic chair — a voice recorder, a laptop, a mug on the table. Across from me sat an engineer named Caleb. His job was to ask me questions.

"Why PostgreSQL over MongoDB?"

"Why is the retry interval 450 milliseconds?"

"How did you calculate that alert threshold?"

I answered. He wrote it down. The red light on the recorder stayed on.

They called it a "Knowledge Transfer Initiative." The CTO's all-hands email was polished: We're preserving decades of institutional knowledge for the next generation of engineers.

In plain English: your experience is too expensive. We're packaging it into a Skill.

I didn't take it seriously at first. Every company has knowledge management. Then month three rolled around. Caleb stopped asking questions. He started validating.

He pulled up an incident I'd debugged the year before — a production outage. He asked me to reproduce the diagnosis. Then he let the system do it.

The system said: Message broker latency spike detected. Retry logic at 450ms interval will amplify under queue backpressure. Recommend adaptive backoff.

I read it three times. It was right. It was my chain of reasoning — but it said it faster than I could.

The CTO stood next to me, smiling. I've seen that smile too many times in Silicon Valley. It's not happiness. It's sign-off.

Act 2 · "96.8% Accuracy"

The moment I knew something was wrong was the day the validation report came out.

A full row of people sat across the table — CTO, HRBP, VP of Engineering, and a woman I didn't recognize with a consulting firm logo on her blazer.

The projector displayed one number: Knowledge Retention Rate: 96.8%

Caleb delivered his findings. "After three rounds of validation, the AI Skill achieved 96.8% diagnostic accuracy across 312 historical failure scenarios. The remaining 3.2% deviation is suboptimal recommendations due to insufficient context — correctable with guidance."

Applause. The CEO turned to me and said something I'll never forget.

"Mark, you created your own replacement."

Everyone laughed. I laughed too. What else was there to do?

After the meeting I sat in the parking lot for a long time. My coffee had gone cold. Outside the window, that unchanging California blue — the same one it's always been. My phone buzzed. A text from my wife: What time are you coming home?

I typed: "Soon." Then I deleted it.

I didn't know how to tell her I'd spent three months turning myself into a manual. And once you write the manual, you don't need the original.

Act 3 · The Severance

The Monday after, the CTO closed the door for our one-on-one.

"Mark, I won't sugarcoat it. Your position's been eliminated as part of the restructuring."

He slid an envelope across the table.

N+3. Three months of severance. The standard Silicon Valley we're not firing you, we're helping you transition treatment.

I signed without stopping. Not because I didn't care — because I'd already played out how this ended before I walked in.

My last day was a Friday. It took me under forty minutes to clear my desk. One backpack. One monitor stand. Three pen caps — I still don't know how three pen caps ended up in my drawer.

I sat in the car and searched three things on my phone:

LLC registration (California, single-member, $70, 15 minutes)
Professional liability insurance (industry standard for consultants, ~$1,200/year)
Rate benchmarking for senior infrastructure consultants (median: $215/hour)

That night I registered a company. Mark Johnson Consulting LLC. No office, no employees, no VC pitch deck. One clause, written clear: project-based only. No AI in the delivery chain.

Act 4 · AI Skill Goes Live

The first full quarter was a showpiece.

The AI Skill absorbed 70% of tier-2 operations tickets. New engineers went from six months to three weeks to reach productivity. The CEO used a slick slide at the all-hands:

Before: 12 years of Mark's experience locked in his head.
After: 12 years of Mark's experience available as a prompt.

Nobody mentioned Mark himself. They didn't need to. He was already packaged.

My wife noticed before I did. The weekend after I left, she looked at me across the kitchen and said something I wasn't expecting.

"You seem different."

"Different how?"

"When you sit at your computer and don't talk — that tension's gone. You used to be waiting for things to break. Now you're waiting for clients."

She was right. I was waiting.

I saw on LinkedIn they'd hired a new Platform Lead. First thing he did was rebuild the monitoring stack. The Skill? Nobody re-ran validation after they migrated to Kafka. Why would they? The person who built it wasn't there anymore.

Act 5 · Crash

It happened at 3:47 AM on a Wednesday.

The distributed tracing system lit up. P99 latency on the core payment chain went from 80 milliseconds to 12 seconds. Three minutes in, the first transaction timed out and rolled back. At ten minutes, the payment queue started backing up. At twenty-three minutes, the rollbacks triggered a cascade — two downstream services began refusing requests.

The on-call engineer pulled up the AI Skill. It scanned the logs, identified the pattern, and returned a diagnosis.

Detected message broker latency. Applying known mitigation: activate retry queue with 450ms backoff.

That diagnosis had been correct 312 times out of 312 historical scenarios.

This was number 313.

The 450ms retry window was a compatibility shim I'd written five years earlier for RabbitMQ. The number wasn't random — I'd spent two weeks load-testing on a RabbitMQ cluster to find the exact gap that cleared its Erlang VM GC cycle.

But that was RabbitMQ. Retired for three years. They were running Kafka now.

Kafka's consumer groups use a poll-based protocol — every consumer has to keep polling the coordinator at a configured interval, or the coordinator marks it dead and triggers a rebalance. My retry worker was synchronous — process one message, pull the next. At 450ms a round, a queue of a few dozen messages stretched the poll window by multiple factors.

# What the Skill did (the 450ms retry window)
def handle_queue():
    while queue_not_empty():
        msg = fetch_one_message()          # one at a time
        retry_with_backoff(450ms)          # ⏱ 450ms per message
        process(msg)                       
        poll_consumer()                    # "still alive?" → into debt

# What happened with 60 queued messages
poll_timeout = 5000                        # 5s before coordinator marks dead
time_per_round = 60 * 450                  # 60 messages × 450ms = 27,000ms
# 27,000ms > 5,000ms → poll timeout missed → rebalance triggered

Once the rebalance kicked in, the whole group started oscillating. New consumers claimed partitions. Old messages got redelivered. Latency climbed again. A sticky problem turned into a snowball.

The AI executed a strategy that was 100% correct on the old architecture. On the current one, it was pouring gasoline on the fire.

The AI didn't fail because it was wrong. It failed because it was right about yesterday — and yesterday wasn't running anymore.

Act 6 · The Call

My phone rang at 4:12 AM.

A name I hadn't called in months — my former CTO.

His voice was calm. Too calm. When an engineer calls at 4 AM and sounds that steady, something's on fire.

"Mark. I need to ask you something."

"Ask."

He laid out the incident in three sentences. I thought he'd ask about rollback strategy. Damage containment. He didn't.

"You left a note in the RabbitMQ migration docs. At the time, nobody understood it. We thought it was a stale config comment. I think I understand it now."

I leaned back against the headboard. The only light in the room was the phone screen. My wife stirred beside me. Didn't wake up.

The note said: 450ms matches RabbitMQ GC window. Do not reuse outside this context.

Silence on the other end. About four seconds.

"I know," he said. "I'm out of time, Mark. Come back."

"I'm not asking if you'll come back." His voice shifted. "I'm asking you to. I'll pause everything for the rest of the month. You call the shots. Name your price."

Act 7 · The Price

"Five times my old salary."

I heard his breath catch.

"… Deal. But I need you on-site for two weeks. Contract. Bring your own equipment. No AI touches your delivery chain."

"Done. Send the contract to my lawyer."

I didn't have a lawyer. What I meant was: this runs at my pace.

After I hung up, I scrolled to a name I hadn't called in nearly a year — Mike, college roommate, law school grad. Sent one message.

Got a contract. Mind glancing at it?

He didn't reply. At 4 AM, who's awake except people in IT?

I waited. Then I typed five more words. I sent them to him, but they were really for me.

This time I set the price.

You've been through this before? Your knowledge extracted, packaged into a Skill, turned into the reason you weren't needed anymore. And when the Skill hit something it didn't understand, they came back. Is there a better answer than "no"?

The Skill knew every past scenario. It just couldn't see that the infrastructure had changed.

The company saved six figures on headcount. They just forgot validation only works when the person who wrote it is still there to re-run it.

When the system went down — who did they call?

Follow for more stories about AI extraction, implicit knowledge, and what happens when companies discover their Skill only remembers the past.

I couldn't teach the AI Skill to understand context. But the stories stay open. buy me a coffee ☕

Our VP Said AI Would Test Itself. Day 3 Cost $2.8M. I Had the Screenshots Ready.

xulingfeng — Sun, 07 Jun 2026 09:19:56 +0000

This story is based on a submission from a community member. If you have a similar story or something you need to get off your chest — reach out. The next one could be yours.

Act 1 · The Tech Meeting

"Starting today — no more hand-written code."

Marcus, the new VP of Engineering, put a slide up on the big screen.

Four words: WRITING BY HAND IS OVER.

I was sitting in the back row, against the wall. Seven years at this company. Three core modules that I'd built from scratch. Two production systems that ran the company's primary revenue stream. Now someone was telling me — don't write anymore.

The room went quiet for about five seconds. Then people started whispering. Someone pulled out a phone and took a picture of the slide.

Marcus added: "AI coding isn't optional — it's a mandatory development standard. We benchmarked this. AI writes code 400% faster than humans. Anyone still typing manually is wasting the company's time."

I raised my hand.

"Who reviews the code?"

"AI reviews it."

"Who writes the tests?"

"AI tests itself."

"What if AI writes something wrong?"

Marcus laughed. Not a polite laugh. The kind of laugh you give someone whose question you've already decided doesn't matter.

"Let me ask you something."

He paused.

"Do you really think — you, one person — have more training data than Orion-7? "

People started laughing. Not supportive laughter. Pile-on laughter.

"Or do you think the world's AI companies — hundreds of billions in investment, tens of thousands of GPUs — built something that's less reliable than one backend developer?"

Nobody was looking at me anymore. Everyone was watching him, waiting for the kill shot.

He didn't take it. He just smiled. "Starting next sprint, it's AI across the board. Anyone who has concerns — my door's open."

Act 2 · The Reassignment

I didn't go to his door.

HR notified me the next day: I was being moved to the Legacy Systems team.

Three people. Three projects that hadn't been touched in five years. No new requirements. No QA handoffs. No deadlines — because nobody used the software.

I handed off my work and sat down at my new desk.

A beat-up laptop sat there. Screen cracked in the bottom-right corner.

I opened my email — three new messages. All from the repo: my write access had been revoked on all three legacy repositories. Review-only.

I went to find Marcus.

"About the repo permissions —"

"Oh, right." He looked like he'd been waiting for me. "New system code runs under the new policy. You're not familiar with the AI workflow, so I set you up with read-only access to the legacy repos."

He leaned back in his chair.

"Oh, and I cancelled your Copilot license. Those three legacy projects — not worth the GPU cycles for AI anyway. Write it by hand. You don't trust AI anyway."

He smiled. He knew exactly what that line would do. He was waiting to see if I'd say something.

I didn't.

I went back to my desk, opened the cracked laptop, connected to the internal network, and cloned the new system's main repo. Public read-only access. He'd forgotten to lock it down.

Act 3 · Building a Case

The new team delivered the order module in two weeks.

The CEO posted in the company-wide Slack channel:

"New order module is live. AI-generated code end-to-end. Zero-defect delivery. Let's give the team a round of applause."

The channel filled with clapping emojis. Someone @'d Marcus: "That's insane efficiency."

I was in the channel. I didn't say anything.

What the CEO called "live" wasn't a full cutover. It was a canary — the new module running alongside the old system. Low traffic. Simple orders. Nothing that would hit an edge case.

But the code was sitting on the server.

I opened a terminal in the new system's code directory and ran one line:

pytest tests/ --tb=short -q

Output:

..............F.......FFF......
====== 8 failed, 15 passed in 2.87s ======

15 passed, 8 failed.

I ran another line:

grep -rn "TODO" src/ | wc -l

47.

One more — I hit the API:

curl -s https://new-api.xxx.com/v2/order/123456 | jq '.discount_detail'

Response:

{
  "order_id": "ORD-2026-0614-0123456",
  "subtotal": 12500.00,
  "line_items": [
    {"sku": "A-200", "qty": 10, "price": 800.00},
    {"sku": "B-150", "qty": 5, "price": 900.00}
  ],
  "discounts": [
    {"type": "promo_early", "amount": 500.00},
    {"type": "member_gold", "amount": 1875.00},
    {"type": "volume_tier_3", "amount": 1250.00}
  ],
  "discount_detail": null
}

The API docs said: "discount_detail: object, required. Contains breakdown of all applied discounts." It returned null.

I stared at the screen for thirty seconds.

Then I created a folder.

Called it evidence/.

I opened my email. Wrote to Marcus with the subject "New Order Module — Test Failures Found." Attached the pytest output, the TODO count, and the null response.

He didn't reply.

For the next month, nobody touched that code. Marcus was preparing the Q3 board deck. The AI team was writing prompts for the next module. And me — I was on Legacy Systems. My job was to not touch anything.

Act 4 · The Glory

The new order module ran canary for a month. Low traffic. Simple orders. Zero incidents.

The CEO was happy. Marcus asked for a product demo.

The screen in the conference room showed a perfect happy path. Clean data. Discount calculations stacking correctly. Coupons applying. One-click ordering. Everything worked.

The CEO said: "You're two months ahead of my timeline."

Marcus smiled.

"Because I used AI. Some people are still writing code by hand. You can see the difference, right?"

He glanced at the back row.

I was in the back row. What that look meant — everyone in the room knew.

I stood up.

"The module — "

Marcus cut me off.

"— has a problem? Zero incidents in canary. Full green on the test dashboard. You've been on legacy — have you even looked at the new code?"

The CEO glanced at me. Then turned back.

"Marcus — run through the full Q3 cutover plan again."

I stood there for maybe three seconds. Nobody looked at me. Nobody asked if I had more to say.

I sat down.

The CEO made the call: "Full Q3 cutover. Start sunsetting the legacy systems."

Marcus got his scope expanded — he now owned the entire engineering org. No more CTO sign-off needed.

People patted him on the shoulder on the way out. "Well deserved." He said thanks. He walked right past me — he didn't need to look at me anymore.

Act 5 · The Crash

Day 3 of the full cutover.

4:27 PM. The CFO walked straight into the CEO's office.

Not a complaint. Worse.

"A major client called me directly. Every order we've processed in the last three days has the wrong discount amount. Not some of them — every single one. "

Root cause analysis:

AI-generated code was computing compound discounts
with no defined calculation order.
The prompt didn't specify:
  "calculate line-item discounts first → then subtotal → then member tier."
The model guessed an order.
It guessed a different one on every request.

Canary hadn't caught it — because canary traffic only hit simple orders with single discounts. Full production on day one brought in bulk orders from major clients. Every single one computed discounts differently.

Impact: 3 consecutive days. $2,800,000 in unreconciled billing.

Metric	Value
Orders affected	1,247
Average discrepancy per order	$2,246
Largest single order discrepancy	$18,740
Enterprise clients impacted	16
Longest uncorrected window	72 hours

The CEO called Marcus into his office. I wasn't in the room — but someone told me later he came out looking pale. CEO had HR schedule a postmortem.

Act 6 · The Reckoning

Postmortem. The CEO chaired. Marcus sat in the front row. The CTO, CFO, finance team, and the enterprise account managers filled the room.

I was there too. Back row.

Marcus opened: "The prompt missed a business rule. The team is fixing it now."

The CTO asked: "Canary ran for a month. How did nobody catch this?"

A pause. "The canary environment didn't have complex discount data."

"So your logic is — if the data wasn't in canary, it wouldn't be a problem in production?"

The room went quiet.

I raised my hand.

"Can I share something?"

Marcus turned and looked at me.

The CTO nodded. "Put it on screen."

I walked to the front. Plugged my laptop into the projector.

Slide 1:

# src/orders/discount_calculator.py:127

def apply_compound_discount(order: dict, user: dict) -> float:
    """
    Apply tiered discounts to an order.
    TODO: handle compound discount ordering — currently iterates
          over discount fields in JSON insertion order, which means
          the calculation sequence depends on how the client serialized
          the payload. This is not guaranteed to be consistent across
          requests. Needs explicit ordering: line_item → subtotal → member.
    """
    total = order["subtotal"]
    for key, rule in order.get("discounts", {}).items():
        total -= rule["amount"]  # ← order depends on dict insertion order
    return max(total, 0.0)

"This isn't a bug. It's a TODO. There were 47 of these in the codebase when it shipped to production. Distributed across every critical path in the module."

Slide 2:

# tests/test_discount_complex/test_discount_order.py — last modified 2019

def test_compound_discount_sequence():
    """Verify discount application order: line_item → subtotal → member.

    This sequence was confirmed with Finance in 2019 and must not
    be changed without cross-team approval.
    """
    order = sample_order(
        items=[{"sku": "A", "price": 100.00, "qty": 5}],
        subtotal_discount_pct=10,
        member_tier="gold",
    )
    result = legacy_discount_engine.apply(order)

    # Expected: 500 * 0.9 (subtotal) * 0.85 (member) = 382.50
    assert abs(result.final - 382.50) < 0.01
    assert result.application_log == [
        "line_item",
        "subtotal",
        "member",
    ]

"The old system had this test. Written five years ago."

Slide 3:

vibe-coding-take-down/evidence/
├── screenshot-01-unit-test-failures.png  (15 pass, 8 fail)
├── screenshot-02-todo-list.png           (47 TODOs)
└── screenshot-03-api-null.png            (API returning null)

Generated: 27 days before production cutover.

"Twenty-seven days before this went to production — I ran the tests. Fifteen passed, eight failed. Forty-seven TODOs. I screenshotted every single one."

I let that sit.

"Twenty-seven days. These tests sat in CI failing for almost a month. Did anyone open them? "

Silence.

I asked again: "Did anyone look at the test report?"

Marcus didn't look at me. He looked at the CTO.

"During canary… we were focused on business metrics. Unit tests weren't fully executed. The timeline was tight."

I didn't answer him. I put up slide four.

Slide 4: a screenshot of a Slack message.

Marcus's own message, sent before AI development began.

"Don't spend too much time on testing. AI generates its own code — it'll verify itself. Let's ship first, we'll backfill tests later."

Nobody moved. Nobody spoke.

I closed my laptop.

"Marcus. You said AI would test itself. Forty-seven TODOs unaddressed. Eight failing tests. Two point eight million dollars lost. What exactly did your AI verify? "

Marcus didn't answer.

The CEO spoke. He wasn't looking at Marcus. He was looking at me.

"When did you build that folder?"

"Twenty-seven days before production."

"Why didn't you say anything?"

"I did. At the demo. I stood up and tried to say something. Marcus cut me off. Nobody let me finish. "

Act 7 · The Aftermath

The CEO sent a company-wide email the next morning:

"Effective immediately: all AI-generated code must pass a line-by-line review by a senior engineer before merging to main. Test coverage requirements are non-negotiable. This policy cannot be overridden."

Marcus kept his title.

But the same day that email went out, HR released an updated org chart. A new Code Review Board, independent from Engineering, reporting directly to the CTO. I was put in charge.

Three people on the board. The other two came with me from Legacy Systems — solid engineers who never learned to play the room.

Marcus's team no longer had final say on what got merged.

Someone asked me later: "You waited a month for one meeting? Was it worth it?"

I said: "I didn't wait a month for a chance to prove I was right."

"I waited for him to say 'AI will test itself' in front of the whole company. I wanted to watch him swallow every word."

I deleted the evidence folder later. Didn't need it anymore.

The next day I walked past Marcus's office. The door was open. He was sitting at his monitor, reading a document about the new code review process — the one I'd written.

Forty-seven TODOs. Not a single one became a bug. The AI just guessed a calculation order where no rule was specified — a different guess on every request. When the process itself doesn't exist, AI will help you prove that "doesn't exist" is good enough.

What's the worst "AI will handle it" decision you've seen on your team? Drop it in the comments.

It took me thirty seconds to create an evidence/ folder — and one second to delete it. Forty-seven TODOs taught me one thing: the only calculation order I trust is the one my coffee cup follows.Buy me a coffee ☕

Follow for more stories about AI testing, quality engineering, and what happens when the code is generated but not verified.

I Tried to Fix a Vulnerability. A $1,400,000 AI System Said No. Twenty Days Later, That Vulnerability Cost $4,200,000.

xulingfeng — Sat, 06 Jun 2026 00:44:28 +0000

This story was shared by a fellow developer on DEV who asked to remain anonymous. If you've got a story to tell — come find me. Your name won't appear anywhere.

Based on real microservice security design patterns. About an engineer whose PR got blocked by an AI security system — he thought he was fixing a vulnerability. Turns out, someone had a vested interest in that vulnerability staying open.

1. $1,400,000

All-hands meeting. CTO James stood at the front, a number on the screen:

$1,400,000

"This is what we're spending on security this year." He pointed at the number. "The biggest piece — right here."

He clicked the remote. VoidSentinel's architecture topology appeared on screen.

"VoidSentinel — an AI security platform. Integrated into our CI/CD pipeline. Starting today, every PR involving internal service-to-service calls — it reviews them automatically."

The CEO didn't show up today. James didn't mention it. He looked straight at Mark — VP of Security.

Mark took the mic. "VoidSentinel has been running in our pre-production environment for three weeks. It's caught 47 high-risk patterns. Zero false positives."

He paused.

"— Of course, some people might feel uncomfortable when their PR gets blocked. But this isn't personal. This is the security standard. "

He wasn't looking at me. But I knew who he was talking about.

2. High Risk. Denied.

The story started three weeks earlier.

We had a payment service and a user service that talked to each other internally. They shared an old API key — one key across thirty-plus services, unchanged for five years.

It wasn't that nobody knew. It just never made it to the top of the backlog.

On Day 1, I opened a PR: add independent service-to-service auth between the payment and user services. Not much code — a new token exchange module, three call sites modified.

Five minutes later, VoidSentinel's automated comment hit:

"High-risk alert: Unauthorized internal access pattern change detected. This PR has been automatically rejected. Contact the security team."

I stared at that comment for a long time.

This isn't "high risk." This is adding independent auth between two services. This is fixing a hole that's been open for five years.

I replied under the PR: "This PR fixes a shared credential vulnerability between services. It is not introducing a new attack surface. Please escalate for manual review."

The next day, the reply came:

"VoidSentinel's verdict is final. Please reference the system's suggested modifications to adjust your code."

The system's "suggested modification" was: don't change anything.

3. This Is Final

I walked to Mark's office.

"That PR — it's not an attack. It's a fix. The payment service and user service are still sharing a five-year-old API key. If any one service gets compromised, the other thirty are exposed."

Mark didn't look up.

"VoidSentinel's model analyzed your code and flagged it as high risk. I trust its judgment."

"Its model can't read intent. It just sees someone modifying service-to-service auth — which looks exactly like what an attacker would do."

Mark finally looked up.

"So what's your suggestion? Everyone can just modify internal auth whenever they want, because 'I had good intentions'?"

" — Add a manual review gate. High-risk PRs go through AI first, then human review if rejected."

"I am the human review. The system flagged high risk, and I agree. PR denied."

I went back to my desk.

I tried a different approach: instead of modifying the auth layer, add a call allowlist at the gateway level. That wouldn't trigger VoidSentinel's "service auth change" detection pattern.

Submitted the PR.

VoidSentinel verdict: High risk. Denied.

— It recognized I'd changed the approach. The model's coverage was deeper than I thought.

But it still couldn't cover the real vulnerability.

4. PIP

Day 3. I got the PIP notice.

Subject: "Assessment of Alignment with Company Security Protocols"

Mark's office. He slid the PIP notice across the desk.

"Two attempts to bypass VoidSentinel's security review. This isn't a technical issue — it's an attitude problem."

"I submitted fixes. Not bypasses."

"From the system's perspective, you modified internal auth paths. The system flagged high risk. You changed the approach and submitted again. That's a bypass."

I looked at the PIP notice.

"30-day improvement plan. Success criteria: zero compliance violations. Fail — you're out."

I signed it.

Walking out of Mark's office, I ran into Jay from ops in the hallway. He lowered his voice:

"— You know Mark's a board appointment, right? The CEO's been trying to move him for almost a year. Watch your back. Also — VoidSentinel? He picked it. Sold the board on it for six months before they approved it. You tell him he picked wrong — he's not going to admit it."

I nodded.

Back at my desk, I opened VoidSentinel's architecture documentation.

5. Nobody Knew

For the next 24 days, I did three things.

One: went through the PIP motions during the day.

No more touching service-to-service auth in any PR. VoidSentinel stopped flagging my code. Mark sent PIP progress emails every Friday. I replied every Friday with "on track."

Two: built a proof of concept at night.

VoidSentinel's core blind spot — I confirmed it — wasn't that it was too sensitive. It was that it applied the exact same detection rules to fixes and attacks. The code that fixes a vulnerability and the code that exploits one — to VoidSentinel's model, they're indistinguishable.

That means: it can't tell who's fixing something and who's breaking something. It only knows someone's modifying service-to-service auth. Doesn't matter who you are, or why you're doing it.

Three: set up a monitor.

It didn't block anything. It just logged. Logged every "high-risk" PR VoidSentinel intercepted. Logged the PR content. The submitter. The resolution.

I piped it to a server I controlled. Auth call logs from 138 service nodes, synced in real time.

Mark didn't know.
Jay didn't know.
Nobody knew.

6. Zero Incidents

Day 10. Mark sent a company-wide email:

"VoidSentinel has been live for two weeks. 217 high-risk PRs intercepted. Zero security incidents. This is the security standard we need."

I read that email. Then I opened my monitor.

Out of 217 interceptions, 43 were security fixes that got falsely blocked. And 3 of those fixes were for vulnerabilities that — if exploited — VoidSentinel couldn't detect at all. Because those vulnerabilities weren't in north-south traffic. They were in the gaps between services.

VoidSentinel can't see what it isn't deployed to look at.

7. No

Day 22.

Mark's office. "You're more than halfway through the PIP. How are things?"

"On track."

"Anything you want to talk about?"

"No."

He looked at me.

"Alex. Your technical skills are not in question. If you're willing to put something in writing — acknowledging VoidSentinel's security review process — the PIP can end early."

I stood up.

"VoidSentinel's security review process — 43 false positives. 3 of them blocked real fixes. That's the only thing I can put in writing."

I didn't wait for his response.

8. $4,200,000

Day 27. 2:47 AM.

My phone lit up. Not a VoidSentinel alert — my own monitor.

"Anomaly detected: service-to-service auth call — credential KT-9f4 — flagged as 'high risk' by VoidSentinel, then automatically cleared three minutes later."

I sat up. Opened my laptop.

At 1:12 AM, someone had initiated an auth call on the payment service's dev interface. Credential KT-9f4 — a service account belonging to an employee who left four months ago.

VoidSentinel flagged it. Then automatically cleared it.

Reason: "Credential is valid. Call frequency normal. Classified as normal operation."

— The attacker used a legitimate credential. Normal frequency. Normal path. They just accessed a service they shouldn't.

VoidSentinel recognized "someone's modifying service-to-service auth" — but after evaluating the call frequency and credential validity, its model decided this was a routine operation.

It did nothing.

I opened VoidSentinel's dashboard. Threat score: 0.02. All green.

Then I opened the Slack channel.

"Payment reconciliation is off by $4.2 million. Anyone looking?"

I took three screenshots:

My monitor log — credential KT-9f4 at 1:12 AM, cross-service call
VoidSentinel's audit log — "high risk" → cleared three minutes later → "normal"
The Slack message — $4.2 million missing

Then I opened the PR I'd submitted on Day 1.

VoidSentinel's verdict: High risk. Denied.
Fix content: independent auth between payment service and user service.
— The exact same link that was exploited at 1:12 AM.

I sent the three screenshots and that PR to CTO James. CC'd the CEO.

One line in the body:

"Day 27. The thing I tried to fix on Day 1 — someone used it tonight. VoidSentinel flagged it for three minutes, then let it through. PIP has 3 days left. Your call."

9. RCA

CTO James. VP Mark. VP of Finance. VP of Legal. The CEO — everyone in the room. One person I didn't recognize sat in the corner. Someone called him a "board observer."

James ran through the incident timeline first. When he said "the attack vector exploited an internal auth gap," the CEO raised a hand and stopped him.

"Alex. When did you submit your first PR?"

"Day 1."

"What was it for?"

"Independent auth between the payment and user services."

"What was VoidSentinel's verdict?"

"High risk. Denied."

"And then?"

I didn't answer. I opened the PR.

I flipped to the first page — the code diff. A new token exchange module. Three call sites changed.

"This was a fix. VoidSentinel flagged it as high risk."

Then I flipped to the 1:12 AM attack log.

"This was an attack. VoidSentinel flagged it as high risk — then automatically cleared it. Reason: 'credential valid, frequency normal.' Same system. Same vulnerability. Same detection model. One blocked. One let through."

I closed the laptop.

"— Because VoidSentinel can't tell the difference between a fix and an attack. It only knows someone's modifying things. Whether that person is patching or exploiting — it doesn't know. It can't."

The room went quiet for a few seconds.

The CEO didn't look at Mark. He looked at me.

"When were you put on PIP?"

"Day 3."

"What was the reason?"

"Two attempts to bypass VoidSentinel's security review."

The CEO didn't say anything. He turned to Mark and asked quietly:

"He submitted a fix. You gave him a PIP. Is that right?"

James's eyebrow twitched.

"Mark. The report you gave me said 'two attempts to bypass security review.' You didn't mention he was submitting a fix."

Mark opened his mouth. "His submission method — "

"He proposed independent auth. You said no. He tried a gateway-level approach. You said no. The vulnerability he reported — someone exploited it tonight. Your system — flagged it for three minutes, then let it through."

The CEO's voice was flat. Flat enough that the silence after it felt heavier than any shout.

Mark couldn't find words.

The CEO stood up.

"Alex. My office."

I stood up. In my peripheral vision — Mark stayed seated. He didn't follow.

10. Eleven Months

The CEO walked fast down the hall. I followed. He didn't turn around.

"PIP rescinded. Effective immediately."

"Okay."

"Your fix gets deployed today."

" — VoidSentinel will block it."

"VoidSentinel gets reconfigured this afternoon. Your PR gets re-reviewed after the config update."

"Okay."

He stopped.

"That monitor you built. How long has it been running?"

"Day 3."

"The day you got the PIP?"

"The day I got the PIP."

He looked at me. Maybe five seconds.

"You know what it means — being on PIP and still building something that caught an attack before a $1.4M system did?"

" — It means those 27 days on PIP weren't wasted."

He smiled. Not a "good job" smile. A different kind.

" — You know how Mark got that seat?"

I paused.

"Board appointment, I heard."

The CEO nodded.

"I waited eleven months for a reason."

He didn't finish the sentence. He didn't need to.

I went back to my desk. Mark's desk was empty. HR had completed everything by 3 PM.

I never replied to Mark's last PIP progress email. The last one sat at Day 27.

Subject line: "On track."

Later, I reopened that Day 1 PR. VoidSentinel ran it again — this time it passed, with a note: "Low risk. Recommended to merge."

I didn't feel good about it.

I just remembered what Jay said to me in the hallway that day.

— "The CEO's been trying to move him for almost a year. Watch your back."

The reason he was waiting for was never about me.
It was about someone finally handing him a board he didn't have to wait on anymore.

Folks, when you submit a fix that gets blocked —
are you fixing a bug, or finishing someone else's chess move?

👇

To the person fixing vulnerabilities at 3 AM — this one's for you. Buy me a coffee ☕

Our VP's AI Wrote 3,000 Tests. Production Cost $700K. I Deleted Every Single One.

xulingfeng — Thu, 04 Jun 2026 23:50:06 +0000

Based on real industry trends. About an AI testing tool promising 300x efficiency, a VP who rebranded hand-written automation as "manual testing," and a $700K SLA bill nobody saw coming.

Act 1: The All-Hands

VP Harrison stood in front of the screen, the AI testing dashboard glowing behind him.

"Three days. Three thousand test cases. Zero human intervention."

He paused. Let his eyes sweep the room. They landed on me.

"And some people — six years. Maintained four hundred automated test cases. That's less than twenty per person per year."

A few people looked at their phones. Others studiously avoided my direction.

"I'm not here to debate efficiency. I'm here to ask — why does your team still exist?"

I opened my notebook.

"Mr. Harrison. What's the coverage on those three thousand AI-generated tests?"

"One hundred percent."

"And how many bugs did they find?"

A beat.

"The first phase is about regression coverage, not —"

"Zero." I cut him off. "Three thousand tests, zero bugs. You ran three thousand checks on 'what the code does' and not a single one on 'what the code should do.' That's coverage, not quality."

VP Harrison smiled.

"I understand your anxiety. When new technology threatens your domain, people always rationalize resistance. But the data doesn't lie — three hundred times the efficiency, zero incremental cost. What took you six years to prove — AI did in three days."

He didn't look at me again. Clicked to the next slide.

Act 2: The Sideline

That afternoon, HR notified me: my team was being reassigned from Quality Assurance to the AI Engineering Group. My reporting line now went through VP Harrison's deputy.

I walked to his office.

"Mr. Harrison. The AI testing tool needs a three-week trial run. I need to verify its behavior under production traffic patterns —"

"You don't need to verify it. I already did."

"What environment?"

"Staging. One hundred percent pass rate."

"Staging doesn't simulate real traffic shapes —"

"Are you telling me your four hundred manual test cases are more effective than three thousand AI-generated ones?" He leaned back. "Do you actually believe that?"

I stood at his office door. He didn't ask me to sit.

"Your new desk is on the third floor. AI Engineering Group. Report tomorrow."

Act 3: The Report

I spent three nights pulling and reviewing all 3,000 AI-generated test cases.

The AI tool itself wasn't bad. The problem wasn't the technology.

The problem was the configuration. VP Harrison's deputy had set the input boundary to "90th percentile of historical production data." The AI faithfully generated tests within that boundary. All three thousand cases lived inside the 90th percentile. Inside that range, the AI validated "what the code does according to the config." It couldn't — by design — validate "what the code should do at the boundaries." The config never asked it to look there. The AI didn't err. It flawlessly executed a flawed instruction set.

I wrote a full analysis report with configuration screenshots and comparative data.

Sent it to VP Harrison. No CC.

Twenty-three minutes later, his reply landed:

"Noted. The edge scenarios you identified have an estimated probability below 0.3%. Per our risk prioritization framework, we will not allocate resources to cover them. I suggest you focus on learning the new tools rather than finding reasons to reject them."

I read that line twice.

Then I filed the report into a folder called RCA_2026Q3 and went back to maintaining my test suite.

Act 4: The Rollout

Three weeks later. The AI tests went live on the main release pipeline.

VP Harrison published a piece in the company newsletter:
"Why We Retired Manual Testing — And Why Your Team Might Be Next"

One line found me in the company-wide email:

"Some people spent three weeks trying to prove AI wouldn't work. Two weeks in production — zero incidents. Sometimes, what you're resisting isn't the technology's flaws. It's your own insecurity."

The one remaining tester on my team walked over to my desk.

"Boss... that part about insecurity. Was that about you?"

I closed the email tab.

"Two weeks zero incidents. Let's see what week three brings."

Act 5: The $700K Breakdown

1:14 AM. PagerDuty lit up like a Christmas tree.

A module the AI tests had cleared — thanks to that "90th percentile" boundary — hit a data race condition under real traffic. Every AI-generated test ran inside "normal traffic" parameters. Not a single test covered "resource contention when call frequency exceeds threshold." Because the AI's configuration never told it to check.

Cascading failure. Core transaction pipeline down. Nine hours of data recovery.

Initial damage: $700K.

The CTO called an RCA. Meeting time: Monday, 9 AM. Attendees: VP level and above... and me.

Act 6: The Meeting Room

9 AM. The conference room.

The CEO walked in. Didn't sit. Stood at the head of the table and placed a printed report on the surface.

"Mr. Harrison. You go first."

VP Harrison cleared his throat.

"This was a tool-level edge case. The AI testing framework lacks built-in detection for this scenario. We've contacted the vendor — the next release will include a fix."

The CEO listened standing. He didn't interrupt.

When Harrison finished, he closed the folder and looked around the table.

"Before we went live — did anyone raise a concern like this?"

Silence. VP Harrison said nothing. The CTO studied his laptop.

Three seconds.

I opened my notebook.

"Yes. One month ago."

The CEO's eyes shifted to me.

"A report analyzing the AI testing tool's configuration — input boundary set at the 90th percentile, leaving twenty-three categories of low-probability, high-impact scenarios uncovered. Including the race condition that caused last night's outage."

CEO: "Who did you send it to?"

"Mr. Harrison."

I plugged my laptop into the conference room projector. The email screenshot filled the screen.

"To: Mr. Harrison. Sent: June 7, 11:23 PM."

Several people pulled out their phones to photograph it.

The CEO glanced at the screen, then back at VP Harrison.

"You received it?"

"I did."

"And?"

A pause.

"At the time, the assessment was —"

I flipped to the next page.

The CEO read it. Nodded once. No raised voice. No theatrics.

He placed the report back on the table and looked at VP Harrison.

"He sent it to you. Why didn't you escalate?"

VP Harrison had no answer.

Act 7: The Aftermath

VP Harrison submitted his resignation two days later.

Internal memo from the CTO: Quality Assurance restored as an independent division, reporting directly to the CTO. Budget doubled. I was appointed department head.

That afternoon, I walked back to my old desk on the fifth floor. Still empty. Everything where I'd left it.

A yellow sticky note on the corner of my monitor. Not mine — left by a former teammate who'd left the company months ago.

It read: "Don't let them touch your tests."

I never knew if he meant the VP, or the AI.

I peeled it off and tucked it into the first page of my notebook.

I opened the AI testing platform. Typed test_case list --all --source ai. 3,000 records.

Select all. Delete. Confirm.

"This action cannot be undone. Delete 3,000 test cases?"

Confirm.

Three thousand cases. Three seconds. Gone.

The one remaining tester on my team stood behind me.

"Boss... you just deleted everything?"

"The configuration was wrong. Every test was built on a broken foundation. If the foundation is crooked, no test on top of it will save you."

He didn't answer.

I opened our four hundred automated test cases — six years of writing, one line at a time. Four hundred cases on the legacy system. Zero production incidents in six years.

"What about the new module?" he asked.

"We write them. Starting today."

But I didn't close the AI testing platform.

I opened its test generator. Pasted the new module's API spec. Changed the boundary config from the default 90th percentile to — unlimited. Generate everything. I'll curate.

The AI generated 87 candidates.

I reviewed every single one. Kept 42. Deleted 45. Added 8 boundary scenarios the AI never considered.

Fifty cases merged into our test suite.

Four hundred human-written, plus fifty AI-assisted — running together.

The tester looked at the green checkmarks on the dashboard.

"Boss... you're using AI too?"

"The AI isn't the problem. The problem is who decides what it tests."

"The VP bought AI as a mask. I use AI as a microscope."

"AI makes mistakes. Humans make mistakes. But the worst part is — someone stacks both mistakes on top of each other, then blames it all on the AI."

AI-generated tests pass at 100%. They verify what the code does — not what it should do. When the code itself is wrong — AI will prove the wrong code is right.

Have you seen someone package a process failure as a technology failure? What happened next?

Follow for more stories about AI testing, quality engineering, and what happens when the tools are smarter than the process.

If these stories made you think or saved you time, buy me a coffee ☕ — currently maintaining 400 automated test cases and manually reviewing every AI-generated one. Caffeine is the only config I trust at 💯.

Our CTO Guaranteed His $2.8B AI Gateway Was Safe. I Quietly Kept a Different Record. For 8 Months.

xulingfeng — Wed, 03 Jun 2026 18:09:56 +0000

A story about formal verification and adversarial testing.
About systems that are mathematically safe, and systems that don't fail in the real world — and which one regulators actually care about.

Has your company ever made a decision using AI that just felt wrong? A loan that got approved when it shouldn't have. A compliance recommendation that made you do a double-take.

Do you trust the system with the green checkmark that says "verified safe" — or are you quietly running your own tests in the background?

This is that story.

1 · The Green Signature

Day one. Drew was demoing his AI gateway at the all-hands tech meeting.

He opened a terminal. Typed a command. Loaded the model. Fed it test data. A line of green text appeared on screen:

FORMAL VERIFICATION PASSED. This system is mathematically proven safe. No boundary violations possible.

Two seconds of silence. Then someone started clapping.

Drew Chen, co-founder and principal architect. Stanford PhD, 15 top-tier conference papers. The company's AI decision system was his baby — a formal-verification gateway that constrained every model output. Running for two years, handling over $2.8B in transaction volume.

The room applauded. Three people raised their hands with questions. Drew answered each one.

I was sitting in the corner.

I was hired to replace the previous QA lead. He got laid off — second on the list in the last round of cuts. When I showed up, my desk still had someone else's stuff in it. An unopened bag of tea. A style guide nobody ever read.

Our QA team, including me: five people. Buried under Ops on the org chart. Not even Engineering.

A co-founder built the AI gateway that carries $2.8B. He doesn't need QA, because he proved his system is mathematically safe.

And me? I was a compliance hire. A box the company had to check.

I didn't know yet that the next eight months would be spent proving that green signature would cause an incident serious enough to bring regulators through the front door.

Second week. I submitted my test pipeline proposal.

Adversarial testing. Distribution drift monitoring. Injection attack detection. Inference chain auditing. A complete AI validation framework.

Drew finished reading in three minutes.

"Formal verification already covers every possible output path. If an output exceeds the boundary, the gateway blocks it. Everything you're talking about — input injection, distribution drift — it's either allowed inside the boundary or blocked outside. There's no third category."

"There's a gray zone between inside and outside," I said. "After three months of live service, the model develops feature drift on its own. The boundary constraints don't change — but the semantic layer does. Formal verification doesn't check semantics."

"That's ops monitoring, not architecture."

I opened my notebook. Three P0 incidents since the system went live.

Incident one: Input distribution drift caused sustained output quality degradation. Triggered on day 52 of live service.
Incident two: Model output was syntactically valid but recommended a decision that violated business rules. Triggered on day 87.
Incident three: Semantic drift in a specific scenario. No boundary was breached. Triggered on day 103.

Three incidents. All post-deployment. The formal verification signature was valid for every single one.

Drew glanced at them.

"These aren't system design problems."

I closed my notebook.

He put the report back on the desk. His fingers paused at the edge of the paper. Then he said something, without looking at me.

"You've got good data. But data isn't architecture."

I knew he was giving himself an out. I didn't push.

2 · Drift

I didn't have the standing to argue with him. So I built my own adversarial testing pipeline on a personal cloud account.

Non-blocking. Recording only.

April 2025. The pipeline caught the first AI distribution drift.

I had been watching Drew's system for three months. Same input class, 1,000 runs. The output syntax was always correct. Parameters in range. Confidence scores passing. But the semantic distribution had shifted — the model's judgment criteria for the same category had quietly changed over those three months.

The formal verification signature was still valid. The gateway saw nothing wrong.

I sent the report to Drew.

24 hours later, his reply came back. He confirmed the drift. But:

"This is normal model evolution. The architecture allows ±3% semantic drift. I've added an input normalization layer — should fix it."

He fixed it over a weekend. Drift metrics zeroed out.

But I knew this wasn't a fluke. The root cause was the model's online learning mechanism adjusting parameters without any guardrails. He fixed one tree. The forest was still burning.

I didn't say anything. I just added a trigger to the pipeline:

「When drift accumulates past threshold, auto-save a complete audit snapshot.」

Then I archived 5,000 test records, dozens of distribution comparison charts, and a full traceability script — all to an encrypted volume outside company storage.

Not to expose anyone. So that if things went wrong, someone would know what happened.

3 · Immunity

After the drift incident, I wired my pipeline into CI. Every PR ran through it before merge. No pass, no deploy.

The dev team hated it within a week. A PR that took minutes to unit-test now took almost four times longer. Minor issues blocked merges, waiting for me to review, waiting for Drew to decide.

June 2025. Drew walked straight into the CEO's office.

His argument: the QA team's adversarial testing was slowing releases by 4x. Every minor anomaly triggered a change review. The dev team was losing it.

The CEO signed off: Quality Immunity. Drew's architecture would assume full quality responsibility. External testing pipelines were advisory only — not release-blocking.

Drew announced it at the weekly standup. He looked at me as he said it. Not a challenge. Just a statement of fact.

"I guarantee the system is safe."

I said nothing. Went back to my desk. Switched my pipeline to passive mode. Record only, don't block.

Worked faster that way.

Within three weeks, the pipeline logged:

7 minor drifts that CI self-monitoring hadn't caught
2 data mismatches between test and production environments
1 label leak in a staging environment

I wrote a report. Put it in my drawer.

I didn't send it. The CEO didn't even know my name.

But here's the interesting part: Drew started adding adversarial training modules to the architecture. He added distribution drift metrics to the monitoring dashboard. He told the CEO, "We've strengthened our internal AI validation."

I watched my data show up in Drew's slides.

He said he didn't believe in this approach. His hands were already building it.

What does that tell you?

It tells me he knew formal verification wasn't enough. But he couldn't say it. Because he built this system. If he admitted it wasn't sufficient, then every decision made on this system for the past two years would be questionable.

He wasn't arguing with me. He was defending his own legacy.

4 · Inside the Boundary

September 2025.

The company's AI system handles intelligent approvals in a financial compliance context — loan applications, risk recommendations, compliance suggestions. Tens of thousands per day. At that volume, $2.8B in two years makes sense.

The incident took seconds.

The on-call engineer saw it first — a compliance score ticked up to 92% on the monitoring screen. Glanced at it. Moved on. He didn't know that score corresponded to a loan that should never have been approved.

The system generated a compliance recommendation for a customer's loan application. Syntax correct. Format valid.

The recommendation violated regulations.

Drew's architecture didn't alert. The boundary was intact. Format was clean. Confidence was passing. The system saw nothing wrong.

My pipeline had flagged this input 48 hours earlier — its distribution characteristics had deviated significantly from the top 1% of the training set. The trigger was lit. But during the immunity period, nobody processed it.

The customer got the result and found the problem. They had their own compliance team.

They reported it to the regulator.

Three days later, the regulator's representatives showed up at the front desk.

5 · The Audit Trail

They didn't ask for architecture documents.

Complete model version history — what model, what parameters, when it changed
Audit owner for every change — who approved it, based on what data
Traceable inference chains — how each decision was made, what version it rolled back to

Drew's architecture team spent three days trying to pull inference records from the logging system.

Nothing.

Because the design philosophy of a formal verification gateway is: input → check boundary → output. It doesn't record decision paths. It records "verified" or "not verified." If it's inside the boundary, it passes. If outside, it blocks. There's no "it's inside the boundary but semantically wrong" state.

Kate was the compliance officer. She came to find me.

"Does your system keep an audit trail?"

"From day one."

"And Drew's?"

I didn't answer directly. I opened my laptop and showed her.

My pipeline recorded a chain:

Input sample → Distribution check ✓ → Model inference → Drift detection → 
Semantic validation → Compliance check → Audit snapshot → Encrypted storage

Every step with a timestamp, model version, parameter hash. Started eight months ago.

Kate looked at it for five minutes. Then she said:

"Send this to the regulator's legal counsel. I want confirmation they've received it by nine tomorrow morning."

At the hearing, Drew made one last effort.

"Formal verification is the industry-accepted gold standard for AI safety," he said. "Our verification approach is at the highest academic and engineering level available."

The regulator's representative flipped through his report.

"100% pass rate. Covers all boundary conditions."

The representative set the report down.

"I understand. But regulation doesn't require mathematically perfect systems. Regulation requires systems that can be investigated when something goes wrong."

"Your system, by design, cannot be investigated."

The room was quiet for a long time.

Drew didn't say another word.

6 · The Door

Here's how it landed.

Drew kept his title and his team.

But his technical decision-making authority was transferred to a new AI governance committee. I was put in charge of it.

Adversarial testing pipeline was formally integrated into the company's AI governance framework. Equal on paper. In practice, the center of gravity had shifted.

A few days after the results came out. In the hallway.

Drew stopped me. First time in eight months he used my name.

"Ryan. You won this round. But your method has no mathematical guarantee."

"You're right."

"Mathematically safe systems and systems that don't fail in the real world — they're not the same thing."

He stopped walking. Looked at me.

"Your AI doesn't make mistakes — inside the boundaries you defined for it." I said.

"The problem is — after deployment, the AI walked itself right out of those boundaries."

"And you didn't even install a door."

He stood there.

I walked away.

Heard later that he redesigned the architecture. Added an audit logging layer.

Not because formal verification couldn't do it. Because he never thought to put a door on it in the first place.

You can't prove a system is safe. What you can prove is: when it isn't — you're watching.

Formal verification says: "My math can't be wrong."

Adversarial testing says: "Your math doesn't tell me when it will change."

When the auditors show up — which one do they look at?

I couldn't install a door in Drew's system. But the stories stay open. buy me a coffee ☕

I Asked for $500/Month and got turned down. My Company Spent $470K on AI Instead. Then I Quit.

xulingfeng — Wed, 03 Jun 2026 08:11:00 +0000

I asked for $500/month and got turned down. The company spent $470K on an AI platform instead. Then I quit.

A story about what $500 couldn't buy, what $470K couldn't fix, and an 87-day countdown on a door that wouldn't open again.

Your company bought AI this year. How much did it cost?

Are you sure that money bought technology — or did it buy a "we're doing AI too" slide deck, and a nicer-looking résumé for someone else?

I asked for $500/month and got turned down. The company spent $470K on an AI platform instead. Then I quit.

The pay cut notice went out to the whole department by email.

Red title bar. Three lines of body text. "Optimizing cost structure — evaluating role efficiency." My name was seventh on the list. Not first. Not last.

30%.

I closed the email and went back to fixing the log.

Mike, at the desk next to mine, pushed his keyboard away. "Thirty percent? Are they serious?"

I didn't answer.

He turned to look at me. "You're not mad?"

"I was mad," I said. "Past tense."

— I wasn't mad today. I was mad three months ago, when the VP announced at an all-hands meeting that the company was buying an AI platform for $470K a year. Someone checked quietly — similar platforms on the market cost about a third of that.

After the meeting I followed the VP to his office and put a budget request on his desk.

Cloud servers: $500/month. A model validation pipeline. Stress tests, evaluation runs, regression checks — the full stack. Two pages, every line item accounted for.

The VP flipped through it for three seconds.

"Not needed. The AI platform comes with built-in validation."

"That's black-box testing. Feed in data, get a score. A pipeline needs traceability — which model layer, what parameters, what version of the eval script."

"I hear you. But the budget won't clear." He slid the request back across the desk, like he was mentioning something that didn't really involve him. "Run a prototype on your own account first. Get some results, then we'll talk about budget."

I looked at him.

He knew those words were empty. I knew he knew. Neither of us said anything else.

For the next three months, the entire floor was busy with the vendor's platform deployment. Nobody paid attention to what I was doing. I spent a weekend putting the pipeline together and the rest of the time feeding it data.

"We'll talk about budget" — the VP never brought it up again.

The VP hired someone to take over the AI platform — Mark. The whole engineering team's AI work was reassigned to him, including mine. In his first week, he put up a slide deck on the big screen.

Three slides. First one: market size. Second one: customer logos. Third one: one sentence.

"— Covers approximately 72% of the engineering team's routine development work."

That evening I came back to my desk and found Mark standing next to it — holding a piece of paper.

"Unauthorized Systems List." He set it down next to my keyboard. One line was highlighted in yellow: Model Validation Pipeline / Registered Under: Personal Account / Budget Allocation: None.

"From the IT audit. No purchase record, no license, personal account — I'm guessing you haven't read the company's compliance policy."

"You can check with the VP," I said. "He told me to run the prototype three months ago."

Mark didn't respond to that.

He glanced at my screen — the stress test I'd been running for the last twenty minutes.

"The VP resigned," he said. Quietly. Like he was stating something that had nothing to do with him.

The VP left without a goodbye email, without a handover meeting. One day he was at his desk. The next day it was empty.

I didn't react.

"You registered this on a free personal tier?"

I didn't look up.

He stood there for two seconds, folded the paper, and put it back in his pocket. He took three steps toward the door, stopped, and said without turning around:

"You can keep it around. As a reference."

Then he walked out.

I watched him disappear around the corner.

The platform had been live for a month. Every week, Mark sent out a progress email with a dashboard screenshot — model accuracy at 98.7%, response latency under 200ms, coverage climbing steadily. The numbers looked better every time.

But every number was the platform testing itself. Its own test set, on its own models, producing its own report. Not a single data point from actual production feedback.

The only data set in the whole company that could independently validate whether a model actually worked — that was on my cloud server. The test suite had been built from five years of production failures, one incident at a time. Every label hand-reviewed. Every model that passed it, you knew exactly how it would perform on real traffic.

Mark knew how important this thing was — it was outside his audit list, and that's exactly why he needed it. He was auditing my pipeline. What he really needed was five years of my mistakes.

I closed my laptop and went to the rooftop.

Two empty chairs and a waist-high wall. I sat there for fourteen minutes. The wind was strong enough to keep my shirt collar flapping.

My phone rang. I thought it was a delivery.

It wasn't. It was a recruiter.

"Simon? This is Kate. Mia suggested I reach out — she said you might be exactly who we're looking for."

Mia. A recruiter I'd worked with six months ago. She'd put me forward for two positions, neither worked out. But she said better opportunities would come in three months.

Three months.

"What kind of role?"

"AI infrastructure. VP level."

"And the comp?"

She said a number.

Then she said: "Want to think about it?"

"Don't need to. Send me the JD."

I hung up and stood on the rooftop for a long time. You could see the whole campus from up there. Four buildings. I was in Building Two, fourth floor, west side — one of those windows was my desk for five years. Five years. How many PRs had I written? How many bugs had I fixed? How many internal tools had I built? I'd lost count. But the company remembered a different number: what percentage of my work could be automated.

72%.

One number erased every line of code I'd written in five years.

I didn't look at the windows again. I walked back down to my desk.

The keyboard was still warm. The log was still running.

A few days later, I took a half-day off.

The interview was in a small conference room at the new company.

No slide decks, no market sizes. The engineering director pushed a laptop across the table with a production incident reproduced on screen: an AI model's recommendation accuracy had dropped from 93.7% to 68.2% over two weeks. No errors in the inference logs. No memory leaks. No latency changes.

He asked: "How would you investigate this?"

I looked at it for three minutes. Three layers.

First, the feature computation pipeline. A text processing library the pipeline depended on had received an auto-patch two weeks earlier. The patch fixed an edge-case bug — but it also changed the vector mapping for those inputs. The edge cases had been a tiny fraction of traffic, barely affecting overall accuracy. But after the patch, the routing shifted. More and more inputs got mapped to the new feature space, and accuracy drifted down, slowly, invisibly.

Second, the input monitor — was the online pipeline running a real-time feature distribution comparison against the training pipeline? A threshold breaker that would shut things down automatically? No.

Third, the human fallback — below a confidence threshold, was there a manual review flow? No. Fully automated. Full black box. Nobody knew anything was wrong until the incident hit. An auto-patch had silently changed the path the model was using to read its own data.

I wrote seven lines of pseudocode. One checkpoint per layer.

The director read it. Then he said one sentence:

"When can you start?"

"A month. Handover."

"What's your notice period?"

"Four weeks."

He picked up his phone, sent a message. Thirty seconds later he put it down.

"We'll buy out your notice period. The offer starts the day you sign. Your salary runs from today."

I stared at him.

"Your time is worth more than the company you're at. I'm not waiting four weeks."

I said: "Deal."

Two weeks later, the offer was signed. Sixty percent more than my current salary. No fancy title — "VP" was just a placeholder; I'd start with a team of three.

No need to serve the notice period — the new company bought it out in full. I nodded to HR, signed the handover documents, and that was it.

I sat at my desk, let the last log finish, committed the code, and closed my laptop.

Then I packed. A mug. A few technical books. Sticky notes that hadn't run out. No framed photos, no souvenirs, no "5-year anniversary" plaque — that was at the bottom of a drawer somewhere. I didn't even remember to take it.

I walked to the elevator and pressed B2. The doors closed. Floor 4. Floor 3.

The elevator stopped at 3. The doors opened. Mark was standing there.

He didn't step in. He looked at my backpack.

"You're leaving?"

"Yeah."

He stood there for two seconds, then walked in. The doors closed. He didn't look at me — he watched the floor numbers change.

"That pipeline of yours — the test data and labels — where are they stored?"

In three months, this was the first technical question he'd ever asked me. Not when he was standing in front of the whole engineering team with his audit list. Now. In an elevator. Nobody else.

I didn't answer.

"How do I get access after you're gone?"

"You don't have permissions."

"I can add you as a collaborator — "

"Personal account. One owner. I don't add you, nobody gets in. Call IT, call support — the account is registered under my personal email, and support only talks to me."

He was quiet. Two seconds. Three seconds.

The elevator went from floor 3 to floor 2.

"...How many days do we have?"

"Key rotation every ninety days. You've got eighty-seven."

He didn't say anything else.

The elevator reached B2. I stepped out. He held the door open with one hand — didn't let it close. He watched me walk away and said something that wasn't to me — it was to the elevator wall:

"Eighty-seven days."

The doors closed.

I was about ten steps into the B2 corridor when I heard footsteps behind me — not walking, running.

I turned around.

The CEO was coming down the stairwell.

"Wait."

I looked at him.

"Who's poaching you?" His voice was quiet. Not an accusation — he was still catching his breath.

"I signed."

"I know. Who?"

I didn't answer. He stood at the stairwell door for two seconds, caught his breath, and walked toward me.

"I said, who? Give me a name. I'll double it — I'll approve the budget myself. Tell me where you're going right now."

He didn't say "don't leave." He didn't say "we need you." He said "tell me the name of my competitor."

That told me more than any farewell speech ever could.

I looked him in the eye for a moment.

"That AI platform — the one Mark is running — the free trial ends in thirty days. Renewal is $470K a year. You knew that, right?"

He didn't answer.

"When it went live, it was all promises. Now the trial is almost up — has anyone actually verified those dashboard numbers?"

I stepped forward. He stepped back — not because I pushed him, but because he moved.

"Whatever your new company is paying, I'll double it. Report to me directly. No layers."

I stopped and looked at him.

"You signed the pay cut notice. The 30% one."

He froze. "That was company-wide — "

"I know. You didn't stop it."

He had nothing to say.

I said:

"The new company bought out my notice period. The three minutes you've spent talking to me — you just cost them."

He didn't say another word.

I held his gaze for two seconds, then turned and kept walking toward the parking garage exit.

After a few steps, I looked back. He hadn't followed. He was still standing at the stairwell door, tie crooked, one hand on his knee.

The lights were on at B1. He hadn't taken the elevator — he'd run down the stairs.

I didn't look back again.

Week four at the new company. My tool passed the full end-to-end stress test.

The engineering director stopped me in the hallway and said exactly one sentence:

"Twelve-week probation — you passed it in four. Want an extra equity grant?"

"The third architecture proposal is still running. Results next week. Let's talk then."

He nodded and walked off.

Before I left, I scrolled past a message from an ex-colleague. The old company's AI platform free trial had expired. They didn't renew. Mark's team had submitted three renewal requests — all three were kicked back. The last one had three words in the approval field: "ROI?"

I opened the AI platform's website. The VP's photo was on the team page. Third screen down.

"72% coverage."

What was left was the 28% that nobody else could do.

I didn't reply to the message. I locked my screen and packed up. The pipeline would break sooner or later. Whether Mark ever got the data — I don't know.

What $500/month couldn't buy, $470K a year couldn't buy either.

The wind on the rooftop is the same everywhere.

Some things don't show up in a budget request. They're not in the slide deck, and they're not in the purchase order. That doesn't mean they don't exist.

So — are you the VP? Are you Mark? Are you the CEO? Or are you me?

Follow for more stories about AI, engineering, and what happens when code meets corporate reality.

I Built Open-Source AI. Our New CTO Spent $8M on His Old Company's Product and Fired My Team. Two Weeks Later, the CEO Called.

xulingfeng — Tue, 02 Jun 2026 08:40:51 +0000

Based on a real production incident. About an $8M AI platform sourced from the new CTO's former employer. A legacy system that ran five years without a single P0. And what happens when nobody remembers where the load-bearing walls are.

The news broke on a Friday.

Henry was called to HR first. Came back with a piece of paper. White-faced.

"Severance package. Sign. Out."

Mike went second. Walked out and tossed his badge on the desk.

"Five years. Huh."

Then Jay, our ops engineer. Will, our tester. Chris, our frontend guy.

One by one. One piece of paper each.

Finally, HR called me. She didn't look at me.

"Leo. Company decision — the entire backend team is being optimized out. Your position is on the list."

I said, "I ran a six-month AI pilot that was internally validated. What about that?"

"…That's a management decision."

"Whose decision?"

She didn't answer.

"Severance package. Fine."

That afternoon, our team of six had our last round at the pub downstairs.

Mike raised his glass: "Leo. That six-month AI pilot — I really wanted to see it go live."

Henry said, "Go live? The code's already been wiped clean. They don't even know what we built."

I smiled. Didn't say a word.

Three weeks earlier. The all-hands meeting.

The CEO stood on stage. Next to him, a man I'd never seen before. Navy blazer. Metal-rimmed glasses. "Brought in from outside" written all over him.

The slide read: "AI First — A Complete Technology Architecture Upgrade."

"We're honored to welcome James — former AI VP at a major tech company. As of today, he's our new CTO. He's going to lead us into the next era."

The room applauded. Our table didn't.

James took the mic. Scanned the room — his gaze paused on us for two seconds.

"I spent a week reviewing our current system. Let me be blunt — it's a pile of 2021 relics. It runs. It's stable. So what? You have no idea what AI can do today. This stack can't handle it."

He clicked to the next slide. A gear getting crushed by an AI chip.

"Two weeks. Full AI replacement. Old system — wiped clean, rebuilt from zero."

Mike muttered beside me: "Bullshit."

James heard him. Turned, looked at Mike, and smiled. "Something to add?"

Mike didn't answer. But his face was red.

I didn't say anything either.

Because that system — I wrote it.

Five years ago. By myself. No help. No oversight. Three months of all-nighters, and I took the entire business line from zero to profit. The company went from losing money to making money. Not a single P0 the whole time.

And now someone was calling it "antique."

After the meeting, I went back to my desk. Packed the entire legacy system from the production servers onto my old laptop — the one I bought myself back when I was writing it. 500GB. Stuffed it in a drawer.

Not for revenge. Just — someone needed to remember how this thing was built.

Our team had six people.

Mike, backend — four years with me, his code more reliable than documentation.
Jay, ops — would wake up at 3 AM to check an alert.
Will, testing — ran full-load stress tests before every release, no exceptions.
Chris, frontend — once he fixed a bug, it never came back.
Henry — the youngest, but six months ago he became the lead on our AI pilot.

That's right. Six months ago, we'd already started working on AI. Not AI as a slogan. Real AI. Line by line. Version by version.

Back then, the previous CTO was still around. We proposed a plan: don't replace the legacy system, but add an AI assistance layer on critical paths. He approved a pilot.

Three months. Two business scenarios — order routing and anomaly detection. All based on open-source models, fine-tuned in-house. No procurement. No budget approval. We labeled the data ourselves, trained ourselves, tested ourselves. The numbers — I remember every single one:

Intent recognition accuracy: 91.3%
Average response latency: 180ms
Zero hardware procurement cost (ran on existing server idle capacity)
Three-person cross-validation consistency: 98.2%

The results passed internal validation. The previous CTO approved the next phase.

Then he was restructured out. And in came this guy.

Two weeks ago. The technical strategy meeting. James asked me to attend.

I brought six months of our team's data.

"James, this is our team's AI pilot results from the past six months. All open-source. Zero procurement cost."

I put the comparison table on screen:

Metric	Our Pilot	Their Proposal (Initial Quote)
Intent recognition accuracy	91.3%	Claimed ≥95%, actual 76.8%
Avg response latency	180ms	2.1s (peak 3.8s)
Daily concurrent load	4,200 (already running)	Not live yet, promised 5,000
Inference hardware cost	$0	$3.8M (4 GPU servers + rack)
Model licensing cost	$0 (open-source)	$4.2M (first year)
Time to production	Already running 3 months	Promised 4-6 months
P0 incidents	0	Not live, no data

"91% versus 76%. 180 milliseconds versus 2.1 seconds. Zero dollars versus $8M. James — I'm not against AI. I'm against spending $8M on something that performs worse than what we already built."

James stared at the screen for five seconds. Then he smiled. The same smile from the all-hands meeting — pity from above.

"Leo. That 91% — you ran it on your own 2,000-label dataset. Do you know what production data distribution looks like? What long-tail cases will do to that 91%?"

Henry stood up. "James, those were production logs. Full de-sensitized data from Q3 last year to Q1 this year. 470,000 records. Labeled by our team."

James turned to him. "Your labels? Who validated them?"

"Three-person cross-validation. 98.2% consistency."

James paused. Changed angle. "And that 180ms — how many machines were running it?"

"One. Two cores from an existing application server."

James's throat moved. He didn't answer.

Then he said the line I'll never forget:

"Leo. I'm not dismissing what you did. But — putting a smart lock on an old house doesn't make it a smart home."

Before I could respond, Henry's voice cut through: "James, then tell us — where exactly are we wrong? If the data is wrong, point to the data. If the approach is wrong, point to the approach. You're not saying no — you're saying you don't want to look."

The room went quiet. James stared at Henry. Henry stared back.

Finally James turned to me. "Leo. I respect what your team did. But what this company needs isn't patching. It's rebuilding."

I wanted to say: Rebuilding and replacing are two different things.

But I didn't.

Because I saw his slide close — and there it was in the bottom-right corner. The procurement department's approval number.

I knew that format. Eight digits.

$8 million.

And I knew the vendor — his former company's product.

From that day, James's team took over the production environment. First order of business — clean up the old code.

"Redundant systems. Incompatible with the new architecture. Remove everything."

I watched five years of my code get deleted line by line from the servers. Git repos archived and locked. CI/CD pipelines redirected. Even the wiki pages about the legacy architecture were marked "Deprecated."

Someone joked, "Your antique — it's officially a museum piece now."

Mike sent one message in the group chat: "He's burning bridges."

I didn't reply.

Because the bridge wasn't fully burned — I still had a copy in my drawer.

Then came Friday.

After that last round at the pub, I went back to the office.

Swiped my badge one last time. Opened a workstation.

Copied the AI pilot code, the fine-tuned model weights, the 470,000 labeled records, the test reports — all of it — onto my old laptop.

Not a backup.

Evidence.

Two weeks later. 2:47 AM.

The first week ran smoothly. Monitoring all green. The CEO patted James on the shoulder in the hallway — "Nice work."

Then, on day thirteen — some accumulated threshold was triggered.

My phone rang. The CEO's private number.

I picked up. Three seconds of silence.

"Leo. James's system… crashed."

His voice didn't sound like a CEO. It sounded like a middle-aged man who'd just been dragged out of bed.

"All six GPU servers went OOM. Three AI agents started stealing each other's context. Order duplication rate — 37%. Customer complaints — 2,300 in four hours. Estimated refund liability — $2.7 million."

He kept going: "A competitor's 'offer of assistance' landed in our client's CEO inbox. First line: 'Your current architecture presents significant risk. We can complete migration within one week.'"

I was already at the door, putting on my shoes. But I asked anyway: "What did James say?"

CEO paused. "…He said they need two weeks."

I blinked. "Two weeks?"

"They said the prompts need retuning."

I couldn't help it. I laughed into the phone. "$8 million. Two weeks before it collapsed. And the excuse is — the prompts weren't tuned."

I thought to myself, His old company's product. Never disappoints.

CEO didn't respond. Long silence. Then he said, "That old system… do you still have it?"

"Yes."

Another pause. Then he said something I didn't expect: "We can't find anything. The Git repos are empty. The servers are all running his new stack. Even the deployment docs were deleted. Five years of your work — there's nothing left in this company. "

I didn't say anything.

"The code. Your team's six months of work. Is it still there?"

"All of it. On one laptop. 500GB."

He let out a long breath — like he'd been holding it for two full weeks.

"Name your price."

"I'll be there in 30 minutes."

3:12 AM.

I was sitting in the server room. The new ops guy didn't know me — he was standing next to me, hands shaking, holding a cigarette.

I connected my laptop to the switch. Changed one Nginx route.

"Cut it over."

He stared at me. "…Cut over to what?"

"The legacy system."

"Wasn't that… decommissioned?"

I said: "No. I kept it."

30 seconds.

The monitoring dashboard went from blood red to green.

Orders started flowing normally again. The entire AI stack — taken offline.

I also deployed the AI pilot modules from my laptop onto three regular servers. Running as an assistance layer.

140ms. 90.8%. Zero new hardware.

I put the comparison on the big screen:

Metric	The $8M AI (Before Crash)	Legacy + AI Assist (After Recovery)
Response time	2.1s → timeout	140–210ms
Accuracy	64% → gibberish	90.8%
Concurrent load	Crashed at 62	4,200 normal
Error rate	100%	0.3%
Recovery time	Promised "two weeks"	30 seconds
Hardware cost	$8M	$0 (existing hardware)

The CEO stood at the server room door, still catching his breath. He stared at the green line on the screen — a full half-minute.

Then he asked, "Your six people — how many can come back?"

I said, "How many can come back depends on why you let them go in the first place."

CEO didn't answer. He stared at the green line, then at my old laptop in my hand. Finally, he asked the question he'd been holding back:

"That smart lock — is it running?"

He was talking about James's joke — "putting a smart lock on an old house doesn't make it a smart home." He remembered. So did I.

I didn't turn around. "It's running. 140ms. 90.8%. And —"

I closed the laptop.

"Putting a smart lock on an old house doesn't make it a smart home. But the $8M new house you couldn't build — the old house is what held it up."

CEO didn't say a word. He looked down the hall at the brightly lit "Special Projects" office. Then he turned and walked away.

Two steps. Stopped.

"Your team — I'll make the call. Come back."

A week later. The client renewed. With a new clause in the contract: Any core architecture changes must be communicated in advance and approved in writing by the client.

Nobody mentioned the competitor's "offer of assistance" again. Someone sent me a screenshot of the email with a note: "$8M vs 30 seconds. Leo, you win."

James was moved to "Special Projects" — no budget, no team, no OKRs. Just a desk and a chair.

HR called me again. "Leo. The company wants you back. The entire backend team — full recall. Name your salary."

"Fine. Three conditions."

"First — cancel the paid AI. We don't need it."

"Second — our AI pilot becomes official. But it stays as an assistance layer. It doesn't touch the main pipeline."

"Third." I paused. "From now on — how the system is built and what tools we use — the engineers decide. Not the people who write the checks."

HR was quiet for a long time. "I'll ask the CEO."

The CEO's response — HR relayed it to me — six words:

"Say yes to all of it. Bring him back."

Henry came back. Mike came back. Jay, Will, Chris — the whole team recalled.

Same desks. Not the same salaries — everyone doubled.

The CEO called me into his office. Closed the door. Three sentences:

"Leo. The seat — it's yours."

"CTO."

"The team is yours. The budget is yours. The technical direction — yours."

One late night, Mike walked over with two cups of instant noodles. "Leo. What do you think — if they'd taken one look at what we built back then, what would have happened?"

I thought about it. "Nothing. They wouldn't have looked."

Mike smiled. "Yeah. That's why you kept it."

I said: "Someone has to remember. How this system was built — and who built it. Not a PowerPoint. Not $8 million. Six people."

That afternoon, I ran into James in the hallway.

He was walking out of "Special Projects," carrying a cardboard box. Inside: his metal-rimmed glasses. A framed photo — him and the CEO on his first day.

He saw me. Paused.

I kept walking. Didn't look back.

Five steps. I heard his voice behind me. "That pilot project… were the numbers real?"

I stopped. Didn't turn around.

"Real. Putting a smart lock on an old house doesn't make it a smart home. But if you're going to tear the house down — you should at least know where the load-bearing walls are."

I walked away.

That old laptop is still in my drawer.

500GB.

Auto-syncs every night.

Not for revenge.

So the next "James" knows — this system? Someone knows it better than anyone.

You can spend $8M on an "AI-first" transformation. Or you can spend six months with a small team, build an open-source pilot that costs nothing, and keep the people who actually understand the system. The best AI strategy isn't replacing your team — it's giving them tools to do what they already do best.

Your company is rolling out AI across the board, isn't it?

"Full AI replacement." "Two weeks to production." "95% automation." Sound familiar?

James wasn't the first CTO to do this. He won't be the last.

So I'll ask you two questions —

Will your job be replaced by AI?

When a system goes down — does the human save it, or does the AI?

Drop your real thoughts below 👇

The CTO spent $8M on his old company's product. My open-source solution cost $0. If you enjoyed this story, meet me in the middle — buy me a coffee ☕ ($8M not required.🤣)

[Boost]

xulingfeng — Tue, 02 Jun 2026 01:26:01 +0000

David Ingraham

Jun 1

The Most Valuable QA Skill in the Age of AI Is Thinking

#ai #testing #programming

8 min read

[Boost]

xulingfeng — Tue, 02 Jun 2026 00:30:39 +0000

zerodrop

Jun 1

Testing email flows in Playwright without a mail server

#playwright #testing #webdev #javascript

4 min read

[Boost]

xulingfeng — Mon, 01 Jun 2026 23:51:18 +0000

Tijo Gaucher

Jun 1

Your AI Agent Doesn't Have a Model Problem — It Has an Ops Problem [The 20% Reliability Trap]

#ai #agents #automation #devops

4 min read

My Company Bought a $660K AI Platform. I Was Replaced. On Friday at 2:58 AM, It Fixed Everything. Then It Rolled Back the Wrong Patch.

xulingfeng — Mon, 01 Jun 2026 03:47:40 +0000

Based on real system architecture decisions. About a $660K AI platform, three AI agents that kept the dashboard green, and a P0 incident that cost $3.15M over one weekend.

Act 1 · The All-Hands Meeting

Wang Lei, VP of Product, stood in front of the big screen, a smile on his face.

Behind him, a dashboard rolled data from the "Axon AI Client Engineering Platform — Q1 Performance Report." Numbers cascaded across the wall:

Metric	Axon Platform	Human Team (Last Q1)	Improvement
Avg daily tickets processed	847	312	+171%
Avg first response time	12s	4h 17m	↓ 99.92%
Customer satisfaction	4.8/5	4.1/5	+17%
Monthly operating cost	$52K	$133K	−61%

Twelve department heads sat in the room. Dead silence.

Wang Lei planted both hands on the table and scanned the room. His eyes landed on me.

"Alex. Your team processed 312 tickets last Q1. Axon processed more than that in a single day last month."

He smiled. Not a friendly smile. A sentencing smile.

"And Axon costs less than a third of your team's operating expense."

"We invested $660K in the whole platform. At current operating costs, it pays for itself in eighteen months."

"After management review — the Client Engineering technical liaison function is being fully transitioned to the Axon platform."

He clicked to the next slide.

"Employees in replaced roles will complete exit interviews within the week."

Someone inhaled sharply.

I didn't. I opened my notebook to page 37.

"Wang, what dimensions are these numbers from?"

"What do you mean, 'what dimensions'?" His smile tightened.

"Of those 847 daily tickets — how many are auto-tagging and routing, and how many are actual technical resolutions?"

The room went quiet for about five seconds.

Wang Lei looked at me. "Axon's ticket closure rate is ninety-three percent."

"What's the reopen rate?"

He paused. "What?"

"After Axon replies — how many customers reopen the same ticket within twenty-four hours?"

"We're still collecting that —"

"Let me save you the trouble." I turned my notebook toward the room. Three lines in handwriting:

Axon ticket closure rate: 93%
24h reopen rate: 41%
Human escalation rate: 37%

"Out of every 100 tickets Axon closes, 41 customers had their issue unresolved and came back. 37 of those ended up needing a human anyway."

"847 tickets × 37% = 313. That's exactly what my team handled manually last Q1."

People in the room started checking their phones.

"Your AI didn't replace anyone. You just put a voice assistant in front of every ticket I was already handling."

Wang Lei's face went red.

After the meeting, the HR notification hit my phone. Time: 3 PM today.

Act 2 · The HR Signing

Zheng, the HR Director. Mid-forties, sharp chin. Her smile felt like an assessment.

She slid a paper across the desk: Voluntary Separation Agreement. Severance: the legal minimum.

"Sign it."

"I want to see the real Axon operating data."

"This isn't a negotiation." She uncapped the pen and set it on the paper. "Your desk needs to be cleared by 6 PM. Company laptop, keycard, all storage media — hand them over on-site."

"Storage media?"

"Per Wang Lei's specific request — seven years of accumulated technical materials and client communication records in Client Engineering. All company property. All to be turned over."

I looked at her.

"Are you serious?"

She didn't blink.

I picked up the pen. Signed.

Then handed it back.

"All client-related local files on my laptop — already deleted."

Her face flickered. "What?"

"Archived backups are in the company knowledge base. Local cache, work notes, technical scoping docs — wiped clean before I walked in here. The company assets I already submitted. Everything I ever uploaded to the knowledge base."

"What's left is my personal engineering notebook."

I pulled a hardcover notebook from my bag. The cover was worn white, corners frayed.

"Twenty-three client requirement analyses. Seventeen POC architecture scopes. Seven years of post-mortems written at 3 AM after every phone call. All in here."

"Not company property. I wrote it."

I closed the notebook and stood up.

"If Wang Lei needs this data — his AI can generate it."

I walked out of HR and went back to my desk. The old sticky note was still under the monitor, the ink faded — from four years ago, my first POC with Mike, CTO of MedTech. I'd written my number and told him, "If something breaks at 3 AM, call this." He saved it in his phone. I kept the note under my monitor — a reminder of what I'd promised. A phone number. Next to it: 3 AM. Call this.

I looked at it. Folded it twice. Put it in my jacket pocket.

Act 3 · Seven Years of Weight

2 AM. I sat in my car, engine off.

Seven years.

Seven years of late-night calls I'd lost track of. MedTech's compliance audit — four rounds. I found a log bug buried three years deep before their own compliance officer did. FinTech's payment system migration — I slept on the data center floor three nights straight. Manufacturing's IoT protocol stack failure — I sat in a remote session for 11 hours, diffing logs line by line.

Not because I wrote better code than anyone else.

Because when those clients had an emergency, the first person they called was me — not the support line.

My name in the ticketing system: 214 P0+P1 incidents resolved.

193 of them happened outside business hours.

They'd never appear in Wang Lei's PowerPoint. Because Axon's "847 daily tickets" only counted business hours. It didn't count 3 AM.

At 3 AM, Axon is off duty.

Act 4 · The Crash

Three weeks later. Friday. 2:58 AM.

My phone lit up on the nightstand.

Not a number I didn't recognize — Mike, CTO of MedTech.

"Alex. Our compliance pipeline is stuck. Core transaction modules are erroring out. Payment gateways are all timing out. We've got 20,000 orders queued."

"What did Axon do?"

"It ran diagnostics automatically. Rolled back the last two deployments, restored from snapshot. All surface metrics turned green. Then the compliance pipeline crashed again fifteen minutes later — and this time the data was completely corrupted. It sent an auto-reply — 'Your case has been escalated to our technical team, expected response within 48 hours' — and marked the ticket as 'Resolved.'"

"48 hours?"

"48 hours. And here's the fun part — the rollback also removed a compliance hotfix from three weeks ago. The one your team applied manually because it never made it into the deployment pipeline. Finding and reapplying it is going to take at least two more days."

"Finance just ran the numbers — direct losses so far: $630K. If this isn't restored by Monday morning, including SLA penalties and compliance fines, we're looking at over $3.15M. Do you have anyone over there who can take a remote look? The on-call engineers can't even tell me what hotfixes were applied."

I sat up. Opened my notebook.

"I don't have access anymore. I returned my laptop the day I was let go."

Two seconds of silence on the line.

"I'll give you a temporary account. MedTech-side ops portal. You built the integration layer — you know it better than anyone on my payroll."

"Fifteen minutes."

That fifteen minutes turned into a weekend.

Act 5 · The Data Doesn't Lie

Sunday, 5 PM. I sent an email.

To: Former CEO, VP Wang Lei
CC: Mike, FinTech CTO, Manufacturing CTO

Attachment: Axon_vs_Reality_2026Q2.md

Comparison	Axon Claimed	Actual (3-week production data)	Gap
Auto-resolution (surface-level)	95%	83%	−12pp
Auto-resolution (architecture-level)	—	6.8%	6.8% actual
Ticket closure (no reopen in 48h)	93%	61%	−32pp
P0/P1 diagnostic accuracy	—	Rollback restored surface → left root cause untouched	3/3 misdiagnosed
P0/P1 manual hotfix preservation	—	Auto-rollback overwrote without detection	0%
Off-hours coverage	24/7	89% ticket accumulation between midnight and 6 AM	Effectively absent
Client opt-in willingness	—	Top 3 clients all refused Axon handoff	100% rejection

The email body had one line:

"This is what your $660K bought."

Twenty minutes later, Mike's reply was one sentence:

"Monday, 9 AM. My office. Bring your notebook."

Act 6 · The Phone Call

Sunday, 11:47 PM. My former CEO's number.

He'd never called me at this hour.

"Alex."

"Yeah."

"I saw the email."

Silence.

"Are those numbers real?"

"There's a full audit trail in our internal systems. Check it yourself."

More silence.

"Wang Lei says he didn't know."

I paused.

"He says he didn't know — and you believed him?"

A long silence on the line.

"Were you at the Axon procurement meeting?"

"No."

"Then how do you know the $660K figure?"

"...The budget was Wang Lei's proposal. The board approved it."

"And when you signed off — did you know his PPT numbers were cherry-picked? Did you know the '95% auto-resolution' only counted tickets the AI could open — not tickets it could actually close?"

He didn't answer.

"Did you know — when you signed?"

"No."

"Now you do."

"Alex."

"Yeah."

"Come back."

"Come back to what?"

"Your old role. Wang Lei — I'll handle it."

"No thanks."

"What?"

"MedTech's offer is already in. Principal Architect. Double the compensation. Title bump. Signed Monday morning."

"Tell Wang Lei — his Axon is great at generating beautiful reports."

"It's just not great at answering the phone at 3 AM."

I hung up.

Act 7 · The New Office

Three months later. MedTech's new compliance engineering center.

No monitor on my desk. Just a laptop and a worn-hardcover notebook.

Mike walked over with two coffees.

"Did you know your old HR called to verify your background?"

"Which HR?"

"Your former company. They wanted to confirm you were actually employed here."

"What did you say?"

"I said — he's in the office next to mine. Want to say hello?"

I laughed.

An email notification popped up. FinTech's CTO.

"Subject: Your report made it to our board. We're putting our engagement with your former company on hold. Want to talk about a consulting contract?"

I glanced at the time. 1 AM.

I locked the screen. Didn't reply.

I'll answer tomorrow morning when I get to the office.

Trust doesn't come with "please hold while the system generates a response."

It comes with a phone number — and when you call it, a real person picks up.

——

Alex's notebook still has that sticky note inside. The phone number hasn't changed. The only difference is — this time, nobody's AI gets in the way.

Have you ever watched a system you built get handed over to someone else's algorithm? What happened next?

My dashboard has been green for 3 hours. I'm terrified🤣. — buy me a coffee ☕