DEV Community: CPDForge

Why Building Software Is Like Leading an MMO Raid

CPDForge — Thu, 11 Jun 2026 12:38:07 +0000

Why Building Software Is Like Leading an MMO Raid

A few years ago, if you'd told me that thousands of hours leading MMO raids would end up helping me build software products, I'd have laughed.

Today I'm not so sure.

I've spent years leading raids in games like Star Wars Galaxies and Star Wars: The Old Republic. What surprised me is how often the same lessons show up when building software.

The technology changes.

The tools change.

The jargon changes.

But the principles?

Not so much.

After building products, leading teams, surviving deployments, and spending more hours than I'd care to admit staring at architecture diagrams, I've realised that software development and MMO raid leadership have far more in common than most people think.

Most Wipes Are Self-Inflicted

One of the first lessons every raid leader learns is this:

Most wipes are self-inflicted.

Not every wipe.

Not all wipes.

But most of them.

The boss usually isn't the problem.

The raid wipes because:

Somebody got greedy
Somebody ignored the mechanic
Somebody panicked
Somebody thought they could improvise

Software projects are exactly the same.

Most projects don't fail because the technology was impossible.

They fail because:

Requirements weren't clear
Priorities kept changing
Validation was skipped
Scope grew uncontrollably
Assumptions went unchallenged

The technology rarely kills the project.

The team usually does.

Don't Pull Extra Mobs

This one has become a genuine software development philosophy for me.

Every MMO player knows this moment.

The group is making steady progress.

Everything is under control.

Then somebody says:

"While we're here..."

Five minutes later you're fighting three extra packs, the healer is out of resources, and half the raid is dead.

Software development has exactly the same trap.

You're implementing a reporting feature.

Someone says:

"While we're here, we could also..."

Then suddenly you're discussing:

New dashboards
New permissions
Notifications
Analytics
Exports
AI summaries

Congratulations.

You just pulled extra mobs.

One of the most useful questions I've learned to ask is:

Does this solve the problem we're trying to solve right now?

If the answer is no, it probably belongs in the backlog.

Finish the current fight first.

Do The Mechanic

Every raid eventually has that player.

The one topping the damage charts.

The one doing incredible numbers.

The one who dies first because they ignored the mechanic.

The mechanic doesn't care how talented you are.

It doesn't care how much damage you're doing.

If you don't do the mechanic, you're dead.

Software projects have mechanics too.

Things like:

Architecture reviews
Testing
QA
Security checks
Deployment validation

Nobody gets excited about them.

Everybody wants to write code.

But the mechanic still needs to be done.

The teams that survive long term are usually the ones that respect the boring parts.

Don't Stand In Fire

Another timeless lesson.

There is always fire.

Sometimes it's literal fire.

Sometimes it's:

Hardcoded secrets
Direct production database edits
Skipping tests
Ignoring warnings
Undocumented architecture

Everybody knows it's dangerous.

People still stand in it.

Repeatedly.

The lesson is simple:

If something has already burned you three times, stop standing there.

The DPS Meter Lies

One of the most underrated lessons from raid leadership.

The highest DPS player is not always the most valuable player.

Sometimes the most valuable person is:

The healer who prevented a wipe
The player handling mechanics
The person explaining strategy
The one spotting problems before they happen

Software teams work the same way.

The most commits don't necessarily create the most value.

The engineer who prevents a production outage may contribute more than the person who writes thousands of lines of code.

The person who simplifies a system may create more value than the person who adds five new features.

Not all contributions show up on the meter.

Wait For Loot

This might be my favourite lesson.

The boss dies.

Everyone gets excited.

Then the raid leader says:

"Wait. Don't loot yet."

Usually because:

Loot rules aren't set
Someone is still running back
A screenshot is needed
Something still needs checking

Good raid leaders don't sprint to the next boss the moment the current one dies.

They stop.

Review.

Recover.

Then move on.

Software projects should do exactly the same.

A feature ships.

Great.

Now:

Test it
Monitor it
Validate it
Learn from it

Only then start the next thing.

Too many teams treat deployment as the finish line.

It's usually the start of the learning phase.

The DPS Hero Is Usually The First To Die

Every raid has one.

The player convinced they're the main character.

The one who thinks mechanics are for everyone else.

The one who says:

"I've got this."

Thirty seconds later they're face down on the floor.

Software projects have these moments too.

The engineer who insists:

Documentation is unnecessary
Testing is optional
Architecture reviews are bureaucracy
Deployment checklists are for other people

Often ends up discovering why those things existed in the first place.

Usually in production.

Usually on a Friday.

The Boss Usually Isn't The Problem

This is probably the biggest lesson of all.

Most of the time, the boss isn't what kills you.

It's everything around the boss.

The lack of planning.

The lack of coordination.

The avoidable mistakes.

The unnecessary complexity.

Software projects are no different.

The technology challenge is often only a small part of the problem.

The bigger challenge is usually:

Focus
Discipline
Prioritisation
Communication

The boring stuff.

The raid-leader stuff.

Don't Touch Anything, I Forgot To Set Random

If you've ever led a raid, you've probably said something like:

"Don't loot yet. I forgot to set random."

Everyone freezes.

Nobody touches anything.

Because everyone understands that a tiny process mistake can create a much larger problem.

Software has these moments too.

They're usually disguised as:

"Don't deploy yet."

"Don't restart that."

"Don't touch production."

"Wait, I forgot to update the environment variables."

The principle is the same.

Slow down.

Verify.

Then proceed.

The Best Raid Leaders Aren't The Best Players

This took me years to understand.

The best raid leaders aren't necessarily:

The most skilled
The best geared
The highest DPS

They're usually the people who:

Stay calm
Keep everyone focused
Prioritise correctly
Reduce unnecessary chaos

The same is true in software.

The best technical leaders aren't always the smartest people in the room.

They're often the people who stop the team from creating problems for themselves.

Final Thoughts

After years of leading raids and building software, I've ended up with a surprisingly simple development framework:

Do the mechanic.

Don't stand in fire.

Wait for loot.

Don't pull extra mobs.

Most wipes are self-inflicted.

It's not a bad framework for software development.

And honestly, it's probably saved me more projects than some of the formal methodologies I've used over the years.

Although it's admittedly harder to explain during architecture reviews.

Then again...

The longer I build software, the more I think good architecture and good raid leadership are really the same thing.

Reduce unnecessary chaos.

Focus on the current objective.

And for the love of all that is holy...

Don't pull extra mobs.

Beyond the Prompt: Building a Proposal-Based Workflow for AI Content Updates

CPDForge — Tue, 09 Jun 2026 15:30:55 +0000

AI should propose changes. Systems should decide whether those changes become reality.

Most AI systems are designed to generate content.

The problem is that production systems rarely need content generation.

They need content maintenance.

And maintenance is where AI becomes dangerous.

When you're building software for regulated environments, the challenge isn't creating version one.

It's safely updating version one after it's already been approved, deployed, audited, referenced, and relied upon.

That's where we started rethinking how AI should interact with content.

The Problem With Regenerate

Imagine you have a compliance training course containing:

Regulatory references
Knowledge checks
Real-world scenarios
Internal procedures
Approval history

A regulation changes.

You need to update a single section.

Most AI systems approach this by sending the lesson to an LLM and asking it to regenerate the content.

Technically, it works.

Operationally, it creates a new problem.

The AI might:

Rewrite surrounding content unnecessarily
Change instructional tone
Remove important references
Alter lesson structure
Introduce subtle inaccuracies

The update itself might be correct.

The rest of the lesson might no longer be.

In high-stakes environments, that's not acceptable.

Content Is Not A String

One of the design decisions we made while building CPDForge AI was to stop thinking about content as large blocks of text.

Instead, we treat content as structured documents.

A simplified model looks like this:

Course
 └── Module
      └── Lesson
           └── Section
                └── Content Block

That structure allows updates to be targeted precisely.

Instead of telling AI:

Rewrite this lesson

We can tell it:

Update this section

That distinction turns out to be surprisingly important.

Because once content becomes structured, AI no longer needs to touch everything.

It only needs to touch the thing that changed.

From Generation To Proposals

The next decision was even more important.

The AI never directly edits production content.

Instead, it generates a proposal.

Production Content
        ↓
AI Analysis
        ↓
Proposed Change
        ↓
Validation
        ↓
Human Review
        ↓
Approved Update

The model suggests.

Humans decide.

That simple shift dramatically improves trust.

Instead of treating AI as an author, we treat it as a contributor.

Granular Path Targeting

Because content is structured, updates can be scoped to specific locations rather than entire lessons.

Conceptually:

{
  "path": "modules[2].lessons[0].sections[4]",
  "action": "update",
  "reason": "regulatory_change"
}

The AI receives the affected section rather than the entire document.

This reduces content drift, limits unintended changes, and makes updates easier to review.

More importantly, it keeps the blast radius small.

The Preservation Problem

Even targeted updates create risk.

An AI can still:

Remove required knowledge checks
Drop regulatory references
Break expected structure
Change instructional intent

So every proposal must be validated before it reaches a reviewer.

Deterministic Validation

LLMs are non-deterministic.

Compliance systems shouldn't be.

Before a proposal can be approved, it must pass validation checks such as:

Schema Validation

Does the proposal still conform to the expected structure?

Required Component Validation

Are mandatory elements still present?

Citation Validation

Have required references been preserved?

Structural Integrity Validation

Does the update still fit within the expected hierarchy?

The objective is simple:

Prevent the AI from damaging content while attempting to improve it.

Human Review Still Matters

There is a temptation to automate everything.

We've found the opposite works better.

The AI identifies potential improvements.

The platform validates the proposal.

Humans make the final decision.

For regulated content, that distinction matters.

Not because humans are perfect.

But because accountability still matters.

AI As A Pull Request

The more we developed this workflow, the more it started to resemble modern software development.

Developers don't usually push unreviewed code directly into production.

They create pull requests.

Those pull requests are reviewed, validated, tested, and approved before being merged.

We're increasingly treating AI-generated content updates the same way.

The AI creates the equivalent of a pull request.

The platform validates it.

Humans decide whether it should be merged.

That model feels significantly safer than allowing direct mutation of production content.

The Bigger Lesson

This pattern extends far beyond compliance training.

The same principle applies to:

Documentation systems
Knowledge bases
Legal content
Internal policies
CMS platforms
Enterprise workflows
Any system where correctness matters

The most valuable AI systems won't necessarily be the ones that generate the most content.

They'll be the ones that help organisations maintain complex information safely.

Content generation is rapidly becoming commoditised.

Content governance is not.

As builders, we spend a lot of time thinking about generation.

Increasingly, I think we should be thinking about maintenance instead.

Because once version one exists, the real challenge begins.

Questions For Other Builders

If you're building AI into a production system:

Are you allowing direct mutation of live content?
How are you validating AI-generated changes?
What safeguards exist when the model gets it wrong?
Are you treating AI as an author or as a reviewer?
Have you adopted a proposal-based workflow similar to pull requests?

I'd be interested to hear how others are approaching this problem.

The Most Dangerous AI Output Isn’t Wrong — It’s “Almost Right”

CPDForge — Sat, 04 Apr 2026 09:19:47 +0000

Most people think the biggest risk with AI is hallucination.

Completely wrong answers.

Obvious mistakes.

Stuff you can spot instantly.

That’s not what caused problems for us.

The real issue showed up later — once things looked like they were working.

The outputs weren’t wrong.

They were almost right.

And that’s a much harder problem to deal with.

Why “Almost Right” Is Worse Than Wrong

If something is clearly wrong, you catch it.

You fix it.

You move on.

But when something is:

90% correct
Well structured
Confidently written

…it passes through unnoticed.

And that’s where systems start to break.

What This Looks Like in Practice

These weren’t big failures.

They were small, subtle ones:

A field slightly misclassified
A rule applied in the wrong context
A structure that looks valid but doesn’t align with the system

Individually, they don’t matter.

At scale, they compound.

The Real Problem: AI Stabilises Its Own Mistakes

Here’s what we realised:

AI doesn’t just generate errors — it reinforces them.

Once a slightly incorrect pattern appears, the model tends to:

Repeat it
Expand on it
Make it look more consistent over time

So instead of random errors, you get:

Clean, consistent, wrong outputs.

Which are much harder to detect.

Why This Happens

AI isn’t reasoning in the way we expect.

It’s optimising for:

Coherence
Pattern completion
Internal consistency

Not correctness.

So if an early assumption is slightly off, the model will build a very convincing version of reality around it.

Where This Breaks Real Systems

This becomes critical when AI is used for:

Structured content generation
Compliance or policy outputs
Anything reused or scaled

Because now you don’t just have an error.

You have:

A repeatable error
A scalable error
A system-level error

What We Changed

We stopped trusting “good-looking outputs.”

Instead, we built around one principle:

Every output is suspect until proven stable.

1. Pattern Detection Over Single Output Review

Instead of asking:
“Is this output correct?”

We ask:
“Is this pattern consistently correct across outputs?”

This exposes hidden drift fast.

2. Intent vs Output Validation

We separate:

What the system is supposed to do
What the AI actually produced

Then compare them explicitly.

If they don’t align, it fails — even if it looks right.

3. Breaking the Feedback Loop

We avoid feeding AI its own outputs without checks.

Because that’s how:

Small errors become reinforced patterns become system behaviour

The Counterintuitive Bit

Making outputs more polished made the problem worse.

Cleaner language increases trust.

More trust reduces scrutiny.

Which allows bad patterns to survive longer.

Why This Matters Right Now

A lot of AI tooling is focused on:

Making outputs better
Making them more human
Making them more polished

But that increases risk if you’re not validating underneath.

The Takeaway

If your AI outputs look great but your system still feels unreliable:

You’re probably dealing with “almost right” errors.

And those are much harder to catch than obvious failures.

Question for Anyone Building with AI

If you’re using AI in production workflows:

What breaks first when you scale?
Do you validate outputs, or just trust them if they look good?
Have you run into “clean but wrong” behaviour?

Genuinely curious how others are handling this.

From Prompts to Systems: Fixing AI Agent Drift in Production

CPDForge — Mon, 30 Mar 2026 13:50:55 +0000

Why My AI Agent Kept Getting Things Wrong (And What Actually Fixed It)

At first, it worked.

I gave the AI a clear prompt. It responded well. Structured, relevant, even a bit impressive.

Then I tried again.

Same prompt. Slightly different output.

Then again — and something felt off.

Not completely wrong… just inconsistent.

That’s when it became a problem.

Because I wasn’t building a demo. I was building a product.

The Problem: “Almost Right” Is Not Good Enough

When you’re working with LLMs in isolation, variability is fine. Even interesting.

When you’re building something people rely on — it isn’t.

I started seeing patterns:

Outputs drifting in structure
Key instructions being ignored
Tone and formatting changing between runs
Occasionally… things just made up

Nothing catastrophic. Just unreliable.

And that’s worse.

Because you can’t trust it.

The Context: This Wasn’t Just a Chatbot

One important detail — this wasn’t an internal tool or a sandbox experiment.

This was a user-facing AI agent, interacting with both:

logged-in users (with context, data, and history)
prospective users (with no context at all)

Which meant I effectively needed two behaviours:

one that could operate with structured internal data and constraints
one that could explain, guide, and respond more openly without access to that context

Trying to handle both with the same prompt quickly broke down.

The agent would:

assume context that didn’t exist
overreach when it should stay generic
or lose structure when switching between modes

That’s when it became clear the issue wasn’t just prompting — it was context control and behavioural separation.

Why This Happens (and Why It’s Not a Bug)

It took a bit of stepping back to realise:

The model wasn’t failing — I was asking it to behave like something it isn’t.

LLMs are:

Stateless (unless you force context)
Probabilistic (not deterministic)
Context-sensitive (and context degrades fast)

What I was treating as “rules” were really just:

Suggestions with good intentions

Even system prompts didn’t fully solve it.

They help — but they don’t enforce behaviour.

What I Tried First (and Why It Didn’t Work)

Like most people, I went through the usual iterations:

Making prompts longer
Repeating instructions
Adding “IMPORTANT:” everywhere
Trying to be hyper-specific

It improved things slightly… but not enough.

The problem wasn’t clarity.

The problem was control.

The Shift: From Prompts to Systems

The breakthrough came when I stopped thinking in terms of prompts and started thinking in terms of structure.

Instead of:

“Tell the model what to do”

I moved to:

“Define how the model is allowed to behave”

That’s a completely different mindset.

What I Built: A Structured Instruction Layer

I ended up creating what I originally called an “instruction bible”.

In reality, it’s closer to a structured instruction system layered on top of the model.

1. Persistent rules (not buried in prompts)

Instead of mixing everything into one prompt, I separated:

Role definition
Behaviour rules
Output constraints

Example:

{
  "role": "compliance_ai",
  "rules": [
    "Do not invent regulations",
    "Flag uncertainty explicitly",
    "Prioritise clarity over completeness"
  ],
  "output_format": "structured_sections"
}

This becomes the source of truth, not just part of the conversation.

2. Modular instructions

Different tasks = different instruction sets.

Instead of one giant prompt, I used:

Generation mode
Review mode
Analysis mode

Each with its own constraints.

This reduced cross-contamination between behaviours.

3. Controlled outputs

I stopped accepting “natural” responses.

Everything had to follow a structure.

For example:

Sections must exist
Headings must match
Lists must be formatted consistently

If the output didn’t comply, it was rejected or reprocessed.

4. Reduced ambiguity

I removed anything vague.

No:

“be helpful”
“be clear”
“be concise”

Instead:

Define structure
Define constraints
Define boundaries

The model performs much better when it has less room to interpret.

What Changed

Once this layer was in place, the difference was immediate.

Outputs became consistent
Structure stabilised
Hallucination dropped significantly
Reuse became possible

Most importantly:

I could actually trust the output in a product setting

Not perfect — but predictable.

The Bigger Realisation

The real lesson wasn’t about prompts.

It was this:

Prompt engineering doesn’t scale. Systems do.

You can get good results with clever prompts.

But if you want:

reliability
repeatability
product-grade output

You need structure.

Where This Fits in the Bigger Picture

This lines up with a broader shift happening right now:

From chatbots → agents
From prompts → orchestration
From “AI responses” → controlled systems

We’re moving away from:

“Ask the model something”

Toward:

“Design how the model operates”

Final Thought

LLMs are powerful — but they’re not plug-and-play components.

If you want to build something real with them, you have to accept:

You’re not just writing prompts
You’re designing behaviour

And once you start treating it that way, everything changes.

If you’re building with AI and hitting similar issues, I’d be interested to hear how you’re handling it — especially where things break.

We tried to generate a compliance course with AI. It didn’t go well.

CPDForge — Wed, 25 Mar 2026 08:07:10 +0000

We started off trying to build a compliance course.

We ended up building the system required to trust one.

Turns out they’re not the same thing.

That’s when everything changed.

🧪 The First Version (Looked Fine… Until It Didn’t)

The initial idea was simple:

Use AI to generate a compliance training course.

Pick a topic like:

risk assessment
workplace safety
ESG fundamentals

Feed it into a model, get a structured course out.

And technically — that worked.

We got:

modules
lessons
headings
even quizzes

On the surface, it looked decent.

But once you actually read it properly…

❌ What Was Broken

Shallow Content

It explained things, but didn’t really teach anything.

No depth. No real-world context. No edge cases.

Inconsistent Structure

Some lessons were detailed. Others felt like placeholders.

No consistency across the course.

No Instructional Flow

It wasn’t designed — it was assembled.

Content chunks, not a learning journey.

And the Big One: Reliability

In compliance training, “almost correct” isn’t acceptable.

It’s a risk.

⚠️ The Realisation

We assumed the problem was:

“How do we generate better content?”

It wasn’t.

The real problem was:

How do we make that content consistent, reliable, and safe to use?

AI was doing exactly what it’s good at:

producing plausible output
filling gaps convincingly
sounding right

But that’s not the same as being trustworthy.

🔧 What Broke First

Our original pipeline looked something like:

Prompt → LLM → Output course

And for a moment, that felt like enough.

Until we started testing it properly.

Sections contradicted each other
Concepts repeated in different ways
Terminology drifted across lessons
Some parts were strong, others clearly weak

You could generate a course.

You just couldn’t rely on it.

🧱 What We Had to Build Instead

The moment things changed was when we stopped treating this as a generation problem.

We started treating it as a system problem.

The pipeline evolved into something more like:

Input
→ Structured Generation
→ Validation Layer
→ Targeted Rewriting
→ Enrichment (quizzes, scenarios, examples)
→ Compliance Checks
→ Output

Each layer existed for a reason.

Because every time we skipped one — something failed.

🧩 The Hard Parts (That Don’t Show Up in Demos)

Structure Enforcement

We had to stop the model from improvising.

That meant:

fixed lesson frameworks
defined section types
controlled outputs

Targeted Improvement (Not Regeneration)

Regenerating everything just moved the problem around.

Instead:

identify weak sections
rewrite only those
preserve what already works

Cross-Course Consistency

This was harder than expected.

We needed to deal with:

duplicated concepts
mismatched terminology
uneven difficulty

Which meant introducing:

internal rules
pattern checks
consistency constraints

Compliance Awareness

This is where most tools fall down.

We needed:

alignment with recognised frameworks
the ability to adapt as guidance evolves
detection of weak or risky content

🧠 The Shift

At some point, we stopped thinking in prompts.

We started thinking in systems.

AI became one part of the process — not the solution.

🛠️ If You’re Building with AI

It’s very easy to focus on:

better prompts
better outputs

But the real leverage is in:

constraints
validation
iteration
control

Because generation is easy.

Making it usable is not.

🚀 Where This Landed

What started as “generate a course” became:

structure
validation
rewriting
enrichment
compliance
delivery

Not because we wanted more features —

but because without them, none of it worked.

That was the real lesson.

AI doesn’t remove complexity.

It just hides it — until it matters.