DEV Community: Mostafa Fathy

The Course Production Pipeline: How I Combine Instructional Design and AI Without Losing Quality

Mostafa Fathy — Sun, 12 Jul 2026 10:46:24 +0000

Most conversations about AI in education are stuck on the wrong question: "Can AI create a course?"

Yes, it can. In about twenty minutes. And that's exactly the problem — because what it creates in twenty minutes is content, not learning. The real question worth answering is: where does AI belong inside a serious course production process, and where does it not?

After building AI-assisted production pipelines for learning content, I've landed on an answer that I'll lay out in full here. It comes down to a hard boundary: humans design the architecture, AI produces inside it.

Why "AI-generated courses" fail

When you ask a language model to create a course on any topic, it produces something that looks remarkably like a course: modules, lessons, quizzes, even learning objectives. The structure is grammatical, so to speak.

But look closer and you'll find three failures, every time:

1. Objectives that describe topics, not behaviors. "Understand the fundamentals of marketing" is not a learning outcome — it's a table of contents entry. You can't observe "understanding," you can't practice it, and you can't verify it happened.

2. No designed forgetting-resistance. Human memory doesn't respond to exposure; it responds to retrieval, spacing, and application. Generic AI output arranges information logically — but logical arrangement is for reference material, not for learning. A well-designed course sequences practice, not just information.

3. Uniform difficulty. Real learners arrive with uneven prior knowledge and get stuck at specific, predictable points. Designing for those friction points is the craft of instructional design. AI can't predict them because they aren't in the text — they're in the learners.

None of this means AI doesn't belong in course production. It means AI can't be the architect.

The pipeline: five stages, one boundary

Here's the structure I use. The boundary between human and machine work sits between stages 2 and 3.

Stage 1 — Outcome mapping (human)

Before anything else, decompose the course into atomic units, where each unit maps to exactly one behavioral outcome. The test I use: the outcome must fit in one sentence with one verb, and it must describe something the learner does, not knows.

Weak: "Learn about pricing strategies"
Strong: "Price a new product using value-based anchoring"

This stage is slow, and it should be. It's where all the quality lives.

Stage 2 — Constraint specification (human)

Turn each atomic unit into a production spec: the outcome, the audience and their prior knowledge, the one practice activity that exercises the outcome, the checkpoint that evidences it, tone, length, and format constraints.

This is the step almost everyone skips, and it's the step that changes everything. A language model given a topic will invent a structure. A language model given a spec will fill a structure. The difference in output quality is not incremental — it's categorical.

Stage 3 — Constrained generation (AI)

Now, and only now, AI enters — and it's genuinely excellent here:

Drafting explanations against the spec
Generating scenario variations (the same negotiation exercise set in healthcare, retail, SaaS)
Producing distractor options for assessments that reflect real misconceptions
Adapting reading level and tone for different audiences

For larger projects I run this as a multi-agent system: separate agents for drafting, for reviewing against the spec, and for consistency checking across units. Each agent has one job and a narrow definition of success — which is, not coincidentally, the same atomic principle the course itself is built on.

Stage 4 — Human review at the friction points (human)

Not a full rewrite — a targeted review. I check the three places AI output reliably fails: the practice activities (are they actually doable and actually aligned to the outcome?), the transitions between units (does the sequence build?), and the examples (are they real-world plausible or plausible-sounding?).

Stage 5 — Instrument and iterate (human + data)

Ship, then watch where learners stall, quit, or fail checkpoints. Those friction points feed back into Stage 1 of the next revision. The pipeline is a loop, not a line.

What this changes in practice

The honest pitch for this approach isn't "AI makes courses cheaper." It's this: the pipeline moves human effort to where it's irreplaceable.

In a traditional process, a designer spends most of their time on production mechanics — writing, formatting, generating variations. Architecture gets whatever hours are left. This pipeline inverts that: production is largely automated, so the designer's time concentrates on outcome mapping, constraint design, and friction-point review — the work that actually determines whether anyone learns anything.

Faster production is real (in my experience, dramatically so). But speed is the side effect. The point is that quality stops depending on heroic effort and starts depending on system design — which means it survives scale, team changes, and deadline pressure.

Where to start

If you're building learning products and want to move toward this model, don't start by buying AI tools. Start here:

Take one existing course and rewrite every module objective as a one-verb behavioral outcome. Most won't survive the rewrite. That's the finding.
Pick the three units where learners struggle most and write full constraint specs for them.
Regenerate just those three units with AI working inside your specs. Compare against the originals — not for polish, but for whether the practice activities changed.

That comparison will tell you more about AI's real role in learning design than any tool demo.

I design learning systems and AI-assisted course production pipelines. If you're building courses or educational products and wrestling with where AI fits, my messages are open — I'm always happy to talk shop.

Originally published at mostafafathy.com.

The Multi-Agent Architecture for Education

Mostafa Fathy — Wed, 01 Jul 2026 14:08:02 +0000

The Problem With One Prompt

When we first started automating curriculum production , the obvious move was to throw a big prompt at Gemini and ask it to produce a full lesson.

It worked - sometimes. The quality was inconsistent. A great lesson on Monday, a mediocre one on Tuesday, and nobody could explain why.

The root issue: one LLM doing everything is like one developer handling design, backend, QA, and deployment simultaneously. You get output, but it's noisy.

The Multi-Agent Solution

We split the pipeline into four specialized agents, each with a narrow job:

1. Planner Agent
Takes the learning objective and breaks it into a structured outline - topics, subtopics, learning outcomes, estimated time per section. No content yet, just architecture.

2. Writer Agent
Receives the outline section by section and writes the actual content. Has no visibility into other sections - this forces consistency through structure, not context.

3. QA Agent
Reviews the full draft against the original learning objective. Flags gaps, redundancies, and places where the content drifted from the outcome. Returns a structured diff.

4. Formatter Agent
Takes the approved content and outputs it in the exact format our LMS expects - SCORM metadata, section markers, media placeholders.

Why LangGraph

LangGraph lets us define the flow as a state machine - each agent is a node, and we can branch conditionally. If QA flags major issues, the loop goes back to Writer. If it passes, it moves to Formatter.

This is something you can't do cleanly with a linear LangChain pipeline.

`python
from langgraph.graph import StateGraph

workflow = StateGraph(CurriculumState)
workflow.add_node("planner", planner_agent)
workflow.add_node("writer", writer_agent)
workflow.add_node("qa", qa_agent)
workflow.add_node("formatter", formatter_agent)

workflow.add_conditional_edges("qa", should_revise, {
"revise": "writer",
"approve": "formatter"
})
`

Results

After switching to this architecture:

Content consistency score went from 6.8/10 to 8.5/10 (internal rubric)
Average production time per lesson dropped by ~40%
QA revision loops average 1.2 iterations instead of 3+

The key insight: specialization improves quality even for AI agents. A model doing one focused task outperforms the same model doing five tasks in a single prompt.

Originally published at mostafafathy.com

Why I Treat the Curriculum Like a Software Architecture

Mostafa Fathy — Wed, 01 Jul 2026 14:07:22 +0000

The Insight That Changed How I Work

I spent years thinking about curriculum as content â€” topics to cover, videos to produce, exercises to write.

Then I started building software systems at Armstrong, and something clicked: a curriculum is an architecture problem, not a content problem.

Both have:

Components that do one job (a lesson, a microservice)
Dependencies that must be satisfied in order (prereqs, API contracts)
Failure modes that cascade if not handled (a confusing lesson breaks everything after it)
Interfaces between parts (how a module hands off to the next)

The Problems This Reframing Solves

1. The "Zipped" Curriculum Problem

Most curricula are built like a monolith â€” everything is coupled. Change one topic and you have to rewrite three lessons. Add a new module and the learning path breaks.

Software has solved this with microservices. We applied the same idea: every lesson is self-contained. It has defined inputs (what the learner must know before), defined outputs (what they can do after), and no hidden dependencies.

Now we can swap, update, or reorder lessons without touching anything else.

2. The No-Tests Problem

Software engineers write tests. Instructional designers... review? Hope?

We introduced learning outcome validation at the unit level â€” every lesson has an exit assessment that maps directly to its stated outcome. If learners fail the assessment, the lesson failed, not the learner.

This is just unit testing for education.

3. The "Who Owns This" Problem

In a big curriculum, nobody knows who's responsible for which part. Bugs (bad content, outdated info) go unfixed for months.

We use a CODEOWNERS-style assignment. Every module has an owner. Changes require review from the owner. Same as a pull request.

What This Looks Like in Practice

Our curriculum now has:

A dependency graph (built in Notion, visualized in Figma)
Module specs that define inputs/outputs before a single word is written
An automated QA pipeline (our AI system) that checks new content against the spec
A versioning system â€” curriculum v2.3, not just "the updated version"

The Result

We went from ad-hoc content production to a system that can onboard a new SME in two days and have them producing spec-compliant content by day three.

The curriculum is no longer a document. It's a system. And systems can be improved systematically.

Originally published at mostafafathy.com

Why Nobody Finishes Your 1-Hour Video

Mostafa Fathy — Wed, 01 Jul 2026 14:01:18 +0000

The Attention Problem Is a Design Problem

Originally published at mostafafathy.com

The Information Dump Problem

Mostafa Fathy — Wed, 01 Jul 2026 14:01:15 +0000

The Mistake Every New EdTech Platform Makes

Originally published at mostafafathy.com