DEV Community: Hansheng Li

A Better Workflow for AI-Assisted Coding: Lessons from Matt Pocock

Hansheng Li — Mon, 19 Oct 2026 05:34:10 +0000

I recently watched Matt Pocock's workshop, Full Walkthrough: Workflow for AI Coding.

The main takeaway: Good AI-assisted coding is not about writing better prompts. It's about applying solid software engineering practices to AI agents.

Here are the key lessons.

1. Keep AI Tasks Small and Context Fresh

LLMs tend to become less reliable as their context grows.

Matt describes two zones:

Smart Zone: Fresh context, focused tasks, better decisions.
Dumb Zone: Bloated context, reduced reliability, more mistakes.

He uses roughly 100K tokens as a practical warning threshold, not a universal technical limit.

His recommendations:

Break large tasks into smaller, independent tasks.
Keep system instructions minimal.
Prefer clearing context and restarting with well-defined tasks rather than repeatedly compacting long conversations.
Use subagents for isolated exploration without filling the main agent's context.

2. Align Requirements Before Writing Code

Instead of immediately asking AI to implement a feature, Matt uses a custom "Grill Me" skill.

The AI interviews the developer, asking questions one at a time and recommending possible answers.

For example, when building a gamification feature:

Which activities earn points?
Should existing users receive retroactive points?
How should levels work?
Where should the UI appear?

The goal is to reach a shared understanding before implementation.

Planning requires human involvement. Coding can be delegated much more easily.

3. Turn Requirements into a PRD and Independent Tasks

Once the requirements are clear, create a Product Requirements Document (PRD) covering:

Problem and proposed solution
User stories and acceptance criteria
Implementation decisions
Testing requirements
Out-of-scope features

The PRD describes the destination.

Next, break it into independent issues with explicit dependencies. This creates a Kanban-style backlog that agents can work through, potentially in parallel.

4. Build Vertical Slices, Not Horizontal Layers

AI often prefers implementing features horizontally:

Build the entire database layer.
Build the APIs.
Build the frontend.

The problem is that meaningful end-to-end feedback comes too late.

Matt recommends vertical slices (tracer bullets) instead.

For example, implement one small feature that includes:

Database changes
Backend logic
API integration
A minimal frontend
Tests

Each slice delivers something functional and testable.

This makes integration problems visible much earlier.

5. Use Ralph Loops for Automated Implementation

Once the backlog is ready, implementation can become an automated loop:

Select an unblocked issue.
Explore the relevant code.
Implement the feature.
Run tests and type checks.
Commit the changes.
Repeat until the backlog is complete.

Matt demonstrates this with a Ralph-style loop.

For parallel execution, he also introduces Sand Castle, which uses isolated Git worktrees and Docker sandboxes for agents.

Separate agents can handle planning, implementation, review, and merging.

6. Test-Driven Development Makes AI More Reliable

Matt strongly recommends TDD (Test-Driven Development) for AI agents.

The process follows:

Red → Green → Refactor

Write a failing test.
Implement the minimum code needed to pass.
Refactor while keeping the tests green.

Writing tests first makes it harder for AI to generate weak tests that merely confirm its existing implementation.

More importantly, AI needs fast feedback loops:

Unit and integration tests
Type checking
Linting
Automated code review

The quality of your feedback loops often determines the quality of your AI-generated code.

7. Design Deep Modules with Simple Interfaces

Another important lesson comes from A Philosophy of Software Design.

Matt recommends deep modules:

Small, clear public interfaces
Substantial functionality hidden internally
Well-defined responsibilities
Easy-to-test boundaries

This is preferable to many tiny, tightly coupled modules.

The developer should understand and control the overall architecture while delegating internal implementation details to AI.

This helps prevent losing architectural understanding as AI generates more code.

8. Never Skip Human QA and Code Review

Automated tests passing doesn't guarantee a good product.

Matt emphasizes that humans still need to:

Verify actual user behavior.
Review important architectural decisions.
Inspect tests and critical code.
Evaluate frontend usability.
Identify missing requirements.

He also recommends using a fresh AI context for code review rather than asking an agent to review its own work after a long implementation session.

QA findings can become new backlog issues, creating a continuous improvement cycle.

The Complete Workflow

Idea
  ↓
Grill Me (Requirements Alignment)
  ↓
PRD (Define the Destination)
  ↓
Independent Issues (Vertical Slices)
  ↓
AI Implementation (Ralph Loop)
  ↓
TDD + Automated Feedback
  ↓
AI Code Review
  ↓
Human QA
  ↓
New Issues / Improvements
  ↓
Repeat

Final Thoughts

The biggest lesson from this workshop is that AI doesn't replace software engineering fundamentals. It makes them even more important.

Good architecture, small tasks, clear requirements, automated tests, and continuous feedback are what make AI coding agents effective.

The developer's role increasingly shifts from writing every line of code to defining requirements, designing systems, providing feedback, and maintaining quality.

AI can handle much of the implementation, but humans still need to own the decisions and the final product.

Source: Matt Pocock — Full Walkthrough: Workflow for AI Coding (YouTube)

Suggested DEV.to tags: #ai #programming #productivity #softwareengineering

From AI Agent Demos to Production: 6 Essential Concepts for Agentic AI System Design

Hansheng Li — Mon, 19 Oct 2026 04:41:49 +0000

Building an AI agent demo is easy. Building one that is reliable, scalable, secure, and production-ready is much harder.

I recently watched Agentic AI System Design was HARD until I Learned these 6 Concepts and summarized six important concepts for designing production AI agents.

Let's use a customer support agent as an example. Imagine a customer asking:

"I never received my order. Please cancel it and refund me."

The agent needs to understand the request, retrieve the order, check refund policies, execute authorized actions, and respond correctly.

Here's what makes this challenging.

1. Agent Loop: Plan → Act → Observe → Repeat

Unlike a traditional LLM application that simply generates a response, an AI agent can reason, call tools, observe results, and decide what to do next.

A typical agent loop:

Plan: Determine what needs to happen.
Act: Call a tool or execute an operation.
Observe: Examine the result.
Repeat: Continue until the task is complete.

The challenge is reliability decreases as the number of steps increases.

If each step has a 90% success rate, assuming independent failures:

3 steps: 72.9% success
5 steps: 59.0% success
8 steps: 43.0% success

This illustrates why unnecessary complexity can make agents unreliable.

Key takeaway: Minimize unnecessary steps, validate intermediate results, and implement retries, fallback strategies, and clear stopping conditions.

2. Model Routing: Use the Right Model for Each Task

Using the most powerful LLM for every task increases cost and latency unnecessarily.

Different tasks require different levels of intelligence.

Task	Recommended approach
Classify customer intent	Small, fast model
Extract order ID	Small model or code
Look up order information	Database/API
Understand complex complaints	More capable LLM
Validate refund eligibility	Deterministic business logic
Generate customer response	Small or mid-tier LLM

This technique is called model routing.

The objective is to balance:

Quality ↔ Cost ↔ Latency

Key takeaway: Use expensive reasoning models only when additional reasoning improves the outcome.

3. Tool Design: Treat Tools Like Production APIs

Tools allow agents to interact with external systems.

Our customer support agent might have:

get_order_details
search_refund_policy
check_refund_eligibility
issue_refund

However, adding more tools also creates more opportunities for incorrect tool selection and execution failures.

Well-designed tools should provide:

Clear names and descriptions.
Strict input/output schemas.
Structured error responses.
Authentication and authorization.
Separation between read and write operations.

For example, retrieving an order is low-risk, while issuing a refund changes financial records.

The video also introduces MCP (Model Context Protocol), an open standard for connecting AI applications with tools and external data.

One additional production consideration is idempotency.

If a refund request succeeds but the API response times out, retrying it should not accidentally issue another refund.

Key takeaway: Let the LLM propose actions, but use backend code to validate and safely execute them.

4. Memory, State, and RAG: Give Agents the Right Context

These concepts solve different problems.

State represents the current workflow execution:

Current step.
Order ID.
Approval status.
Refund status.

Memory represents longer-term information:

Previous conversations.
Customer history.
User preferences.

RAG (Retrieval-Augmented Generation) retrieves relevant external knowledge, such as company refund policies.

Instead of loading an entire policy handbook into the prompt, RAG retrieves only the relevant passages.

A typical RAG pipeline:

Documents → Chunking → Embeddings → Retrieval → Ranking → LLM

But retrieval quality matters.

If the system retrieves an outdated or incorrect policy, the LLM might confidently generate an incorrect answer.

Important RAG considerations include:

Chunking strategy.
Metadata filtering.
Retrieval accuracy.
Reranking.
Document freshness.

Loading too much information into the context window can also introduce context rot, where irrelevant information makes reasoning less effective and increases token costs.

Key takeaway: Store workflow state separately, retrieve memory selectively, and provide only relevant context to the model.

5. Orchestration: Workflow vs. Single Agent vs. Multi-Agent

Orchestration coordinates models, tools, memory, and execution steps.

There are three common approaches:

Approach	Best for
Deterministic workflow	Predictable business processes
Single agent	Tasks requiring dynamic decisions
Multi-agent	Complex tasks with distinct specialist responsibilities

For example, a refund process usually follows a predictable workflow:

Classify → Retrieve → Validate → Approve → Execute → Respond

A predefined workflow may be more reliable than allowing an autonomous agent to plan everything.

Multi-agent systems introduce additional coordination, latency, and failure risks.

Each handoff between agents creates another opportunity for context loss or mistakes.

Key takeaway: Start with a simple workflow or single agent. Introduce multi-agent architecture only when it provides measurable benefits.

6. Evals, Observability, and Security: Production Controls

This is what separates a working demo from a production system.

Evals: Is the agent doing the right thing?

Traditional software uses deterministic tests.

AI agents introduce nondeterministic behavior. An agent can return valid JSON while selecting the wrong tool or applying the wrong refund policy.

Evaluation datasets should include:

Normal requests.
Ambiguous questions.
Incorrect or missing information.
Tool failures.
Unauthorized requests.
Prompt injection attempts.

Rerun evaluations whenever prompts, models, tools, or workflows change.

Observability: What actually happened?

When something goes wrong, we need to identify exactly which step failed.

Track:

Model and prompt versions.
Tool calls and results.
Workflow execution traces.
Latency and token costs.
Errors, retries, and fallback decisions.

Tracing is especially important because one agent request may involve multiple LLM calls and external APIs.

Security: What if the agent makes a mistake?

Not all actions have the same risk.

Reading order information is different from issuing a refund.

For high-risk actions, follow:

LLM proposes → Backend validates → Human approves (if required) → Tool executes

Business rules and permissions must be enforced by application code, not just prompts.

Also treat customer messages, retrieved documents, and external tool responses as untrusted input to defend against prompt injection.

Key takeaway: Measure agent quality, trace execution, and restrict what the agent can do.

Putting It All Together

A production customer support agent might use this architecture:

Layer	Responsibility	Example technology
API	Receive requests	FastAPI
Orchestration	Coordinate workflow	LangGraph
Models	Classification and reasoning	LLM APIs
Tools	Execute operations	Internal APIs
State	Persist workflow progress	PostgreSQL
Cache	Temporary data	Redis
RAG	Retrieve relevant policies	pgvector
Observability	Trace execution	OpenTelemetry
Monitoring	Detect errors	Sentry
Approval	Human-in-the-loop	React/Next.js

These are possible implementation choices rather than requirements from the video.

Final Thoughts

The biggest lesson is that production AI agents are still software engineering systems.

LLMs introduce flexible reasoning, but traditional engineering principles remain essential:

Reliability and fault tolerance.
API and database design.
State management.
Testing and evaluation.
Logging and observability.
Security and authorization.

A successful AI agent combines probabilistic AI reasoning with deterministic software controls.

The goal is not to build the most autonomous or complicated agent. It is to build the simplest agent that reliably solves a real problem.

Reference: Agentic AI System Design was HARD until I Learned these 6 Concepts — YouTube

These are my learning notes based on the video, with a few additional engineering considerations and implementation examples.