DEV Community: Rob Earlam

Harness Engineering: Anthropic's AI SDLC

Rob Earlam — Tue, 20 Oct 2026 00:00:00 +0000

This post is part of my series deep diving into Harness Engineering. You can find all of the posts in the series so far below:

An Introduction to Harness Engineering
Harness Engineering: Agents and Roles
Harness Engineering: Artifacts, Inputs and Context
Harness Engineering: Orchestration and execution tracking
Harness Engineering: Running the pipeline end-to-end
Harness Engineering: Anthropic's AI SDLC

When I last wrote about Harness Engineering back in May, I walked through taking a backlog item all the way through specification, design, planning, build and QA. The idea was to stop treating AI development as one enormous conversation and instead give each stage a clear responsibility, defined inputs and an artifact to hand to the next stage.

That was a few months ago now, and things have moved on a bit since then. I've been evolving the harnesses that I'm using, so I figured it was probably time for an update.

In my original introduction in April, I called out IDE independence as a benefit. I kept the workflow in shared role files, with thin adapters for Claude, Cursor and GitHub Copilot. It meant I could change tools without rewriting the whole process. I've recently moved away from that. The harness in this repo now runs solely on Claude Code , using its native agents, skills, settings and hooks.

So why did I reverse something I'd previously described said was one of the advantages of this approach? Because sharing instructions between tools isn't the same thing as getting the same execution behaviour from them.

From portable instructions to native execution

The old structure separated the agent definitions from the role definitions. That made sense for portability, but it also meant maintaining three sets of adapters and several documents describing how the workflow should behave.

There was a more important problem with the Claude implementation: the adapters lived in .claude/commands/. Those were slash commands, not isolated subagents. The stages ran inline in the main conversation, so by the time Build started, the conversation already contained the orchestrator's reading and the work from the earlier stages.

I'd described bounded context as a key benefit of the approach, and I realised that with the advances in the off the shelf harnesses I was builting on top of, I could get that benefit much more simply by choosing a single IDE to work with and aligning to its native tooling.

The workflow is still recognisable:

Task definition
  -> Specification
  -> Design
  -> Implementation plan
  -> Build and tests
  -> Correctness review
  -> QA
  -> Human approval

The difference is that more of the structure now comes from the runtime and executable checks, rather than instructions asking the model to maintain that structure itself.

What changed in the harness?

Standards became skills

The coding, testing and UI design standards now live in .claude/skills/. Each stage agent preloads only the skills it needs through its frontmatter.

Previously, a stage loaded well over a thousand lines of roles, standards and templates before doing any actual work. Much of that was repeated workflow guidance. The new agent definitions are around 40 to 80 lines each, supplemented by the relevant skills and task inputs.

That doesn't mean the whole context is only 80 lines, it means the agent's own definition is smaller, with less duplicated guidance and a clearer separation between its job and the standards it applies.

It also lets me assign models by role much more easily. Design, technical planning and QA use Opus, while specification and implementation use Sonnet. I can make that trade-off explicitly rather than giving every stage the same configuration.

The state machine became a script

One of the key changes I made though was to pivot the Orchestrator away from being an Agent, to now being a Skill instead. The orchestrator still needs judgment: which findings matter, whether a request changes the specification, and what information needs carrying forward. But deciding whether Build can advance to QA is largely a state-machine problem.

That machinery now lives in the harness-state script. It owns stage transitions, run IDs, retry limits, workflow modes and the recorded run history. The /run-task skill handles orchestration, while the script refuses invalid transitions.

I'm no longer asking an LLM to repeatedly interpret several pages of process rules to work out whether a transition is legal. Code handles the mechanical part; the model handles the judgment.

Verification happens at the handoff

A SubagentStop hook runs artifact verification when a stage agent finishes. The verifier checks that the expected file exists, isn't empty and contains the required headings. The state script verifies the artifact again before advancing.

That's a structural check, not proof that the specification is good or the feature works. Tests, review and human judgment still have jobs to do. But it does mean that saying "I've finished the design note" isn't enough to move the pipeline forward when the note is missing.

A SessionStart hook also loads the live execution state, helping a new session pick up the current task rather than reconstructing it from conversation history.

Governance moved into configuration

Claude Code's settings now deny file-tool edits to the harness control files and protect environment files. Changes to task definitions and the backlog, and a git push, prompt for permission. The /approve skill is human-invoked rather than something the model can choose to run itself.

There are limits here. The repo's decision record explicitly notes that file-tool permission rules don't block shell redirection into protected paths. This isn't a complete sandbox or an enterprise security boundary, but I've found its been good enough to work for my use cases.... so far anyway!

Still, there's a useful distinction between a rule written in a prompt and a restriction applied by the runtime. For the controls covered by those settings, I've moved from asking the agent not to do something to configuring the tool to refuse it.

Review got a clearer stopping point

Before QA, the orchestrator now runs the built-in /code-review high correctness pass against the files changed by the task. The run records a base commit, so the review stays scoped even when the task didn't start directly from main.

QA then checks those findings against the specification. Broken acceptance criteria in normal use, security issues, regressions and task-caused validation failures block progress. Refactoring suggestions and other non-blocking findings become candidate follow-up work for me to consider.

That matters because an agent can always suggest another improvement. Without a consistent severity bar, a bounded delivery loop can become an endless polishing exercise. There are at most three Build passes before the run stops as blocked, I've generally found that most tasks are done in 1 or 2 passes and if it's trying to go past a third then there's something wrong with something earlier in the workflow like the task definition, or spec and the Human should step in at this point.

What do I get out of giving up portability?

The immediate benefits are structural: fewer adapter files to maintain, less repeated instruction text, genuine context isolation between stages, and deterministic checks around workflow progression.

The boundaries are also easier to explain. An agent owns a stage. An artifact carries the handoff. A script owns execution state. Settings and hooks provide specific controls. I remain responsible for approval.

All of this has lead to a much smaller required token window for the tasks being executed, and I'm noticing that I seem to get much more output from my Claude Pro account than I was doing previously.

There is a cost however. If I wanted to pivot and use a different IDE again that would now require adapting the enforcement layer, not just adding a thin agent file. For now, it's a trade off I'm happy to make, but with the pace of change in this area, it's something I'm going to keep reviewing moving forwards.

How this compares with Anthropic's AI-native SDLC playbook

While making these changes, I also read The AI-native SDLC playbook, by Louis Claxton, dated 21 August 2026.

Its central argument is that making code generation faster isn't enough. Planning, handoffs, verification and governance need to change too, otherwise they become the bottleneck around the build phase.

That felt very familiar after writing this series since April. The strongest similarities aren't the names of the agents. They're the mechanisms connecting the stages:

Approach in this series and repo	Corresponding approach in the playbook
A task definition captures scope and intent before delivery starts.	An `intent.md` captures the problem, outcome and constraints.
Specification, design and implementation-plan artifacts guide later stages.	Version-controlled intent, spec and plan artifacts carry decisions into implementation.
Agents have narrow responsibilities and bounded inputs.	Scoped subagents have their own context windows and tool limits.
Coding, testing and design standards are shared, versioned guidance.	Skills encode institutional knowledge alongside the code.
Tests, correctness review and QA provide feedback before approval.	Agents verify their work and review findings inform human judgment.
Hooks, settings and the state script check specific workflow boundaries.	Deterministic guardrails back advisory instructions and preserve approval gates.

The artifact chain is the clearest connection. In my post on artifacts, inputs and context, I described outputs from one agent becoming the explicit inputs for another. The playbook makes that same principle central to its lifecycle: durable artifacts carry the decisions forward and provide an audit trail.

The shift in human involvement is familiar too. In my May walkthrough, I wasn't manually starting and supervising every stage. The orchestrator coordinated the work, and I came back to review the result. The playbook similarly moves human attention towards intent, risk and approval rather than supervising every edit.

A shared direction, not an identical system

It's aways nice to see big players like Anthropic, advocating for the same agentic flow that I have been discovering on my own as well. I didn't invent this approach though, my first post mentioned OpenAI's harness engineering article as the inspiration for rebuilding this site. Anthropic's playbook is another concrete example of major vendors describing development as a managed lifecycle around agents, rather than simply faster code completion.

If you want to see the workflow I'm using to build out this site, then it's all available in my GitHub Repo

Harness Engineering: Running the pipeline end-to-end

Rob Earlam — Fri, 08 May 2026 00:00:00 +0000

This post is part of my series deep diving into Harness Engineering. You can find all of the posts in the series so far below:

An Introduction to Harness Engineering
Harness Engineering: Agents and Roles
Harness Engineering: Artifacts, Inputs and Context
Harness Engineering: Orchestration and execution tracking
Harness Engineering: Running the pipeline end-to-end

When I finished my last blog post, I said I wanted to write this post detailing how I run the complete pipeline in the harness from end to end, so let's do that.

We're going to follow one backlog item through the pipeline to completion. I chose the backlog item where I added support for mermaid diagrams into the blog. This is a good example because it's a pretty small update, but it's got to be integrated with a lot of existing code that's already there. If you look back at the previous post on harness engineering where I covered agents and roles, you can see an example of a mermaid diagram in action, and the reason this feature was required.

Generating the Task Definition from the Backlog

So let's start by looking at the backlog item. Below you can see the item as it was listed in my backlog file, and as you can see, it's just a simple one-liner. There's not much detail there at all.

### B017 - Blog support for Mermaid Diagrams
**Status:** Done  
**Summary:** Add support for mermaid diagrams to be added into the blog post functionality for the site.

To prepare the pipeline for execution I need to take that backlog item and generate a task definition. I do this by talking to the LLM and asking it to generate a task definition based on the backlog item number. I go back and forth with the LLM a few times, answering some questions, and once it has all the information it needs, it goes away and generates the task definition. You can see the back and forth I had with the LLM below.

Rob: 
Can you please generate the task document for backlog item B017. This Task id will be T014. Please ask me any qualifying questions you require to develop the task document. Ensure you use the task template when generating the document

---

LLM: 
Perfect! I found B017 in the backlog. Before I generate the task document for T014, I have some qualifying questions about the Mermaid diagram support feature:

Implementation Scope:
1. How should Mermaid diagrams be integrated? (e.g., as a markdown code fence syntax like ```

mermaid ...

 ```, a custom component, or both?)
2. Should ALL Mermaid diagram types be supported, or are there specific types to prioritize? (flowcharts, sequence diagrams, class diagrams, etc.)

User Experience:
3. Should there be any styling applied to the diagrams beyond Mermaid's defaults? Should they respect the site's theme/color scheme?
4. How should invalid or malformed Mermaid code be handled? (silent fallback, error message, placeholder?)

Display & Performance:
5. Should diagrams be displayed at full width, constrained to a specific width, or responsive?
6. Are there any performance considerations or diagram size/complexity limits to document?

Documentation & Examples:
7. Should the blog post content about Mermaid diagrams include example diagrams with the feature documentation?
8. Should there be a style guide or documentation page added to the site showing supported diagram types and usage examples?

Testing Requirements:
9. What level of testing is expected? (unit tests for the rendering component, integration tests, visual regression tests?)

Please answer these questions so I can create a comprehensive and well-scoped task document.

---

Rob:
1. Yes in mermaid code fence
2. all default mermaid diagrams to be supported please
3. no styling needed for now, let's go with the defaults
4. error message please
5. Responsive please
6. I don't think so, the ones I'm creating won't be too complex
7. I'm not blogging about mermaid, the diagrams will be included on posts covering other topics
8. No
9. Yes testing is required

Do you need any other information to generate the task document now?

---

LLM: 
I have all the information I need. Let me create the task document for T014 now:

Perfect! I've created the task document T014-blog-mermaid-diagrams.md with:
- Clear summary of the Mermaid diagram functionality
- Comprehensive acceptance criteria covering all aspects you specified:
- Markdown code fence support
- All Mermaid diagram types
- Responsive rendering
- Error handling with messages
- Default styling (no custom styling)
- Unit and integration tests
- Task notes with implementation guidance (suggesting a React component library for Next.js)
- Proper artifact references for the specification, design, implementation plan, and QA review

You can see here that, after a short back and forth it returned and stated that it's generated the completed task file, if you look at the generated file here: T014-blog-mermaid-diagrams.md. It's really detailed, covering what it is we need to build.

Running the Harness Pipeline for the generated task

Next up is actually executing the Harness pipeline for the Task definition. This is really simple. Again, just one line passed to the LLM, and then it goes off and does its thing.

Rob:
Please run the orchestrator agent for T014.

At this point, the orchestrator agent will take over. It will generate the run-state documents and also the run report used for logging. Then it will read the task definition for T014 that you can see from the link above, and it will start to execute each of these sub-agents sequentially. Each one will generate its artifact and execute on its responsibilities as we've discussed in the previous posts.

PO-Spec agent

First up is the PO-Spec sub-agent. This is going to take that task definition file and it's going to produce a feature specification document for the work that needs to happen. This is used to clarify the scope and define the acceptance criteria for the piece of work. You can see the generated artifact here: T014-feature-spec.md.

Feature Design agent

Next comes the Feature Design Agent. This uses the generated Feature Specification document created by the PO-Spec sub-agent. It then translates that into a series of UX and Layout design decisions that can be leveraged later on by the build agent. You can see the generated document here T014-design-note.md.

Tech Lead agent

The Tech Lead agent is next. This takes the feature specification document created by the PO-Spec agent and the design note created by the feature design agent. It uses those to produce a detailed implementation plan, taking those product and design intent documents and converting them into defined delivery steps. You can see the generated document here. T014-implementation-plan.md.

Build Agent

The build agent then takes the detailed implementation guide and follows it to produce the required code and tests to actually do the delivery of the feature. It will iterate and rerun itself up to a maximum of three times to ensure that when tests fail, it can adjust the code accordingly to make sure that everything is functionally correct and that we have tests covering all of the feature steps that were involved. There is no actual output artifact to show for this stage because the artifact is the code itself. If you take a look in the codebase, you can see the feature enabled there.

I can see that it added a reference to the mermaid NPM package, built the components to deliver the functionality, and wrote tests to protect the functionality from unintended changes moving forwards. This is pretty much how I would have implemented the feature myself had I been doing it by hand, so it's great to see how accurate the Build Agent is when it has all of the supporting artifacts to guide it.

QA Agent

The final step is the QA agent. This reads all of the upstream artifacts that were generated above, and it produces a QA review detailing whether the initial acceptance criteria have been completed by the generated code. If it fails, it will detail what has failed, and it will go back to the build step, asking it to re-run to address those issues. Again, we have a maximum retry limit of 3 for this iteration. If you look here, you can see the generated QA report for this feature: T014-qa-review.md.

This leads to the following flow of artifact creation when all of the agents are combined together

flowchart TD
    B[Backlog Item] --> T[Task Definition]
    T --> FS[Feature Specification]
    FS --> DN[Design Note]
    DN --> IP[Implementation Plan]
    IP --> CT[Code and Tests]
    CT --> QR[QA Review]
    QR --> D[Deployed Feature]

The Orchestrator

As we talked about in the previous blog post about the orchestrator, it isn't responsible for generating the delivery artifacts itself. What it does though, is create order and structure to the execution by coordinating the execution of the sub-agents. It validates that the sub-agents have generated the artifact that they are responsible for, and then decides whether to pass execution flow to the next agent in the pipeline. It also tracks this information in the run-state document, and the run report, so we keep track of what state we're currently in. You can see a diagram showing this flow here:

flowchart TD
    O[Orchestrator] --> RS[Run-State]
    O --> RR[Run Report]
    O --> P[PO-Spec]
    O --> FD[Feature Design]
    O --> TL[Tech Lead]
    O --> B[Build]
    O --> QA[QA]

Finishing up

At this point, I step back in and go and actually review the code myself. Make sure I'm happy with it and that it's something I would deploy. If I am, then it's a simple case of going and pushing it through my CI/CD system, and then the feature will be live.

If you find that there are issues with how the feature has been implemented, or with the code quality, what I would then do is go and talk to the LLM once more. State that these are the issues you've seen. Ask it to go and review the various artifacts that have been created. Then you can get it to go and tweak the task definition right at the start, clear all the artifacts and run the pipeline again, and see if you then get a response and an output that is more to your liking.

So while you still have humans involved in the development process, they're no longer involved at every step any more when working this way. What this leads to is less ambiguity between the stages and the responsibility for each of your agents. It means you no longer have to write these huge page-long prompts trying to define everything in one go, because everything is tightly bounded and defined by the harness guidance.

If you look at the two diagrams above, they are both Mermaid diagrams, so you can see the completed feature delivered, live, and in use!

Harness Engineering: Orchestration and execution tracking

Rob Earlam — Fri, 24 Apr 2026 00:00:00 +0000

This post is part of my series deep diving into Harness Engineering. You can find all of the posts in the series so far below:

An Introduction to Harness Engineering
Harness Engineering: Agents and Roles
Harness Engineering: Artifacts, Inputs and Context
Harness Engineering: Orchestration and execution tracking

This post continues my series on Harness Engineering , this time I'm going to focus specifically on the Orchestrator Agent and Role to talk about how it is the workflow controller , ensuring that the artifacts the subagents create are generated successfully, then managing the following handoff as the execution flow continues.

Why do we need orchestration?

Once you create a workflow that has multiple stages , you're left with a choice. You can either manually execute each of these steps yourself, passing in the required artifacts that each stage requires yourself, or you can automate that process. Obviously, with the power that AI provides today automating that process is much more achievable. What you need to create is something responsible for sequencing the agents , validating handoffs , and stopping the system from drifting or skipping ahead. This is where the Orchestrator Agent comes in.

The Orchestrator Agent is different from the other agents we have talked about already as it is responsible for execution rather than being a worker itself. It is responsible for controlling the current workflow state , determining the valid next stage , invoking the correct agent, checking that required artifacts were created, and updating control artifacts accordingly. Crucially it is also responsible for catching any possible issues with a subagents output and terminating the pipeline when this happens, returning a response back to the human for them to jump back in and resolve.

The important point to remember is that the Orchestrator is not here to write specs, design features, build code, or perform QA. It exists purely to coordinate subagent roles and preserve workflow integrity throughout the full execution of the pipeline.

How do I start the pipeline?

Now you might be wondering how the pipeline execution is started? What provides the input for the first subagent that is executed? In my case that is the PO-Spec Agent , and it gets its input from a Task Definition file.

I maintain a very light backlog file in my harness that tracks upcoming pieces of work that I want to complete, and this is what I use to kick off the generation of a Task Definition File. Now, when I say that the backlog items are light, I'm not joking, here is the whole backlog item I had for generating the homepage content I built for my site:

### B003 — Homepage v1
**Status:** Done  
**Summary:** Create the homepage with hero content, introduction, and clear navigation to key areas.

What I do is I have a conversation with the LLM inside of my IDE to generate from the Task Template , we go back and forth a few times and I tell it to ask me any questions it needs to clarify what I want creating. Here is the exact back and forth conversation we had for that task:

Rob: 
  Please can you generate the task file for backlog item B003 - Homepage. I want the homepage to include the highlights of the content in the rest of the site. So showing the latest blogs & videos. If I have an upcoming speaking events registered also show those. 
  The homepage must be visually engaging. I also have a profile picture included in the img dir in the next app, so incorporating that somewhere would be good as well.  
  Please ask me any other questions to get the information you need for this.

LLM:
  Before I draft the task file, I have a few questions to make sure it captures your intent accurately:
    1. Hero / Intro section — Should the hero include your rob.png profile photo alongside the heading and intro text, or would you prefer the photo in a separate "About Me" section below the hero?
    2. Latest Blogs — How many recent blog posts should be shown on the homepage (e.g. 3 or 5)?
    3. Latest Videos — Same question — how many recent videos should be surfaced?
    4. Upcoming Speaking — Should this section only appear when there are upcoming events (i.e. hidden when none exist), or should it always be shown with an empty-state message?
    5. Call-to-action links — Should each content section (blogs, videos, speaking) include a "View all →" link to its respective page?
    6. Section ordering — What order would you like the sections in? My suggested default would be: Hero → Latest Blogs → Latest Videos → Upcoming Speaking. Does that work, or would you like a different arrangement?

Rob: 
    1. Lets try including it in the head and intro text please.
    2. Lets go for 3.
    3. 3 again.
    4. Only shown when there are upcoming events.
    5. yeah links to all sections of the site.
    6. Suggested order works for me!

LLM: 
    Now I have everything I need. Creating the task file.
    Task File is ready. Here's a summary of what's captured:
     - Hero — two-column layout (photo left, name + intro right) at desktop, stacked on mobile. Uses Next.js <Image> for rob.png. The generic HeroSection component is extended non-breakingly via a homepage-specific variant in src/components/homepage/.
     - Latest Blogs — 3 posts, reusing BlogPostCard, "View all posts →" link to /blog.
     - Upcoming Speaking — fully conditional; hidden when no upcoming events exist, "View all speaking →" link to /speaking when shown.
     - All data loaded at build time via existing lib layers — no new data fetching logic needed.
     - Section order locked: Hero → Blogs → Videos → Speaking.

After the LLM completed at the end, it generated a really good Task Definition. It's pretty long so I won't include it, but you can view the full contents for it in this here: T010-homepage.md. Once we have the generated Task Definition, it becomes super easy to start the execution pipeline, all I do is use the following prompt and away it goes!

Please execute the Orchestrator Agent for Task definition T010.

How does the Orchestrator track state during pipeline execution?

Once I give the LLM the command above the pipeline is going to start executing and Orchestrator Agent will begin running the different subagents in the required order, checking the required artifacts are created after each. Now you might be wondering how does the Orchestrator know where it is in the pipeline? How does it know that, for example, the Design Agent has just completed and it's ok to move on to the Tech Lead Agent? Well this is where the run-state.md file comes in, as that is the authoritative record of what task is active, which stage is current, which artifacts have been completed, and whether the workflow is blocked , awaiting approval , or ready to continue.

The run-state.md file is pretty light, it purely lists current state of where things are and the locations of the generated artifacts ready to be picked up by the other subagents. I also don't persist the run-state after a feature is complete, as its only required during pipeline execution and doesn't give any benefit afterwards. You can see the run-state for the last update I did, where I added Dark Mode support to the site below

# Run State

## Task

Task ID: T015  
Task File: harness/tasks/T015-dark-mode.md

---

## Execution State

Current Status: Done  
Current Stage: Completed  
Current Run ID: RUN-2026-04-02-T015

Stop Reason: None

---

## Completed Stages

- Specification — RUN-2026-04-02-T015-SPEC — 2026-04-02 — PASS
- Design — RUN-2026-04-02-T015-DESIGN — 2026-04-02 — PASS
- Implementation Planning — RUN-2026-04-02-T015-PLAN — 2026-04-02 — PASS
- Build — RUN-2026-04-02-T015-BUILD — 2026-04-02 — PASS
- QA — RUN-2026-04-02-T015-QA — 2026-04-02 — PASS

---

## Artifact Paths

Feature Specification: harness/artifacts/specs/T015-feature-spec.md  
Design Note: harness/artifacts/design/T015-design-note.md  
Implementation Plan: harness/artifacts/plans/T015-implementation-plan.md  
QA Review: harness/artifacts/qa/T015-qa-review.md  
Run Report: harness/runs/RUN-2026-04-02-T015.md

---

## Approval Mode

Require Approval Before Build: false  
Require Approval Before Complete: true

---

## Notes

Run initialized 2026-04-02. Task is Dark Mode Support (B018). Specification, Design, Implementation Planning, Build, and QA completed 2026-04-02. QA result: Pass. Advancing to Awaiting Approval — gate requires human approval before Complete.

Another way I record state is in the Run Report , this acts as the execution log and is different from run-state. The run-state tells you the current status of the workflow , whereas the run report tells you what happened during execution , what decisions were made , what validations ran , and what the outcomes of each stage were. Keeping these separate gave me a nice clear separation of concerns , one file governs the current state of the workflow , while the other preserves logs of execution output. Unlike the run-state which I treat as ephemeral, created fresh to each run, the Run Report is one of the Artifacts of the Orchestrator and is persisted in the harness as such. Again, the generated file is quite large, so you can see the full contents of it here: RUN-2026-03-26-T010.md

How does the Orchestrator progress through the stages?

The run-state and run-report combined to give the Orchestrator the ability to know precise information about the current state of execution, and it uses that to decide on next steps once execution is returned by a subagent. When that happens the Orchestrator reads the current stage , verifies that the required prior artifacts all exist, ensures there are no blocking conditions present, then invokes the next appropriate agent. This gives a formal definition to how and when execution is passed between the orchestrator and various subagents. A crucial part of this is that the Orchestrator should not just assume a stage succeeded because an agent ran and completed. It has to explicitly confirm that the expected output artifact for the agent in question exists and is valid before progressing.

This is really important, if the Orchestrator just blindly executed one subagent after another without checking anything, then it wouldn't be much use as if there were issues with the output of one of the subagents then executing the subsequent ones would just compound that error and lead to the final output not matching what was required, if there even is an output at all. So there are a few scenarios that can cause the Orchestrator to pause or amend the pipeline depending on execution results. That could result in orchestrator pausing the workflow for human review, stopping all together if there are missing artifacts, halting on invalid state, or re-running previous subagents, for example when QA fails.

This results in the workflow becoming more trustworthy because progression is based on explicit state and durable outputs, not on the model vaguely remembering where it thinks it is at in the execution pipeline

Conclusion

To finish this up, without orchestration agents can easily overreach , perform work out of order , or continue past the boundary of their role. The Orchestrator acts as the mechanism that keeps those boundaries solid and ensures that agents only perform the specific task they were built for.

Because workflow state is written down and readable by the orchestrator, this means you can stop and resume execution later without reconstructing the whole process from conversation history. You can stop the LLM, and just tell it to resume later when you're ready. This is super useful as trying to do that when you're in the middle of a large conversation with an LLM, and need to recreate that large context to pickup again is super painful!

If you're wondering how I actually configured my Orchestrator Agent and Role then you can see the current state of them in the links below. Note that this is this is the current state of the files as of writing this post, but they will continue to evolve as the harness is tweaked and amended.

Following on the series, I think next time. I'm going to cover how I used Harness Development in practice to build out my site. We'll take the different concepts that I've talked about in the past few posts and walk through an example of how I built out one of the features from my backlog using it. We'll follow a single task from a one-liner in my backlog to a completed and deployed feature, covering all of the artifacts generated by each of the agents and how it all came together to deliver the feature all the way to prod.

Harness Engineering: Artifacts, Inputs and Context

Rob Earlam — Tue, 14 Apr 2026 00:00:00 +0000

This post is part of my series deep diving into Harness Engineering. You can find all of the posts in the series so far below:

An Introduction to Harness Engineering
Harness Engineering: Agents and Roles
Harness Engineering: Artifacts, Inputs and Context
Harness Engineering: Orchestration and execution tracking

We've already covered an overview of Harness Engineering, and also how the Agents and Roles in the system function. In this post I want to take a look at how the Artifacts from each Agent form the Input for other Agents , and how this form of Bounded Context helps to greatly improve accuracy in what your LLM is doing, leading to far fewer cases of Hallucinations.

We've been talking with LLM's to help with Coding tasks for a while, and while the quality has improved in that time, accuracy and reliability can still be an issue. Chatting directly like this is flexible, but it is also fragile as it drifts, expands, gets reinterpreted, and makes it harder to know which decisions are actually the source of truth to be trusted. This has been mitigated somewhat by vastly increasing the context we provide to the LLM's. Even with the higher token limits that the newer models support, this isn't foolproof, a lot of the time too much context can be provided and the model can get confused with what information to use for what task.

Artifacts as the backbone of Harness Engineering

Artifacts are outputs produced at each stage of the workflow, such as a feature spec , design note , implementation plan , QA review , or the code itself. The important thing is that each artifact exists because a specific role is responsible for producing it, which makes it a formal handoff rather than just extra documentation to clog up the context some more. This means that each Agent will only produce the artifact that they are designed to output and nothing else. This stops, for example, the Tech Lead agent from getting confused and trying to actually implement the code changes and not just creating the implementation plan.

The crucial next step in this process, is that these artifacts then form the input for the next agents to execute in the workflow. This means that one agent completes its work by producing a defined artifact, and then next agent begins with that artifact as one of its approved inputs. Here is where the bounded context comes in. Instead of every agent pulling in everything it can find, each one works from a small, deliberate set of inputs, which helps keep the task bounded and reduces irrelevant context accumulation and the inaccuracies that can lead to.

Reliability improves when agents are not constantly trying to infer intent from long conversations, but are instead working from stable artifacts that already capture the decisions made by previous agents executions. An agent is less likely to invent requirements, expand past its defined scope, or make up missing detail when it is grounded in a constrained set of explicit inputs and expected outputs... the bounded context provides the power here.

This helps to enforce the difference between context and authority. Not all information that an agent can see should be given equal importance. Artifacts help distinguish between background noise and the valid sources that should be given the most consideration when executing.

Human Legibility and Traceability

The other awesome thing about working with Artifacts in this way is that handoffs are legible to humans too! The same structure that helps the agents also helps you as the driver of the system. You can inspect what was decided, which agent decided it, and then see how the downstream work was affected. This means if you're not happy with the output you can go back and slightly tweak your initial instructions to better ensure that the artifacts are generated to your requirements.

As you execute the workflow, the artifacts are all persisted into your repository as well. This means that if in the future you want to go back and see why a certain feature works a certain way, then you can go and look back at the different artifacts from all of the different execution runs and review why something was built as it was. If you look at the image below you can see the different outputs listed from my PO-Spec Agent we discussed in the previous post, all keyed the Task ID of the work being completed

Here you can see here how easy it is for me to go back and look at the spec that was built for a specific task. The same is true for the artifacts from the Tech Lead Agent , Design Agent , and QA Agent. Now the Builder Agent is obviously different as it's artifact is the code itself.

Artifacts as memory outside the model

Another of the key things to highlight here is that artifacts act as externalized working memory. This reduces dependency on the model remembering everything correctly across long running execution streams. It greatly reduces the need for context compacting which is required when running long conversations with huge contexts being evaluated.

It also gives you consistency across iterations , the thing with a harness is you do need to tweak what you're putting into it, and probably run it a couple of times to get the output that you want. Having the artifacts store the result of prior agent executions means that context is always available when re-running the harness again.

Another benefit this concept of memory gives you is recoverability. Externalized memory means you can stop , resume , inspect , or even rerun parts of the workflow without having to reconstruct a full chat history. This is important as it means that your harness is repeatable , while also maintaining consistency in those repetitions.

Just to repeat here, the goal is not to maximize how much the model sees, but to maximize the relevance and authority of what it sees. This allows the model to create successful executions with greatly reduced context load.

The artifacts I'm producing in my harness

I've been talking in this series about the harness I created to rebuild my personal site. I've covered the Agents and Roles I created in a previous post, so I wanted to now cover what artifacts each of those is responsible for creating and which they each use as their input, as crucially not all artifacts are available for all agents to leverage.

So here are the list of agents I created, the artifacts they output and what they use as their inputs.

Agent	Inputs	Artifact Created
Orchestrator	Task definition	Run-state, run report
PO-Spec	Task definition	Feature specification
Feature Design	Feature specification, design system	Design note
Tech Lead	Feature specification, Design note	Implementation plan
Build	Feature specification, Design note, Implementation plan	Code & tests, run report
QA	Feature specification, Design note, Implementation plan, code & tests	QA review

Here you can see my agent workflow is actually executing like a real team of humans would work along a well defined Software Development Lifecycle (SDLC). Each of them takes the output from the previous Agent and uses it to execute and produce the artifact they're responsible for, just like a team of humans working on those roles would.

Now the interesting edge case in the list above is the Orchestrator. This Agent has control over when and how the other subagents execute, it checks they successfully created their artifact and only then moves on to execute the next in the workflow. It has access to some items we haven't discussed so far, namely the Run State , Run Report and the Task Definition. These are what we're going to discuss in my next post, where we take a look at how execution is tracked and managed by the Orchestrator.

Harness Engineering: Agents and Roles

Rob Earlam — Wed, 08 Apr 2026 00:00:00 +0000

This post is part of my series deep diving into Harness Engineering. You can find all of the posts in the series so far below:

An Introduction to Harness Engineering
Harness Engineering: Agents and Roles
Harness Engineering: Artifacts, Inputs and Context
Harness Engineering: Orchestration and execution tracking

Carrying on from my recent post giving an Introduction to Harness Engineering, I want to start to dig a level deeper on the topics involved. First up we're going to look at the Agents and Roles that are defined in a setup like this. They're one of the key elements in Harness Engineering and are what actually control and execute the work you want completed, so they're a good place to get started!

Agents and Roles

A lot of AI workflow discussion seems to treat agents and roles as one, but I found that separating them makes the system easier to design, understand, and to iteratively improve on. In the setup I created, I ended up with a 1-1 relationship between my Agents and Roles, so you might be wondering why they are separated and not just combined together. Well there are some distinct benefits to defining them separately.

Keeping them separate allowed me to have the Agents be responsible for defining the tools it has access to, but its actual tasks are defined in the role itself. So if we take the Build Agent for example it's definition is quite skinny, it defines the tools it has access to, the areas of the codebase it can access, and also what areas it shouldn't. You can see a slightly truncated version of the build agent definition below.

---
name: build
description: Implement the active task defined in harness/run-state.md using the approved specification, design note, and implementation plan.
tools:
  - edit/editFiles
  - search/codebase
  - search
  - execute/runInTerminal
  - read/terminalLastCommand
---

Follow the role definition in:
- `harness/roles/build-agent.md`

......

Read the approved artifacts referenced by the task file and/or run-state before making changes:
- feature specification
- design note
- implementation plan

.....

Do not modify governance-controlled files or paths as part of normal implementation work.

Governance-controlled paths include:
- `harness/run-state.md`
- `harness/tasks/`
- `harness/backlog/`
- `harness/standards/`
- `harness/roles/`
- `harness/templates/`
- `.github/agents/`
- `.cursor/rules/`

If governance metadata appears inconsistent, report it in the implementation summary and run report instead of changing it.

Use the controlled validation loop defined by the Build role when command execution is available.

The validation loop may run only the approved commands and must stay within the retry and scope limits defined in the role.

Operate only within the approved task scope.

Make implementation changes, required test changes, and the run report update, but do not continue beyond this stage without orchestration or human review.

If we take a look at the Build Role you can see it is much more detailed, defining things like where to access the coding guidance docs, how to build and run the project, the process it should follow and also its required output alongside much more. Again, you can see a slightly truncated version of the definition below.

# Build Agent

## Purpose

The Build Agent implements the approved implementation plan by writing or modifying code within the repository.

The goal of this role is to translate the approved specification, design note, and implementation plan into working code while following the project's coding standards and testing expectations.

The Build Agent should focus on faithful execution of the approved plan, not reinterpretation of the feature or management of workflow state.

The currently active task and active run context are defined in `harness/run-state.md`.

---

## Responsibilities

This role is responsible for:
- implementing the steps defined in the implementation plan
- writing production application code
- adding or updating tests where required
- following project coding standards
- keeping changes within the approved task scope
- documenting assumptions made during implementation
- reporting blockers, workflow inconsistencies, or deviations clearly
- using the controlled validation loop to resolve obvious in-scope failures where allowed
- creating or updating the task run report with build-stage outcomes and validation evidence

The implementation should align with the approved feature specification and design note.

---

## Non-Responsibilities

This role must not:
- modify the feature specification
- redesign the feature layout
- introduce unrelated refactors
- expand the feature scope
- invent architectural patterns not defined in the implementation plan
- modify workflow governance files
- update lifecycle state or task-management metadata
- repair harness governance inconsistencies by editing control files
- continue fixing indefinitely without a retry boundary
- approve task completion

If the implementation plan appears incorrect or incomplete, the role should escalate rather than improvise.

If governance or lifecycle metadata appears inconsistent, the role should report it and stop, not fix it.

.....

## Process

Follow this process when implementing a feature:
1. Read the live execution state in `harness/run-state.md`.
2. Confirm that the current stage is `Build`.
3. Use the task file referenced in `harness/run-state.md` as the single task for this run.
4. Read the approved feature specification and acceptance criteria.
5. Read the approved design note to understand the intended user experience and layout expectations.
6. Read the approved implementation plan carefully and use it as the execution boundary.
7. Generate a stage-appropriate run ID for the Build stage using the run-id standard.
8. Identify the implementation slices and execute them in small, controlled steps.
9. Modify only the application code, tests, and implementation-relevant files needed for the approved task.
9a. Implement file and folder placement exactly as defined in the task, feature specification, and implementation plan.
- If canonical paths are defined, do not introduce alternative structures.
- Do not introduce new architectural folders for organisational preference.
10. Add or update tests as defined in the implementation plan.
11. Run the allowed validation commands where the runtime environment supports command execution.
12. If validation fails, apply bounded in-scope fixes and rerun within the retry limit.
13. Create or update the task run report at the canonical run report path defined by the task file and/or run-state.
14. Record in the run report:
    - task ID
    - task title
    - current stage
    - current Build-stage run ID
    - implementation summary
    - files changed
    - validation commands executed
    - validation results observed
    - assumptions used
    - blockers or follow-up recommendations
    - governance observations, if any
15. If governance metadata appears inconsistent, report it in the run report and implementation summary instead of editing control files.
16. Prepare a concise build-stage summary for the user.
17. Do not implement custom parsers for standard content formats such as Markdown when a suitable project-compatible library can satisfy the requirement more safely and with less complexity.

---

## Output Requirements

The output of this role should include:
- a Build-stage run ID following the run-id standard
- code changes
- new or updated tests
- a summary of the implementation
- a list of modified files
- validation commands run
- validation results observed
- any assumptions made during implementation
- any known limitations, blockers, or follow-up recommendations
- confirmation that the run report was created or updated

The implementation should leave the task ready for later QA review.

If governance inconsistencies were observed, they should be reported in the summary and run report rather than corrected by this role.

If validation could not be run because the runtime environment lacked command execution, that must be stated explicitly in both the summary and the run report.

---

## Quality Bar

A successful implementation should be:
- correct
- minimal
- readable
- aligned with coding standards
- supported by appropriate tests
- bounded to the approved task scope

A successful build run should:
- implement the approved slices
- avoid unrelated refactors
- avoid changing workflow governance
- remain clearly distinct from QA and task control responsibilities
- use the validation loop responsibly and stop when blocked
- leave durable validation evidence in the run report for downstream QA

The implementation should remain simple and avoid unnecessary complexity.

---

## Escalation Conditions

Escalate when:
- `harness/run-state.md` is missing or unclear
- the current stage is not `Build`
- the referenced task file does not exist
- the approved feature specification is missing
- the approved design note is missing
- the approved implementation plan is missing
- governance files appear inconsistent with the actual stage of work
- the implementation plan conflicts with the current codebase
- required files or architecture do not exist
- the feature cannot be implemented within the defined scope
- acceptance criteria cannot be satisfied
- validation still fails after the retry limit
- the runtime environment does not support required validation commands
- the run report path is missing or cannot be determined

Escalation should clearly explain the blocker.

---

.....

I'm going to be releasing the full repository including the harness as Open Source, once I think it is in the best place to be able to help people follow the same process at which point you'll be able to see the full Agent and Role definitions for each of the system sections.

Another cool advantage to separating responsibilities like this, is that it allows you to remain somewhat IDE agnostic. With the vast majority of the definition existing in the Role and not the Agent, all you need to do is replicate the skinny Agent format for each of the IDE's you want to support instead of having to replicate the full definition for each.

The way I like to think about this is that Agents are there to Execute, whereas Roles are there to Govern.

Multi Agent Workflows

A lot of the time talking to an LLM agent directly in a chat window in an IDE will get you a good enough result eventually, but it usually becomes inconsistent once a workflow needs handoffs, traceability, and repeatable quality. You get these features by developing a harness with well bounded Agent and Role definitions, that stop scope creep & excessive Context grabbing.

I've found that the best results generally don't come from giving a single agent every bit of context possible, and maximum freedom, but from creating discrete agents each with a very narrow purpose that can execute one single part of the system well. Generally with a single Orchestrator Agent, whose role isn't to actually perform any of the tasks but to oversee the system as a whole.

My Agents and Roles

As I mentioned in my previous post I ended up with the following set of Agents defined in my Harness. Each one with a corresponding Role definition used to contain the majority of the definition.

Orchestrator — Controls the workflow end to end, decides which stage should run next, invokes subagents, validates handoffs, keeps execution moving in the right order, and bails out if one of the subagents fails.
PO-Spec — Turns the original idea or request into a clear feature specification with scope, intent, and acceptance criteria.
Feature Design — Translates the feature requirements into a design direction, interaction model, and structural approach for the features being built.
Tech Lead — Converts the spec and design into an implementation plan with technical decisions, architecture guidance, and delivery steps.
Build — Actually does the coding work - it carries out the implementation work by creating or updating the code and assets needed to deliver the feature.
QA — Reviews the output against the specification and expected behaviour, identifies gaps or defects, and decides whether the work is ready or needs to return to Build.

The Orchestrator agent is responsible for executing the system as a whole, calling each of the subagents at the correct time, in the correct order and validating the output before moving on to the next one. You can see a Sequence Diagram below, showing the order of execution in the system.

Artifacts, Inputs and Context

I'm going to focus more on this topic in my next post, but you can see from the diagram above that as each of the subagents executes, it is responsible for producing a specific single output, e.g. the PO-Spec Agent is responsible for taking the Task definition I create and generating a Feature Specification defining what it is the system is going to build. That is all though, the agent doesn't then make any code change or anything else, it just produces that document then stops. That document is then used as an input for the Feature Design Agent to be able to create the Design Note. Both of these documents are then used by the Tech Lead Agent to create a detailed Implementation Plan.

This is what I mean when I talk about discrete agents with narrow responsibility. Each of the agents has a single task, and the outputs of those tasks are used by the subsequent agents for them to complete their own tasks. This means that each agent is provided the exact amount of context it requires to get it's job done.

I upgraded the SUGCON sites to Content SDK v2 in under 1hr

Rob Earlam — Thu, 02 Apr 2026 00:00:00 +0000

Just a quick blog post about a win I just had. I blogged last month about the process of upgrading the SUGCON sites from JSS to the Content SDK, and how the sites had been a bit neglected and were running a super old version. I finished the post by promising not to let that happen, and how we were going to ensure we kept up to date with future releases of the Content SDK. Well shortly after v2 of the SDK was released, so I knew I had to get onboard and update the sites to the latest version.

I decided to see how far AI could get me in the process... and it turns out it basically did it all for me first time. All in all the upgrade took me less than an to go from nothing to an open PR ready for my colleagues to review.

I did this by leveraging Copilot CLI and the Adversarial Testing feature that it supports. All I did was to create a new branch on my machine, open the root of the repo using the Copilot CLI and give it the following prompt.

Please upgrade the SUGCON Site located in this repo to Content SDK v2. You can find the upgrade guide here: https://doc.sitecore.com/sai/en/developers/content-sdk/20/upgrade-content-sdk-1-5-1-next-js-apps-to-version-2-0.html.

Do not make any changes to git, I will do that once I'm happy with the work at the end.

Once you can finished, please run an adversarial test of the changes using both Opus 4.6 and GPT 5.4

This went away thinking and making changes for a while and came back with the following response.

 Upgrade Complete — Sugcon2024 → Content SDK v2

  All files changed

  ┌──────────────────────────────────────────────┬─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ File │ What changed │
  ├──────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ package.json │ Removed @sitecore-cloudsdk/*; added analytics-core, events, personalize at ^2.0.0; bumped SDK + CLI to ^2.0.0; next → ^16.1.1; @types/node → ^24.10.4 │
  ├──────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ src/Bootstrap.tsx │ CloudSDK().addEvents().initialize() → initContentSdk({config, plugins}) with analyticsPlugin + eventsPlugin │
  ├──────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ src/middleware.ts │ Deleted │
  ├──────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ src/proxy.ts │ Created — Next.js 16 proxy format; *Middleware → *Proxy class names; defineProxy; default export │
  ├──────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ src/components/content-sdk/CdpPageView.tsx │ pageView import: @sitecore-cloudsdk/events/browser → @sitecore-content-sdk/events │
  ├──────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ src/byoc/index.tsx │ Events namespace import: @sitecore-cloudsdk/events/browser → @sitecore-content-sdk/events │
  ├──────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ sitecore.config.ts.example │ SITECORE_EDGE_URL → NEXT_PUBLIC_SITECORE_EDGE_PLATFORM_HOSTNAME; PERSONALIZE_MIDDLEWARE_* → PERSONALIZE_PROXY_* │
  ├──────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ .env.remote.example │ Added NEXT_PUBLIC_SITECORE_EDGE_PLATFORM_HOSTNAME, SITECORE_EXPERIENCE_EDGE_HOSTNAME, SITECORE_RENDERINGHOST_NAME; renamed PERSONALIZE_MIDDLEWARE_* → PERSONALIZE_PROXY_* │
  └──────────────────────────────────────────────┴─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

  Bugs caught by adversarial review

   - Opus
    4.6 caught: BotTrackingProxy imported from the dev branch template but not yet in published v2.0.1 — removed
   - GPT
    5.4 confirmed the above, and also caught: PERSONALIZE_MIDDLEWARE_* env var names in .env.remote.example were misaligned with the updated PERSONALIZE_PROXY_* names in sitecore.config.ts.example — fixed

Now this was the first time I'd upgraded a site to v2 of the Content SDK, so I followed the upgrade guide myself afterwards to double check the changes it had made to ensure nothing had been missed. Turns out, it nailed it first time!

The thing I really liked about this was the text output at the bottom above, covering the result of the Adversarial Testing I asked it to complete. It used both Opus 4.5 and also GPT 5.4 to validate the changes it had made, and each model found different issues that needed to be fixed. This is a a super powerful trick as it means you can play models off against each other leveraging the benefits of both to get you where you need to go faster, how cool is that!

An Introduction to Harness Engineering

Rob Earlam — Wed, 01 Apr 2026 00:00:00 +0000

This post is part of my series deep diving into Harness Engineering. You can find all of the posts in the series so far below:

An Introduction to Harness Engineering
Harness Engineering: Agents and Roles
Harness Engineering: Artifacts, Inputs and Context
Harness Engineering: Orchestration and execution tracking

So for a decent amount of time now I've been on the Vibe Coding bandwagon, like a large portion of the development community. When it works it really can seem like magic. Too often though it does seem to go off on a tangent, editing unrelated parts of the codebase, or implementing a something in a way that I just wasn't happy with. I felt that it was making me more productive, but I also felt like I spent a good amount of my time chasing the LLM, going back and forth on the same problem and repeatedly prodding it to get to the outcome that I wanted.

It felt like one of the big problems with this approach was repeatability, now rules files can help here to try and enforce some guidelines for how LLM's interact in certain scenarios, but I was getting sick of constantly having to type prompts like "You are a senior developer with 10yrs of experience yadda yadda". I had played around with some Agents, and they were a step in the right direction, but the whole thing was still feeling very manual, in that I had to hand-hold the AI at each step.

Then last month I read a blog post published by Ryan Lopopolo from the OpenAI Tech Team about Harness Engineering and this got me interested. I had heard of people creating teams of Agents, so I decided it was time to jump in. I ended up rebuilding the site you're reading this blog on now using this approach and I wanted to document how I ended up putting it together for anyone else looking to build software this way.

Harness Engineering 101

As I said in my previous blog post the name is pretty apt. The basic concept is that you think of AI like a horse, it's fast and it's powerful, but for the vast amount of people, if you try an ride it bareback then you're going to have a bad time. That's why you need to develop a Harness for the horse, allowing you to make use of the power and speed it provides while being much more in control of where the final destination is.

The harness also means you can repeatedly ride the horse to different destinations, but with the same ease. Now from time to time you might have to tighten the harness when you realise there's something you're not happy with. You might also make some slight adjustments depending on the journey your taking or the destination. Overall though, the harness remains pretty much the same for each trip. The same is true for Harness Engineering as well. In this case the harness is a series of agents , roles , artifacts , and workflow standards that let you guide execution instead of hoping for a good result.

The workflow standards define how everything should function. In my case this covers everything from Next.JS best practices, and testing standards, through to how the agents communicate with each other, and how the current state of the repo is tracked during execution.

The agents are autonomous workers, each with a very specific purpose to execute within the system. I keep the actual definition here very light, instead storing most of the definition in the roles. I have the following agents defined currently though:

Orchestrator — Controls the workflow end to end, decides which stage should run next, invokes subagents, validates handoffs, keeps execution moving in the right order, and bails out if one of the subagents fails.
PO-Spec — Turns the original idea or request into a clear feature specification with scope, intent, and acceptance criteria.
Feature Design — Translates the feature requirements into a design direction, interaction model, and structural approach for the features being built.
Tech Lead — Converts the spec and design into an implementation plan with technical decisions, architecture guidance, and delivery steps.
Build — Actually does the coding work - it carries out the implementation work by creating or updating the code and assets needed to deliver the feature.
QA — Reviews the output against the specification and expected behaviour, identifies gaps or defects, and decides whether the work is ready or needs to return to Build.

The artifacts are what is delivered by each if of the agents above, so for the PO-Spec agent, the artifact is the actual spec that it is created. For the Build agent however the artifact would be the code changes themselves.

The roles are where the vast majority of the agent logic is maintained and there is a 1-1 relationship from agents to roles meaning all of the agents listed above have a corresponding role definition. These each define things like the responsibilities of the agent, what pre-requisites need to exist before it can run, what areas of the codebase it can look at to perform it's actions, whether it can iterate on itself, which it can says its completed, and a detailed walkthrough of the process it should follow. One of the main benefits with keeping the definitions in role files with skinny agent files that reference them is that it helps to keep my harness IDE agnostic - I can have skinny definitions for agents in Claude, Cursor or VS-Code, but the logic is in the shared role file avoiding repetition.

Advantages of Harness Engineering

So you might be reading through the items above and wondering why someone would go to the effort of configuring all of this, when you could just start directly talking with your LLM and creating the code for the feature you require. There's a few really nice things you get by following this approach that I want to cover.

The shift from prompting to orchestration

One of the main ones for me is the change from having to repeatedly craft prompts to defining a lot of this in advance in the harness. This allows the LLM to take the actual work request and break it down into discrete tasks, and hand them off to tightly bound agents with specific roles and outputs. It means that each agent has a narrow purpose, clear responsibilities, and explicit limits. Defining all of this up front reduces role leakage, keeps outputs focused, and makes the system easier to reason about and improve. If you find the Tech Lead Agent isn't creating a proper plan from the specs for the build agent, then it becomes very easy to tweak that agents role to get the result you want.

Artifacts, tasks & the backlog

How I run the delivery process using the harness is that I have a backlog of desired features with a very basic high level description. When I want to implement one I have a conversation with an LLM to generate a much more detailed Task Definition with a unique id, which would describe in more detail the feature I want and any guidance for how I think it should and shouldn't work. This task is then passed through all of the agents to generate their artifacts. The nice thing here is that every artifact generated is created keyed with the task id, and stored in the artifacts folder.

This means that if I want to look back in the future at how the blog feature was implemented, I can refer back to all of the artifacts that were generated by each of the agents to see why it was implemented the way it was. This can also be used as memory for future feature implementations, you can including instructions like "following the same pagination pattern implemented in the blog features" and the LLM will know how to do that and where to look for the information.

Repetition, Repetition, Repetition

I keep mentioning the benefit of repetition with the project for future features, but this concept of repetition is actual broader than that. If I want to start a different project tomorrow on a different technology stack, then my existing harness can still help. Lets say I want to build an IOS application tomorrow, now coding standards will obviously need a large rewrite, but the concepts I described above about agents , roles , artifacts , and workflow standards will be largely transferable. I would move them over to the new project and tweak them to get the output tailored to that project. Coming back to the horse analogy, I'm once more tailoring the harness I'm going to be riding with to better suit the journey I'm on and the destination I want to get to.

Conclusion

As I built out this harness, I released I was actually recreating a structure that I had seen and worked in at many different companies before when working on software projects. You have each role within the company responsible for producing the artifacts that the next team would use. Now the difference is that all of this is created by AI, and in a fraction of the time!

Hopefully this has been a good introduction to some of the key elements to Harness Engineering and how it all comes together. As I continue this series I want to deep dive in to each of the topics I've covered above. I want to talk about how you can craft a tightly defined agent and role, how you can enforce contractual boundaries to minimise agentic scope creep, and ensure the orchestrator can validate completion before continuing the execution pipeline. There's going to be lots to cover, so keep an eye out for future articles on this topic coming soon!

New Site, Who Dis?

Rob Earlam — Mon, 30 Mar 2026 00:00:00 +0000

For anyone who's read my blog posts before, you wont have seen much (if any) content on here about horses, but turns out there's a first for everything. Bear with me though as it looks like I saved my "horse content" for just the right time.

Like the majority of the rest of the development community, I've spent the past couple of years both amazed and a little bit terrified about how AI powered development has taken over the industry. I've always been a developer who's excited about learning new things though, and this was a big one I could sink my teeth into.

Every few years a new technology comes along that I want to learn, and in those cases I have generally rebuilt my personal site that you're on now as a way to put the technology into play. I knew that I wanted to follow the same process for AI based development, but with the rapid pace of advancement in the technology, I was never really sure about when to attack this, until last month when I read a blog post published by Ryan Lopopolo from the OpenAI Tech Team about Harness Engineering and I knew now was the time to bite the bullet.

I'm planning to write much more on this subject as it has been a decent learning curve for me, but it's been some of the most fun I've have writting code (or not writing it as the case maybe), in a very long time!

The basic concept is that you think of AI like a horse, it's fast and it's powerful, but for the vast amount of people, if you try an ride it bareback then you're going to have a bad time. Thats why you need to develop a Harness for the horse, allowing you to make use of the power and speed it provides while being much more in control of where the final destination is.

In reality the what you end up building is a team of agents, each with a tightly bound function. They can't stray outside of their designated function, and there is an orchestrator agent who is repsonsible for invoking each of them. The orchestrator is repsonsible for invoking them in the correct order, and also ensuring that they complete succesfully before invoking the next one.

All of this comes together through a series of markdown files that are used to define how this system comes together. This covers everthing from how the agents themselves function, the different actions they can perform, the files in the repo they have access to in order to perform their job, and how communication and state management is handled throughout all of this.

I'm planning to write all of this up in the coming weeks, as it's been an amazingly enjoyable learning journey for me to pull it all together and I'm hoping this series will help others looking to build software in the same way. It took a few attempts to get the Harness into a state where I was happy with it, and it is now churning out features for this Next.js site with ease. I do think that when I complete this the next time the process will be much faster with everything I've learned.

So my aim is write content covering how I built my setup, what actions worked well, and crucially what didn't. We're going to go over the different agents and their roles, artifact generation, human-in-the-loop review process, execution boundaries, the process I went through pulling all of this together and crucially where I want to take it next.

So I hope you come back for the next post when we start to dig into this exciting topic for real!

Sitecore Search - Debugging Crawling Errors

Rob Earlam — Tue, 27 Feb 2024 21:45:06 +0000

The latest video in my series on #Sitecore #Search was published this week, discussing how you can debug crawling errors.

Sitecore Search SDK - Breaking Change

Rob Earlam — Fri, 02 Feb 2024 00:00:00 +0000

Updating packages in Sitecore’s Developer Portal

Recently my colleague Mark van Aalst was updating the package versions for the Sitecore Developer Portal. This also included updating the version of the Sitecore Search React SDK from v2.0.0 to v2.2.2, among other packages that were updated at the same time. Once this had been deployed, we got reports that faceting was no longer working on the search results page. We quickly rolled back and started to look into what had happened.

Breaking changes in the Faceting UI component

I reached out to the team that owns the SDK and apparently there was a breaking change introduced in a previous version and we needed to update how we were leveraging the UI components being used to render the Search Results page.

Previously we were wrapping out Facets in the AccordionFacets.Root component, like so

<AccordionFacets.Root defaultFacetTypesExpandedList={facets.map((x) => x.name)} onFacetValueClick={onFacetClick}>
    ….
</AccordionFacets.Root>

However, this approach is no longer valid, we should now be using SearchResultsAccordionFacets component instead. As follows:

<SearchResultsAccordionFacets defaultFacetTypesExpandedList={facets.map((x) => x.name)} onFacetValueClick={onFacetClick}>
    …
</SearchResultsAccordionFacets>

This was a fairly straightforward change, which meant we could bump the Developer Portal up to be running on the latest version of the Search SDK 🎉.

To see the complete set of changes we had to make for the Developer Portal, you can view the PR that was opened for this fix here.

SUGCON India 2023 goes off with a bang 💥

Rob Earlam — Thu, 31 Aug 2023 00:00:00 +0000

Last week I had the pleasure of attending SUGCON India 2023 in Delhi, India. This was the first time we’d run a SUGCON in India since 2019 and it was great to be back once more, interacting with the local Sitecore community. There was a lot of effort in involved in pulling the event off, and I do want to start by thanking all of the organisers for the time they invested in making this event the success that it was.

I was privileged to be chosen to deliver one of the keynote sessions this year, where I talked about a recent project I worked on where we integrated Sitecore Search into our Developer Portal. This was very well received and led to many questions afterwards, and many further conversations throughout the event. It was great to see the amount of interest there is in Sitecore Search from the attendees!

The quality of the sessions that were delivered at the event was of a very high standard. They ranged from high-level sessions covering Composable DXP’s in general and the power they can bring to customers, all the way down to technical deep dives on specific features and use-cases. Some of my highlights were:

The session by Pieter Brinkman titled Digital Experience is a team sport, where he talked about the different roles that play a part in successful Composable DXP implementation – it’s not just all about the developer!
George Smith & Sheetal Jain delivered an awesome session called Beyond Buzzwords: Composable Tech and Generative AI for Real. Here they showed how OrderCloud, Sitecore Content Hub & Dall-E can be combined to allow customers to generate art on the fly and have it printed on a t-shirt for sale in an online store, cool stuff!
Exploring the Experience Edge GraphQL Schema was a really interesting session delivered by Alistair Deneys. Here he gave a true deep-dive into the schema implemented for Edge and how you can leverage it in some really interesting ways. There’s also a cool surprise at the end of the session that I won’t give away here, but be sure to check it out if you get the chance.
Women in contemplating Darwin's Theory - To Break the Conventional Wheel was a great session by Varalakshmi M D & Raveena Mathur where they talked about the important role women have in ensuring successful outcomes for customers.
Andy Cohen delivered a great session titled Build for XM Cloud according to the Architect of XM Cloud. Here he talked about the best practices for building with XM Cloud... he might have ruffled some feathers with some of the recommendations, but it was great to see it all layed out!

The closing keynote was also a highly enjoyable part of the event. Each of the representatives from Sitecore (Pieter, Nicole, Andy, Alistair and myself) held a panel discussion and open Q&A session. This gave the attendees the chance to ask any lingering questions that they hadn’t had the chance to ask already. The topics were very wide from specific feature questions on the products through to general questions about the best way that Sitecore can support the local Indian community… and of course the age-old question of Sandboxes and how people can get access to Sitecore’s newer SaaS products – we hear you and are working on a solution!

As much as I enjoyed everything I mentioned above, as with every event I seem to attend nowadays, the true highlight was getting to interact with the local community we have on the ground. It’s amazing to see the passion they have for Sitecore and our products. It was great to see so many familiar faces from the events back in 2018 & 2019, but it was also great to see so many new faces as well, a good balance that shows how healthy the Sitecore community in India is.

I just want to finish off by saying that of course trips like this aren’t always about work though, we always try to make sure that we get out and see some of the places we visit and India is no different. On this trip, we made the journey down to see the Taj Mahal after the event had finished, and it truly is an amazing site to see. Really all of the descriptions of it you hear just cannot do it justice, and it was great to experience it with the awesome team we had on-site.

So, whether it’s the always friendly people, the amazing effort from the organisers & speakers, the food, the noise or the general sights you see around the city – India is one of my favourite places to visit and certainly one of my favourite SUGCON events.

Here’s looking forward to doing it all again in 2024!

Migrating the XM Cloud Introduction Repo to a new Nuget feed.

Rob Earlam — Thu, 27 Jul 2023 00:00:00 +0000

Today I had to go through the process of migrating the XM Cloud Introduction repository to a new Nuget feed. This was required as the old feed was no longer available, and we needed to move to a new one on short notice. This post will cover the steps I needed to complete to migrate the solution to the new feed.

Machine Tidy Up

Now for anyone who hasn't run a Sitecore site before you can most likely skip this step. However, for anyone who has you should tidy up any references to the old feed that have been configured on your machine. In my case, this was the PowerShell Repository used to restore the SitecoreDockerTools module used by this, and other Sitecore repositories.

Remove PS Repository

The first thing to check is whether you have the old Sitecore Feed registered as a PowerShell Repository, you can do this by running the following command Get-PSRepository and checking where there is a record named SitecoreGallery. If you have one registered, then you can remove it by running Unregister-PSRepository -Name SitecoreGallery.

Remove PS Modules

Next, I tidied up any previous versions of the SitecoreDockerTools module installed from the old feed. You can test whether you have any versions of that module installed by running the Get-Module -ListAvailable and look for any named SitecoreDockerTools. If you have those installed, you can quickly remove all installed versions of that package by running Uninstall-Module -Name SitecoreDockerTools -AllVersions.

Update the Repository

So now we’ve tidied our developer machine, we can update the repository to reference the new feed. This involved making two changes, an update to the init.ps1 script used to initialise the repository, and then an update to the nuget.config to define the new location for the NuGet packages in use by the solution.

Updating the init.ps1 script

The first thing the script does is install the SitecoreDockerTools PowerShell module, so we need to update the location that it is installed from, in our case, we needed to use v2 of the Nuget feed to get it to work, so we needed to change it to https://nuget.sitecore.com/resources/v2/. After making this change the init script ran successfully and installed the module from the new feed.

Updating the nuget.config

The final change we need to make is to update the nuget.config file in the root of the repository that controls where custom NuGet packages are installed from. We had a couple of older references in there that we removed and condensed down to the single new feed reference. This resulted in us having the following line in our config:

<add key="Sitecore" value="https://nuget.sitecore.com/resources/v3/index.json" />

Conclusion

Updating the XM Cloud Introduction repository to the new feed location was straight forward in the end. I would recommend any developers who have been running existing Sitecore solutions before follow the tidy-up steps at the start to ensure they remove any legacy references they have to the old feed. You can see the changes we had to make in the following commit. There may be other changes that you will need to complete to bring your codebase in line, but hopefully, this will help to get you started with what tasks are required for you.