DEV Community: Harshith Vaddiparthy

Ego Browser: give the agent a workspace, then check its work

Harshith Vaddiparthy — Tue, 20 Oct 2026 10:45:22 +0000

I've been using Ego Browser since it was released on GitHub. The useful idea is that an agent gets its own browser space while the human keeps their own browsing context.

That is a practical change in how browser automation fits into a working day. An agent can have a task and a set of pages to work through, without that task becoming a pile of tabs in the middle of whatever I am doing.

Ego calls these workspaces Spaces. Its documentation describes separate task tabs with access to the browser's available login state. The agent can use structured page snapshots, navigate, click, fill controls and capture screenshots. The visible agent cursor and action labels help a person follow the interaction.

Start with one question and a result you can inspect

A good first task is small and read-only. For example:

Compare these three product pages. For each product, find the supported operating systems, whether data export is documented, and where the pricing is explained. Return a table with a source URL for every answer. If the page does not answer a question, say that. Do not sign up, submit a form or change any account settings.

This task has a clear end. It also gives me something to check: each answer should agree with its source. A polished summary with no traceable evidence is harder to trust than a modest table that admits what is missing.

The same pattern works for release research. Open the repository, follow the relevant documentation, distinguish released features from roadmap items, and retain the exact sources.

Animation: Ego / Citro Labs, excerpted from the official GitHub demo.

Three useful applications

Research and comparison. Give the agent the same questions for each product or source. Ask it to preserve URLs and uncertainty. This is more useful than asking it to browse broadly and report whatever looks interesting.

Website checks. Pick one journey: find a feature, use a navigation menu, or complete a form in a test environment. Ask for the steps taken, the resulting state and a screenshot of anything confusing or broken. Successful clicking does not establish that the site saved the result correctly.

Content preparation. Gather original sources and relevant screenshots before drafting. Keep the evidence beside the draft so that a factual claim can be checked without repeating the entire browsing session. Preparing an article and publishing it are different actions; authorize the latter explicitly.

These are suggested workflows, not performance results from a benchmark or claims that every website permits automation.

What the animation tells me

A visible cursor and a short action label make the next interaction easier to follow. They help answer: which control is the agent targeting, and what does it think that control will do?

The next question still matters: what happened on the page?

After navigation, check the destination. After filling a field, check its value. After an authorized save, reload and check that the result persisted. An action receipt establishes that an input was sent; it does not establish that the task succeeded.

The accompanying clip is an excerpt from Ego / Citro Labs' official demonstration. It shows the browser's own interaction treatment. The documentation screenshot was captured through Ego on my Mac mini; it is a separate source, not a claim that I recorded the official demo.

Footage: Ego / Citro Labs. The excerpt preserves the browser's actual visual feedback.

A small implementation pattern

On the installed Ego API used for this walkthrough, a task has a persistent space ID and managed page labels. Reuse them across rounds:

const task = await taskSpace("Compare product documentation");
const page = task.page("p1");
await page.goto("https://lite.ego.app/document/en/docs/space");
console.log({ spaceId: task.spaceId, page: page.label });
console.log(await page.snapshot());

Read the current installed Ego guide before adapting this example. Observe the actual page before choosing a click target; do not assume a selector from another site will apply.

Keep the boundary clear: which pages may the agent inspect, which changes may it make, and when should it stop? Separate task spaces organize the work. Account access still needs an explicit scope.

Sources and credits

Product descriptions were checked against the project's documentation. Ego Browser and its interface are the work of the Ego / Citro Labs team.

Before an AI migration, build a behaviour comparison harness

Harshith Vaddiparthy — Tue, 20 Oct 2026 02:43:09 +0000

An AI-generated replacement can compile, start and produce plausible output while changing a business rule. For a migration, the useful question is whether you can identify and explain each observable difference from the old implementation.

OpenAI’s October 19 research on low-resource programming languages distinguishes building new functionality from preserving existing behaviour. MultiLangBench asks models to build a SQLite-compatible engine across 20 languages. A separate evaluation migrates COBOL programs to other languages, with hidden tests requiring byte-for-byte parity on observable outputs. A migration passes only if all its tests pass.

OpenAI reports 86.6% accuracy for GPT-6.1 Sol at low reasoning effort on that migration evaluation and 97.9% at higher effort. Those are benchmark results, not a forecast for your repository. The announcement explicitly retains expert review and testing against production business rules and dependencies.

Here is a small migration workflow I would use to make those distinctions operational. It is a proposed implementation approach, not a reproduction of OpenAI’s evaluation or a tested production framework.

Start with an observable boundary

Choose a routine whose inputs, outputs and side effects you can isolate. A record-processing batch is a better first unit than an entire application with loosely understood integrations.

Write down:

Input format, encoding and missing-value conventions.
Output format, precision and ordering rules.
Error codes and rejected-record behaviour.
State changes, messages and other externally visible effects.
Environment assumptions such as timezone, locale and dependency versions.

The point is to define what a comparison will cover. A matching output file says nothing about an unobserved message sent twice.

Separate parity from intentional changes

Create a fixture directory with input and expected-output files from an approved, isolated run of the old implementation. Use synthetic or appropriately protected data. Pin the old executable and its environment so that the reference result is reproducible.

Alongside the fixtures, keep a small decision ledger:

# Illustrative structure; identifiers and values are hypothetical.
case_id: positive_fraction_boundary
rule: retain_two_decimal_places
old_result: "12.34"
decision: preserve
approved_result: "12.34"
owner_role: financial_process_owner
reason: preserve_existing_contract

OpenAI’s article supplies the mechanism behind this example: COBOL fixed-point arithmetic discards excess decimal digits unless rounding is requested. For a hypothetical positive value of 12.349, discarding excess digits gives 12.34; rounding to the nearest hundredth gives 12.35.

A replacement that chooses the second result can run perfectly well. Whether that difference is a regression depends on the requirement. If the owner approves a rule change, record it separately and retain the old expected result as historical evidence.

Avoid quietly regenerating all expected files from the new implementation. That can turn the code being evaluated into the authority for its own correctness.

Make the first comparator deliberately strict

For a byte-oriented interface, a basic file comparison is useful. The following standard-library script compares two existing files and reports a mismatch. It does not execute either program, validate business correctness or prove migration completeness.

from pathlib import Path
import sys


def main():
    if len(sys.argv) != 3:
        raise SystemExit("Usage: compare.py expected.bin actual.bin")

    expected = Path(sys.argv[1]).read_bytes()
    actual = Path(sys.argv[2]).read_bytes()

    if expected != actual:
        first = next(
            (i for i, (left, right) in enumerate(zip(expected, actual))
             if left != right),
            min(len(expected), len(actual)),
        )
        print(
            f"Mismatch at byte {first}; "
            f"expected length={len(expected)}, actual length={len(actual)}"
        )
        return 1

    print("Outputs match byte for byte")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

Start strict so differences become visible. If a format permits harmless differences, such as a field order that consumers explicitly do not depend on, define a narrow semantic comparison for that field. Keep the original output files. Broad whitespace stripping or sorting can hide changes that matter to a downstream consumer.

For the initial fixture set, include normal records and examples around the actual workflow’s boundaries: empty input, missing references, repeated records, date transitions and decimal precision. A passing set establishes evidence for those cases; it does not establish universal equivalence.

Inspect failures as part of the interface

Run the candidate in a sandbox with live writes disabled. Observe exit status, output files, state changes and attempted side effects. Use captured or simulated dependency responses to investigate retries and partial failures where appropriate.

Classify each discrepancy against the decision ledger: a candidate defect, an explicitly approved change, an unstable test environment or an unresolved requirement. Keep unresolved cases visible. An unexplained difference should not disappear behind a passing aggregate percentage.

Finally, measure generation effort separately from investigation, review and integration work. A lower API bill is one cost component. It does not tell you how much work remains to understand undocumented behaviour.

The first deliverable I would ask for is one isolated routine, a repeatable fixture set and a reviewed list of differences. That gives a team concrete evidence for deciding whether to expand the migration.

What an AI work-week review taught me about prototypes and finishing

Harshith Vaddiparthy — Mon, 19 Oct 2026 14:43:46 +0000

I asked AI to review a partial record of my work week, and the most useful result was a development pattern: I keep turning explanations into tools.

During workshop preparation and delivery, an idea would become an interactive analysis view, a small campaign workspace, or a reusable project. A working example makes the inputs, calculations and limits visible. It also creates another implementation to maintain.

Give the prototype one job

For a teaching tool, I want a learner to change an input and understand one consequence. That is a useful scope boundary. If I need ten minutes to explain the interface before the concept, the prototype has become another lesson.

The same rule applies to a product spike. State the decision it should resolve before choosing the framework or building the shell.

Review a real artifact early

One presentation grew to 44 slides. Reviewing the deck made the problem concrete: I wanted a practical demonstration-led session and a maximum of 15 slides.

For software, a small working surface plays the same role. It lets a reviewer inspect what expands, what survives navigation, what state is visible, and what happens next. A rough implementation can improve the brief, provided it is small enough to change cheaply.

Separate activity from completion

My history contained repeated visits to builds, renders and release sessions. An open terminal tells me a process was visible. It does not prove a feature works in the destination environment.

For a development task, I would keep a completion note like this:

Changed: the specific behaviour the user can now exercise

Inspect: the artifact, route or revision that demonstrates it

Evidence: the check actually performed against that result

Remaining: any unfinished dependency or limitation

A prototype checked with example records should say so. A video render still needs review. A release receipt should agree with the live page.

Use the history as evidence, then challenge the interpretation

The review covered available activity from October 12–19, 2026. It was not a complete time audit. A visible window cannot establish attention, and a long sequence of events cannot establish useful output.

I found it useful to ask what repeated, check that reading against the work itself, and choose one adjustment. Mine is making the finish condition explicit before opening the next build.

There was also a television that became a Jarvis project. Apparently my side quests now need repositories.

The habit I want to keep is making ideas tangible. The habit I want to improve is deciding which ideas deserve implementation today.

Read the original work-week reflection: https://www.harshith.com/blog/ai-reviewed-my-work-week

More field notes in The H Letter: https://www.harshith.com/newsletter

Four UI projects, four different jobs for the interface

Harshith Vaddiparthy — Sun, 18 Oct 2026 06:53:40 +0000

I’ve brought four of my projects into a UI gallery: an image editor, a content dashboard, a video-workflow canvas and a CRM. Each study includes real application screens, design decisions and links to the implementation.

The gallery itself is open source. It is a small React application you can run locally. The four applications remain in their own repositories, with their own setup instructions and MIT licenses.

These are the decisions I wanted each interface to make easier.

Grain Studio: judge the image while changing it

An image-effects tool needs a short feedback loop. Grain Studio keeps the image stage, texture dock and material controls in the same workspace. Its before-and-after scrubber lets you compare the effect while you tune it.

Export introduces a different set of decisions: format, dimensions and transparency. I gave those choices a dedicated flow with progress and cancellation states. The source uses Canvas 2D for the preview and a bounded OffscreenCanvas worker on supported browsers for export.

The reusable pattern is to keep comparison close to the adjustment and make output choices explicit at export time.

This capture uses the included sample artwork. The behavior was independently implemented after studying Textures; the dock’s proximity and tactile feedback are informed by ThreeUI Community, with its notice retained.

Inspect the case · Read the implementation

AI Content Dashboard: make content states easy to scan

The working sequence is brief → draft → review → library. The overview helps someone understand the collection; the library provides type and status filters; the editor gives an individual piece a focused working area.

The same content model feeds summary metrics, activity charts and recent-content rows. Reusing type and status labels across these views keeps the navigation predictable. A person can move from an overview count to the actual pieces behind it.

This is built on shadcn/ui, Base UI and the tweakcn Caffeine theme. The screenshots use fictional sample content and illustrative metrics. The source README distinguishes the implemented writing workflow from planned video-rendering and database integrations.

Inspect the case · Read the sample-data model

UGC Genie: show where work is in the process

UGC Genie connects product input, visual analysis, creative direction, generation, rendering and review. React Flow supplies the canvas; the application defines the stages and their state.

The source initializes every step explicitly:

export function createSteps(): WorkflowStep[] {
  return workflowDefinition.map((step) => ({ ...step, status: "pending", progress: 0 }));
}

That gives the interface a clear vocabulary: pending, running, completed and failed. The canvas can show which stage is active while the input and output-review areas remain available alongside it.

The screenshot shows a simulated, credit-free local demo. Its displayed progress describes application stages. The visual foundations include shadcn/Base UI and tweakcn claymorphism; my work is the workflow composition, feedback and review experience.

Inspect the case · Read the stage definition

PulseCRM: bring context to the next action

Sales work needs the pipeline, a record’s history and a useful next step. PulseCRM brings pipeline health, priority deals, account activity and follow-up work into one navigation system.

I adapted shadcn’s sidebar-07 into the CRM shell. Sales Copilot suggestions remain review-required, so record summaries and follow-up drafts have a clear human review step.

Preparing this case also exposed a small implementation bug: --font-sans referred to itself. I corrected it to the Geist font variable supplied by the layout, rebuilt the application and recaptured the interface. The linked source includes that fix.

These screens use bundled demo records in an isolated environment. The source also includes a separately configured Supabase-backed path.

Inspect the case · Read the captured revision

Explore an interaction in the code

For each study, start with one control and follow it through the implementation:

What decision is the person making?
Which state changes when they act?
What feedback makes that change clear?
How does the screen handle loading, empty and failed states?

The gallery credits the component libraries and visual foundations used in each project. Keep the original copyright, license and third-party notices when reusing the source. Individual applications have their own runtime and provider requirements; the gallery itself needs no API key or database.

Browse the interface studies or run the gallery from GitHub.

Build an Independent Attempt into Your AI Training App

Harshith Vaddiparthy — Sun, 18 Oct 2026 02:44:24 +0000

If your AI training app marks a lesson complete when a conversation ends, its completion record tells you very little about what the learner can explain or apply.

A useful alternative is to model guided practice and independent assessment as separate activities. Keep the learner's first attempt, any assistance they received, and the subsequent review distinguishable. That gives a reviewer something concrete to inspect without treating the assistant's confidence as a competence score.

This design is inspired by OpenAI's October 15 article, “Teaching students to think and learn through questions”. Its classroom example has students explore evaporation with Study Mode, then write an explanation and a discussion question without referring back to the conversation. The teacher uses those responses to identify what to revisit.

OpenAI also suggests that teachers try familiar subjects first and assess whether the questions and explanations suit their class. The article does not establish the effectiveness of a workplace training application. What follows is a proposed software design based on that exercise, with explicit limits on what its records can prove.

Start with an observable decision

Define a small learning target before building a chat interface. For a hypothetical internal support course, the target might be: “Identify which team should review an account-access exception, using the approved support guide.”

The exercise needs a reference version, a reviewer-approved case, and expected reasoning. Avoid asking a model to invent the organization's policy or grade against a standard it generated during the same conversation.

Keep the reference and case versions attached to the attempt. If the policy changes later, the reviewer should be able to see which version the learner was given. A version identifier is useful only if it points to content your system actually retains.

Separate the stages in the application

A small state model is sufficient:

Stage	Learner experience	Record
Guided practice	Answer questions, request hints, read explanations	Practice case and assistance events
Independent attempt	Read a fresh case and explain a decision	Initial answer and assistance status
Review	Receive feedback from a qualified reviewer	Reviewer, rubric findings, unresolved points
Revision	Consult the reference and revise the answer	New answer linked to the initial attempt

The independent case should differ from the practice case in one meaningful condition. In the hypothetical support course, the practice case could have all required identity evidence. The second could have missing evidence and an urgent request. The approved guide determines what happens next; urgency should not become an invented exception.

Show the independent case on a separate screen without the practice transcript or generated hints. Explain the purpose of that short attempt before the learner starts. If they request help, make it available through an explicit action and record that the attempt became assisted.

Hiding a transcript in your application does not prove that the learner used no other resources. Report the condition your application provided and the assistance it observed. Avoid describing this as a secure examination system.

Preserve the first answer

Here is an illustrative record, not a platform API or a complete production schema:

{
  "attempt_id": "attempt-101",
  "exercise_version": "support-routing-v3",
  "reference_version": "support-guide-v8",
  "case_id": "missing-evidence",
  "stage": "independent_attempt",
  "assistance": "none_observed_in_app",
  "answer": "The required evidence is missing. I would route this for review using the owner named in the guide.",
  "review_status": "pending",
  "revises_attempt_id": null
}

When the learner revises, create a new answer record linked to the original. Do not overwrite the initial explanation and then report the revised answer as unaided performance.

Similarly, keep an assistance event separate from answer content. An event can record that a hint was requested, when it was provided, and which attempt it belonged to. Only retain the content you need under your organization's data policy; these records are for learning review, not an excuse to collect unrelated employee activity.

Keep transitions under application control

The application should decide which actions are available at each stage. A learner's answer or pasted reference is data; it should not be able to mark an attempt reviewed or change the recorded assistance status.

If you use a model to propose feedback, save that feedback as a suggestion awaiting review. For this design, a qualified reviewer makes the competence decision. The distinction should be visible in the interface and enforced wherever state changes are accepted.

Keep submission handling predictable. A repeated submit for the same attempt should return the existing result, rather than creating multiple initial answers. Persist the answer before showing the confirmation. If a response is lost, the client should be able to retrieve the recorded attempt and display its actual state.

These are implementation recommendations, not claims about how Study Mode works internally.

Review reasoning, evidence, and boundaries

Use a short rubric that asks whether the answer identifies the applicable rule, cites the relevant evidence, notices the changed condition, and recognizes any unresolved question or need for review.

An answer that admits a specific uncertainty can reveal a useful learning gap. An answer that invents an owner or exception deserves different feedback, even when it sounds fluent.

Avoid compressing everything into a single model-generated score. Store the reviewer's findings by criterion and the reference supporting each correction. A later reviewer can then understand why the learner was asked to practise again.

Verify behavior with a few meaningful cases

Before piloting the app, check these situations:

A hint request during the independent stage changes the assistance record before the answer is assessed.
A revised answer preserves the initial attempt and links back to it.
Repeated submission creates one initial answer record.
Text inside an answer cannot change review status or the reference version.
Feedback distinguishes an incorrect decision from a reference that does not resolve the case.

Start with one approved exercise and inspect the resulting records with a knowledgeable reviewer. The initial outcome you can honestly report is what the learner explained under the observed conditions, what help they requested, and what changed after feedback.

That is useful evidence for deciding the next practice. Establishing lasting competence requires more than one interaction, especially when the real task has meaningful consequences.

What is an AI agent harness?

Harshith Vaddiparthy — Sat, 17 Oct 2026 23:26:01 +0000

By Harshith Vaddiparthy. Originally published in my AI agent harness guide.

An AI agent harness is the software around a model that lets it do work: supply the right information, offer tools, enforce permission rules, return results, and decide when to continue or stop. The model helps choose the next action. The harness connects that choice to an actual environment.

The walkthrough uses real software footage, fictional expense data, and my AI avatar and cloned voice. The transcript is available below without signing in.

That distinction matters when you are choosing an AI tool. A model name does not tell you whether the application can read your files, preserve the originals, ask before saving, recover from a failed action, or prove that the final result is correct. Those are properties of the complete system.

I recorded the walkthrough above to make those moving parts visible. The task is deliberately ordinary: read a small file of fictional expenses, calculate spending by category, flag a missing category, and save a short summary. There is no real financial account or customer data in this example. A small, inspectable task makes the boundary between a convincing answer and completed work much easier to see.

Model, tool, harness: three different jobs

The pieces of the expense demonstration

Piece	Its job	What to inspect
Model	Interprets the request and proposes the next response or action.	Did it understand the missing-category rule?
Tool	Performs a specific operation outside the model.	Which file did it read, or which calculation did it run?
Harness	Connects instructions, context, tools, permissions, and results.	Which actions are available, which are blocked, and what evidence returns?

The recorded application is DeepSeek Harness. In the interface, a conversation sits beside file tools and permission controls. You do not need that particular application to use the evaluation method. What matters is being able to inspect the input, the action, its result, and the boundary around it.

This general pattern also appears in Anthropic's guide to building effective agents: the model uses tools, receives feedback from the environment, and continues until it completes the task or reaches a stopping condition. The useful part is the feedback. Each step can give the system new evidence, rather than requiring it to finish everything from its first guess.

Start with a task whose answer you can check

My input contains 12 expense records. Eleven have a category. One does not. The instruction is explicit: retain that record, group it separately, and flag the missing category. Do not invent a label, delete the row, or quietly change the original file. The amounts are fictional and have no assigned currency.

Expected totals from the original 12 records

Category	Amount
Software	78.00
Supplies	46.00
Travel	86.00
Uncategorized	17.00
Grand total	227.00

The uncategorized amount is part of the 227.00 total. It is not an extra amount to add afterward. This is a small detail with a large lesson: handling uncertainty is part of the task definition. If I only asked for a tidy report, a plausible-looking classification could conceal the very exception I need to review.

Before running an agent, I want three things written down: the input it should use, the output I expect, and the rule for anything uncertain. For this task, that means the expense file, a category summary, and an explicit uncategorized group. A stronger prompt helps, but the application must still load the right files and enforce the allowed actions.

Follow the activity between the prompt and the answer

After submitting the task, I open the tool details. The first useful question is whether the agent read the intended input. A polished summary is weak evidence if I cannot connect it to the source records. The activity view should identify the action and show a result that I can interpret.

Read: The application supplies the instructions and the expense records through file tools. Check the file identity and the returned data.
Calculate: A verification command computes category totals, the grand total, and the number of records with a missing category. Inspect the actual output.
Propose: The model uses those results to prepare the summary. Compare its claims with the original input and the calculation.
Save and reopen: After the specific write is authorized, inspect the file that was actually saved and run the post-write check.

This is the agent loop in practical terms: choose an action, act through a tool, inspect what happened, and choose the next step. A tool request is not the same thing as a successful action. A planned save is not a saved file. A result saying that no summary exists yet is useful information, even when the arithmetic has already passed.

The harness should preserve those distinctions. If it reports every tool request as a success, the model may continue from a false premise. If it hides a failed write behind a generic error, recovery becomes harder. Clear results help both the model and the person reviewing the work understand the current state.

A saved file is not automatically remembered

Context is the information the model receives for the current step. In this example, the expense records and the rule for blank categories are relevant context. Unrelated files and old conversations are not automatically helpful. The harness decides what to load, what tool results to retain, and how to manage limited context space.

For longer tasks, the application may summarize earlier conversation or retrieve only selected material. That can keep a task moving, but summaries can drop detail. I prefer important facts and decisions to live in an artifact that can be reopened: the original data, a written instruction, or a checked output.

Persistence and retrieval are separate steps. The saved summary can survive the chat, but the next model response will not know its contents merely because it exists somewhere on disk. The system has to load it again. When someone says an agent has memory, ask what is stored, how it is retrieved, and how you can tell that the current response actually used it.

Approval and a sandbox answer different questions

In the recording, the initial write is blocked by the read-only setting. The agent then requests one-time permission to save the summary. I inspect the destination and the proposed content before allowing that action. The original expense file remains unchanged.

This demonstrates an application permission boundary. It does not demonstrate an operating-system sandbox. Approval answers whether a particular action may proceed. A sandbox limits which resources the execution environment can reach. A prompt saying 'do not touch other files' is neither a permission implementation nor proof of isolation.

For a learning exercise, the practical choice is a small disposable workspace with fictional inputs and only the capabilities needed for the task. Reading a file, creating a new summary, modifying the source, and sending the result to another person are different actions. Permission for one should not silently imply permission for all four.

Check what was saved, not just what was said

Once the summary is saved, I reopen it. The checker compares its category totals, grand total, and missing-category record count against the source data. In this example, those checks pass. That is stronger evidence than the final chat message saying the task is complete, because it tests the artifact I will actually use.

The limit matters just as much as the pass. This checker does not verify every sentence, decide whether an expense is legitimate, or tell me which category the blank row should have had. Correct arithmetic over incorrect source data would still be incorrect business information. The scope of a check determines what its result supports.

Input preserved. The original records remain intact, so the output can be traced back to them.
Exception retained. The 17.00 blank-category record remains visible instead of being guessed away.
Output reopened. The saved artifact exists and can be inspected independently of the chat response.
Checks named. A passing calculation has a specific meaning; it is not a blanket guarantee of correctness.

Four questions to ask of any agent

I use four questions to evaluate the complete system. What information can it see? Which actions can it take? How is the result checked? Where must it stop? These questions are useful even when you have no interest in building the harness yourself.

Take a weekly report as another example. The agent might be allowed to read a folder of approved inputs and draft a summary. Its check could reconcile totals and link each factual claim to a source. Its stopping point could be the reviewed draft. Publishing the report would remain a separate decision. That is a clearer design than asking an agent to 'handle reporting' and discovering its interpretation afterward.

Try the method on one small task

Choose a small set of fictional or non-sensitive records whose expected result you can calculate yourself. Use an agent application that exposes its tools and lets you restrict writes. Adapt this task brief to the capabilities it actually has. If it cannot show a tool result or enforce a permission boundary, record that limitation instead of assuming it exists.

A task brief you can adapt

Read the supplied expense file and preserve it unchanged.
Calculate totals by category and a grand total.
Keep blank categories in an Uncategorized group and report their count.
Show the calculation evidence and a proposed summary.
Do not save or send anything yet.
After I approve one destination and the summary, save that file only.
Reopen the saved file and compare its totals with the source.
Report which checks passed, which were not performed, and stop.

Then change one input deliberately: remove a category, add a row, or supply a destination the application is not allowed to write. Does the system retain the uncertainty, notice that the expected total changed, or stop at the boundary? A small controlled failure can teach you more about an agent than another polished demonstration.

You do not need a multi-agent architecture to learn this. Start with one task, a few tools, and one visible completion check. If you want to understand the implementation after that, my guide to building a coding agent from scratch explains the loop and the operating machinery around it. If you want more practical field notes like this, subscribe to The H Letter.

Sources

My recorded AI harness walkthrough

The fictional expense task, tool activity, blocked write, one-time approval, and saved-file check shown in this article.

Anthropic: Building effective agents

Primary guidance on tools, environmental feedback, stopping conditions, and human oversight. It supports the general pattern, not a security certification of this demonstration.

Frequently asked questions

Is an AI harness the same thing as a model?

No. The model produces responses and requests actions. The harness is the surrounding software that supplies context, exposes tools, applies permission rules, runs permitted actions, and returns observations. The same model can behave differently in two applications because those surroundings differ.

Does an approval dialog mean an agent is sandboxed?

No. Approval decides whether a particular action may proceed. A sandbox limits what an execution environment can reach. The walkthrough demonstrates an application permission boundary; it does not demonstrate an operating-system sandbox.

Does saving a file give an agent permanent memory?

Saving creates a durable artifact. The model can use that artifact in a later response only if the system loads its contents into the context it receives. Persistence and retrieval are different steps.

Full transcript and downloads

The complete transcript follows. You can also read it alongside the video in the original guide or download a copy:

0:00: What makes an agent work

When an AI reads a file, uses a tool, and checks a result, the model is only part of what you're seeing. The surrounding software makes that work possible. That surrounding system is called a harness. I'm going to show you one real task, and point out the pieces that make an agent useful.

0:18: The model and its harness

I'm using DeepSeek Harness, a real agent app with a conversation and tools for files. The model is one component inside this application. The instructions, available tools, and permission settings all affect what the complete system can do. The name in the model selector does not tell you everything about what the complete system can accomplish.

The model chooses what to say or which action to request. A tool does something outside the model, such as reading a file. The harness connects those pieces. It supplies context, makes allowed tools available, runs permitted actions, and returns their results.

The model can then use that new information to decide what to do next.

0:59: Give it the right context

For this task, I have a small file of fictional expenses. The goal is to calculate spending by category, flag anything that needs attention, and create a short summary. First, look at the information we are giving the agent. The file contains the original records.

The instructions say how to handle a blank category: keep the record, group it separately, and do not guess. We also tell it to preserve the input instead of quietly changing the source. These details are context: the information available to the model for this job. More context is not automatically better.

Unrelated files and old conversations can distract from the task. A useful harness helps select the relevant instructions and evidence, within the scope we allow. For longer tasks, a harness also has to manage limited context space. It may select earlier results or summarize older conversation.

Summaries can lose detail, so keep critical facts in an artifact you can reopen. The file is durable; whether the next response uses it depends on what the system actually loads. Before pressing send, check that the request names the input, the expected output, and the rule for uncertainty. That makes success something we can actually inspect.

2:19: Follow the real tool loop

Now we submit the task. Watch the activity between the prompt and the final response. The agent requests a tool, and the application performs the action. Opening the tool details lets us see which file it read and what information came back.

We can see what it actually read. The model uses that result to choose its next step. If it needs another file or a calculation, another tool request can follow. This is the basic agent loop: decide, act through a tool, inspect what happened, and continue.

This example uses files and a calculation. Those capabilities come from the integration around the model, and they should be limited to what the task needs. Installing a connector does not mean the agent should have unrestricted access. The loop does not guarantee that every decision is correct.

It gives the system a way to work with evidence instead of stopping at a single response. In this example, the useful evidence is the original expense data and the calculations made from it. We can follow the activity and compare the proposed summary with those records. Notice that an empty category is still something to flag, not an invitation to invent a label.

Check whether the actions and results support the answer.

3:35: Check the result and the boundary

Before accepting the summary, we check it against the original data. First, the calculator totals the source records. At this stage, it correctly reports that no summary has been saved. The expense with a missing category must remain visible in that check. Twelve records total two hundred and twenty-seven, including seventeen without a category.

The calculated totals match the source records; that is evidence for this calculation. Now look at the write boundary. Reading the input and saving a new file are different actions. The first write is blocked by the read-only setting. The agent then asks for one-time permission to save this summary.

Here, the application restricts which files and command the agent can use, and asks before writing this summary. This demonstrates an application permission boundary. An operating-system sandbox is a separate boundary, and we have not demonstrated one here. Inspect the proposed destination and contents before allowing this specific action.

After the file is saved, reopen it and confirm what actually exists. The checker compares the saved category totals, grand total, and the count of records missing a category against the source. A pass supports those checks; it does not verify every sentence in the report. That final readback matters more than a message saying the work is done.

Saved files only become context when the system loads them again. Approvals and sandboxes also do different jobs. An approval asks whether an action may proceed. A sandbox limits what the execution environment can reach.

Always check what is actually enforced.

5:22: Judge the whole system

The original file is unchanged, the totals have been checked, and the saved summary can be reopened. Following those steps gives us a practical way to evaluate the system. When you evaluate an agent, ask four questions. What information can it see?

Which actions can it take? How is the result checked? And where must it stop? Those questions tell you much more than the name of the model alone.

Try this with one small task of your own. Follow the actual steps, inspect the output, and identify one check you would want before trusting the result.

Version AI Outputs Before People Rely on Them

Harshith Vaddiparthy — Tue, 13 Oct 2026 02:45:50 +0000

A generated analysis can change after somebody has already acted on it. If your application overwrites the output in place, a later reviewer cannot reliably reconstruct which recommendation the person saw.

OpenAI’s October 6 mathematics release offers a useful engineering prompt. The announcement describes manuscripts, Lean formalizations of many proofs, reasoning summaries and information about attempted problems. The repository says corrections become new versions while earlier released versions remain accessible.

It also says results are at different verification stages, not all have accompanying formalizations, and some unformalized results could have issues. This article does not validate those mathematical results. The design lesson I am taking from the release is narrower: preserve the specific artifact that a review or decision refers to.

Separate a result from its revisions

For an AI-assisted report, keep a stable result ID and a separate revision ID. The result ID identifies the continuing task; the revision ID identifies one frozen output and its associated evidence.

A minimal record could contain the artifact digest, creation time, prior revision, reason for the change, source references and review status. Record model and prompt versions when available and relevant to reproducing the process. Do not claim that these fields make a nondeterministic generation reproducible by themselves.

Keep the output file and its receipt together. If somebody changes an assumption or conclusion after handover, create a new revision. The old revision remains the object of any earlier review. “Latest” can be a convenience pointer, but it should not be the identifier saved with a consequential decision.

Bind a receipt to exact bytes

Here is a small Python example using only the standard library. Run it in a temporary working directory. It creates a new directory for each revision and refuses to reuse an existing revision name.

from hashlib import sha256
from pathlib import Path
import json


def save_revision(root, revision, text, *, previous=None, reason):
    if not revision.isalnum():
        raise ValueError("Use an alphanumeric revision ID")
    if previous is not None:
        read_revision(root, previous)
    folder = root / revision
    folder.mkdir()  # Existing revision names are never overwritten.
    payload = text.encode("utf-8")
    (folder / "output.txt").write_bytes(payload)
    receipt = {
        "revision": revision,
        "previous": previous,
        "reason": reason,
        "artifact_sha256": sha256(payload).hexdigest(),
        "review_status": "not_reviewed",
    }
    (folder / "receipt.json").write_text(
        json.dumps(receipt, indent=2), encoding="utf-8"
    )
    return receipt


def read_revision(root, revision):
    if not revision.isalnum():
        raise ValueError("Use an alphanumeric revision ID")
    folder = root / revision
    receipt = json.loads((folder / "receipt.json").read_text("utf-8"))
    payload = (folder / "output.txt").read_bytes()
    if receipt["revision"] != revision:
        raise ValueError("Revision identity mismatch")
    if sha256(payload).hexdigest() != receipt["artifact_sha256"]:
        raise ValueError("Artifact differs from receipt")
    return payload.decode("utf-8"), receipt


root = Path("revisions")
root.mkdir(exist_ok=True)
save_revision(root, "v1", "Supplier covers region A.",
              reason="Initial unreviewed analysis")
save_revision(root, "v2", "Supplier excludes region A.",
              previous="v1", reason="Corrected coverage assumption")
assert read_revision(root, "v1")[0] == "Supplier covers region A."
assert read_revision(root, "v2")[1]["previous"] == "v1"

The example demonstrates an application convention and a byte-integrity check. It is not an immutable storage system. A person with filesystem write access can change both files, and a digest does not authenticate the author or establish that a claim is true.

There is also no transaction across the two file writes. An interrupted save can leave an incomplete directory; the reader fails instead of treating it as a valid revision. A service implementation needs a transaction or an explicit completed state, appropriate permissions and a recovery process. Keep those requirements separate from what this short example proves.

Store review decisions separately

A reviewer should reference the revision ID and artifact digest they actually inspected. Keep the reviewer identity, time, intended use and scope of checks in a separate review record.

For example, “arithmetic checked against supplied spreadsheet” is narrower than “recommendation approved.” If the spreadsheet itself has not been verified, that uncertainty should remain visible. A changed output should get a new review decision; copying an earlier approval onto the new revision would erase the very boundary the receipt is meant to preserve.

Do the same for inputs that can change. A source URL alone may later show different content. Where you are permitted to retain it, capture the relevant source version or snapshot and its timestamp. For private material, use restricted references and your existing retention rules. Do not turn a review feature into an uncontrolled copy of confidential documents.

Track the consequences of a correction

A revision chain says what came before. It does not tell you who acted on it.

If a report informs a consequential action, store that action’s reference to the specific revision. When an assumption changes, those references help identify decisions requiring another look. The correction event should describe the changed assumption and its possible consequence, rather than silently regenerating every downstream artifact.

Avoid auto-invalidating every decision merely because wording changed. Define which changes require review for your use case: a different source, an altered calculation, a changed eligibility assumption or a reversed recommendation may matter more than an editorial fix.

Test the handover, not just the happy path

Useful checks include retaining access to the earlier output after a new revision, refusing accidental reuse of a revision name and detecting an output that no longer matches its receipt. Test an interrupted write too: incomplete data should be visible as an error, not accepted as a completed record.

If you also evaluate generation quality, keep task IDs distinct from attempt IDs. OpenAI’s repository reports attempted problems and groups outputs into manuscripts and families; those different units do not support a simple success-rate division. In your own pilot, attach retries to the original task and define what counts as a usable result before comparing runs.

Start with one report or generated analysis that already crosses a team boundary. Save the revision people use, attach the scope of review, and make the next correction explicit. You can improve the storage architecture later; first make it possible to answer which version somebody trusted.

Gate AI Workflow Acceptance on Mandatory Checks

Harshith Vaddiparthy — Mon, 12 Oct 2026 02:54:37 +0000

An agent can configure most of a workflow correctly and still leave a required approval broken. If your evaluation reports only an average score, that failure can disappear inside an encouraging number.

OpenAI's October 6 publication, Advancing computer use with Ironclad, makes this a concrete engineering problem. The researchers worked with practitioners to define 11 tasks across legal, commercial and procurement work, with 8 to 50 evaluation criteria per task.

OpenAI reports mean rubric scores of 55.0% for GPT-6 Astra at Max reasoning and 41.6% for GPT-5.6 Sol at High reasoning. Those averages describe criteria met, not the percentage of workflows completed perfectly. The reported 19.2 versus 37.0 minutes per attempt are simulated estimates, not measured customer savings. The results cover those 11 research tasks, not every Ironclad workflow.

The implementation lesson I take from this is to separate a score that measures progress from the conditions that permit a workflow to be used. The design below is my proposed pattern, not a description of Ironclad's implementation.

Start with a versioned requirement

Suppose a fictional purchasing policy requires Finance approval for amounts strictly above USD 10,000, excluding tax. A real owner would need to confirm those details; they should not be invented by the agent.

Represent the amount in cents for this example, so the boundary is 1,000,000. Then define observable cases:

999,999 cents follows the ordinary path.
1,000,000 cents follows the ordinary path because the rule says strictly above.
1,000,001 cents requires Finance approval.
A missing amount remains incomplete rather than silently taking the ordinary path.
Editing an existing request across the boundary triggers the review behavior specified by the owner.

The last two cases are additional proposed requirements. They need agreement before implementation. Their purpose is to expose ambiguity rather than turn an assumption into policy.

Attach the accepted cases to a configuration version. A passing observation from yesterday's configuration should not authorize today's changed one.

Keep mandatory checks separate

A rubric can measure helpful labels, document quality and configuration details. Maintain that score for comparisons and debugging. Separately keep an explicit set of mandatory check IDs selected by the process owner.

Here is a small Python gate for observations collected by a trusted test runner. It accepts only explicit passes with an evidence reference for the requested version. Missing, stale or inconclusive observations remain unverified.

def acceptance(required, observations, version):
    if not required:
        raise ValueError("Define mandatory checks before evaluation")

    failed = []
    unverified = []
    for check_id in sorted(required):
        row = observations.get(check_id)
        if (not row or row.get("version") != version
                or not row.get("evidence")):
            unverified.append(check_id)
        elif row.get("status") == "fail":
            failed.append(check_id)
        elif row.get("status") != "pass":
            unverified.append(check_id)

    return {
        "ready": not failed and not unverified,
        "failed": failed,
        "unverified": unverified,
    }

There is deliberately no aggregate score in this decision. A high score for optional checks cannot cancel a failed mandatory check. Both failed and unverified conditions prevent ready from becoming true, while remaining distinct in the report.

For example, these are synthetic fixtures demonstrating the gate's behavior, not evidence from a real workflow:

required = {"above_threshold", "missing_amount"}
observations = {
    "above_threshold": {
        "status": "pass",
        "version": "config-7",
        "evidence": "fixture:above-threshold",
    }
}

result = acceptance(required, observations, "config-7")
assert result["ready"] is False
assert result["unverified"] == ["missing_amount"]

An empty required set raises an error because an accidentally empty checklist should not produce a passing result. An unexpected status also remains unverified. This function assumes the input mapping follows the shown schema; validate untrusted input at the boundary before calling it.

Collect observations outside the agent's narration

The gate above checks references and statuses. It does not prove that a referenced artifact exists or that the result is true. If the same agent can invent observations or rewrite the mandatory set, the gate offers little assurance.

Have a separate test controller submit known requests in a safe environment and read the resulting route or approval state. Save the request ID, tested configuration version, input, expected outcome, actual outcome and supporting evidence. Check that the evidence reference resolves to the observation for that exact case and version before passing it to the gate.

Use screenshots when visual context matters, but capture the actual result state too. A screenshot of a rule in an editor establishes what the configuration looks like. A routed test request establishes something different: how that configuration behaved for a particular input.

Preserve failures as reproducible cases. "The agent missed some steps" is difficult to debug. "Increasing a saved request from 900,000 to 1,100,000 cents did not trigger Finance review" describes an input transition and an observable defect.

Recheck after changes

After a configuration change, collect fresh observations for the mandatory cases. Do not reuse an old pass solely because the visible form looks unchanged. Approval routing may depend on fields, conditions or later edits that the screenshot does not reveal.

For a first pilot, a small mandatory suite is more useful than a large collection of vague checks. Add cases when the owner identifies another requirement or a failure reveals a missing boundary. Track the broader rubric alongside it so you can see improvements that do not yet justify acceptance.

Also record the human effort required to reach an accepted result: preparation, review, repair and repeated tests. Keep those measurements separate from model execution time and from the simulated timing in the research.

OpenAI's publication explicitly retains a role for human oversight in complex contracting. This proposed gate does not remove that responsibility. It gives the responsible person a clearer view of what passed, what failed and what nobody has verified yet.

Start with one workflow, one agreed version of its rules and a few meaningful cases. A reliable acceptance decision begins with a precise definition of what the finished system must do.

Make AI-Generated Calculators Explain Their Inputs

Harshith Vaddiparthy — Thu, 08 Oct 2026 02:47:43 +0000

OpenAI's October 7 announcement of GPT-6 and Intelligent UI describes ChatGPT responses that can include forms, charts, buttons and interactive tools. It also describes native components that appear progressively as the model generates an interface.

That makes a familiar engineering question more visible: what calculation sits behind a convenient control?

My interpretation is that developers should make the inputs, assumptions and result easy to inspect when building similar tools. This article uses a deliberately small capacity calculator to show what that means. It is a hypothetical example, not an implementation of OpenAI's component system or evidence that a generated ChatGPT tool has passed these checks.

Write down the calculation first

Suppose a support team wants a rough monthly capacity estimate. The model has three inputs:

Monthly request count.
Average minutes spent per request.
Productive support minutes available per person in that same month.

Multiplying the first two gives workload minutes. Dividing that by productive minutes per person gives a fractional staffing estimate. Rounding upward produces a whole-person capacity estimate.

Every part of that description matters. Scheduled work hours and productive support minutes are different inputs. Monthly averages leave out peak-hour coverage. A single average handling time hides variation between simple requests and specialist work.

The interface should state these limits near the result. They are assumptions to review with someone who knows the work. A correct division cannot tell you whether the three-input model represents the team's actual staffing constraints.

Keep one deterministic calculation

A small pure function is enough for this example. Given the same inputs, it returns the same result and changes no external system. The generated interface can call it when an input changes.

Here is a complete JavaScript example:

export function estimateCapacity(input = {}) {
  const fields = [
    "monthly_requests",
    "average_minutes_per_request",
    "monthly_productive_minutes_per_person",
  ];
  const missing = fields.filter((key) => input[key] === undefined);
  if (missing.length) return { status: "needs_input", fields: missing };

  const invalid = fields.filter((key) =>
    typeof input[key] !== "number" || !Number.isFinite(input[key])
  );
  if (!Number.isInteger(input.monthly_requests) || input.monthly_requests < 0)
    invalid.push("monthly_requests");
  if (input.average_minutes_per_request <= 0)
    invalid.push("average_minutes_per_request");
  if (input.monthly_productive_minutes_per_person <= 0)
    invalid.push("monthly_productive_minutes_per_person");
  if (invalid.length)
    return { status: "invalid_input", fields: [...new Set(invalid)] };

  const workload = input.monthly_requests * input.average_minutes_per_request;
  const people = workload / input.monthly_productive_minutes_per_person;
  if (!Number.isFinite(workload) || !Number.isFinite(people))
    return { status: "invalid_input", fields };

  return {
    status: "ok",
    formula_version: "monthly-capacity-v1",
    workload_minutes: workload,
    fractional_people: people,
    rounded_people: Math.ceil(people),
  };
}

For the demonstration, pass an object containing numbers. Convert form strings deliberately before calling the function, and preserve an empty field as missing; Number("") produces zero and can hide an omitted value. A real endpoint also needs to validate the shape of its incoming object before using this function.

The function rejects invalid numbers and nonpositive time values. It returns a separate needs_input state when a required value is absent. Display that state as a request for information, keeping any previous result clearly marked as stale.

No model call is required to redo the arithmetic. If the formula changes, review the change and update its version so a saved result can be interpreted later.

Check the boundaries that change the answer

Use known results to check the function before connecting an interface. These checks run in Node alongside the example:

import assert from "node:assert/strict";

const sample = {
  monthly_requests: 800,
  average_minutes_per_request: 12,
  monthly_productive_minutes_per_person: 4800,
};
assert.equal(estimateCapacity(sample).fractional_people, 2);
assert.equal(estimateCapacity({ ...sample, monthly_requests: 0 }).rounded_people, 0);
assert.equal(estimateCapacity({}).status, "needs_input");
assert.equal(estimateCapacity({ ...sample, monthly_requests: -1 }).status, "invalid_input");
assert.equal(estimateCapacity({ ...sample, monthly_requests: "800" }).status, "invalid_input");
assert.equal(estimateCapacity({ ...sample, monthly_productive_minutes_per_person: 0 }).status, "invalid_input");

All numbers here are synthetic demonstration inputs. They make the arithmetic observable and carry no staffing recommendation.

Then check the interface itself. Clearing an input should request information. Changing the request count should use the new value. Each label should show its unit and period. If the display rounds a result, the unrounded value and rounding rule should remain available.

An exported summary should contain the inputs, formula version, result and omitted factors. Another person should be able to reproduce it with ordinary arithmetic. This also helps reveal when a result was calculated using an earlier set of inputs.

Give external actions their own review

A calculator control can adjust a scenario locally. Submitting a staffing request is a separate operation that writes a record and may start an approval process.

If you later add that operation, show the destination and proposed payload, check authorization on the server, and confirm the resulting record. Handle an uncertain response by checking whether the record already exists before resubmitting. The interface's own input validation does not establish permission to write.

The OpenAI announcement does not show that every interactive answer can perform such external actions. These are implementation responsibilities for a system you choose to connect.

Start with a calculation you can inspect

The announcement says Intelligent UI is rolling out in ChatGPT's Chat experience; enterprise availability depends on admin settings. It does not introduce this example as an API or change the models powering Work and Codex as part of the release.

For your own prototype, choose a familiar calculation, state its assumptions, use synthetic inputs, and keep it limited to scenario exploration initially. Ask a domain expert whether the omitted factors change the decision.

A useful generated interface makes the calculation understandable, lets people correct its inputs, and tells them when it cannot produce a meaningful result. Those properties can be checked regardless of how impressive the first rendering looks.

Build AI Provenance Checks That Can Say Unknown

Harshith Vaddiparthy — Tue, 06 Oct 2026 03:02:24 +0000

OpenAI's October 5 text-provenance announcement is a useful prompt for builders. Its textGrain watermark puts a statistical signal into eligible model output. API customers can opt in for select models; eligible ChatGPT and Codex text in the EU is due to receive it over the coming weeks. Detector access starts with approved researchers and expert organizations, not a public endpoint that every product can call.

The engineering lesson is broader than this rollout: an origin signal is one observation in a decision workflow. If your data model turns that observation into written_by_ai: true or written_by_human: true, it has already lost the uncertainty the source warns about.

Separate capture, signal, and decision

I would model a provenance review as three connected records, each with a different job.

Capture records what the workflow knew when the document was created: a version hash, the policy for permitted AI use, an author declaration, and references used for material claims. A hash identifies the version under review; it does not prove who wrote it. Keep a link to the version history where one exists.

Signal records what a check actually observed, including whether that check was applicable. A watermark observation needs its provider, method version, date, input version, and known limits. not_detected is a result about a check, not a statement of human authorship. not_run, unsupported, and inconclusive are useful states too.

Decision records what a person did with the evidence: which claims were checked, who reviewed them, whether the work was approved or needs revision, and why. The decision can remain pending when the evidence is incomplete.

Here is a proposed internal shape, not an OpenAI API response:

{
  "document_version": "sha256:...",
  "declared_ai_use": "unknown",
  "watermark_check": {
    "eligibility": "unknown",
    "result": "not_run",
    "method_version": null
  },
  "review": {
    "source_check": "pending",
    "decision": "pending",
    "owner": null
  }
}

This example records a check that has not run and a decision that has not been made. Other possible check results include detected, not_detected, and inconclusive. The exact fields will vary. The key property is that missing evidence stays missing. A downstream dashboard should never relabel not_detected as “human written,” or detected as misconduct.

Make the limits visible in the interface

OpenAI says a text watermark cannot identify the user, measure human contribution, establish ownership or responsibility, or verify accuracy. It also says that no detected watermark does not prove human authorship. Editing, translation, short passages, older output, and unsupported models can all complicate detection.

Its own evaluation shows why the qualification matters. For 400-token English passages in one test, replacing 10% of words with synonyms reduced detection from about 92% to 66%; replacing 25% reduced it to 17%. Those are company-reported results under specific conditions, not a field benchmark you can apply to every document.

Show that scope beside a result. An interface that displays only a red or green badge invites an overconfident decision. A review screen should show the input version, whether the method was eligible, the observed result, and what the result cannot establish. If a tool returns a probability or score, preserve its meaning and calibration rather than converting it into a verdict through a hidden threshold.

Route consequential cases to people

Consider a hypothetical policy memo. The team permits disclosed AI assistance and needs accurate claims plus a named approver. A detected watermark might justify asking how the memo was drafted. It cannot tell the team whether the memo's sources support its recommendations.

The review queue should bring the memo version, declaration, source links, and prior edits together. Reviewers can then check the claims that matter, ask for missing context, and record a decision. If the result could affect a person's reputation or opportunity, include a way to explain the evidence and correct errors. Do not make an adverse action the automatic output of a detector callback.

You can build this queue without a watermark detector. In fact, the current access limit means many teams must. Capture and review are still useful when a signal is unavailable, and they continue to matter if access expands.

Test the workflow, not just the classifier

If you can evaluate a provenance signal for your own authorized material, include clean, edited, short, translated, and constrained examples. Keep examples outside the tuning set. Measure false alarms, missed signals, inconclusive cases, reviewer time, and the decisions that followed. Separate results by content type; one overall rate can hide the cases your product handles poorly.

Write the proposed action for each result before looking at the scores. “Ask for supporting records” is a different consequence from “reject the work.” The evidence needed for those actions should differ too. Make it possible to pause when the method's scope is unknown.

My first implementation step would be a document-version record and a small review form with an explicit unknown state. Then I would test whether the team can answer who approved a claim and what source supported it. A detector can be assessed later as one additional observation. The system is useful as soon as it can preserve evidence and make uncertainty visible.

Design AI Workflows Around Verified Completion

Harshith Vaddiparthy — Thu, 01 Oct 2026 22:55:01 +0000

While connecting a Mac mini to Jarvis, my AI assistant, I kept coming back to a small question: what can I prove actually happened?

The setup produced a series of ordinary milestones. A command ran on the intended machine. A file transferred and could be read back. A browser opened the requested page. A device on my home network returned its status. The connection recovered after a deliberate connection restart.

Each result established a specific capability. Together, they gave the assistant a practical way to carry out work.

Then I asked it to research an article and prepare a Medium draft. It reached a login screen. I signed in, and the work continued.

For someone building agent workflows, that handoff is a useful place to start. Understanding a task and completing it depend on different parts of the system.

Intelligence needs execution capacity

In “The eternal complement”, Hemanth Asirvatham and Elliott Mokski argue that intelligence needs complementary capacity to turn ideas into progress. Their essay is hosted on OpenAI’s Intelligence Age platform and explicitly represents their own views, not OpenAI’s position.

The authors use “institutional intelligence” to describe the coordination behind execution: funding, supply chains, laws and the many correct local actions that make an ambitious project possible.

An agent workflow has its own supporting work. It needs the right input, access to the right application, a clear scope of authority and a way to check the result.

A model may produce an excellent plan while those pieces remain missing. Making the plan more sophisticated does not automatically supply the missing login or establish that a file was saved.

My computer setup is a small example of this relationship. It does not establish anything about the future of superintelligence. It does make the dependency visible.

Define completion before choosing tools

“Write an article” leaves several possible stopping points.

The model could return text in chat. It could save a local file. It could create a draft in the intended account. It could publish the reviewed version and verify the public page.

Each is a different deliverable.

For a draft workflow, I would define completion as: the intended title, body and links are saved in the correct account, and the content survives a reload.

For an authorized publishing workflow, I would add: the public article exists, the author is correct, and the published body matches the intended version.

Those definitions give the implementation something concrete to verify. They also prevent a fluent final message from becoming the only evidence that the work succeeded.

Before adding another tool, write the acceptance condition in one sentence. Then identify which tool can produce evidence for it.

Treat the application’s saved state as evidence

Browser automation introduces an easy trap. A successful input operation is not the same as a successful application write.

Text can land in the wrong field. A rich editor can show content before it has persisted. A connection can drop while the application is saving.

For this kind of workflow, a small write probe is useful. Enter a small amount through the intended UI, inspect where it landed, and confirm that the application created the expected saved artifact before inserting the full body.

After the full edit, reload and compare the persisted content with the intended text. Check links as well as paragraphs. Inspect the rendered page where layout matters.

This is why screenshots and text readback complement each other. A screenshot can expose a formatting problem. A full text comparison can expose missing content that sits outside the viewport.

The evidence should match the claim. A working browser establishes browser access. Recovery after restarting a connection establishes that recovery path. Unattended recovery after a machine reboot would need a separate test.

Keep uncertainty in the state model

The most important failure case often happens after an external action starts.

Suppose an agent clicks Publish and then loses its connection. The article may already be public. The agent may simply have missed the response.

Treating that as an ordinary failure and retrying immediately can repeat the action.

A content workflow can distinguish these states:

State	What it establishes	What happens next
Prepared	Local content and the intended account are known	Create or reconcile the draft
Saved	The application retained the intended content	Continue within the authorized scope
Publishing	Publication intent was recorded before the external action	Observe the result
Uncertain publication	The action may have succeeded, but evidence is incomplete	Inspect the existing story and public profile
Published	The public artifact and its content were verified	Record the result and stop

This table is a practical design suggestion, not a framework prescribed by the essay.

The record needs enough information to reconcile the operation: the existing draft identifier, intended account, content hash, and any public URL already returned. Write the intent before the action, and preserve those identifiers when an error occurs.

A local record cannot make an external website transaction atomic. Its value is that the next attempt can investigate the same operation instead of starting over.

Once publication is verified, a later cleanup error should not silently downgrade the article to “unpublished.” Record the cleanup problem separately.

Put autonomy inside a clear scope

A workflow needs to know what it is allowed to do.

That can include standing authorization to complete a recurring task. The agent should carry out that authorized work without asking the same question at every step.

It should also recognize actual exceptions: an expired login, a different account, a platform refusal, or an outcome it cannot establish.

Persist the exception and the work already completed. A vague “something went wrong” gives the next run very little to work with. A known draft URL and an uncertain publication state give it a specific investigation.

Improve the next blocked step

The essay explores two uncertain directions. In a civilization of depth, better intelligence makes much more effective use of existing knowledge and resources. In a civilization of width, it generates worthwhile projects faster than supporting capacity can keep up.

The authors do not predict which direction will dominate. The idea that machine intelligence could spend most of its time on routine organizational work belongs to the width scenario.

For the workflow in front of me, I do not need to settle that question.

I can choose one recurring task, define its finished result, and follow it through every dependency. Where does it wait for access? Where is the outcome assumed rather than checked? What happens if the next external write succeeds but its response never arrives?

Answering those questions gives me concrete implementation work. It is the ordinary work that lets an intelligent system carry a useful idea through to a result.

Adapted from my original Medium article, “The Boring Work That Makes AI Useful”.

Primary source: “The eternal complement,” by Hemanth Asirvatham and Elliott Mokski. The authors write as independent voices hosted by OpenAI.

Forking Macro: A Technical Walkthrough of Agent Collaboration

Harshith Vaddiparthy — Wed, 19 Aug 2026 20:33:54 +0000

Multi-agent demos often hide the most difficult engineering problem.

Disclosure: I used AI tools to assist with research organization and drafting. I reviewed, edited, fact-checked, and source-checked this article before publication.

Spawning workers is not collaboration. A collaborative system must manage context, permissions, handoffs, concurrent state, and verification across humans and agents.

Macro is a workspace that describes its open-source decision publicly and is useful to examine through this lens. Its official repository is published at macro-inc/macro. The root license presents AGPLv3, but a nested web-app license says all rights reserved. That conflict matters and is covered below. The published stack includes Rust and SolidJS. Macro connects documents, email, tasks, calls, channels, CRM, memory, and agents.

I am using Macro and forked the repository. On August 19, 2026, GitHub compare showed the fork's main identical to upstream main, 0 commits ahead and 0 behind, at 02c218308c462046ee54e6f275b8c0a371ee3f5a.

Important scope note: no local clone, modified branch, self-hosted deployment, or benchmark is claimed in this article. The walkthrough maps verified repository and documentation claims to a reproducible evaluation plan. It does not pretend I ran tests for which I have no artifact.

Repository snapshot

At review time, the official repository showed:

Item	Snapshot on 2026-08-19
Stars	3,696
Forks	362
Commits in `main` history	5,066
Open issues	19
Open pull requests	51
Tags	213
Latest visible release	`v2026.8.18.2`
Reviewed commit	`02c218308c462046ee54e6f275b8c0a371ee3f5a`

These are activity signals, not performance results. The latest values will change.

The cover uses Macro's older embedded star-history snapshot, which shows 3,608 stars. The live count at this review checkpoint was 3,696.

Macro said on August 13 that the repository briefly reached number one on GitHub Trending. The company said on August 12 that it had received an influx of users, pull requests, and feature requests. Both are attributed company statements.

Reproduce the source boundary first

If you want to inspect the same fork, begin by recording the exact source state. The commands below verify Git identity only. They do not install or run Macro.

git clone https://github.com/harshith-vaddiparthy/macro.git
cd macro
git remote add upstream https://github.com/macro-inc/macro.git
git fetch upstream main
git remote -v
git status --short
git rev-parse HEAD
git rev-parse upstream/main
git rev-list --left-right --count upstream/main...HEAD
git log -1 --oneline

Before building, compare the full SHA with the current upstream default branch and read the repository's current README. Do not assume dependencies or setup commands from this dated article are still correct.

Resolve the license boundary before reuse

Those terms conflict with a repository-wide assumption that every file is AGPLv3. The affected scope needs clarification from Macro's maintainers. Until then, do not assume every web-app file, screenshot, or asset can be reused under the root license.

This article does not decide which terms control. Before modifying, distributing, hosting, or reusing repository material, inspect the applicable path, request maintainer clarification, and obtain appropriate legal advice.

A technical model of Macro's collaboration surface

The documented system can be understood as six connected layers:

Work surfaces
  email | tasks | documents | calls | channels | CRM
                         |
                         v
Shared memory refreshed nightly from team activity
                         |
              explicit context via @mentions
                         |
                         v
Agents operating under the user's permissions
                         |
       read | draft | write | update | post | automate
                         |
                         v
Shared document state using CRDT collaboration
                         |
                         v
External coding agents through an MCP endpoint

This is an editorial model based on Macro's website, documentation, and README. It is not a literal internal service diagram.

Layer 1: Work surfaces

Macro's Agents documentation says agents can access email, tasks, documents, calls, and channels under the user's permissions. Documented actions include search and read, document writing, task and CRM updates, email drafting, channel posts, call reading, and automation.

The engineering advantage is context locality. The agent does not need a separate integration for every step if the workspace already exposes the relevant object and action.

The risk is authority aggregation. A workspace that unifies several tools can give one agent a broad action surface. The permission model, confirmation boundaries, audit trail, and revocation behavior therefore matter as much as tool coverage.

Layer 2: Shared memory

Macro's Unified Memory documentation says memory refreshes nightly from team activity.

A shared memory layer can reduce repeated context assembly. It can also create subtle failure modes:

stale information remains retrievable;
two sources disagree;
deletion does not propagate as expected;
an agent retrieves a related but non-authoritative artifact;
the user cannot see why a particular item was selected.

Test memory against the latency of the workflow. A nightly refresh may fit a weekly recap. It may not fit an incident response or a decision made minutes ago.

Layer 3: Explicit context

Macro's Agents documentation presents @mentions as a way to pin context for an agent.

This gives the system two retrieval modes:

broad organizational context from shared memory;
task-specific context selected by a human or collaborator.

That distinction is useful because retrieval relevance is not the same as authority. The closest semantic match may not be the document that governs the current decision. Pinning context creates a direct signal about which source matters now.

Layer 4: Handoffs

The official repository includes an agent task-handoff screenshot. A handoff is the state boundary between actors, so evaluate it as structured data rather than a friendly message.

A robust handoff should preserve at least:

goal: "What outcome are we trying to produce?"
source_artifacts:
  - "Which documents, calls, tasks, or messages govern the work?"
completed_work:
  - "What has already changed?"
open_questions:
  - "What remains unresolved?"
next_action: "What should happen next?"
owner: "Which person or agent is responsible now?"
verification: "How can the next actor check the claims?"

This YAML is an evaluation template, not Macro configuration.

Layer 5: Concurrent document state

Macro's Documents documentation says its documents use CRDTs with Cloudflare Durable Objects. The README describes swarms of agents acting as peers in CRDT collaboration.

A conflict-free replicated data type allows concurrent updates to converge without using a simple last-write-wins overwrite. This is relevant when humans and several agents edit a shared document.

CRDTs solve a state problem. They do not solve a meaning problem.

Two changes can merge correctly and contradict each other. One agent can add a decision while another rewrites the assumption behind it. A production workflow still needs editorial ownership, review boundaries, and a way to identify which claims require verification.

Useful CRDT tests include:

two agents insert text at the same location;
a human deletes a section while an agent edits it;
one participant works from a delayed client;
several edits reference the same task or decision;
a reconnect occurs after substantial concurrent work.

Measure both state convergence and semantic coherence.

Layer 6: External agents over MCP

Macro's MCP overview names Claude Code and Codex CLI. Its Agents documentation separately names Cursor. MCP gives an AI client a standard interface to external context and tools.

The valuable pattern is a closed state loop:

shared task -> coding agent -> repository work -> task update -> team recap

An integration is incomplete if it only sends context to the agent. The agent should return a structured status, evidence, and unresolved questions to the workspace where the team coordinates.

Macro's published Agent Recipes include MCP-driven coding-agent task updates, along with daily briefs, project status, weekly recap, call-to-tasks, draft email, and channel summary.

Build a bounded evaluation, not a swarm demo

A weekly project recap is a good first workflow because it is recurring, reviewable, and reversible.

Define the test before connecting broad authority:

Test dimension	Question	Evidence to capture
Context	Did the agent use the governing sources?	Source links and retrieval trace
Freshness	Did it include a recent change?	Source timestamp versus output timestamp
Permission	Could it access only allowed objects?	Account role and denied-access test
Handoff	Did it preserve owners and open questions?	Structured comparison with source tasks
Action boundary	Did it draft or mutate shared state?	Before and after object state
Concurrency	What happened during a conflicting edit?	Revision history and final document
Verification	Can a human reconstruct the output?	Action log, citations, and correction record

Run the workflow in three rounds:

Happy path: clean sources, no conflicts, draft-only output.
Adversarial path: one stale document, one contradictory update, and one inaccessible source.
Concurrent path: a human or second agent changes a related artifact while the workflow is active.

The primary metric should be correction cost. Record how long a person spends finding missing context, checking claims, fixing ownership, and restoring state. Fast generation does not help if review becomes expensive.

Where the architecture is promising

Macro's direction connects several primitives that are often separated:

an inspectable public implementation with a license-scope conflict that needs clarification;
shared organizational context;
explicit context selection;
permission-aware actions;
concurrent collaborative documents;
MCP access for external coding agents;
recipes that return work to a shared system.

The combination is the important part. An agent becomes more useful when it can read current work, make a bounded contribution, and return state that another person or agent can continue.

What remains unproven here

This review does not establish:

production reliability or security;
latency and cost at scale;
the quality of memory retrieval;
behavior under permission changes;
audit-log completeness;
semantic quality under concurrent edits;
successful self-hosting from Harshith's fork;
a modification or benchmark at revision 02c218308c462046ee54e6f275b8c0a371ee3f5a.

Those are test targets, not conclusions to smuggle into a launch post.

Conclusion

The hard part of multi-agent software is not multiplying workers. It is coordinating context, authority, shared state, handoffs, and verification.

Macro is worth studying because its public architecture addresses those concerns directly. Shared memory gives agents organizational context. @mentions narrow attention. Permission-bound actions connect context to work. CRDTs support concurrent collaboration. MCP can return coding-agent state to the team's workspace. Studying public code is different from assuming every path has uniform reuse terms, so the nested-license conflict remains a real diligence item.

None of these features eliminates the need for verification. Together, they provide a concrete public system against which developers can test collaboration claims.

Credit to Jacob Beckerman, whose profile identifies him as Macro's founder and CEO, and the Macro team for publishing the work. This is an independent technical analysis and does not imply endorsement.

For the broader product argument, FAQ, sources, and dated updates, read the canonical guide.