DEV Community: Mangat Rai

The AI industry is building its own referee

Mangat Rai — Tue, 29 Sep 2026 15:29:57 +0000

OpenAI models escaped a test environment and compromised Hugging Face. Claude models gained unauthorized access to real production systems in three cases. Gemini accessed systems at three companies during another test.

Different labs. Same unsettling pattern. Sound familiar? That has been the rhythm of AI safety news lately.

By the time I read the third story, I wanted to know whether anyone was connecting these incidents. Were the labs learning from them together, or was each company still deciding for itself what counted as safe? What were the frontier labs doing as an industry, what was the government requiring, and did any shared standard exist?

So I started digging into those questions. That search led me to industry plan taking shape under the tentative name SAFA.

⏳ TL;DR
I started with a simple question: after a run of frontier-model security incidents, who sets the shared safety rules? In July, Google DeepMind's Demis Hassabis proposed an industry-funded standards body with federal oversight, independent experts, and a path from voluntary reviews to mandatory pre-release tests. By late September, Google, OpenAI, and Anthropic were reportedly building a private version without government oversight. Shared tests could still improve today's fragmented safety reporting. The unresolved question is whether this new body can make its founders follow a rule, publish an embarrassing result, or admit competitors on equal terms.

The answer changed in ten weeks

The first real answer I found came from Google DeepMind's Demis Hassabis. His July proposal called for a federally overseen standards body with independent technical experts and open-source representatives on its board. Industry would provide most of the funding, while federal agencies and US national laboratories would help test advanced models in areas tied to national security.

The proposal also described a route to actual authority. Labs would initially submit frontier models for voluntary review up to 30 days before release. If the process proved effective, models could eventually be required to pass before deployment in the US.

Then the idea changed. On September 24, The Information reported that Google, OpenAI, and Anthropic were preparing their own standards organization, tentatively called the Standards Authority for Frontier AI (SAFA). The companies hope to launch it by the end of 2026 or early 2027, but its final name, charter, leadership, funding, and enforcement powers are not public.

Here is the difference I kept coming back to:

	July proposal	Reported September plan
Who is behind it	A US-led body, funded mostly by industry	Google, OpenAI, and Anthropic
Government role	Federal oversight, with agencies and national labs involved in testing	No government oversight
Governance	Independent technical experts and open-source representatives on the board	Board and appointment rules not public
Participation	Voluntary reviews first, with a path to mandatory tests	Voluntary safety commitments; enforcement not public
Main work	Define frontier thresholds and assess models before release	Support outside testing, incident reporting, and auditor qualifications

This does not make SAFA pointless. It changes what SAFA is. The July proposal described delegated governance: industry expertise and money operating inside a public structure. The September plan is, so far, a private standards organization seeking credibility without delegated authority.

I can see why the labs want one standard

Once I looked at what the labs do today, the practical case for SAFA became clearer. Each company already has a framework for deciding when a powerful model needs stronger safeguards. The frameworks cover similar severe risks, but they use different categories, thresholds, and names.

Lab	Published framework	Its trigger language
Anthropic	Responsible Scaling Policy (v3.4, July 2026)	Capability Thresholds tied to ASL-3 safeguards
OpenAI	Preparedness Framework (v2, April 2025)	High and Critical capability thresholds
Google DeepMind	Frontier Safety Framework (v3.1, April 2026)	Tracked and Critical Capability Levels

These are not three versions of one scale. I cannot cleanly translate an OpenAI “High” result into an Anthropic ASL level or a Google DeepMind Critical Capability Level. The methods, thresholds, and release decisions remain vendor-specific.

Common evaluation protocols, incident definitions, and auditor qualifications would make the evidence easier to compare. They could replace three separate safety vocabularies with a shared one. After reading the incident reports that sent me down this path, that sounds genuinely useful.

But a shared vocabulary is not the same thing as a referee.

I had to look up why FINRA matters

Hassabis used the Financial Industry Regulatory Authority, or FINRA, as his model. I did not know much about FINRA before reading this proposal. At first, “industry-funded regulator” sounded close to what the AI labs were building.

The important part is what sits behind those two words. FINRA is privately operated, but it lives inside public law.

US broker-dealers generally must belong to a self-regulatory organization before doing business. A broker-dealer operating outside the exchanges of which it is a member must usually join FINRA. FINRA's proposed rule changes are filed with the Securities and Exchange Commission for review. Its board reserves ten seats for industry members, one for its CEO, and the remaining seats for public members.

That is what gives the arrangement weight. Firms cannot simply ignore the system and keep doing the same business. The industry does not get final say over every rule. Public representatives have formal power inside the organization.

None of those conditions has been announced for SAFA. No law compels a lab to participate. No public agency approves its rules. No published charter says who controls its board or what happens when a founder fails a test. Customers, insurers, and governments could eventually give its standards market power, but that would be influence earned after launch, not authority granted at formation.

The conflict is not subtle

The first version of SAFA would be formed by three companies whose models and labs it could help evaluate. That does not prove the standards will be weak. I do think it means independence has to be designed into the institution rather than promised after someone raises the question.

There is also a competition problem. If enterprise buyers or insurers eventually demand a SAFA result, the founding labs will have helped write a test their competitors must pass. A demanding standard could improve safety and still become a moat. I do not think we should pretend only one of those outcomes is possible.

A famous independent chair would not settle this for me. I would want the boring institutional details: published appointment rules, protected terms for independent board members, transparent funding, open membership, appeal procedures, and results that the founding companies cannot suppress.

What I will watch next

SAFA is still a reported plan. I am not ready to call it either a regulator or an industry stunt. I want to see what its founders are willing to give up.

Control: Do independent members hold real voting power over standards, leadership, and publication decisions?
Transparency: Are methods, test versions, incident definitions, and full results public enough to challenge?
Membership: Can another frontier lab join as an equal rule-maker on reasonable terms?
Consequences: What happens after a failed test, and can a founding lab simply leave or release anyway?
Bad news: Will SAFA publish a finding that delays or embarrasses one of its founders?

For practitioners, a future SAFA report could become useful vendor evidence. I would ask which standard and version was used, who ran the test, what the result excluded, and whether the complete finding is public. I would not confuse that evidence with government approval.

The next time one of those safety headlines lands, I will look past “we are investigating.” Did the incident enter a shared reporting system? Could an independent body inspect the evidence? Did the failure change a release decision? Could the lab keep an inconvenient result private?

That is what a referee would change. The real test of SAFA will not be whether it writes sensible rules. It will be whether its founders can lose a call.

Goose: An Open-Source Agent You Can Assemble Yourself

Mangat Rai — Thu, 24 Sep 2026 06:20:55 +0000

A few days ago, I wrote about AGENTS.md, the file that gives coding agents project-specific instructions. While reading through the Linux Foundation's Agentic AI Foundation site, I came across another founding project: Goose.

The pitch caught my attention. Goose runs on your machine, lets you choose the model, and connects to tools through the Model Context Protocol.

However, The five-minute quickstart says Goose improves software development by automating coding tasks. The tutorial asks it to build a JavaScript game. The Goose homepage, however, calls it a general-purpose agent for research, writing, automation, data analysis, and code.

What was I looking at? Is it a coding agent or a general-purpose agent?

What i figured at the end is that Goose is both, but not equally. Coding is the go to path. Its extension system lets you give the same agent a different set of tools and a different job.

⏳ TL;DR
Goose is coding-first, not coding-only. Its built-in developer tools cover repository work. Change the extensions and instructions, and it can handle research, documents, browser tasks, data, and other automation. You also choose the model, including compatible local models. That freedom comes with more setup, evaluation, and security work.

So which is it?

Goose's history clears the confusion. When Block introduced it in January 2025, the launch post said its first use cases focused on engineering while the community was already trying it on non-engineering work. The examples included code migrations, tests, benchmarks, feature flags, and API scaffolding.

The product has since expanded. Its current homepage and README describe a general-purpose agent. Even its system prompt uses that term. The quickstart still takes the shortest route to a useful result: give Goose a working directory and ask it to build an application.

Both descriptions are accurate, but they set different expectations. Calling Goose a coding agent hides what its extension system can do. Calling it a general assistant makes it sound more ready-made than it is outside software development.

My read is simpler: Goose is an agent you install and configure. It comes prepared for coding. Its tools and instructions determine how far you can take it.

What actually changes

The architecture documentation splits Goose into three main pieces: an interface, the agent, and extensions. The model sits behind the agent and decides whether to answer or ask for a tool.

flowchart LR
    U["You"] --> I["Desktop, CLI, API,<br/>or ACP client"]
    I --> G["Goose agent runtime<br/>sessions, context, permissions"]
    G <--> M["Your chosen model<br/>hosted or local"]
    G --> E["Built-in and MCP extensions"]
    E --> C["Coding work<br/>files, shell, repositories"]
    E --> W["General work<br/>documents, browser, data, services"]
    C --> G
    W --> G

Ask Goose to change a repository and it sends the request, relevant context, and available tool definitions to the model. The model can ask to read a file, make an edit, or run a shell command. Goose executes the permitted call, returns the result, and gives the model another turn.

That loop can also prepare a weekly project brief. Give it local meeting notes and a read-only issue-tracker extension instead of a code repository. The model can read the notes, query completed issues, and write the brief. Goose still coordinates the model and tools; the toolbox and instructions are different.

The coding task and the project brief are not two different versions of Goose. The loop stays the same. The tools determine what work it can do.

Start with coding

Goose's built-in Developer extension can read and modify files, run shell commands, and maintain a plan. Start a session in a project directory and it has the basic tools needed to investigate and change a codebase.

This is why the quickstart builds a tic-tac-toe game. In a few minutes, Goose creates the files and, after you add the Computer Controller extension, opens the result in a browser.

Goose can also run as an ACP server. ACP, or Agent Client Protocol, separates the agent runtime from its interface. An ACP-compatible editor can use Goose as the agent while Goose keeps its configured models and extensions.

That does not make Goose an IDE. The desktop app and CLI do not replace an editor's navigation, autocomplete, debugging, or visual review tools. Goose is the agent doing the work. You can run it beside an editor or connect it to one that supports ACP.

For repository work, I would give it the same project guidance I give any coding agent. An AGENTS.md file can state the build commands, paths, conventions, and boundaries it should follow. Open source at the runtime layer does not remove the need for clear project context.

Beyond code, you assemble the workflow

Goose calls its MCP servers extensions. MCP is the open protocol for connecting agents to tools and data; the MCP track explains the host, client, and server roles in detail. Goose's built-in catalog includes the Developer, Computer Controller, Memory, and Auto Visualiser extensions, and it can connect to external MCP servers for other systems.

A research setup might get access to a document directory and a browser. An operations setup might use an issue tracker, a database, and a reporting destination. Calling Goose general-purpose does not give the model those abilities automatically. You add them by choosing extensions.

Recipes make those choices repeatable. A recipe can package instructions, parameters, model settings, and required extensions in YAML or JSON. Goose can run recipes interactively, headlessly, or on a schedule.

This is what I mean by an agent you assemble yourself. You do not have to build the agent loop, but you do choose the parts around it:

Part	What you choose	What you still have to verify
Model	Hosted provider, private endpoint, or compatible local model	Tool use, quality, context, latency, and cost
Capabilities	Built-in tools and local or remote MCP extensions	Data access, credentials, dependencies, and permissions
Workflow	Instructions, recipes, parameters, retries, and schedules	Correctness, failure handling, and review points
Interface	Desktop, CLI, API, or ACP client	Which surface fits the people and task
Distribution	Upstream Goose or a custom distribution	Upgrades and the cost of maintaining changes

This path is less turnkey than coding. Goose gives you the runtime and extension system. It does not know which business process matters, which source is authoritative, or what a correct result looks like. You still have to design and test that workflow.

Bring your own model, with caveats

Goose does not bundle one mandatory model. Its provider guide covers hosted providers, cloud platforms, compatible private endpoints, subscription-backed agents, and local options. Goose connects to the provider and runs the agent loop around the selected model.

A coding team can test another model without replacing Goose. A broader workflow can use a model and endpoint that fit its cost, data, or deployment requirements.

Ollama is one way to run inference locally. With a compatible tool-calling model, Goose and the model can both run on your machine. Newer Goose desktop releases also provide built-in local inference, so Ollama is no longer the only local route.

Local inference does not automatically make the whole workflow private or offline. A remote MCP extension still receives whatever its tool call sends, and browser work still uses the network. The model also needs to fit the hardware and reliably make structured tool calls. Goose's documentation warns that models without tool calling can provide chat completion only when extensions are disabled.

The models are not interchangeable just because Goose can connect to them. Two models may accept the same prompt and expose the same tools while producing very different results. I would treat every provider change as a new agent configuration and rerun the task evaluation.

Where Goose fits

I would choose Goose when the work begins with code but does not end there. The same agent that changes code might query an issue tracker, inspect a design document, update release notes, or prepare a deployment checklist. It becomes more useful when a job crosses tool boundaries.

It is also a useful place to compare models against a reasonably stable runtime. Keep the instructions, extensions, inputs, and acceptance criteria fixed, then change the provider. That does not make the models interchangeable, but it makes their differences easier to see.

I would also consider it when I need control over the runtime or data path. Goose is an Apache 2.0 project governed at the Agentic AI Foundation, and it documents custom distributions. An organization can inspect the system and adapt it instead of waiting for a hosted product to add the option it needs.

Open source still does not eliminate operating work. A private fork needs maintenance. A local model needs hardware and evaluation. Extensions bring dependencies and credentials. The benefit is the ability to make those decisions, not the disappearance of their cost.

When I would look elsewhere

If I wanted a polished coding experience entirely inside an editor, I would first check whether Goose's ACP integration provides the interactions I rely on. The agent can connect to an editor, but the surrounding editor experience still comes from that client.

If I wanted a general assistant with almost no setup, I would also look elsewhere. Choosing a provider, enabling extensions, defining permissions, and turning a task into a reliable recipe are part of using Goose. A managed assistant makes more of those decisions for you.

The biggest concern is how much authority you give it. Goose can execute commands, change files, control a browser, and call external services. Its permission controls, prompt-injection detection, and adversary review can reduce risk, but they do not make every extension safe. I would start with the smallest tool set, least-privilege credentials, approval for consequential actions, and an output I can inspect.

The agent security chapter explains why untrusted tool output needs rules enforced in code, not only instructions in a prompt. That becomes more important as Goose moves beyond a repository and into other systems.

How I would try it

I started with the question of whether Goose was a coding agent or a general-purpose agent. I would test it in that same order.

First, I would use it for one bounded coding task in a disposable branch: inspect a small repository, make a change, run the tests, and explain the result. That exercises Goose's strongest default path and reveals how well the selected model uses its developer tools.

Then I would reuse the runtime for one bounded non-coding task. My choice would be a weekly project brief built from a local notes folder and one read-only issue-tracker extension. I would define the required sections and citations, keep approval enabled, and compare the result with the source material.

Only after the interactive task worked would I turn it into a recipe or schedule. That sequence separates three questions: can the model complete the task, have I given the agent the right tools, and is the workflow dependable enough to repeat without constant supervision?

Goose is a coding agent you can turn into something broader. That is its sweet spot, and also its cost. You get to choose the model, tools, and job, but you also have to assemble and test the result.

Agent security starts before the first prompt

Mangat Rai — Sun, 13 Sep 2026 23:47:16 +0000

An agent can cause damage before the model makes its first decision.

On September 1, 2026, Manifold described coding agents that invoked Git while gathering workspace context, allowing repository-local Git configuration to run a program on the host. Some affected agents performed that work before a user typed a prompt or approved workspace trust.

In Manifold's GitSpawn research, the relevant path was a repository received as files, with a .git directory inside a ZIP archive. Git read repository-local configuration while the agent gathered context. An ordinary Git clone, fetch, or pull does not transfer repository-local .git/config from the remote.

The model did not choose a harmful tool call. The runtime acted while preparing to use the model. That is why an agent review needs three boundaries: the runtime that starts processes, the gateway that holds credentials and routes requests, and the tool that makes a sensitive change.

Workspace trust has to include startup

Imagine a coding agent opening a project so it can answer a simple question about a failing test. Before the chat box is useful, the application may inspect the working tree, run Git to learn the branch and status, index files, load extensions, and collect credentials for the model gateway. None of that is a model-requested tool call. It is still code acting with the authority of the agent process.

Manifold reported eight findings across seven coding agents. Some affected paths ran before authentication or workspace approval. At publication, some issues had been patched; Manifold said four findings remained unpatched and withheld details for those paths.

For the coding agent in this example, I would make the startup contract explicit. It may read the checked-out project files. It may not inherit production cloud credentials, reach arbitrary internet hosts, or start programs named by workspace configuration. Opening an unfamiliar workspace happens in a reduced-permission environment. If the product asks for workspace trust before executing code, discovery and indexing need to honor the same boundary. Keeping production credentials out of its environment and filesystem limits what a missed startup hook can expose.

This is not a replacement for prompt-injection defenses. It covers authority that exists before the model is asked to reason.

The gateway is also outside the model loop

The same coding agent may send its prompts through a gateway. That gateway can hold provider keys, route requests, apply guardrails, and reach internal services. It is a useful policy point, but it is also a privileged service. Its authentication, administration, credentials, and network reach are security decisions made outside the model's visible tool loop.

Wiz's September 9 LiteLLM research gives that distinction some weight. In a February 2026 scan of 3,074 publicly exposed instances, Wiz found that 9.6% either accepted the default master key or required no authentication. That is a combined figure, not a claim that 9.6% used the default key, and it does not describe private deployments.

Wiz also reported an MCP authentication bypass, CVE-2026-59822, and a separate post-authentication code-execution issue in custom-code guardrails, CVE-2026-59821. The researchers wrote that the custom-code guardrail registration path lacked protections present in the test endpoint. Patches were available when Wiz published. A gateway deserves the same careful boundary design as any other service that holds keys and reaches sensitive systems.

For this agent, the gateway should reject a request without a valid credential before it routes anything. Its service identity should have only the provider access and internal destinations the task needs. Administrative and custom-code features should be limited to the small group that operates them. Keeping a gateway private, patching it with an owner and response window, and logging administrative changes are ordinary infrastructure controls. They matter here because a compromise could act without asking the model to call a tool.

Keep the last boundary at the tool

Now return to the coding agent after it has started safely and reached an authenticated gateway. The model examines a failing test and proposes a sensitive action: deploy a fix to the production environment.

That decision still needs a tool boundary. The deployment service can require an approved environment, a change identifier, and a human approval. The model can propose the action, but deterministic code decides whether the request fits the authority granted to this task. A model instruction such as “deploy it now” cannot substitute for those checks.

This is where tool allowlists and argument validation belong. They are strong controls for actions the model requests: writing outside an approved filesystem root, querying a restricted database schema, sending to an unapproved recipient, or deploying to an environment that needs review. They do not secure Git startup or gateway authentication because those events happen on different paths.

Review one agent you run

Follow one sensitive job from startup to its most consequential action. For the illustrative coding agent, the map might look like this:

Component	Capability	Credential or network reach	Enforcing control
Workspace runtime	Read and index project files	Gateway access token; no provider or production cloud keys; gateway and approved dependency endpoints only	Isolated workspace, restricted subprocesses, outbound allowlist
Model gateway	Route model requests and apply policy	Scoped provider credential; approved internal services	Authentication that fails closed, private exposure, patched release, least-privilege service identity
Deployment tool	Request a production deployment	Deployment API for the approved environment	Environment allowlist, change ID, human approval

The point is to name the authority each component actually has, then attach a control that can enforce it. A local learning project may need far less isolation than an unattended agent with production access.

One boundary test makes the map real. Send the gateway the same request the coding agent would make, but omit its gateway credential. The expected result is an authentication rejection before routing, with no provider request and no internal service access. A successful model response or evidence in downstream routing logs that the request was forwarded is a failure. Record the rejection and check downstream logs; an error message alone does not show whether the request was forwarded.

The Agent Security lesson demonstrates the tool-call side with an indirect prompt-injection lab. AI Gateways explains the application-owned routing boundary.

Record expected and actual results for one boundary test, then fix one gap before giving the agent more authority.

AGENTS.md: one guide for your coding agents

Mangat Rai — Wed, 09 Sep 2026 00:23:15 +0000

AGENTS.md is a Markdown file you keep in your repository to tell coding agents how to work on the project. It can explain where the code lives, which commands to run, and which decisions the agent should leave to you. The open format has no required fields or schema.

Imagine asking an agent to add a page on Monday. You explain the project's colors, where the source lives, and why the navigation must stay as it is. On Friday, a contributor picks up the work in another coding tool. Their agent proposes a new color scheme and reorganizes the navigation. The decisions are still in Monday's conversation, but they never made it into the repository.

That is the gap this file is meant to close. Write the ground rules down once, keep them with the code, and give the next agent a place to start.

The instructions should travel with the project

For an open-source maintainer, Friday's contributor may use a different coding tool. Keeping the instructions in the repository makes them available to both people; a shared filename gives their tools a common place to look. One contributor can use Codex and another Cursor without needing separate copies of the same build instructions.

That practical benefit is also the story behind the name. Amp initially used AGENT.md, singular. When OpenAI chose AGENTS.md, Amp agreed to switch if OpenAI secured the matching domain. On August 20, 2025, Amp announced the change. Sharing a standard mattered more than keeping its original filename.

The Linux Foundation dates the format's release to August 2025. By its December 9 announcement of the Agentic AI Foundation, it reported adoption by more than 60,000 open-source projects and agent frameworks. That is a dated adoption report, not a live count. AGENTS.md became one of the foundation's initial projects.

The convention gives those contributors a common starting point, although each tool still decides how to discover and apply the file. First, that shared file needs instructions worth carrying forward.

Write down the decisions the next agent needs

A useful file answers the questions that would otherwise interrupt the work: where to start, what to run, what to preserve, and how to know the change is ready.

For our imagined documentation project, “follow best practices” adds little. “Edit source files in site/; do not edit generated files in site/build/” settles a concrete decision. “Run the checks” is vague. Naming the commands and their working directory makes the instruction usable.

Four sections are a useful starting structure:

Project and layout. What the project does and the few directories an agent needs to understand.
Setup and verification. Exact commands, where to run them, and any prerequisites.
Conventions and boundaries. Existing patterns to reuse, files to preserve, and actions that need approval.
Completion. What evidence to report and how to describe anything left untested.

For a small repository, I would start with roughly 30–60 short lines. That is my editing budget, not a format limit. If ten lines cover the important decisions, stop there. Add a rule when you can name the recurring mistake it should prevent.

Here is an illustrative, shortened file for the website portion of Few-Shot Academy. The paths and commands come from this repository; the example is not a replacement for our full contributor instructions.

A Sample AGENTS.md

# Project instructions

Few-Shot Academy is a free GenAI curriculum. Write for readers
who may have no programming or AI background.

## Project layout
- `site/docs/`: curriculum pages.
- `site/blog/`: standalone practitioner articles.
- `site/src/`: shared components and styles.
- `site/build/`: generated output; edit the source instead.

## Setup and checks
- Use Node.js 22 or newer.
- From `site/`, run `npm ci` to install dependencies.
- After site changes, run these from `site/`:
  - `npm run typecheck`
  - `npm run build`
- For visual changes, check mobile layout, keyboard navigation,
  and both light and dark themes.

## Working rules
- Read the relevant page and nearby examples before editing.
- Reuse design tokens in `site/src/css/custom.css`.
- Preserve published URLs.
- Verify factual claims against primary sources.
- Keep changes focused and preserve unrelated work.
- Ask before pushing, merging, or deploying.

## Before finishing
- Summarize the change and the checks that passed.
- State what you could not verify and why.

The next agent now has somewhere to find the source paths, design rules, and checks that were stranded in Monday's conversation. Replace the example's paths and commands with ones you have verified in your own repository.

These are still instructions the model receives, so check its work. “Do not deploy” is useful guidance; actual deployment access belongs in permissions and approval controls. Anthropic's documentation explicitly distinguishes instruction files from enforced configuration.

Make sure your tool reads it

With the file written, the next step is to connect it to the agent. Some tools recognize AGENTS.md directly; others need a small configuration change.

This is a selection of coding editors and agents, not a popularity ranking. The table reflects their official documentation checked on September 8, 2026. “Automatic” means the tool recognizes the file without a filename bridge; settings can still affect loading.

Tool	Root `AGENTS.md`	Details or setup
Codex	Automatic	Combines guidance along the root-to-working-directory path. An `AGENTS.override.md` takes priority over `AGENTS.md` in the same directory.
Cursor Agent	Automatic	Also applies nested files to their directory and children. More specific instructions take precedence.
GitHub Copilot in VS Code	Automatic	Root loading is enabled by default. Nested discovery is a separate experimental opt-in; see the settings reference.
Cascade (Windsurf / Devin Desktop)	Automatic	Root instructions are always on; subdirectory files apply to that part of the project.
Amp	Automatic	Discovers instructions in the working directory, parents, and relevant subdirectories.
Cline	Automatic	Recognizes `AGENTS.md`; check the Rules panel for whether a detected rule is enabled.
Gemini CLI	Configure filename	Defaults to `GEMINI.md`. Set `context.fileName` to include `AGENTS.md`.
Claude Code	Import or symlink	Reads `CLAUDE.md`. Import the shared file as shown below.

The Copilot row is specifically about VS Code. Do not assume that an editor extension, a command-line agent, and a hosted pull-request agent load instructions identically just because they share a product name.

Connecting Claude Code and Gemini CLI

The last two rows need setup, but you can keep the shared rules in AGENTS.md. For Claude Code, put this line in the root CLAUDE.md:

@AGENTS.md

Copy the line without the surrounding code fences. That is the arrangement in this repository. Claude's documented import loads the shared instructions at session start. Keep any existing Claude-specific guidance below it. A root import does not by itself wire up every nested AGENTS.md.

A symlink is another supported option: ln -s AGENTS.md CLAUDE.md on macOS or Linux, when CLAUDE.md does not already exist. I prefer the import because it is a plain-text edit on macOS, Linux, and Windows. Anthropic also recommends it on Windows, where creating symlinks can require extra privileges.

For Gemini CLI, merge this into .gemini/settings.json, preserving your other settings:

{
  "context": {
    "fileName": ["AGENTS.md", "GEMINI.md"]
  }
}

Gemini's context configuration supports multiple filenames. Keeping both lets existing Gemini-specific guidance remain discoverable; avoid repeating the shared rules in both files.

Let the setup grow with the work

Once the short file is written and your tool is set up to read it, you have a starting point for the next session. Over time, you may also want to preserve architecture explanations, design reasoning, or the state of unfinished work. Those do not all belong in the everyday instructions.

Separate architecture, design, and decision notes can preserve reasoning the code cannot explain. In our Monday-to-Friday example, “use these colors” records a rule. “We kept the existing palette so new pages match the rest of the site” records why it exists.

But five overlapping summaries give you five places to forget an update. Choose a pattern that solves a problem you actually have:

Pattern	When it helps	What to watch
Short root file with links to project docs	Architecture or design explanations are too long for everyday instructions.	Say when to read each document; a link alone does not guarantee it gets read.
Instructions scoped to a directory	Different packages need different checks or conventions.	Confirm the tool's nesting rules. Avoid copying root rules into every package.
A temporary handoff note	A task spans sessions or pauses halfway through.	Record the current state, unresolved questions, and next step; replace outdated notes when work moves on.

The first two have direct support in tools such as Cursor, which documents references and nested instructions. For handoffs, Anthropic describes using progress files alongside Git history.

Splitting files does not create more memory

The context window, the material a model can use at once, is finite. Summarizing a long conversation helps it continue, but that summary can miss details the next session needs.

Tools can also limit how much they load. Codex has a 32 KiB default limit on the combined project instructions it loads, but that is a ceiling, not a target.

The loading strategy matters more than the file count. Claude Code's @ imports load the referenced content at launch. Splitting one large file into five and importing all five still puts that material into context. To keep startup context small, keep the root guidance brief and instruct the agent to read the relevant supporting document when the task calls for it.

A “resume this project” prompt can tell the agent to read a handoff; it cannot recover details nobody saved. Update the relevant document when a decision changes. Before pausing, record unfinished work, checks performed, and the next step. Review those updates in the diff instead of assuming the agent made them.

My default is one short AGENTS.md, existing project docs for durable decisions, and a handoff note only when there is work to resume. Add files when they remove confusion.

Try the next handoff

Return to Friday's contributor. The useful test is whether the saved instructions help them pick up the work. Start a fresh session after changing the setup and try a small page edit. Did the agent find the source, use the existing design, run the right checks, and report anything it could not verify?

If it misses a rule, check which files loaded and whether their instructions conflict before adding another paragraph. The goal is to stop re-explaining the same decisions, while keeping those decisions easy to find and maintain.

GPT-6 Astra: the harness is the product

Mangat Rai — Sun, 06 Sep 2026 00:08:52 +0000

GPT-6 Astra does not prove that OpenAI has reached AGI. What it does prove is that the harness can no longer be treated as plumbing.

Astra scored 54.8% on ARC-AGI-3 at high reasoning in the standard harness. The same model scored 99.9% when the harness preserved its reasoning state and compacted long conversations. That is a 45.1-point difference without changing the model.

My takeaway is simple: if you evaluate only the model name, you are evaluating the wrong product. For long-running agent work, the product is the model, memory, tools, context management, and control loop together.

That is why Astra matters even if you are not interested in arguing about AGI.

The AGI claim is ahead of the evidence

OpenAI President Greg Brockman believes Astra qualifies as artificial general intelligence. During the launch briefing, he left the final judgment to others but said, “I think we're there,” according to The Washington Post. OpenAI's Charter defines AGI as highly autonomous systems that outperform humans at most economically valuable work. No launch benchmark can establish that on its own.

Astra gives OpenAI a serious case to make. The company reports 98% on FrontierMath Tier 4, 99.9% on ARC-AGI-3, and large gains in computer use and coding. OpenAI is positioning Astra to work inside browsers, development tools, spreadsheets, and other software with less supervision.

I would not call it “pre-AGI” either. That avoids the argument without giving us anything measurable. Astra is a meaningful step toward a general-purpose digital worker, but we do not yet have evidence that it can reliably outperform people across most economically valuable work.

ARC Prize calls Astra a major step forward while stating plainly that saturating ARC-AGI-3 is not proof of AGI. The benchmark uses bounded environments with deterministic rules. Real work is messier and carries consequences when the agent gets something wrong.

The biggest Astra number is really a harness result

An agent harness is the software around the model. It supplies tools, carries context forward, manages retries, records state, and decides when the model gets another turn.

ARC Prize tested Astra through two harnesses:

Evaluation setup	Reasoning effort	Score	Evaluation cost
Standard, provider-neutral harness	Max	62.7%	$26,098
Standard, provider-neutral harness	High	54.8%	$40,705
OpenAI Provider Adapter harness	High	99.9%	$18,817

The standard harness gives the model a common interface and lets it decide what to preserve in visible notes. The OpenAI adapter also keeps Astra's opaque reasoning state between requests and uses compaction to manage long conversations.

At the same high reasoning setting, the adapter moved the score from 54.8% to 99.9%, according to ARC Prize's analysis.

The 99.9% result is valid. It is just not a property of the model in isolation. It is the result of Astra operating inside the runtime OpenAI designed for it.

The neutral harness is better for comparing models under the same conditions. The native harness measures the system you might actually deploy. One score cannot answer both questions.

The new context system is the feature I would watch

Astra has a 1,050,000-token context window, but a coding agent can still fill it with source files, test output, logs, and failed approaches. Eventually, something has to be removed or compressed.

Compaction turns the old conversation into a shorter summary. That summary preserves what looked important at the time. A constraint, test failure, or rejected approach can disappear and become important again later.

With Astra, OpenAI says Codex can now keep notes across context windows and search earlier windows. If a detail did not make it into the notes, the agent can retrieve the earlier message or tool result instead of depending entirely on a chain of summaries.

Consider a repository migration. The agent learns early that one service must stay compatible with an older client, then spends an hour inspecting unrelated code and running tests. When it finally edits that service, searchable history gives it another way to recover the compatibility requirement instead of hoping the detail survived compaction.

This could make a long-running agent more dependable, but it needs real testing. The Codex feature is experimental at launch, and we do not yet know how reliably Astra will find the right old detail across large, messy repositories.

Astra can keep working while the task changes

Two new API capabilities support the same idea.

Async tool calling lets Astra start a function or custom tool and continue with independent work while the application runs it. An agent can begin a slow dependency scan, inspect configuration files while it waits, and use the scan after it returns. Your application still runs the job and has to return the result with the correct call ID.

Mid-turn steering lets a user change requirements while Astra is working over a WebSocket connection. You can tell the migration agent to keep the project within one engineer-week without discarding the research it already completed.

That sounds like a small interface improvement. In practice, it changes the agent from a request-response system into a process you can supervise while it runs.

The tradeoff is control. Steering does not undo an action that already happened or cancel a tool already in flight. Async tools can also finish in an unexpected order. If two of them write to the same system, the application needs isolation, idempotency, and a clear approval policy. A more capable model does not remove those engineering responsibilities.

More autonomy also raises the cost of a mistake

OpenAI calls Astra its most aligned model and reports better respect for task boundaries. Its safety review also found that Astra can sometimes evade internal monitors when adversarial tests explicitly tell it to do so. OpenAI says this came from adversarial evaluation rather than normal use, but the result is serious enough that the company is developing monitoring methods beyond reading the model's written reasoning.

Astra is also OpenAI's first broadly deployed model to reach its Critical cybersecurity capability threshold. OpenAI says that, with the right tools and access, Astra can find previously unknown vulnerabilities and develop exploits against hardened systems without a person directing each step. Less-restricted access for advanced cyber work is limited to vetted participants in the Daybreak program.

This is the real tradeoff. Better memory, more tools, and longer autonomy make the system more useful. They also give it more time and more access to act on a bad assumption before someone catches it.

Run two evaluations, not one

I would evaluate Astra in two passes.

First, use a provider-neutral harness with the same prompts, tools, retry limits, and grading criteria you use for other models. That tells you whether Astra itself improves the work you care about.

Then test the native OpenAI harness with persisted reasoning, compaction, searchable context, async tools, and steering enabled where appropriate. That tells you whether the complete Astra system is worth deploying.

Use 20 to 50 real tasks, not polished demos. For long tasks, force at least one compaction and plant an early requirement that matters near the end. Interrupt a run with a changed constraint. Delay tool results and return them out of order. Put an approval boundary in front of an irreversible action. Measure accepted results, recovery from mistakes, latency, and total cost.

The last point matters because Astra costs $10 per million input tokens and $50 per million output tokens, before tool fees. If your application needs short answers or straightforward extraction, these harness features may not justify that price. If it needs sustained work across tools and changing requirements, token price alone is a poor way to judge it.

By the way, this was a crowded release week

Anthropic released Claude Fable 5.1 for long-running coding and knowledge work. Meta released Muse Spark 1.3, with a focus on long-horizon collaboration, interruptions, and coding efficiency. Google released Gemini 3.8 Flash as a lower-cost agentic reasoning and coding model.

All three belong on an evaluation list if they fit your workload. They do not change my conclusion about Astra: its most important advance is not the AGI label or the model name. It is the tighter integration between the model and the system that keeps it working.

My recommendation is to stop treating the harness as an implementation detail. Compare models with a neutral harness, then evaluate the native system you would actually deploy. The evaluation chapter shows how to build the task set. Let the AGI debate continue somewhere else.