DEV Community

Shawn knight
Shawn knight

Posted on Originally published at Medium on

2026 Agents Case Study Series: It’s Just a Conversation

I don’t talk to coding agents very differently than I talk to chatbots.

I explain what I am trying to accomplish, answer questions, and challenge assumptions.

I’ll ask for an analysis, a recommendation, or a plan.

I decide whether the proposed direction makes sense, and then I continue the conversation.

That description simplifies quite a bit, but it also points toward something important.

From my side of the interaction, ChatGPT, Claude, Gemini, Codex, and Claude Code can all feel conversational.

It’s not that one system talks while another acts intelligently.

The difference is what the conversation is connected to.

Knowing About a Project Is Not One Thing

We often say an AI system “has access” to a project as if access were binary.

It isn’t.

There are several materially different ways an AI system can receive information about a project and interact with the environment around it.

1. Described Context

The model knows what I tell it.

I might explain the architecture, paste an error, summarize a feature, or describe what I believe a component does.

That can be enough for planning and conceptual guidance.

But the model also inherits the omissions and misunderstandings in my description.

2. Supplied Context

I may upload a document, paste source code, or attach an analysis report.

The model can reason from more direct evidence, but only from what I’ve supplied.

The missing file may still contain the answer.

3. Retrieved Context

A search system, connector, or retrieval pipeline may locate material related to the question and make it available to the model.

Retrieval-augmented generation, or RAG, is useful because it grounds responses in information retrieved from external sources. Google’s Machine Learning Glossary explains the approach.

That can improve accuracy and provide access to private or current information.

But retrieval is selective.

The system still has to determine what appears relevant before the model can reason over it.

3. Retrieved Context

The AI can actively explore an environment as its questions evolve.

It can list a repository, search for a symbol, inspect a function’s callers, compare implementations, or examine configuration files.

Instead of depending entirely on the information supplied in the initial prompt, it can investigate further when something doesn’t make sense.

5. Executable Context

The AI can run code, invoke tests, reproduce errors, inspect output, and compare actual behavior with its hypotheses.

That introduces a form of evidence that static reading alone cannot provide.

  1. Mutable Context

The AI can change the system and observe the result.

It can edit a file, run a test, inspect the diff, and revise the implementation when the evidence contradicts the plan.

These aren’t rankings. They’re observations about different capabilities.

The first three primarily describe how information reaches the model. The last three describe what the AI can do within an environment.

And more access isn’t automatically better.

A read-only architecture review doesn’t necessarily need permission to modify the codebase. A conceptual discussion may not need repository access at all.

The point is that “the AI has context” doesn’t tell us enough.

We need to know what kind of context it has and what relationship it can form with that context.

Agents Are Not Omniscient Either

Repository access doesn’t mean total understanding.

An agent can search for the wrong phrase, stop after finding the first plausible implementation, or misunderstand which path is canonical.

It can overlook generated code, configuration, migrations, or runtime state.

It can infer intention from an implementation that is itself wrong.

It can run tests that pass while failing to cover the behavior that matters.

And it can modify one subsystem without recognizing an undocumented dependency elsewhere.

Its environment is also bounded.

Permissions determine which files it can read or change.

Network policies determine which external systems it can contact.

The checked-out branch determines which version of the codebase it sees.

Installed dependencies determine what it can execute.

And time and context limits affect how much it can investigate.

The agent doesn’t see the entire system simply because it has access to a repository.

It has a better opportunity to investigate the parts of the system within its reach.

That’s a significant difference.

But it isn’t magical omniscience.

The Conversation Changes When It Can Act

The most important capability isn’t necessarily any single tool.

It’s the feedback loop created when reasoning is connected to tools and consequences:

Inspect → Hypothesize → Act → Test → Observe → Revise

The difference is recursive.

In a conversation without execution tools, an AI can tell me what command I should run.

With the right tools and permissions, it can run that command, receive the output, and use the evidence to decide what to investigate next.

It can explain how to fix a failing test.

Or it can apply a candidate fix, run the test, observe another failure, and revise the implementation.

It can suggest that two modules may overlap.

Or it can search the repository, inspect their callers, and determine which implementation is actually being used.

The distinction isn’t strictly between chatbots and agents as product categories. A conversational interface can have tools, and an agent can operate through a conversation.

What matters is whether the system can act on its reasoning and observe the consequences.

The conversation becomes connected to the environment through repeated observation.

And that changes the working relationship.

This Changes the Human Role Too

Before working with coding agents, I carried most of the actions between the AI’s reasoning and the development environment.

If ChatGPT suggested a change, I made it.

If it needed an error message, I reproduced the failure.

If it needed to know whether something worked, I ran the application and reported what happened.

With coding agents, I increasingly moved toward directing the investigation and evaluating the evidence.

That didn’t remove my responsibility.

It changed where my effort belonged.

My questions became:

  • Is the agent investigating the right system?
  • Does it have the evidence required to support its conclusions?
  • Which assumptions came from the code, and which came from the prompt?
  • Did the test actually validate the intended behavior?
  • Does this implementation preserve the project’s architectural direction?
  • Should the agent have permission to perform the next action?
  • What must remain a human decision?

The more capable the environment became, the more important governance became.

An AI system that can edit, execute, publish, deploy, or communicate with external services isn’t merely more convenient than one limited to providing guidance.

It has a different action surface.

That surface needs boundaries, tests, approvals, logs, protected branches, and explicit ownership.

Capability without governance doesn’t become trust.

It becomes exposure.

The Model Is Only Part of the System

People often compare AI products as if the model alone determines the result.

My experience working with AI has suggested a larger unit of analysis:

Model + Context + Tools + Permissions + Environment + Feedback Loop

The same model can behave very differently when it receives one pasted error versus an entire repository.

Again when it can search that repository.

Again when it can execute the failing code.

Again when it can edit the implementation.

Again when a test gate controls whether the work may continue.

And again when the repository contains clear specifications, architectural boundaries, and durable instructions.

The intelligence matters.

So does the environment through which that intelligence interacts with the system.

That’s why evaluating “which AI is best” without describing the surrounding environment often produces weak conclusions.

One system may appear smarter because it was given better context.

Another may appear more reliable because it could verify its answer.

Another may appear more productive because its product design introduced useful friction before execution.

The visible result belongs to the whole system, not only the model that generated the response.

It Was Still Just a Conversation

I use the word “just” carefully.

A conversation connected to a repository, shell, test suite, version history, deployment environment, and permission system isn’t a small thing.

But the conversational continuity matters.

I didn’t become a fundamentally different person when I moved from ChatGPT to Codex or from Claude to Claude Code.

I still reasoned through questions and made decisions through dialogue.

I still challenged recommendations and asked what followed from the answer.

The interaction remained conversational.

What changed was that the conversation could reach the system it was discussing.

What’s Next?

If you’re new to the series, you can catch up on the story so far:

By this point, I’d learned that the capabilities of an AI system depended on more than the model itself. Context, tools, permissions, and the surrounding environment all influenced what it could accomplish.

But I was still learning what it meant to manage the work those systems produced.

And at the time, I had another goal.

I thought I could finish Nodus by getting it to Version 1.

In the next article, I’ll explore what happened when I tried to reach that milestone in a week, why I started using Claude Code alongside ChatGPT and Codex, and how the experience began turning testing requirements and development constraints into enforceable engineering practices.

The next installment of the 2026 Agents Case Study Series is coming soon.

Explore the Projects

The systems discussed throughout this series are part of the Masterplan Infinite Weave ecosystem. While not every project has appeared in the story yet, you can explore the technology, examine the source code, and follow its ongoing development.

  • Nodus — The programming language that began as the experiment documented in this series. Explore Nodus
  • A.I.N.D.Y. Runtime — The governed execution substrate supporting workflow orchestration, persistent execution, memory, and agent coordination. Explore A.I.N.D.Y. Runtime
  • A.I.N.D.Y. Platform — The application platform built on the runtime, bringing the underlying capabilities into usable applications and interfaces. Explore A.I.N.D.Y. Platform
  • Masterplan Infinite Weave — The broader ecosystem connecting these projects and their ongoing development.

Previous Work

This series builds on the ideas, experiments, and lessons documented in my earlier work.

  • 2025 ChatGPT Case Study Series — The original exploration of learning, building, and executing through sustained human-AI collaboration. Read the Series
  • 2025 Duality of Progress Series — An exploration of the relationships between constraints, creativity, iteration, and progress. Read the Series

Originally published at https://www.linkedin.com.


Top comments (0)