DEV Community: Chetan Sehgal

DeepSeek-V4 Changes the Context Game for Agents — And Your Memory Architecture Should Adapt

Chetan Sehgal — Tue, 28 Apr 2026 08:14:47 +0000

A million-token context window built specifically for agentic workloads. That's the feature in DeepSeek-V4 that stopped me mid-scroll this week — not because big context windows are new, but because this one is engineered for the exact failure mode that plagues every serious agent builder right now.

The Duct Tape Era of Agent Memory

Let's be honest about the state of agent architectures in 2026. Most production agents are held together with aggressive summarization, chunked context windows, and RAG pipelines that were originally designed for search, not for multi-step reasoning.

These patterns exist because we've been building agents under a hard constraint: 128K tokens, sometimes 200K if you're lucky. When your agent needs to reason across an entire codebase, navigate a 400-page contract set, or execute a multi-step plan spanning hundreds of tool calls, you hit that ceiling fast. So you compress. You summarize. You retrieve fragments and hope the model can reconstruct enough coherence to make good decisions.

It works — until it doesn't. And when it fails, it fails silently. The agent confidently acts on incomplete context, makes decisions based on lossy summaries, or retrieves the wrong chunk because the embedding similarity didn't capture the actual semantic dependency. You don't get an error message. You get a subtly wrong output that takes hours to debug.

What DeepSeek-V4 Actually Offers

DeepSeek-V4 ships with a native million-token context window that, according to Hugging Face's technical breakdown, is specifically optimized for agentic workloads. This isn't just a bigger number on a spec sheet. The architecture is designed to maintain reasoning coherence across the full window — meaning the model doesn't degrade catastrophically at token 900K the way many extended-context models do.

For agent builders, this changes the design calculus in a concrete way:

Full codebase reasoning: Instead of chunking a repository into fragments and hoping RAG retrieves the right file, you can feed the agent the entire codebase. It can trace dependencies, understand architectural patterns, and reason about cross-file implications natively.
End-to-end plan execution: Multi-step agents that make hundreds of tool calls can maintain their full execution history in context. No more summarizing previous steps and losing the nuance of why a particular decision was made.
Document-heavy workflows: Legal contracts, technical specifications, regulatory filings — domains where missing a clause on page 312 because it wasn't in your top-k retrieval results can be catastrophic.

This Doesn't Kill RAG — But It Reframes It

I'm not arguing that retrieval-augmented generation is dead. RAG still wins when your corpus is genuinely massive — tens of millions of tokens, entire knowledge bases, continuously updated data streams. You can't fit Wikipedia into a context window, and you shouldn't try.

But here's the reframe: RAG should be a scaling strategy, not a coping mechanism. Too many agent architectures use retrieval because the context window is too small, not because retrieval is the right abstraction for the problem. When your entire relevant context fits within a million tokens — and for a surprising number of real-world agent tasks, it does — native context is simpler, more reliable, and produces better reasoning.

The engineering complexity you save is significant. No embedding pipeline to maintain. No chunk-size tuning. No re-ranking layer to debug. No retrieval failures to handle gracefully. You replace an entire subsystem with a longer prompt.

The Benchmark You Should Run

If you're building or refining an agent memory system right now, here's what I'd actually do: take your current RAG-augmented agent, take the same task, and run it with the full context stuffed into DeepSeek-V4's window. Compare output quality, reasoning coherence, and — critically — failure modes. You might find that the simpler architecture wins outright for your use case.

Sometimes the best engineering decision is removing a system, not adding one.

Key Takeaways

Million-token native context changes the design calculus for agents — many tasks that currently require RAG or aggressive summarization can now be handled with full-context reasoning, reducing architectural complexity and silent failure modes.
RAG should be a scaling strategy, not a default — if your relevant context fits within a million tokens, benchmark native context before adding retrieval layers. Simpler architectures are easier to debug and often produce better results.
Test your assumptions empirically — run your current agent pipeline against a full-context baseline on DeepSeek-V4. The results might justify ripping out infrastructure you assumed was necessary.

If you're designing agent memory systems today, benchmark against million-token native context before reflexively reaching for retrieval. What agent architecture decisions would you revisit with a reliable million-token window?

Generalist Reasoning vs Scoped Autonomy: Why Claude Opus 4.7 and OpenAI Codex Aren't Competing — and Why That Should Change How You Build

Chetan Sehgal — Mon, 20 Apr 2026 08:17:00 +0000

Claude Opus 4.7 and OpenAI Codex aren't competing. They're answering completely different questions about what AI should do — and if you're treating them as interchangeable options on a leaderboard, you're already making the wrong architectural decision.

This distinction — generalist reasoning vs scoped autonomy — is the most consequential fork in AI tooling right now. Getting it wrong doesn't just cost you performance. It costs you months of building on the wrong abstraction.

Two Models, Two Philosophies

Claude Opus 4.7 represents Anthropic's bet on depth of thought. Extended thinking chains, sophisticated ambiguity handling, multi-turn conversational nuance — this is a model designed to sit with hard problems. It doesn't just retrieve answers; it reasons through uncertainty, weighs competing interpretations, and synthesizes across domains. When you throw it a messy research question or a legal scenario with conflicting precedents, it behaves more like an expert analyst than a lookup engine.

OpenAI Codex, by contrast, is built around a fundamentally different loop: constrained, sandboxed execution. It doesn't aspire to be the smartest thinker in the room. It aspires to ship pull requests. Codex operates in a tightly scoped environment — read the spec, write the code, run the tests, open the PR. End to end. Its power comes not from reasoning breadth but from reliable, bounded task completion within a well-defined domain.

These aren't two flavors of the same thing. They're two different answers to two different questions:

Opus 4.7 asks: "What should we think about this?"
Codex asks: "What should we do about this?"

Why This Fork Matters for Your AI Stack

If you're building an agent architecture in 2026, the first decision isn't which model to use. It's what type of intelligence your workflow demands.

Consider these scenarios:

Research synthesis — You're ingesting 40 papers on a niche biotech topic and need a coherent analysis that surfaces contradictions and knowledge gaps. This is a reasoning problem. You need a model that handles ambiguity gracefully, maintains context across long interactions, and produces judgment, not just summaries. Opus 4.7's extended thinking is purpose-built for this.
Automated code migration — You're converting a legacy Python 2 codebase to Python 3 across 200 files, with test validation at each step. This is an execution problem. You need a model that stays in lane, operates deterministically within a sandbox, and integrates into your CI/CD pipeline. Codex's scoped autonomy is exactly right here.
Complex planning under uncertainty — Designing a multi-phase go-to-market strategy where market conditions are ambiguous and trade-offs are real. Generalist reasoning wins.
Repetitive, well-specified engineering tasks — Generating API endpoints from an OpenAPI spec, writing unit tests against existing contracts. Scoped autonomy wins.

The mistake most teams make is benchmarking these head-to-head on the same task and picking the winner. That's like comparing a surgeon and a paramedic on the same rubric — they're optimized for different moments in the same pipeline.

The Architectural Implication

The smarter play is to stop treating model selection as a single choice and start treating it as a routing decision. Your agent orchestration layer should be asking, for each subtask: does this require judgment or execution?

This is where the industry is heading. We're moving past monolithic model selection toward composite architectures where different models handle different cognitive modes. A reasoning model sits at the planning layer. An execution model sits at the action layer. The orchestrator decides who gets the ball.

If you're building with the OpenAI Agents SDK or similar frameworks, this routing logic is where your real competitive advantage lives — not in which model you picked.

Key Takeaways

Generalist reasoning models like Opus 4.7 excel at ambiguity, synthesis, and judgment — use them where the problem is underspecified and the output is insight, not action.
Scoped autonomy models like Codex excel at bounded, reliable execution — use them where the spec is clear and the output is a shipped artifact.
The real architectural decision isn't which model is better — it's building a routing layer that matches cognitive mode to task type. This is the unlock most teams are missing.

The Question That Shapes Everything

Stop asking "which model is better." That question assumes a single axis of comparison that doesn't exist.

Start asking: what type of intelligence does my workflow actually need at each step?

That single question will reshape your agent architecture from the ground up. And right now, it's the question that separates teams shipping real AI systems from teams still stuck on leaderboard debates.

Generalist Reasoning vs Scoped Autonomy: Why Claude Opus 4.7 and OpenAI Codex Aren't Competing — and Why That Should Change How You Build

Chetan Sehgal — Mon, 20 Apr 2026 08:09:50 +0000

Two Models, Two Philosophies

These aren't two flavors of the same thing. They're two different answers to two different questions:

Opus 4.7 asks: "What should we think about this?"
Codex asks: "What should we do about this?"

Why This Fork Matters for Your AI Stack

If you're building an agent architecture in 2026, the first decision isn't which model to use. It's what type of intelligence your workflow demands.

Consider these scenarios:

Research synthesis — You're ingesting 40 papers on a niche biotech topic and need a coherent analysis that surfaces contradictions and knowledge gaps. This is a reasoning problem. You need a model that handles ambiguity gracefully, maintains context across long interactions, and produces judgment, not just summaries. Opus 4.7's extended thinking is purpose-built for this.
Automated code migration — You're converting a legacy Python 2 codebase to Python 3 across 200 files, with test validation at each step. This is an execution problem. You need a model that stays in lane, operates deterministically within a sandbox, and integrates into your CI/CD pipeline. Codex's scoped autonomy is exactly right here.
Complex planning under uncertainty — Designing a multi-phase go-to-market strategy where market conditions are ambiguous and trade-offs are real. Generalist reasoning wins.
Repetitive, well-specified engineering tasks — Generating API endpoints from an OpenAPI spec, writing unit tests against existing contracts. Scoped autonomy wins.

The Architectural Implication

If you're building with the OpenAI Agents SDK or similar frameworks, this routing logic is where your real competitive advantage lives — not in which model you picked.

Key Takeaways

Generalist reasoning models like Opus 4.7 excel at ambiguity, synthesis, and judgment — use them where the problem is underspecified and the output is insight, not action.
Scoped autonomy models like Codex excel at bounded, reliable execution — use them where the spec is clear and the output is a shipped artifact.
The real architectural decision isn't which model is better — it's building a routing layer that matches cognitive mode to task type. This is the unlock most teams are missing.

The Question That Shapes Everything

Stop asking "which model is better." That question assumes a single axis of comparison that doesn't exist.

Start asking: what type of intelligence does my workflow actually need at each step?

Software Engineers Are Building Agents Wrong: Treat Agentic AI Like Distributed Systems, Not Prompt Chains

Chetan Sehgal — Wed, 15 Apr 2026 08:20:19 +0000

Anthropic just shipped Managed Agents. Claude Cowork is GA. OpenAI is pushing deeper into agentic workflows. Every major lab is converging on the same thesis: the future isn't chat — it's agents that do things.

And most engineering teams are going to botch the implementation.

The Problem: Bolting Agents Onto Codebases That Were Never Designed for Them

Here's what I keep seeing. A team gets excited about agentic AI. They wire up a few LLM calls with some glue code. Maybe they use LangChain or a lightweight orchestration framework. The demo works. The PM is thrilled. Then it hits staging, and everything falls apart in ways nobody anticipated.

An agent silently hallucinates a malformed JSON payload and the downstream step swallows it. A retry loop burns through $200 in API calls because nobody set a boundary. An agent "succeeds" but produces a subtly wrong result, and there's no trace to reconstruct why it made the decision it did.

This isn't a model problem. It's an engineering problem. And specifically, it's the problem you get when you treat agentic systems like fancy scripts instead of what they actually are: distributed systems with non-deterministic components.

Agents Are Distributed Systems — Engineer Them That Way

If you've ever built microservices, you already know the playbook. You define contracts between services. You handle partial failures gracefully. You make operations idempotent so retries don't corrupt state. You instrument everything.

Agentic AI demands every single one of these disciplines, arguably more, because the components themselves are stochastic. When a traditional microservice fails, it usually fails loudly — a 500 error, a timeout, a schema violation. When an LLM agent fails, it often fails quietly. It returns confident, well-formatted, completely wrong output. Your system happily passes that output to the next step, and the error compounds.

This is why the engineers winning at agentic AI aren't the ones chasing every model drop and benchmarking GPT-5 against Claude 4. They're the ones building engineering primitives around these models:

Typed input/output schemas between every agent step. Not loose JSON blobs — actual validated contracts. If an agent's output doesn't conform to the expected schema, the pipeline should halt, not silently proceed. Tools like Pydantic, Zod, or even simple JSON Schema validation are non-negotiable here.
Explicit retry boundaries and circuit breakers. Every agent call needs a maximum retry count, a cost ceiling, and a fallback strategy. Without these, a single confused agent can trigger runaway loops that drain your API budget or, worse, take irreversible actions.
Human-in-the-loop checkpoints as a first-class design choice. Not an afterthought bolted on when something goes wrong in production. Build kill switches and approval gates into the orchestration layer from day one. High-stakes steps — anything involving external APIs, financial transactions, or data mutations — should require explicit human confirmation until you've earned enough trust in the pipeline to relax that constraint.
Full observability of intermediate reasoning. Logging only the final output of an agent chain is like logging only the HTTP response of a distributed transaction. When things go wrong (and they will), you need the full trace: every prompt sent, every intermediate response, every decision point. This is how you debug the subtle failures that plague agentic systems.

Why This Gap Exists

The tooling ecosystem hasn't caught up yet. Most agent frameworks optimize for time-to-demo, not time-to-production. They make it trivially easy to chain LLM calls together and painfully hard to add the guardrails that production systems require. Anthropic's Managed Agents and similar offerings from other labs are starting to address this, but the fundamental responsibility still falls on the engineering team.

There's also a skills gap. Many of the engineers most excited about AI agents come from ML or data science backgrounds — brilliant at model selection and prompt engineering, less experienced with the distributed systems patterns that make these architectures reliable. And many seasoned backend engineers haven't yet internalized that non-deterministic components require even stricter engineering discipline, not less.

Key Takeaways

The bottleneck in agentic AI isn't model capability — it's the engineering discipline surrounding the models. Treat agent orchestration with the same rigor you'd apply to any distributed system.
Silent failures are the defining risk of agentic systems. Typed schemas, observability on every intermediate step, and human-in-the-loop checkpoints are your primary defenses.
Build for production from the start. Kill switches, retry boundaries, cost ceilings, and full reasoning traces aren't nice-to-haves — they're the difference between a compelling demo and a system you can actually trust.

Over to You

The gap between "agent demo" and "agent in production" is where most teams stall out right now. The patterns to close that gap already exist — they're just borrowed from distributed systems, not from AI research papers.

What's the hardest agentic failure mode you've had to debug? I'm especially curious about the silent ones — the failures that looked like successes until they didn't. Share your war stories.

AI Agents That Learn on the Job: Why On-the-Fly Evolution Changes Everything About Agent Architecture

Chetan Sehgal — Wed, 08 Apr 2026 17:09:35 +0000

Most AI agents shipped today are frozen the moment they hit production. They execute. They respond. But they don't get better from doing the work.

This is the dirty secret of the current agent boom: for all the hype about autonomous AI, the vast majority of deployed agents are static inference machines wrapped in clever prompt chains. When they fail at a task pattern, someone on your team manually re-prompts, retrains, or rewires the pipeline. The feedback loop between failure and improvement is measured in days or weeks — not the minutes it should take.

That's starting to change, and the implications are significant.

ALTK-Evolve: On-the-Job Learning for Agents

Hugging Face and IBM Research recently introduced ALTK-Evolve, a framework that enables on-the-job learning for AI agents. Instead of relying exclusively on offline fine-tuning or static prompt engineering, ALTK-Evolve lets agents evolve their behavior through real-world task execution.

The core idea: an agent's own execution traces — the sequence of actions it took, the tools it called, the results it observed — become training signal. The agent doesn't just complete a task and move on. It reflects on what worked, what didn't, and adjusts its strategy for the next iteration.

This isn't reinforcement learning in the traditional sense, where you need a carefully designed reward function and a simulation environment. This is learning from production behavior, in production, on real tasks. The feedback loop tightens from weeks to hours, potentially to minutes.

Why This Matters More Than Another Benchmark

The AI community is perpetually distracted by benchmark wars. Model X beats Model Y on HumanEval. A new architecture claims state-of-the-art on MMLU. These numbers matter, but they obscure a more fundamental question: what happens after deployment?

A model that scores 92% on a benchmark but can't improve from its own failures in production is less valuable than a model scoring 85% that compounds its experience over time. On-the-job learning introduces a compounding advantage — agents that have been running longer perform better, not because they were retrained by a human, but because they evolved through use.

Think about the economics of this. Two companies deploy competing AI agents for the same enterprise workflow. Company A's agent is static — every improvement requires an engineer to analyze failure cases, adjust prompts, and redeploy. Company B's agent learns from its own execution traces and adapts autonomously. After three months, the performance gap isn't linear. It's exponential. Company B's agent has been compounding improvements with every task it completes.

What This Demands From Agent Architectures

Here's the practical takeaway that most teams are going to miss: agent architectures need to be designed for mutability from day one.

Most agent frameworks today are built around static components — fixed prompt templates, hardcoded tool chains, rigid orchestration logic. These architectures assume that the agent's behavior is defined at build time and frozen at deploy time. On-the-job learning breaks that assumption entirely.

If you want agents that evolve, you need:

Execution trace logging as a first-class concern — not just for debugging, but as training data. Every action, observation, and decision point needs to be captured in a structured format that can feed back into the agent's learning loop.
Mutable strategy layers — the agent's decision-making logic can't be a monolithic prompt. It needs modular components that can be updated independently as the agent learns new patterns.
Guardrails on self-modification — an agent that can change its own behavior is powerful but dangerous. You need validation gates that ensure evolved behaviors don't violate safety constraints or drift from the intended task scope.
Evaluation infrastructure that runs continuously — not just pre-deployment benchmarks, but ongoing performance monitoring that can distinguish genuine improvement from harmful drift.

Static prompt chains won't cut it when your competitor's agents are compounding their own experience.

Key Takeaways

On-the-job learning closes the feedback loop between agent failure and improvement from weeks to hours, using execution traces as training signal rather than requiring manual intervention.
Compounding experience creates exponential advantages — agents that learn from production use will increasingly outperform static agents, regardless of base model quality.
Agent architectures must be designed for mutability from day one — static prompt chains and hardcoded tool orchestration are incompatible with continuous self-improvement.

The Question You Should Be Asking

If you're building agents today, the most important architectural question isn't which model to use or which framework to adopt. It's this: are you designing for deployment, or for continuous improvement after deployment?

That distinction is about to separate the serious agent builders from everyone else. The agents that win in production won't be the ones that launched best — they'll be the ones that learned fastest.

On-Device AI Is Changing How We Build — With Cover Image Test

Chetan Sehgal — Wed, 08 Apr 2026 09:38:18 +0000

The Shift Nobody Priced In

For the past two years, building AI into products meant one thing: an API call to a cloud endpoint. That assumption just broke.

Google's Gemma 4 is a multimodal model with frontier-level reasoning that runs locally — on a phone, a laptop, an edge device. Not behind a server. Not metered per token. On the device in your hand.

Why This Changes Your Architecture

When inference is local, three constraints flip:

Latency drops from hundreds of milliseconds to single digits
Cost goes from per-call pricing to zero marginal cost
Privacy goes from "we send your data to the cloud" to "it never leaves the device"

These aren't incremental improvements. They change which features are viable to build.

What Practitioners Should Do Now

If you're building AI features today, benchmark on-device models for your use case. The gap between cloud and local quality is closing faster than most roadmaps account for.

Hybrid inference — local for latency-sensitive tasks, cloud for complex reasoning — is likely the architecture that wins.

Key Takeaways

On-device AI is no longer a compromise — it's a viable first choice for many use cases
Gemma 4 signals that frontier capability at the edge is arriving faster than expected
Architects who figure out hybrid inference now will ship faster and cheaper

What's the first feature in your product you'd move from cloud to on-device?