DEV Community: joshinii

State, Memory, and Context: What AI Actually “Remembers”

joshinii — Fri, 23 Jan 2026 06:44:04 +0000

Have you noticed this before?

The model forgets something it clearly knew earlier
The same input gives a different answer later
Adding a bit of “context” suddenly fixes the behavior

These aren’t model quirks.

They’re signals that state is unclear in the system design.

This post builds a clean mental model for how state, memory, and context work in AI-enabled systems—and how to design without accidental coupling.

Traditional Systems: Where State Is Clear

State in traditional applications is usually:

Stored in databases

Permanent or semi-permanent information that the system relies on.

Example: Customer records, order history, or inventory levels in an Oracle or PostgreSQL database.
Cached intentionally

Temporary copies of data to improve performance or reduce repeated computations.

Example: Session data stored in Redis or frequently accessed product catalog data cached in memory.
Passed explicitly through APIs

Data sent between services or components as part of a request.

Example: A service call that includes a user ID and account type to fetch specific account information.
Auditable and recoverable

Every state change can be tracked and traced for debugging or compliance.

Example: Versioned financial transactions, order status changes, or audit logs that can reconstruct system behavior at any point.

Most importantly, this state is:

Explicit – you know exactly where it lives
Addressable – you can query, update, or delete it
Deterministic – the same inputs produce predictable results

When something breaks, you can trace what changed and where, which makes debugging reliable.

AI challenges this—not by removing state, but by hiding where developers expect it to live.

What the Model Does Not Have

Let’s start with a hard boundary.

Models do not have memory.

A model:

Does not remember previous requests
Does not retain conversation history
Does not accumulate state over time

Each call is independent.

If something feels “remembered,” your system supplied it again.

Three Concepts That Must Stay Separate

Many AI bugs come from mixing these up.

Context

Context is input.

Examples:

Prompt text
Instructions
Examples
Retrieved documents
Conversation history you resend

Properties:

Exists only for one request
Token-limited
Consumed, not stored

Think of context like function arguments, not variables.

Memory

Memory is external state you manage.

Examples:

Stored chat history
User preferences
Retrieved embeddings
Cached tool outputs

Properties:

Lives outside the model
Must be fetched intentionally
Must be injected back into context

If memory feels implicit, the design is already fragile.

State

State is system-level truth.

Examples:

Workflow progress
Decisions made
User-visible outcomes
Audit logs

State must:

Persist beyond a single request
Be inspectable
Be owned by the application

AI can influence state, but the actual state must be stored and controlled outside the model.

The Most Common Design Failure

Many systems accidentally treat the model as stateful.

This shows up as:

Relying on conversation flow instead of stored data
Assuming consistency without re-supplying context
Letting decisions live only in generated text

The result:

Non-reproducible behavior
Debugging without ground truth
Silent behavior drift

If restarting your service changes outcomes, state is leaking.

A Cleaner Mental Model

A more stable framing:

Models compute
Systems remember
Applications decide

The model is a stateless function.

Your system decides:

What context to assemble
What outputs to persist
What becomes authoritative state

Once this boundary is clear, complexity drops sharply.

A Simple Rule

If something matters tomorrow, it cannot live only in today’s prompt.

Store it.

Version it.

Own it.

AI can assist—but it should never quietly carry state for you.

Where This Leads Next

Context and Data Flow: Feeding AI the Right Information

That’s where the next post goes.

Who Owns the Decision When AI Is Involved?

joshinii — Fri, 23 Jan 2026 06:19:14 +0000

When AI becomes part of an application, a natural question comes up:

If AI influenced the outcome, who is actually responsible for the decision?

This question matters more than model choice or tooling, because it directly affects how systems are designed, tested, and trusted.

Decision Ownership Has Always Existed

In traditional systems, decision ownership is usually clear:

Business rules decide eligibility
Services enforce constraints
Workflows control state transitions
Databases protect consistency

Even in distributed systems, the system itself owns decisions.

Code executes rules, and responsibility is traceable.

AI changes how decisions are informed, not who owns them.

What AI Contributes — and What It Doesn’t

An AI component does not make decisions in the architectural sense.

It provides:

Interpretations
Classifications
Recommendations
Summaries

These outputs are inputs to a decision, not the decision itself.

Example

An AI labels a support ticket as “high priority.”

The system decides whether to escalate, notify, or auto-respond.

AI informs.

The system acts.

Why This Boundary Matters

When AI is treated as the decision-maker:

Failures become hard to explain
Responsibility becomes unclear
Safety checks are easy to bypass

From an architectural standpoint, “the AI decided” is not a useful explanation.

Clear systems ensure that:

Decisions are enforced by deterministic logic
Constraints live outside the AI
Outcomes remain attributable to the system

A Common Architectural Pattern

In practice, many AI-enabled systems follow this flow:

AI evaluates context and produces a recommendation
Deterministic logic validates constraints
Workflows decide the next action
Auditing records the outcome

This keeps authority where guarantees still exist.

AI contributes judgment, not authority.

AI Components vs Agentic Behavior

This distinction helps avoid confusion.

An AI component:

Produces an output when invoked
Has no control over next steps
Operates within strict boundaries

An agentic system:

Uses AI output to drive actions
Orchestrates multiple steps
Is deliberately designed to act autonomously

Agentic behavior is a system-level design choice.

Responsibility still belongs to the system — not the model.

Practical Implications for Developers

Treat AI output as a recommendation, not a command
Keep final decisions deterministic
Add explicit checks for high-risk actions
Avoid explanations that stop at “AI made the call”

These practices keep systems understandable even when reasoning is probabilistic.

A Simple Mental Model

Think of AI as a senior advisor.

It can:

Surface patterns you might miss
Provide strong suggestions
Add useful context

But it does not approve decisions.

That responsibility remains with the system you design.

Closing Thought

AI can influence decisions, but it should not own them.

So, amongst State, Memory, and Context: What AI Actually “Remembers”?

If AI outputs aren’t guaranteed, how do systems stay reliable?

joshinii — Thu, 22 Jan 2026 23:33:27 +0000

When AI becomes part of an application, the first thing that starts to feel less clear is the contract.

What does correctness mean now?
What can still be validated?
Where do guarantees actually live?

Contracts don’t disappear — they shift.

Traditional Contracts: The Baseline We’re Used To

In most Java-based or similar systems, contracts clearly define expectations:

Input — what data is allowed and in what format
Behavior — what the system does with valid input
Output — what is returned and how it is structured
Errors — how failures are reported
Stability — what remains consistent over time

Concrete example

A REST API that accepts a customerId guarantees:

Only valid IDs are processed
The same request produces the same response
Failures surface as explicit errors

These guarantees are why systems are predictable, testable, and easy to compose.

Where AI Changes the Shape of a Contract

AI components still participate in contracts — but the nature of the guarantees changes.

AI does not execute fixed logic paths.

It evaluates information and produces an output that is likely to be useful.

That difference shows up at the boundary:

Aspect	Traditional Component	AI Component
Input	Strict schema	Context-rich, sometimes incomplete
Behavior	Deterministic execution	Inference-based reasoning
Output	Exact and repeatable	Reasonable, may vary slightly
Failure	Errors / exceptions	Low confidence, ambiguity

Example

A rule-based system classifies a support ticket using fixed conditions
An AI component reads the ticket text and infers urgency and intent

The AI result is often correct — but it is not guaranteed in the same way.

Context and Intent

Two ideas explain why AI contracts feel different: context and intent.

Context

Context is everything surrounding a request:

Previous interactions
Related records
Business constraints

Example

User message: “This hasn’t arrived yet.”

Context may include order history, shipping status, and prior messages.

Intent

Intent is what the system infers the user is trying to achieve:

Checking status
Escalating an issue
Requesting a refund

Traditional systems encode intent explicitly in endpoints or request types.

AI components infer intent from context.

That inference is powerful — and inherently less certain.

AI Components vs Agentic Systems

This distinction is important architecturally.

An AI component:

Produces an output (classification, summary, suggestion)
Has no authority to act
Is invoked within an existing flow

Example

Summarizing a document or extracting intent from a message.

An agentic system:

Uses AI output to decide next steps
Orchestrates multiple actions
May operate over time

Example

A system that:

Reads a support ticket
Decides to fetch account data
Generates a response
Updates a ticketing system

Agentic behavior is a system design choice, not something inherent to AI models.

How Contracts Are Enforced Around AI

Because AI behavior is inference-based, contracts are enforced around the AI component — not inside it.

Three familiar architectural ideas make this work:

Wrapping — AI is accessed through a service layer that prepares inputs and validates outputs
Bounding — AI is limited to specific responsibilities and controlled data access
Supervision — AI outputs are monitored, filtered, or reviewed when needed

Concrete example

An AI suggests a reply to a customer email:

The system controls what data the AI can see
The output is checked before sending
Low-confidence responses trigger fallback logic

The AI assists — it does not own the outcome.

Practical Implications for Developers

Contracts still matter — but guarantees shift from correctness to reasonableness
AI outputs should be treated as recommendations, not facts
Agentic behavior must be designed deliberately
Deterministic systems remain responsible for safety, correctness, and control

Strong systems combine:

Traditional software for rules and guarantees
AI components for interpretation and judgment

Where This Leads Next

Once AI becomes part of the system boundary, another question follows naturally:

Who Owns the Decision When AI Is Involved?

AI Is No Longer a Tool, It’s an Architectural Layer!

joshinii — Thu, 22 Jan 2026 06:28:26 +0000

I’m a software developer, and transitioning into AI-assisted development hasn’t felt natural.

When AI is mostly presented as a tool that generates code from prompts, a question comes up quickly:

If AI can already write code, what’s left for experienced developers to do?

The goal of this series is not to use AI tools better, but to understand how AI fits into modern application architecture.

For most Java and web developers, the systems we build follow a familiar pattern.

A request comes in.

Code runs.

A response goes out.

Even in larger enterprise setups — Spring services, Oracle databases, Kafka pipelines — the underlying assumption is usually the same:

Given the same input, the system behaves the same way.

This assumption is deeply embedded in how we design, test, and reason about software.

AI-powered systems begin to stretch this assumption.

Not because they are unreliable, but because they produce results in a fundamentally different way. Recognizing this difference is the first step toward understanding where AI fits architecturally.

This shift isn’t really about generating code faster.

It’s about treating AI as another system component — one with its own boundaries, failure modes, and responsibilities, much like APIs, databases, or message brokers.

Architectural Perspective: Traditional vs AI-Powered Systems

Aspect	Traditional Web Systems	AI-Powered Systems
Core behavior	Deterministic execution	Reasoning based on likelihood
Logic location	Code (services, rules engines)	Model + orchestration layer
Input handling	Strictly validated	Context-heavy, flexible
Output	Predictable and repeatable	Usually correct, not guaranteed
Failure mode	Errors, exceptions	Degraded or unclear responses

Example for clarity:

Traditional: A REST endpoint receives a user ID, queries the database, returns exact user info, or a 404 error.
AI: A system receives a vague prompt, infers intent, consults multiple sources, and returns a reasonable response — which may vary each time.

This combined view highlights the key shift in reasoning:

Traditional systems answer: “What should I do?”
AI systems answer: “What makes sense here?”

The system is not random; it’s making judgments instead of executing fixed rules.

For developers used to strict control flow, this can feel unfamiliar — not because it’s wrong, but because it solves a different class of problems.

Why This Isn’t a Step Backwards

At first glance, AI systems can feel harder to trust:

Outputs aren’t exact
Testing isn’t always binary
Behavior may vary slightly

At the same time, they handle scenarios where traditional systems often struggle.

AI systems work well for:

Interpreting ambiguous or unstructured input
Connecting information across many sources
Supporting decisions when rules are incomplete

They are generally not suitable replacements for:

Financial calculations
Authorization logic
Transactional consistency

That separation is an architectural choice, not a tooling limitation.

In practice, many systems benefit from:

Deterministic software for control and correctness
AI systems for interpretation and decision support

A Practical Mental Model

One simple way to frame the shift:

Traditional software executes instructions.

AI software evaluates possibilities.

Both approaches can coexist within the same application.

AI doesn’t replace backend systems.

It changes where and how certain decisions are made.

Once this distinction is clear, many AI architecture discussions become easier to follow.

What This Series Is Really About

This series is not focused on:

Prompt techniques
Model comparisons

It focuses on:

Mapping familiar concepts to newer ones
Understanding how application architecture is evolving
Exploring how existing backend skills still apply

Each part will relate new ideas back to systems many developers already know — REST APIs, databases, events, and observability.

What’s Next

When outputs are no longer exact, system boundaries become more important — not less.

AI components need to be surrounded by well-defined interfaces that decide what is trusted, what is validated, and what happens when confidence is low.

So, If AI outputs aren’t guaranteed, how do systems stay reliable?