DEV Community: Shweta Mishra

Your AI Agent Doesn’t Need an LLM for Every Decision — Meet Jev and the Rise of System One Models

Shweta Mishra — Thu, 24 Sep 2026 14:12:07 +0000

Most AI applications have a hidden architectural problem:

We use a language model for decisions that never needed language generation in the first place.

Classify this ticket.

Should this request go to billing?

Is this action safe?

How urgent is this incident?

Does this request need human review?

Which workflow should run next?

These are not really “write me some text” problems.

They are decision problems.

And that distinction is exactly what makes Jev, TypeSafe AI’s first System One model, interesting.

Instead of asking a model to generate text and then forcing application code to interpret that text, Jev is designed around a different interface:

State → Typed Question → Decision + Probability

That sounds like a small API difference.

Architecturally, it is much bigger.

The Problem With Using LLMs for Everything

Imagine a support ticket:

“I was charged twice. Please refund the extra payment.”

A traditional AI workflow might look like this:

User message
↓
LLM
↓
Generate JSON
↓
Parse JSON
↓
Validate schema
↓
Check confidence
↓
Apply business rules
↓
Execute action

The LLM might return:

{
"team": "billing",
"refund": true,
"urgency": "medium"
}

It looks structured.

But the underlying system still asked a generative model to produce a representation that application code needs to interpret.

There are more moving pieces than the JSON suggests.

What happens if the field is missing?

What happens if the value is outside the expected schema?

What happens if the model produces an unexpected category?

What happens when the application needs three independent decisions?

And what happens when the system needs to distinguish between “probably yes” and “definitely yes”?

This is where the System One approach becomes interesting.

Jev Changes the Interface

TypeSafe describes Jev as a model built for “fast, structured decisions that software can use directly.”

Instead of primarily generating strings, Jev works with typed questions.

For example:

Choice:

Which team should handle this ticket?

→ billing: 0.97
→ technical: 0.02
→ other: 0.01

Score:

How urgent is this ticket?

→ 0.18 on a defined scale

Yes/No:

Does this require human review?

→ yes: 0.08
→ no: 0.92

The important part isn't simply that the output is JSON.

Modern LLMs can already produce structured JSON.

The architectural difference is that the possible answer space is defined as part of the decision itself.

Your application defines what a valid answer means.

The model evaluates the state.

Your code owns what happens next.

That separation is powerful.

Think of Jev as a Decision Layer

A useful production architecture could look like this:

Input
↓
Jev — decision layer
↓
Business rules
↓
Code / tools
↓
Action
↓
LLM — only when language generation is required

For a support workflow:

Support ticket
↓
Jev
├── Classify ticket
├── Score urgency
├── Detect refund intent
└── Decide whether human review is required
↓
Decision orchestration
↓
Business rules
↓
Billing API / CRM / Queue
↓
LLM
↓
Customer-facing response

Now each component has a clearer responsibility.

Jev answers bounded questions.

Code applies deterministic rules.

Tools execute actions.

The LLM handles language.

That is fundamentally different from putting an LLM in the middle of every step.

The Most Interesting Part: Parallel Decisions

One of the ideas I find particularly interesting is evaluating multiple focused questions against the same application state.

Consider an incoming incident report.

Instead of creating a chain like:

LLM → classify
↓
LLM → determine urgency
↓
LLM → decide escalation
↓
LLM → determine routing

you can formulate multiple typed questions against the same state.

For example:

Which team owns this incident?
What is its urgency?
Does it require human review?
Is the incident related to a production system?

The application can then combine those results.

This matters because real-world automation rarely depends on one decision.

Production workflows are usually collections of small decisions.

The architecture should reflect that.

Jev Is Not an LLM Replacement

This is where the discussion needs nuance.

Jev is not designed to replace language generation.

If your application needs:

Long-form writing
Code generation
Summarization
Explanation
Creative content
Open-ended conversation

you still need a generative model.

But if your application needs:

Classification
Routing
Scoring
Intent detection
Guardrails
Workflow selection
Human-review gates

a generative model may not be the only reasonable architectural primitive.

The better question is:

“What type of intelligence does this step actually require?”

Not:

“Which LLM should I call?”

The Architecture I Would Use

For an AI agent, I would separate the system into four layers.

1. Perception

The system receives messy real-world information.

Emails.

Tickets.

Documents.

Events.

User requests.

Application state.

This is where language models, retrieval systems, parsers, and traditional software can help.

2. Decision

Now the system needs to answer bounded questions.

Which workflow?

Which category?

What priority?

Human review?

Allowed action?

This is where a typed decision model such as Jev can fit.

3. Execution

The model should not automatically own the entire action.

Code should.

For example:

if billing_probability >= threshold:

route_to_billing()

if human_review_probability >= threshold:

create_review_task()

if refund_request:

check_payment_records()

The application defines the actual policy.

4. Generation

Only after the system knows what happened should an LLM generate the communication.

For example:

“We found that the payment was duplicated. The additional charge has been submitted for review.”

The LLM is now doing what it is naturally good at:

turning structured outcomes into useful language.

Probabilities Change the Architecture Too

Another important idea is that Jev returns probabilities and confidence signals for supported decision types.

That creates an opportunity for threshold-based automation.

For example:

High confidence
→ automate

Medium confidence
→ additional validation

Low confidence
→ human review

But there is an important engineering caveat:

A probability is not the same thing as business correctness.

A model saying 0.95 does not magically make an action 95% guaranteed to be correct.

Thresholds must be validated against real application outcomes.

For high-impact workflows such as payments, account changes, deletion, security actions, or access control, the surrounding application should still enforce independent validation and human-review policies where appropriate.

This is where good architecture matters more than model hype.

The Bigger Shift: From AI That Talks to AI That Operates

For years, the dominant AI interface has been:

Prompt → Tokens → Response

That interface is excellent for humans.

But software often needs something different:

State → Decision → Action

That is the interesting architectural idea behind System One models.

The model doesn't have to write a paragraph.

It can provide a typed signal that another system consumes.

This is closer to treating intelligence as a component inside software rather than treating the entire software system as a chatbot.

And that distinction becomes increasingly important as AI agents move from answering questions to taking actions.

The Future Agent Stack May Be Heterogeneous

I don't think the future is:

“Jev replaces LLMs.”

I think the more interesting direction is:

LLMs for generation.

Decision models for bounded judgments.

Code for deterministic rules.

Databases for state.

Retrieval for knowledge.

Tools for execution.

Human review for uncertainty and high-impact decisions.

The winning architecture may not be one model doing everything.

It may be a system where every component is given the job it is actually good at.

That is why Jev is worth watching.

Not because every AI application needs it.

But because it challenges one of the assumptions we've quietly accepted:

that every intelligent step in an AI system should be implemented as text generation.

Maybe the next generation of AI engineering isn't about making one model do more.

Maybe it's about giving different models smaller, clearer jobs.

And that could change how we design AI agents from the ground up.

What would you change in your current AI architecture if the model making a decision didn't have to generate a single sentence?

I Built an AI-Powered Excel Analyst Because Spreadsheets Still Take Too Much Manual Work

Shweta Mishra — Wed, 16 Sep 2026 15:54:32 +0000

An Excel file can contain thousands of rows of useful information.

But getting from that file to a meaningful answer can still take hours.

You open the spreadsheet, inspect the columns, look for missing values, clean inconsistent data, calculate KPIs, create charts, investigate trends, and finally turn everything into something another person can actually understand.

The analysis itself is often not the hardest part.

The repetitive preparation around it is.

That was the problem I wanted to explore when I built Excel Auto-Analyst, a Streamlit-based application for analyzing Excel and CSV datasets.

Live app: https://excel-auto-analyst-ne9ocshgvqtvqtitbapbjs.streamlit.app/

Source code: https://github.com/Shweta-Mishra-ai/excel-auto-analyst

The problem

Data analysis workflows often start with something deceptively simple:

«“Here is the Excel file. Find out what is happening.”»

In practice, that request expands quickly.

Before answering a single business question, an analyst may need to:

Understand the dataset structure
Identify missing values
Detect problematic columns
Check data types
Clean the dataset
Calculate descriptive statistics
Identify useful KPIs
Explore relationships between variables
Build visualizations
Interpret trends
Prepare a report

None of these tasks are individually complicated.

The problem is repetition.

When the same workflow happens across dozens of spreadsheets, a significant amount of analyst time can disappear into tasks that are necessary but not particularly insightful.

I wanted to build something that could automate more of that workflow while keeping the analyst in control.

What Excel Auto-Analyst does

The application starts with a simple input:

Upload an Excel or CSV file.

From there, the workflow moves through several stages.

Data profiling

The first step is understanding what was uploaded.

Instead of immediately generating charts, the application first looks at the structure of the dataset.

This includes information such as:

Number of rows
Number of columns
Data types
Missing values
Duplicate records
Basic dataset characteristics

This matters because analysis built on a misunderstood dataset can produce misleading conclusions.

Profiling should happen before interpretation.

Data cleaning

Real-world spreadsheets are rarely perfect.

Columns may contain missing values, inconsistent formats, duplicated records, or values that require preprocessing before analysis.

Excel Auto-Analyst includes a cleaning stage so the user can prepare the dataset before moving into deeper analysis.

An important part of this workflow is maintaining visibility into what happens to the data rather than treating cleaning as an invisible operation.

That distinction matters.

Automation should not mean:

«“Trust the black box.”»

It should mean:

«“Here is what happened to your data, and here is what you can inspect.”»

KPI and statistical analysis

Once the dataset is prepared, the next question becomes:

What does the data actually tell us?

The application moves beyond simply displaying rows and columns and provides analytical outputs such as KPIs, statistics, and patterns that can help users understand their dataset.

This is where spreadsheet analysis starts becoming more useful.

Instead of manually writing the same calculations every time, the application can provide a starting point for exploration.

The goal isn't to replace statistical reasoning.

The goal is to reduce the repetitive work required to reach the point where reasoning becomes useful.

Visual exploration

Numbers are useful.

But patterns often become much easier to see when they are visualized.

Excel Auto-Analyst generates interactive visualizations to help users explore the dataset from different perspectives.

Charts can make things such as:

Category differences
Trends
Distributions
Relationships
Outliers

much easier to inspect.

This is particularly useful during the exploratory phase of analysis, where the objective isn't necessarily to prove a hypothesis yet.

Sometimes you first need to discover what questions the dataset is capable of answering.

AI-assisted insights

This is the part where the application moves from traditional analytics toward an AI-assisted workflow.

Instead of requiring the user to manually interpret every visualization, the application can generate AI-assisted observations from the analysis.

The important distinction is that AI should sit after the data and analytical processing, not blindly replace them.

A useful architecture is:

Data → Profiling → Cleaning → Analysis → Visualization → AI interpretation

rather than:

Data → LLM → Answer

That difference is important.

If an LLM receives raw spreadsheet data and is simply asked to “analyze it,” the system has much less control over how calculations are performed and how conclusions are produced.

A structured analytical pipeline provides more opportunities for validation and inspection.

The architecture

The project is built around a relatively straightforward pipeline:

Excel / CSV
↓
File Upload
↓
Data Profiling
↓
Data Cleaning
↓
Statistical / KPI Analysis
↓
Visualization
↓
AI-Assisted Insights
↓
Report Export

The interface is implemented with Streamlit, which makes it possible to expose the analytical workflow through a browser without requiring users to build a local dashboard themselves.

Python handles the underlying data-processing and analytical workflow.

The architecture is intentionally focused on the workflow rather than creating another generic chatbot around a spreadsheet.

Why I didn't want another “chat with your CSV” app

There are already many applications where you can upload a dataset and ask an LLM questions.

That approach is useful, but I wanted to explore something slightly different.

A data analyst doesn't only ask questions.

They also need to know:

What is in the dataset?
Is the data clean?
Are there missing values?
Which variables matter?
What metrics are meaningful?
Which patterns are actually present?
Can the result be visualized?
Can the analysis be reproduced or communicated?

That is why I approached Excel Auto-Analyst as an analysis workflow, rather than simply a conversational interface.

The AI layer is one component.

It isn't the entire system.

What I learned building it

One of the biggest lessons was that automating analysis is harder than generating charts.

A chart can be produced quickly.

A useful analytical workflow requires much more consideration.

You need to think about data quality, assumptions, edge cases, user experience, and how the output will be interpreted.

There is also a fundamental difference between:

“The system generated an answer.”

and

“The system generated an answer that an analyst can inspect and reason about.”

For AI-assisted data analysis, I think the second goal is much more important.

Where this could go next

There are several directions I want to explore further.

For example:

More advanced anomaly detection
Better natural-language data exploration
Stronger validation of AI-generated insights
More analytical templates
Larger dataset support
Reproducible analysis pipelines
More export formats
Better evaluation of generated insights

The interesting problem isn't simply making AI produce more analysis.

It's making automated analysis more reliable, inspectable, and useful.

Try it

If you work with Excel, Python, analytics, or data science, you can try the application here:

Live app:
https://excel-auto-analyst-ne9ocshgvqtvqtitbapbjs.streamlit.app/

GitHub:
https://github.com/Shweta-Mishra-ai/excel-auto-analyst

The project is open source, and I'm interested in feedback from people who work with real datasets rather than only toy examples.

The question I'm exploring is simple:

How much of the repetitive spreadsheet analysis workflow can we automate without taking the analyst out of the loop?

What part of your own data-analysis workflow would you automate first?

RAG Is Not an Architecture: Choosing the Right Retrieval Strategy for GenAI

Shweta Mishra — Thu, 10 Sep 2026 13:44:01 +0000

Retrieval-Augmented Generation (RAG) has become one of the default patterns for building GenAI applications.

But there is a problem.

Many systems treat RAG as an architecture rather than a retrieval strategy.

The typical design looks like this:

User Query
↓
Vector Search
↓
Top-K Chunks
↓
LLM
↓
Answer

It works.

It can even look impressive in a demo.

But production systems are rarely that simple.

The real question isn't:

«Should I use RAG?»

It is:

«What kind of retrieval does this problem actually require?»

RAG Is a Pattern, Not a Complete Architecture

RAG fundamentally means retrieving external information and providing it to a generative model as context.

That's useful—but it doesn't tell you:

how information should be retrieved
whether one retrieval step is enough
whether semantic similarity is sufficient
whether relationships between entities matter
whether the model should decide what to retrieve
whether retrieval is even necessary

Those are architecture decisions.

And choosing the wrong retrieval strategy can create problems with accuracy, latency, cost, and maintainability.

Naive RAG

The simplest implementation is semantic vector search.

Query
↓
Embedding
↓
Vector Database
↓
Top-K Chunks
↓
LLM
↓
Answer

This works well when the user's question can be answered from relatively independent pieces of text.

Typical examples:

Internal documentation
Product manuals
FAQs
Knowledge bases
Simple document Q&A

But semantic similarity has limitations.

Suppose a user searches for:

INC-847291

A semantically similar result isn't necessarily the correct result.

Sometimes the exact token matters more than semantic meaning.

That's where hybrid retrieval becomes useful.

Hybrid RAG

Hybrid retrieval combines multiple retrieval mechanisms, commonly:

Semantic/vector search
Keyword or lexical search

Conceptually:

             ┌── Vector Search ──┐

Query ───────────┤ ├──→ Candidate Results
└── Keyword Search ─┘
↓
Reranking
↓
LLM

This is particularly useful when your data contains:

Product IDs
Error codes
Names
Technical terminology
Exact phrases
Version numbers
Structured identifiers

For many enterprise applications, hybrid retrieval is a more practical starting point than pure vector search.

GraphRAG

Some questions aren't really about finding similar text.

They're about understanding relationships.

Imagine a knowledge base containing:

Customer
↓
Purchased
↓
Product
↓
Affected by
↓
Incident
↓
Caused by
↓
Service

Now consider a question such as:

«Which customers were affected by incidents caused by a particular service?»

This isn't simply a semantic similarity problem.

The answer requires following relationships across multiple entities.

That's where graph-based retrieval can become valuable.

GraphRAG can help when the knowledge domain contains:

Strong entity relationships
Multi-hop questions
Connected knowledge
Organizational structures
Dependency networks
Complex relationships between documents and entities

But GraphRAG also introduces additional complexity.

A graph isn't automatically better just because it is more sophisticated.

Agentic RAG

Now consider a question where one retrieval operation isn't enough.

An agentic system can decide:

User Query
↓
Reason about task
↓
Retrieve information
↓
Evaluate results
↓
Retrieve again if necessary
↓
Use tools
↓
Synthesize
↓
Verify
↓
Answer

The retrieval process becomes dynamic rather than fixed.

This can be useful for tasks requiring:

Multiple information sources
Iterative retrieval
Tool usage
Complex research
Dynamic planning
Multi-step reasoning

But there is a trade-off.

More autonomy means more system complexity.

It can also increase:

Latency
Token usage
Infrastructure cost
Failure modes
Debugging difficulty

Agentic RAG should therefore solve a real problem—not simply make the architecture sound more advanced.

Long Context

There is another option that is frequently overlooked:

Don't retrieve aggressively.

Modern LLMs can process substantially larger contexts than earlier models.

For some workloads, it may be better to provide a large, carefully selected context rather than splitting everything into small chunks and hoping retrieval finds the right pieces.

This can be particularly useful when:

Information is highly interconnected
Chunk boundaries destroy meaning
The relevant document set is relatively small
Retrieval errors are more expensive than additional context
The model needs broader context to reason correctly

This doesn't mean "long context is better than RAG."

It means retrieval and context management should be evaluated together.

The Architecture Should Follow the Problem

A production GenAI system might look more like this:

                User Query
                     ↓
              Intent Detection
                     ↓
          Retrieval Strategy Selection
                     ↓
    ┌────────────────┼────────────────┐
    ↓                ↓                ↓

Hybrid Search Graph Search Long Context
└────────────────┼────────────────┘
↓
Reranking
↓
Context Assembly
↓
LLM Reasoning
↓
Verification
↓
Answer

And even this isn't universal.

Different applications may require completely different architectures.

For example:

Document Q&A

Query → Hybrid Retrieval → Reranking → LLM

Relationship-heavy enterprise knowledge

Query → Entity Extraction → Graph Traversal → LLM

Complex research workflow

Query → Planning → Retrieval → Tool Use → Retrieval → Synthesis

Small, highly connected document collection

Query → Relevant Documents → Long Context → LLM

Don't Choose Architecture by Trend

One of the easiest mistakes in GenAI engineering is selecting technology before defining the problem.

"Let's use GraphRAG."

"Let's build an agent."

"Let's add a vector database."

"Let's use a larger context window."

These aren't architecture decisions until you understand the workload.

The better approach is to evaluate:

Accuracy

Can the system consistently retrieve and use the information required to answer correctly?

Latency

How quickly does the system need to respond?

Cost

How much retrieval, inference, storage, and token usage can the application afford?

Complexity

How difficult will the system be to build, debug, and operate?

Maintainability

Can the architecture evolve as the data, models, and requirements change?

The best architecture is usually the one that provides the right balance across all five.

RAG Should Be a Design Decision

A vector database doesn't automatically make an application well-designed.

GraphRAG isn't automatically better than traditional RAG.

Agentic RAG isn't automatically more intelligent.

And long context isn't automatically cheaper or more accurate.

These are tools and strategies.

The architecture comes from the problem.

Before choosing a retrieval strategy, ask:

Is semantic similarity enough?
Do exact terms matter?
Are relationships between entities important?
Does the system need iterative retrieval?
Would broader context improve reasoning?
What are the latency and cost constraints?
How will retrieval quality be evaluated?

That's a much better starting point than simply asking:

«"Should we use RAG?"»

**RAG is not the architecture.

It is one of the building blocks.**

The engineering challenge is choosing the right combination of retrieval, reasoning, context, tools, and verification for the problem you're actually solving.

Can You Replace ChatGPT Plus With Free AI Tools? I Built a 30-Day AI Stack

Shweta Mishra — Sun, 06 Sep 2026 18:36:58 +0000

You probably don't need one expensive AI subscription. You need the right AI stack.

AI subscriptions have quietly become another monthly expense.

One tool for writing.

Another for research.

Another for coding.

Another for image generation.

Another for PDFs.

Before you realize it, you're paying for several AI services every month — even though you use only a fraction of their capabilities.

ChatGPT Plus alone is currently $20/month. That's $240 a year before adding anything else.

But here's the interesting part:

Do you actually need to pay for all of it?

I decided to approach the problem differently.

Instead of looking for one “best” free AI tool, I built a free AI stack where different tools handle different jobs.

The goal isn't to prove that free AI is better than paid AI.

The goal is much more practical:

How much of a paid AI workflow can you realistically replace with free tools?

The Biggest Mistake: Looking for One AI to Do Everything

This is how most people use AI:

Open ChatGPT → ask everything → hit usage limits → consider upgrading.

But AI tools are increasingly specialized.

A research engine doesn't need to be your coding assistant.

A coding model doesn't need to be your web-search engine.

A writing assistant doesn't need to be your data-analysis environment.

Instead of asking:

“Which free AI is the best?”

Ask:

“Which free AI is best for this particular task?”

That simple change makes the free ecosystem much more powerful.

My Free AI Stack

Here's the architecture I would use for a zero-subscription workflow.

Task	Free Option	Why
General AI assistant	ChatGPT Free	Everyday questions and writing
Web research	Perplexity Free	Search + citations
Coding	Gemini / free coding tools	Code generation and debugging
Research & experimentation	Google AI Studio	Model experimentation
Microsoft workflow	Copilot Free	Web-based assistance
Private/offline AI	Ollama / LM Studio	Local inference

The important point is that these tools don't have identical capabilities or limits.

For example, Perplexity's current free Standard plan provides basic search capabilities, citations, limited Pro Searches and limited file uploads.

Microsoft also provides a free Copilot experience for general questions, writing, brainstorming, summarization and web-based tasks.

Google AI Studio provides access to models under a free tier, subject to model-specific rate limits and current usage policies.

That's enough to build a surprisingly capable workflow.

1. General-Purpose AI: Start With Free Access

You don't necessarily need a paid subscription for every basic AI task.

A free general-purpose assistant can handle:

brainstorming
rewriting
summaries
explanations
basic coding
learning
planning
everyday questions

The limitation is usually usage and feature access, not whether the model can do anything useful.

ChatGPT itself remains available on a free tier, while Plus provides broader model and tool access for $20/month.

So don't automatically upgrade just because you've hit one limitation.

First ask:

Can another free tool handle this task?

2. Use Perplexity When the Job Is Research

One of the biggest reasons people pay for premium AI is research.

But research is a different problem from conversation.

When I want to investigate a topic, I want:

Search → Sources → Citations → Synthesis

That's where a search-oriented AI tool can be more useful than a general chatbot.

Perplexity's current free tier includes web search and citations, although advanced searches, models and other features have tighter limits than its paid plans.

So instead of wasting your general AI allowance on web research:

Use a research-focused tool for research.

3. Use Google AI Studio as an AI Playground

If you're an AI developer, this is where things get interesting.

Google AI Studio can be used to experiment with Gemini models and provides free-tier access subject to current limits.

That makes it useful for:

prompt experiments
structured output
prototyping
model comparisons
API experiments
building proof-of-concepts

For developers, the value isn't simply “free AI.”

It's free experimentation capacity.

And that's a much more useful way to think about these platforms.

4. Don't Ignore Free Coding Assistance

Coding is one area where people quickly become dependent on a single premium AI tool.

You don't have to.

A practical coding workflow can combine:

IDE + local models + free cloud models + your own tests

For example:

Use a free model to generate an initial implementation.
Run your tests locally.
Use another model to review the error.
Fix the implementation.
Run tests again.

The important part is that the AI isn't the validation layer.

Your tests are.

That's especially important when working with production code.

5. Keep Private Work Local

This is the part most “free AI tools” lists completely miss.

Not every task should go into a free cloud AI service.

If you're working with:

confidential code
private documents
client information
internal datasets
sensitive research

consider using a local model instead.

Tools such as Ollama and LM Studio can run models directly on your computer.

That gives you another layer in your AI stack:

Cloud AI for capability.
Local AI for control.

And you don't necessarily have to choose one permanently.

The 30-Day AI Stack Challenge

Here's the experiment I would recommend before paying for another AI subscription.

Week 1 — Replace basic usage

Use free tools for:

writing
brainstorming
summaries
everyday questions

Track where you actually hit limitations.

Week 2 — Replace research

Move research-heavy tasks to a search-focused AI.

Track:

number of searches
quality of sources
citation usefulness
time saved

Week 3 — Replace coding workflows

Try free models and local AI for:

code generation
debugging
documentation
test generation

Track how often you actually need a premium model.

Week 4 — Audit your workflow

Now ask:

What did I genuinely miss?

Not:

“Which premium feature looks cool?”

But:

“Which paid capability actually changed my productivity?”

That's the difference between buying AI because it's impressive and buying AI because it's useful.

Where Free AI Still Loses

Let's be honest.

Free tools aren't magic.

You will encounter:

Usage limits

Free plans often restrict advanced models, searches, file uploads, or overall usage. Perplexity's current free plan, for example, has explicit limits on Pro Searches, research queries and file uploads.

Feature restrictions

Some advanced capabilities are reserved for paid plans.

Inconsistent model access

Platforms can change which models are available on free tiers.

Context limitations

Long documents and large codebases may require more capable models or larger context windows.

Fragmented workflow

Using five different AI tools can become annoying.

That's the biggest hidden cost of the free stack:

complexity.

So, Should You Cancel ChatGPT Plus?

Not necessarily.

If a $20 subscription genuinely saves you several hours every month, it can easily justify itself.

The point isn't:

“Paid AI is bad.”

The point is:

“Don't pay for capabilities you don't actually use.”

If your workload consists mostly of simple questions, writing, occasional research and basic coding, a combination of free tools may be enough.

If you're doing heavy research, advanced coding, deep reasoning, large-file analysis or frequent AI workflows, a premium subscription may still be worth it.

The answer depends on your workload.

The AI Stack Is More Important Than the AI Subscription

This is the bigger lesson.

The future of personal AI probably won't be:

One company owns your entire AI workflow.

It will look more like:

General AI
↓
Research AI
↓
Coding AI
↓
Local AI
↓
Automation

Each layer solves a different problem.

And increasingly, you can mix free, paid and local components depending on what the task requires.

That's a much more flexible architecture than putting everything behind one subscription.

Final Verdict

Can you replace ChatGPT Plus with free AI tools?

For some people, absolutely.

For everyone?

No.

The smarter approach is to run the experiment first.

Use free tools for 30 days.

Track your actual usage.

Identify the tasks where free models are sufficient.

Find the tasks where they aren't.

Then pay only for the capability that genuinely creates value.

Because the real question isn't:

“How can I get AI for free?”

It's:

“Which AI capabilities are actually worth paying for?”

And that is a much better question to ask before your next subscription renewal.

What AI tool would you keep if you were allowed to pay for only one?

Run AI Locally on Your Laptop: What You Actually Get When You Stop Using the Cloud

Shweta Mishra — Wed, 02 Sep 2026 14:05:01 +0000

Private AI, offline access, no API bills — and a much better understanding of what actually happens when an LLM runs on your own machine.

Your AI assistant doesn't need to know everything about you.

Sometimes, it shouldn't even have access to the internet.

Think about the documents you routinely give an AI assistant: private code, business files, contracts, financial spreadsheets, research notes, personal documents, or internal company information.

With a cloud-based AI service, that information has to leave your computer to be processed.

But there is another option.

You can download an AI model, run it directly on your laptop, disconnect from the internet, and continue using it.

This is local AI.

And in 2026, local AI is no longer limited to researchers with expensive GPUs. Tools such as Ollama, LM Studio, and Jan have made running open and open-weight models considerably easier on consumer hardware.

The important question is no longer “Can I run an LLM locally?”

It is:

“What should I actually run locally, and when does it make sense?”

What Does “Local AI” Actually Mean?

A local AI setup has three basic components:

Model → Inference engine → Your application

The model is downloaded to your computer.

The inference engine loads that model into your system's available memory and performs the computation locally.

Your application then provides the interface you interact with — a chat window, coding assistant, document-analysis tool, or API.

Unlike a traditional cloud AI workflow, your prompt does not have to travel to a remote inference server.

For example, LM Studio states that downloaded local models can run entirely offline, including chatting with documents and running its local server. Jan similarly describes its local models as running entirely on the user's machine.

That distinction is the foundation of local AI.

Why Run an AI Model Locally?

There are three major reasons.

1. Privacy

If the model and application are genuinely running locally, your prompts and documents can remain on your machine.

This is particularly useful for:

private source code
internal documentation
confidential research
financial spreadsheets
proprietary datasets
personal notes
offline document analysis

But there is an important caveat:

“Local” does not automatically mean “nothing ever connects to the internet.”

The application may still need internet access to download models, check updates, search for models, or access optional cloud features.

For example, LM Studio's offline documentation distinguishes between local inference, which can work offline, and model discovery/download operations that require connectivity.

So if privacy is your goal, understand exactly which features are local and which are cloud-connected.

2. No Recurring AI API Bill

Local inference doesn't require paying for every prompt through an external API.

You download the model and use your own hardware for inference.

That doesn't mean local AI is literally cost-free.

You still pay for:

your laptop
electricity
storage
potentially additional RAM or GPU hardware

But there is no per-request cloud inference bill for the local workload.

For someone experimenting with AI agents, coding assistants, RAG systems, or automation, this can make local models extremely attractive.

3. Offline AI

This is one of the most underrated advantages.

Once the model is already downloaded, you can use it without an active internet connection.

Imagine being on:

an airplane
a train with unreliable connectivity
a remote location
a network-restricted environment

You can still ask questions, summarize local documents, write code, brainstorm ideas, or work with locally stored information.

LM Studio explicitly supports offline chat, document interaction, and local inference after the required model files are available.

That's a very different experience from depending entirely on a web-based AI assistant.

The Three Local AI Tools Worth Knowing

You don't need to start with CUDA configuration, model servers, or complicated Python environments.

Three tools stand out for different types of users.

1. LM Studio — Best for Beginners

LM Studio is probably the easiest entry point if you want a ChatGPT-style interface without spending your first afternoon in a terminal.

You can:

browse models
download models
load them into memory
chat with them
work with documents locally
expose models through a local API

It supports macOS, Windows, and Linux. Its current documentation also supports local models such as Qwen, Llama, Mistral, Gemma, and gpt-oss.

Best choice if: you want the simplest visual experience.

2. Ollama — Best for Developers

Ollama takes a different approach.

Instead of making the graphical interface the main attraction, it makes local models easy to integrate into developer workflows.

A simple command can download and run a model:

ollama run qwen3

Ollama also exposes local APIs, making it useful for applications, scripts, coding workflows, and AI agents.

Its 2026 releases have expanded hardware support and improved performance, including Vulkan support for additional GPU hardware.

It can also integrate with developer tools and coding agents.

Best choice if: you're a developer, AI engineer, or automation builder.

3. Jan — Best Open-Source Desktop Alternative

Jan is another interesting option for people who want a local-first desktop AI environment.

Its current desktop application supports local models, while its ecosystem also includes local APIs and CLI tooling.

Jan's documentation describes local models as running on your machine without requiring an API key, while cloud models are an optional separate mode.

That separation is important.

Local AI should not mean pretending cloud AI doesn't exist.

It means you get to choose where inference happens.

Best choice if: you want an open-source, local-first AI desktop environment.

Which Model Should You Run?

This is where many beginner guides oversimplify things.

There is no universal “best local model.”

The right model depends on:

Model size + quantization + context length + RAM/VRAM + workload

A smaller model can be much faster on a laptop.

A larger model may produce better reasoning or coding results, but can become painfully slow if your hardware cannot keep up.

For example, current Qwen3 variants range from very small models to models requiring substantially more memory.

OpenAI's gpt-oss models are another interesting current option. The smaller gpt-oss-20b model is designed for local use and has a published memory target around 16 GB, while the 120b version is far more demanding.

A practical starting point looks like this:

Hardware	Practical Starting Point	Good For
8 GB RAM	Small 1B–4B models	Summaries, simple Q&A, lightweight writing
16 GB RAM	7B–14B class models	Coding, writing, analysis, RAG
32 GB+ RAM	Larger 20B–30B+ models	More demanding reasoning and development
Dedicated high-memory GPU	Large quantized models	Advanced local experimentation

These are starting points, not hard limits.

Model quantization, GPU memory, CPU performance, context length, and operating system all affect the actual experience.

A Better 5-Minute Local AI Experiment

Don't begin by downloading five different models.

Start with one.

Step 1: Install a local AI runner

Choose:

LM Studio if you want a graphical interface.

Ollama if you want developer/API integration.

Jan if you want an open-source desktop experience.

Step 2: Download one model

Choose a model that actually fits your hardware.

Don't download a 60+ GB model just because its benchmark score looks impressive.

Step 3: Run a simple test

Ask the model to:

summarize a document
explain a Python function
rewrite a paragraph
analyze a small dataset
generate a SQL query

Step 4: Disconnect your internet

Now test the same workflow offline.

This is the moment when local AI becomes more than a concept.

Step 5: Compare it with your normal cloud workflow

Measure:

response speed
answer quality
memory usage
context handling
coding ability
document analysis
ease of use

Don't assume local AI is automatically better.

Measure it.

What Can You Actually Do With Local AI?

The most interesting applications aren't just chatting.

Private RAG

Store documents locally and let the model retrieve information from them without uploading the documents to a cloud AI service.

Local Coding Assistant

Connect a local model to your development environment and use it for code explanation, generation, debugging, or repository analysis.

Offline Research Assistant

Keep a collection of papers, notes, and technical documentation on your machine and query them locally.

Data Analysis

Combine a local LLM with Python, pandas, or a local application to create private analytical workflows.

Local AI Agents

Develop agents that interact with files, APIs, databases, or developer tools while keeping the core inference on your own hardware.

Ollama and Jan now provide local API and developer-oriented workflows, while LM Studio offers OpenAI-compatible local endpoints.

This is where local AI becomes particularly interesting for AI engineers.

You're no longer simply using an AI application.

You're building your own AI infrastructure.

But Local AI Has Real Limitations

This is the part many “run AI locally for free” articles skip.

Local AI isn't magic.

Smaller models can be weaker

A laptop-friendly model may not match the strongest cloud models on difficult reasoning, coding, multimodal tasks, or long-context workloads.

Speed depends on hardware

A model that feels instant on a powerful GPU can feel painfully slow on a CPU-only laptop.

Storage matters

Models can consume several gigabytes each, and larger models can consume tens of gigabytes.

You don't automatically get live information

A fully offline model doesn't know what's happening on the internet right now.

If you enable web search or other external tools, you've introduced an online component again.

Model licenses differ

“Open,” “open-source,” and “open-weight” are not interchangeable terms.

Always check the license for the specific model before using it commercially.

Even LM Studio's documentation explicitly points out that different models can have different degrees of openness and different licenses.

So, Should You Switch to Local AI?

Probably not completely.

The better approach is hybrid AI.

Use local models when:

privacy matters
the task is routine
you are offline
you want predictable local costs
you're experimenting with AI infrastructure

Use cloud models when:

you need frontier-level capability
you need live web information
the task requires substantial compute
you need advanced multimodal capabilities
local hardware isn't sufficient

The future isn't necessarily:

Cloud vs. Local.

It is:

Cloud when you need scale.
Local when you need control.

Final Takeaway

Running an AI model on your laptop changes your relationship with AI.

Instead of thinking of AI as a website you visit, you can start thinking of it as software you own, operate, and integrate into your own environment.

The technology has also become significantly more accessible.

LM Studio gives beginners a polished interface.

Ollama gives developers a lightweight local model layer.

Jan provides another open-source, local-first option.

And newer open-weight models are making increasingly capable local inference possible on consumer hardware.

But don't fall for the “everything is free and unlimited” narrative.

Local AI has trade-offs.

The real advantage is something more valuable:

control.

Control over your data.

Control over where inference happens.

Control over your AI stack.

And, increasingly, control over how much you depend on someone else's API.

The interesting question isn't whether local AI can replace cloud AI. It's where you should stop sending your data to the cloud in the first place.

What would you run locally first — a private RAG system, coding assistant, data-analysis agent, or something else?

MCP + Plugins: Stop Connecting Tools. Start Building Developer Workflows.

Shweta Mishra — Wed, 19 Aug 2026 17:25:39 +0000

Your AI coding assistant does not become powerful because you connected more tools.

It becomes powerful when those tools are connected to the right workflow, with the right permissions, and enough context to make useful decisions.

That distinction matters as AI moves beyond generating code and starts interacting with repositories, databases, documentation, APIs, CI systems, and other development infrastructure.

This is where MCP and agent plugins become genuinely useful.

But there is a problem: developers can easily turn integrations into a shopping list.

GitHub. Database. Browser. Slack. Documentation. Another MCP server. Another plugin.

More connections can look impressive while making the actual system harder to control.

The goal should not be maximum connectivity.

It should be useful connectivity.

MCP is a connection layer

The Model Context Protocol (MCP) provides a standardized way for AI applications to interact with external tools and data.

An MCP server can expose capabilities such as repository access, documentation retrieval, database operations, APIs, or other development tools.

The important separation is:

The model reasons. The tool performs the operation.

Instead of copying information from five different systems into a prompt, the AI application can interact with those systems through defined interfaces.

That makes tool use more structured and reusable.

But MCP itself does not magically create an intelligent developer agent.

The workflow around those tools still matters.

Plugins solve a different problem

MCP and plugins should not be treated as the same thing.

MCP is primarily a protocol for connecting AI applications with capabilities.

A plugin can package a broader capability or workflow, potentially combining skills, instructions, and MCP-based tools into something easier to distribute and reuse.

Think about a developer task such as:

«“Investigate why the production deployment failed.”»

That is rarely a single-tool operation.

The agent may need repository information, CI status, logs, recent commits, changed files, documentation, and issue history.

The useful capability comes from combining these pieces into a workflow.

Stop asking “Which tools should I install?”

This is probably the biggest mindset shift.

Don't start with:

«“Which MCP servers are available?”»

Start with:

«“Which developer workflow am I trying to improve?”»

For example:

CI debugging

GitHub → failed workflow → logs → recent commits → changed files → project context

Database investigation

Database → schema → relevant tables → query → result analysis

Documentation-aware development

Repository → official documentation → API reference → implementation → tests

Release preparation

Git → issues → pull requests → CI → changelog → release

Now every integration has a reason to exist.

What “using MCP like a pro” actually looks like

Imagine asking:

«“Why did my latest CI pipeline fail?”»

A basic AI workflow might ask you to paste the error.

A connected workflow can inspect the failed run, identify the failing job, retrieve relevant logs, inspect the latest commit, examine changed files, and compare the failure with the existing test structure.

The agent can then explain the likely root cause and propose the smallest safe fix.

If a modification is required, it should not automatically change production code simply because it has access to the repository.

It should propose the change, show the impact, run appropriate validation, and request approval when the action has meaningful consequences.

Automation should not mean unrestricted autonomy.

More tools can make agents worse

There is a common assumption that an agent becomes better as you give it more tools.

That is not necessarily true.

If an agent has access to 30 tools but needs only three for a specific task, the remaining capabilities add decision complexity and potentially irrelevant context.

There is also a security problem.

A tool that reads documentation is fundamentally different from one that can modify infrastructure or execute destructive database operations.

A production system should therefore use:

Least privilege

Give each workflow only the permissions it actually needs.

Read before write

Inspect and understand the system before modifying it.

Approval gates

Require appropriate confirmation for high-impact actions.

Traceability

Record important tool calls so developers can understand what the system did.

Build around workflows

A mature developer platform might eventually support dozens of integrations.

The developer should not have to manually manage all of them for every task.

Instead, the platform can activate capabilities according to the workflow.

Choose:

Debug CI

and expose the tools needed for CI investigation.

Choose:

Investigate a database issue

and expose database-related capabilities.

Choose:

Prepare a release

and activate the relevant Git, issue, CI, and documentation capabilities.

This creates a much cleaner architecture:

Intent → Workflow → Required capabilities → Context → Action → Verification

That is far more useful than:

User → giant toolbox → hope the agent chooses correctly

Treat every integration as a security boundary

Before connecting a new MCP server or plugin, ask:

What data can it access?
What actions can it perform?
Does it really need write permission?
What happens if the model chooses it incorrectly?
Can its actions be audited?
Can access be revoked?
What happens when the external service fails?

The question is not:

«“Can my AI agent access this?”»

The better question is:

«“Should my AI agent access this, for this workflow, with this level of permission?”»

The bigger shift

MCP and plugins are interesting because they move AI development systems beyond isolated chat interfaces.

The next generation of developer tools will increasingly combine:

Models + context + tools + memory + verification + human control.

MCP can provide an important connection layer.

Plugins can make capabilities easier to package and reuse.

But neither automatically creates a reliable AI developer.

The difficult engineering work is designing the workflow around them.

Connect fewer capabilities.

Give them clear responsibilities.

Control their permissions.

Verify their outputs.

Measure whether they actually reduce developer effort.

The real question is no longer whether AI can use your tools.

If your AI coding agent could access every tool in your development stack tomorrow, which capability would you trust it to use autonomously—and which one would you still require it to ask permission for?

I Built an Open-Source Fraud Detection API That Scores Transactions in Under 10ms

Shweta Mishra — Tue, 18 Aug 2026 13:06:00 +0000

Most fraud detection stacks make you pick a lane.

Rule engines are fast and predictable, but rigid — they miss anything that doesn't match a pattern someone already wrote down. ML models catch the unknown stuff, but they're slow to run at scale and impossible to explain to a compliance team ("the model said 0.87, trust me"). And almost nobody looks at the relationships between transactions — so a fraud ring using ten stolen cards from the same device just looks like ten unrelated flagged purchases.

I built FraudShield to stop treating these as separate problems. It's an open-source, real-time fraud detection platform that runs a rule engine, an ML ensemble, and graph analytics together, on every transaction, in under 10ms.

🔗 Live demo: https://fraudshield-blue-seven.vercel.app/
🔗 Source: https://github.com/Shweta-Mishra-ai/fraudshield
📜 License: MIT

The architecture: three detection layers, one decision

Instead of picking rules or ML or graph analysis, FraudShield runs all three in parallel and combines them into a single weighted decision:

Rule engine — 9 behavioral fraud rules covering things like transaction velocity, geographic anomalies, and device/IP mismatches. Fast, deterministic, and easy to audit.
ML ensemble — XGBoost + Isolation Forest working together. XGBoost handles supervised classification on labeled fraud patterns; Isolation Forest catches anomalies that don't look like anything the model has seen before.
Graph analytics — built on NetworkX, this layer looks for fraud rings: shared devices, shared IPs, and connection patterns across otherwise-unrelated transactions. This is the layer that catches coordinated fraud, not just one-off bad actors.

Every transaction gets scored by all three, and the result is one of three decisions: ALLOW, REVIEW, or BLOCK — along with a risk score and a plain-language reason.

Real-time streaming, not batch scoring

The detection engine runs on Pathway, a Rust-based streaming framework, which is what gets latency down to sub-10ms per transaction while handling 100k+ transactions/second. There's also a polling-based fallback engine for environments where the native streaming mode isn't available (Windows/WSL), so it doesn't fall over outside a Linux box.

Explainability isn't an afterthought

Every flagged transaction ships with SHAP-based explanations, so "why was this blocked" has an actual, inspectable answer instead of a confidence score nobody can act on. For a live example, the demo shows real output like:

Merchant category 'electronics' is elevated risk
Card-not-present on 'electronics' (risk 0.65)

That's the kind of reasoning an analyst — or a compliance auditor — can actually work with.

Security, because it's handling transaction data

PII is SHA-256 hashed before it ever touches the database
API key auth via X-API-Key header
Rate limiting (HTTP 429 on abuse)
SQL injection prevention
Standard security headers (CSP, X-Frame-Options, X-Content-Type-Options)

Tech stack

Layer	Tech
API	Python 3.10+, FastAPI, Uvicorn
Streaming	Pathway (Rust engine)
ML	XGBoost, Isolation Forest
Graph analytics	NetworkX
Storage	PostgreSQL / SQLite
Web dashboard	Next.js 14, Tailwind CSS
Analyst console	Streamlit

Calling the API

Integration is one POST request. JSON in, decision out:

import requests

response = requests.post(
    "https://fraudshield-api.onrender.com/api/v2/transactions/analyze",
    headers={"X-API-Key": "your-api-key"},
    json={
        "user_id": "USER_001",
        "amount": 299.99,
        "currency": "USD",
        "merchant_id": "SHOP_001",
        "merchant_category": "electronics",
        "location": "US",
        "device_id": "DEVICE_001",
        "ip_address": "192.168.1.1",
        "channel": "online"
    }
)

result = response.json()
print(result["decision"])     # ALLOW / REVIEW / BLOCK
print(result["score"])        # 0.0 - 1.0
print(result["reasons"])      # Why it was flagged
print(result["latency_ms"])   # < 10ms

Other endpoints worth knowing about:

Method	Endpoint	Purpose
POST	`/api/v2/evaluate`	Real-time transaction evaluation
GET	`/api/v2/alerts`	Retrieve flagged transactions
POST	`/api/v2/alerts/{id}/review`	Submit analyst review
GET	`/api/v2/stats`	System throughput metrics
GET	`/api/v2/health`	Health check (no auth)

Repo structure

fraudshield/
├── apps/api/          # FastAPI backend, detection engine, streaming
├── apps/web/          # Next.js 14 web dashboard
├── dashboard/         # Streamlit command center
├── docs/              # Architecture documentation and diagrams
├── requirements.txt   # Python dependencies
└── ...                # setup & contribution guides

Getting started

Self-hosting is free, forever, under MIT license:

git clone https://github.com/Shweta-Mishra-ai/fraudshield.git
python -m venv venv && source venv/bin/activate
pip install -r requirements.txt
streamlit run app.py   # dashboard on localhost:8501

Or skip setup entirely and hit the cloud API — sign up, get a free API key instantly, no credit card, and you can send your first transaction in about 5 minutes. There's also an "Instant Demo Key" option if you just want to poke at it without creating an account.

Currently sitting at 180 passing tests, with the analyst dashboard, review queue, and fraud ring visualization built in — no extra tooling required.

Where this is headed

Right now this is a solo open-source project — rule engine, ML layer, and graph layer are all working together, but there's plenty on the roadmap: more fraud rule coverage, expanded graph ring detection, and (eventually) paid tiers for higher transaction volumes on the cloud API, while the self-hosted version stays free forever.

If you're working on anything in fraud, risk, or trust & safety, I'd genuinely like to hear how you're approaching the rules-vs-ML-vs-graph trade-off — drop a comment or open an issue.

Repo: https://github.com/Shweta-Mishra-ai/fraudshield
Live demo: https://fraudshield-blue-seven.vercel.app/

Built with FastAPI, Pathway, XGBoost, and NetworkX. MIT licensed — self-host it, fork it, break it, tell me what's missing.

The Next AI Engineering Shift: From Prompts to Loops to Graphs

Shweta Mishra — Sun, 16 Aug 2026 14:54:00 +0000

For years, AI engineering revolved around one question:

“How do I write a better prompt?”

That question is changing.

The evolution of modern AI engineering is moving through three increasingly powerful abstractions:

Prompt Engineering → Loop Engineering → Graph Engineering

Not because the previous layer suddenly becomes useless, but because the complexity of AI systems keeps moving upward.

Prompt Engineering: Optimizing the Instruction

The early LLM application architecture was relatively simple:

Input → Prompt → LLM → Response

The engineering challenge was to make the model produce a better answer.

We optimized:

System instructions
Few-shot examples
Context
Output formats
Reasoning strategies
Constraints
Prompt templates

This worked extremely well for summarization, classification, extraction, generation and question answering.

But there was a limitation:

The model answered and stopped.

That is fine for:

«“Summarize this document.”»

It is much harder for:

«“Investigate this production bug, inspect the repository, modify the code, run tests, analyze failures, fix the implementation and verify the result.”»

That is no longer primarily a prompting problem.

It is an execution problem.

Loop Engineering: Optimizing the Execution Cycle

As AI agents became capable of using tools and performing multi-step tasks, the engineering abstraction changed.

Instead of manually providing every next instruction, we started designing systems that could continue working toward a goal.

The fundamental pattern became:

Plan → Act → Observe → Evaluate → Retry

This is the core idea behind Loop Engineering.

The engineer now has to design:

Tool usage
State management
Verification
Retry policies
Stop conditions
Error handling
Resource budgets
Human escalation

The unit of work changed.

Prompt Engineering: one interaction.

Loop Engineering: one autonomous execution.

Consider a coding agent.

Instead of manually telling it:

«Inspect the code.»

Then:

«Run the tests.»

Then:

«Fix the failure.»

Then:

«Try again.»

The system can execute the cycle itself:

Understand → Modify → Test → Observe → Repair → Test → Stop

The human becomes less of a runtime operator and more of a system designer and reviewer.

But there is another problem.

Why Loops Start Becoming Graphs

Real production workflows rarely look like:

A → B → C → A

They branch.

Consider an AI research system.

It might need to:

Search multiple sources
Retrieve internal documents
Run data analysis
Ask specialized agents for investigation
Combine results
Validate evidence
Repair unsupported conclusions
Request human approval
Publish the final output

Some operations can happen in parallel.

Some depend on previous results.

Some failures need different recovery strategies.

Some decisions require a human.

At this point, putting everything into one giant loop becomes difficult to reason about.

The loop starts becoming a graph.

Graph Engineering: Optimizing the Execution Topology

Graph Engineering is an emerging way of thinking about complex AI workflows as explicit execution graphs.

The graph can contain:

AI agents
LLM calls
Tools
Retrievers
Databases
Deterministic functions
Validators
Evaluators
Human approval points
Recovery handlers

The important abstractions become:

Nodes + Edges + State + Routing + Recovery

A node performs work.

An edge defines what can happen next.

State carries information across execution.

Routing determines which path should execute.

Recovery defines what happens when something fails.

Now we're not simply engineering an agent.

We're engineering the topology through which intelligence moves.

This is particularly useful for:

Multi-agent systems
AI coding platforms
Research agents
Complex data pipelines
Enterprise automation
Long-running workflows
Human-in-the-loop systems
Production agent orchestration

Prompt vs Loop vs Graph Engineering

The simplest mental model is:

Prompt Engineering
→ controls the instruction

Loop Engineering
→ controls the execution cycle

Graph Engineering
→ controls the execution topology

Is Prompt Engineering Dead?

No.

That would be an oversimplification.

Prompt engineering is becoming less dominant, not irrelevant.

A graph node can still have a sophisticated system prompt.

An agent still needs behavioral instructions.

Retrieval still requires good query construction.

Structured outputs still depend partly on careful prompting.

The change is architectural.

In the earlier LLM era, the prompt was often the primary interface to intelligence.

In modern agentic systems, the prompt is increasingly one component inside a larger execution architecture.

That's the important shift.

The Real Evolution

I think the progression is better understood as:

Prompt Engineering

Prompt → Response

Optimize what the model says.

Loop Engineering

Goal → Loop → Outcome

Optimize how the agent works.

Graph Engineering

Goal → Graph → Verified Outcome

Optimize how multiple processes coordinate.

So the future isn't necessarily:

Prompt OR Loop OR Graph.

It is:

Prompt inside Loop inside Graph.

A graph can contain loops.

Loops can contain prompts.

Prompts can control individual agents.

The abstraction is simply moving upward.

What Changes for AI Engineers?

The interesting engineering questions are no longer limited to:

«“What prompt should I use?”»

They increasingly become:

State: What information should persist?

Routing: Which agent or tool should execute next?

Evaluation: How do we know the output is correct?

Recovery: What happens when a component fails?

Observability: Why did the system take this path?

Governance: Where should humans retain control?

Cost: How do we prevent runaway agent execution?

These are systems engineering problems, not just prompt-design problems.

And that is why I believe the next generation of AI engineering will be increasingly focused on architecture.

The Bigger Shift

The evolution is not really:

Prompt → Loop → Graph

It is:

Response → Execution → System

We started by optimizing the answer.

Then we started optimizing the agent's behavior.

Now we're increasingly optimizing the system through which intelligence flows.

The prompt didn't disappear.

It became a component.

And perhaps the most valuable AI engineering skill of the next few years won't be writing the cleverest prompt.

It will be designing the most reliable path from:

Intent → Reasoning → Action → Verification → Outcome

Prompt Engineering → Loop Engineering → Graph Engineering.

The next question is:

What comes after Graph Engineering?

What Auditing My AI Content Engine Taught Me About Building Production AI Systems

Shweta Mishra — Fri, 07 Aug 2026 20:56:48 +0000

Everyone loves building AI applications. Connect an LLM, write a prompt, add a clean UI, and within a few hours you have something impressive to share on LinkedIn. I've built several AI projects myself, and like many developers, I was initially focused on adding features quickly.

Recently, I decided to do something different.

Instead of building another feature, I audited one of my own projects—Healthy Gut AI, an AI-powered medical content generation platform. The goal wasn't to find bugs for the sake of fixing them. I wanted to answer a simple question:

If someone used this application in production today, would I trust it?

That question completely changed how I looked at my own code.

The Project

Healthy Gut AI isn't just an article generator. It's designed as a complete AI content pipeline.

The application includes:

FastAPI backend
Multi-LLM provider support
Retrieval-Augmented Generation (RAG)
SEO-focused article generation
Batch content generation
Human review workflow
Quality scoring
Analytics dashboard
DOCX export

On paper, everything looked good. The application generated articles successfully, the dashboard displayed useful metrics, and the review workflow functioned exactly as expected.

From a feature perspective, I could have considered the project "finished."

But production software isn't judged by feature count.

It's judged by reliability.

The Difference Between Working and Production Ready

One lesson became obvious almost immediately.

A project can work perfectly during development and still not be ready for production.

For example, every article generated successfully.

No exceptions.

No crashes.

No failed requests.

Yet during testing, one multilingual article unexpectedly contained a Chinese character inside a Hindi sentence.

Technically, nothing had failed.

The API returned a successful response.

The database stored the article.

The dashboard showed another successful generation.

But no real user would want to publish that content.

That was my first reminder that successful execution is not the same as successful output.

Architecture Matters

One thing I was genuinely happy with during the audit was the project structure.

Instead of putting every feature into one large application file, the project was organized into small modules responsible for specific tasks.

The backend separated routing, configuration, database access, caching, quality evaluation, metrics, and LLM providers.

That decision made the code much easier to understand and maintain.

Adding new features required changing only one or two modules instead of touching the entire application.

Clean architecture doesn't make screenshots more impressive, but it makes future development significantly easier.

Multi-LLM Support Was Worth It

One design decision I don't regret is supporting multiple AI providers.

Instead of relying on a single API, the application can switch between providers whenever one becomes unavailable.

Anyone building AI applications eventually experiences:

rate limits
temporary outages
provider instability
API changes

Building redundancy into the system makes it much more reliable.

Production systems should never depend entirely on a single external service.

Quality Is Harder Than Generation

Initially, I thought article generation would be the difficult part.

It wasn't.

Modern LLMs are already very good at producing readable text.

The real challenge is determining whether that text is actually good enough.

The application assigned quality scores to generated articles.

At first glance, that seemed useful.

But during the audit I realized a problem.

If an article receives a score of 72, what does that actually mean?

Does it have poor SEO?

Weak citations?

Grammar issues?

Medical inaccuracies?

Without explanation, the score becomes little more than a number.

That experience taught me that AI systems should explain their decisions whenever possible.

Transparency builds trust.

Validation Is More Important Than Prompts

Like many developers, I spent a lot of time improving prompts.

Prompt engineering certainly matters.

But the audit showed that validation matters even more.

Instead of asking:

"How can I generate better content?"

I started asking:

"How can I detect bad content before users see it?"

Those are two completely different engineering problems.

Validation includes checking:

encoding issues
incomplete sections
hallucinated references
formatting problems
language consistency
citation quality

Good AI applications don't simply generate.

They verify.

Human Review Still Matters

One reason I included a review workflow from the beginning is that medical content deserves an additional layer of verification.

The application supports:

Pending
Approved
Rejected

This simple workflow ensures AI-generated content isn't automatically published without oversight.

As AI improves, human review may become lighter.

I don't think it disappears completely—especially for domains involving health, finance, or legal information.

Small Improvements Make a Big Difference

Interestingly, many improvements identified during the audit weren't exciting new features.

They were engineering improvements.

Things like:

stronger input validation
better logging
clearer quality reports
safer API configuration
improved Unicode handling
production-ready CORS settings

None of these changes produce flashy screenshots.

All of them improve reliability.

My Biggest Takeaway

The biggest lesson wasn't about FastAPI.

Or LLMs.

Or RAG.

It was about mindset.

When I first built the project, I kept asking:

"Does this feature work?"

After the audit, I started asking:

"What happens when this feature fails?"

That single question changes how you design software.

It encourages better validation.

Better logging.

Better testing.

Better security.

And ultimately, better user experience.

Final Thoughts

Auditing my own project reminded me that building AI applications isn't just about connecting models to prompts.

Real engineering starts after the demo.

It's about making systems reliable, maintainable, and trustworthy when people actually depend on them.

Healthy Gut AI still has room to grow, but the audit gave me confidence that improving production quality is often less about adding more features and more about strengthening the foundations that users rarely see.

If you're building AI applications, I'd encourage you to audit your own project before someone else does.

You might discover that your biggest opportunities aren't where you expected them to be.

TokenMizer: Giving LLMs a Memory That Doesn't Forget Between Sessions

Shweta Mishra — Tue, 04 Aug 2026 17:01:05 +0000

How a graph-memory proxy sits between your app and the OpenAI API, quietly remembering everything your LLM would otherwise lose.

Every conversation with a large language model starts from zero. Close the tab, start a new session, and the model has no idea who you are, what you discussed yesterday, or what you decided last week. Most tools work around this by stuffing more and more chat history into the context window — which is expensive, slow, and eventually hits a hard limit.

I built TokenMizer to solve this differently: instead of remembering by re-reading everything, it remembers by building a graph.

The Core Idea: A Proxy, Not a Plugin

TokenMizer sits as a proxy in front of any OpenAI-compatible API. Your application doesn't change how it calls the model — it just points its API base URL at TokenMizer instead of directly at OpenAI. Every request and response passes through, gets analyzed, and gets stored before continuing on to the real model.

This design choice matters more than it looks. It means TokenMizer works with any framework or app that already speaks the OpenAI API format, with no SDK changes and no rewritten integration code. You add memory to an existing app by changing one URL.

How Memory Actually Gets Built

Two systems do the heavy lifting.

File Intelligence watches what code, documents, or files are referenced during a conversation and builds context around them — so if you're debugging the same file across three sessions, TokenMizer already knows its history in the conversation, not just its current contents.

Graph Memory is the more interesting part. Instead of storing conversation history as a flat log, TokenMizer extracts entities and relationships — people, projects, decisions, dependencies — and stores them as nodes and edges in a graph, backed by SQLite. When a new message comes in, TokenMizer doesn't search through old transcripts; it queries the graph for relevant nodes and pulls in only what's connected to the current topic.

The practical difference: a flat-log memory system gets slower and more expensive as history grows, because it has to search more text. A graph memory system stays fast, because it's traversing relationships, not scanning transcripts.

Seeing the Graph: The D3.js Explorer

Memory systems that live entirely inside a database are hard to trust, because you can't see what they're doing. TokenMizer ships with a Graph Explorer built on D3.js — a visual, interactive map of every entity and connection the system has learned. You can watch new nodes appear as a conversation progresses, or trace why the model suddenly "remembered" something from three sessions ago by following the edge back to its source.

This turned out to be more useful for debugging than for demos. When memory retrieval pulls in the wrong context, the graph view shows exactly why — which node matched, and through which relationship.

Two Ways In: CLI and MCP

TokenMizer is distributed as a pip-installable library, so it drops into an existing Python environment directly. For day-to-day use there's a CLI to inspect, query, and manage the memory graph without writing code.

The second integration path is MCP (Model Context Protocol) server support, which lets TokenMizer's memory be used as a tool by MCP-compatible clients — including editors like Cursor. This was a deliberate design decision: memory shouldn't be locked to one app. Whether you're calling the API directly, scripting through the CLI, or working inside an AI-assisted editor, the same graph is behind all of it.

What Broke, and What I Learned Fixing It

Building a proxy that intercepts every API call raises the stakes on reliability — if TokenMizer fails, your app's LLM calls fail with it. An internal audit surfaced real problems worth naming honestly:

Async race conditions in request handling, where concurrent calls could read or write memory state out of order.
Silent crash patterns repeated across 12 files — errors that failed quietly instead of surfacing, similar in spirit to the fail-closed lesson from my other project, GitHub Autopilot.
A regex regression that broke entity extraction for a subset of inputs after a refactor.
A data-destruction bug in the MCP installer — an edge case where installation could overwrite existing memory data instead of merging with it.

All nine verified issues were fixed and tested. The installer bug in particular changed how I think about setup scripts generally: anything that touches a user's existing data on install needs to default to the safest possible behavior, even if that means asking one more confirmation question.

Why Build This Instead of Using Mem0 or Zep

Existing memory layers for LLMs mostly work — but they usually mean adopting their SDK and their storage model. TokenMizer's proxy-first design means it can sit under tools you already use without asking you to restructure how you call the model. The graph-based retrieval is also a deliberate bet: as conversations and codebases grow, relationship-based lookup scales in a way flat retrieval doesn't.

It's early — this is a project built in the open, not a finished product — but the architecture is stable enough now to be worth explaining properly, and worth other developers trying against their own workflows.

TokenMizer is open source and pip-installable. Code, CLI docs, and the Graph Explorer are on GitHub: Shweta-Mishra-ai/tokenmizer.

I write about building developer tools and AI infrastructure at TechNova World. If you need someone who can build a system like this and explain it clearly to your team or your users, let's talk.

What Auditing My Own AI Projects Taught Me About Shipping Production Code

Shweta Mishra — Sun, 02 Aug 2026 19:01:22 +0000

Two projects, thirteen serious bugs, and one pattern that kept showing up everywhere.

Most engineers find out their code has problems when a user reports one. I found out earlier — by going back through two of my own projects and auditing them properly, on purpose, before anyone else had to.

The projects were different in almost every way. GitHub Autopilot is a Flask app that automates code review and PR management through GitHub webhooks. TokenMizer is a proxy that intercepts LLM API calls to give them persistent memory. One handles webhook traffic; the other handles model requests. Different languages of failure, you'd think.

They weren't. The same failure pattern showed up in both, wearing different clothes each time. Here's what I found, and what it taught me about the difference between code that works and code that's actually safe to ship.

The Pattern: Failure That Doesn't Announce Itself

In GitHub Autopilot, I found 27 places where the code caught an exception and did nothing with it — no log, no alert, just silent continuation. In TokenMizer, the same shape of bug appeared across 12 files as silent crash patterns, plus an async race condition where concurrent requests could read or write memory state out of order without anyone noticing until the data was already wrong.

Neither of these is a bug you catch by staring at the code. They're bugs you catch by asking a different question than "does this work?" The right question is: when this fails — and eventually it will — does the failure tell anyone?

Code that fails loudly is annoying but honest. Code that fails silently is comfortable to write and dangerous to run, because it lets small problems compound into large ones with no trail to follow back. Every fix I made across both projects, at its core, was about converting a silent failure into a visible one.

Three Specific Failures, One Underlying Habit

GitHub Autopilot's auth bypass: the MCP authentication check caught its own exceptions and let the request through — failing open instead of closed. A slow auth service became an accidental backdoor.

GitHub Autopilot's rate limiter: it tracked request counts per IP but never cleaned up stale entries, so the defense meant to stop abuse became a memory leak that could crash the app under sustained traffic.

TokenMizer's installer bug: an edge case in the MCP installer could overwrite a user's existing memory graph on install, instead of merging with it — a data-destruction bug hiding inside what looked like routine setup code.

Three different subsystems, three different consequences — a security hole, an outage vector, a data-loss bug. But look at the shared habit underneath: in all three cases, the code assumed the happy path would hold, and didn't plan for what happens when it doesn't. The auth check assumed the auth service stays up. The rate limiter assumed traffic stays bounded. The installer assumed there's nothing to lose.

Good defensive code doesn't assume the happy path. It asks "what's the safe default when I don't know what's going on?" — and for anything touching security or user data, the safe default is almost always: deny, don't proceed, don't overwrite.

Testing Isn't the Finish Line — It's How You Prove the Fix

Finding bugs is one skill. Proving they're actually fixed is a different one, and it's the part that's easy to skip under deadline pressure. For GitHub Autopilot, that meant writing targeted tests that reproduce each original failure — a forged request header, an auth-service timeout, sustained traffic from many IPs — and raising overall coverage from 62% to 76% along the way, with all 654 tests passing across an 834-test suite. For TokenMizer, it meant verifying all nine issues individually rather than assuming a broad refactor had swept them up.

The number that matters isn't the coverage percentage. It's whether each specific failure mode you found has a test that would catch it coming back. Coverage without targeted regression tests is a vanity metric; targeted tests without coverage tracking mean you don't know what you haven't checked. You need both.

Why I Wrote This Down

I could have fixed these bugs quietly and moved on — nobody was demanding an audit, no user had filed a report. I did it because the fixing wasn't actually the valuable part. The valuable part was noticing the pattern across two unrelated codebases, and writing it down so it's a reusable lesson instead of a one-off cleanup.

That's the habit I'd recommend to any engineer, and it's also, honestly, the reason I enjoy technical writing as much as building: a bug fix helps one project. A clearly explained pattern helps every project you touch after it. If there's one thing worth taking from two audits and thirteen bugs, it's this — the question that finds real problems isn't "does it work in the demo?" It's "what happens when the thing I'm depending on doesn't behave?" Ask that question early, and you fix bugs before they have users attached to them.

GitHub Autopilot and TokenMizer are both open source: github-autopilot · tokenmizer.

I build developer tools and write about what breaks while building them, at TechNova World. If you want documentation that comes from someone who's actually shipped and audited production code, let's talk.

Building a GitHub App That Reviews Its Own Code: Lessons in Security Hardening

Shweta Mishra — Sun, 26 Jul 2026 08:44:16 +0000

How I turned 27 silent failures into logged ones, closed three real attack surfaces, and pushed test coverage from 62% to 76% while building an automated code-review bot.

A few months ago I set out to build something simple on paper: a GitHub App that reviews pull requests, scans for leaked secrets, applies safe autofixes, and responds to slash commands like a teammate would. I called it GitHub Autopilot. It runs on Flask, uses Redis for job queuing, and talks to GitHub through webhooks.
The first working version took a couple of weeks. Making it safe enough to trust with someone else's repository took much longer - and taught me more about security engineering than any tutorial could.
This article walks through three real vulnerabilities I found and fixed during an internal audit, why each one mattered, and what the process looked like end to end.
Why a Code-Review Bot Is a Security Target
A GitHub App that can read code, comment on PRs, and push autofixes sits in a privileged position. It has write access, it processes untrusted input (every PR, every webhook payload), and it often talks to other services - in my case, an MCP (Model Context Protocol) server for AI-assisted review.
That combination means three things need to be airtight: authentication, request validation, and resource limits. I had working code for all three. What I didn't have, until I audited it properly, was proof that each one failed safely under attack.
Problem 1: Authentication That Failed Open
The MCP integration handled requests from an external service. My original authentication check worked like this: if a token was present, validate it; if validation itself threw an error - say, the auth service was slow or unreachable - the code caught the exception and let the request through.
This is called failing open, and it's one of the more common mistakes in systems that bolt security onto an existing code path. The intention was reasonable: don't let a flaky dependency take down the whole app. The result was dangerous: an attacker who could trigger an auth-service timeout could skip authentication entirely.
The fix was to flip the default. Any exception during authentication now results in an automatic denial, not a pass-through. If the auth service is unreachable, the request is rejected, logged, and retried - never silently trusted. This is the standard fail-closed pattern, and it should be the default for any security check, full stop.
Problem 2: A Content-Length Bypass
Webhook payloads come in with a Content-Length header, and the app used it to enforce a size limit before processing - a reasonable defense against oversized or malicious payloads. The gap: the check trusted the header value itself rather than the actual bytes received.
A request could declare a small Content-Length while streaming a much larger body, slipping past the size check entirely. This is a known class of bug in HTTP handling, and it's easy to miss because the code "looks" correct - it reads a header and compares a number.
The fix was to validate against the actual size of the data read from the stream, not the client-supplied header. It's a small code change, but it closes a real gap between what a client claims and what a server receives - a distinction that matters anywhere you're parsing untrusted input.
Problem 3: A Rate Limiter That Leaked Memory
The app rate-limits requests per IP address to prevent abuse. The original implementation stored a counter per IP in memory, incrementing on each request. What it didn't do was clean up entries for IPs that stopped sending requests.
Under normal traffic this is invisible. Under sustained traffic from many different IPs - which is trivial to generate - the counter dictionary grows without bound. Eventually the process runs out of memory and crashes. This turns a defensive feature into an attack vector: the very thing meant to stop abuse becomes the tool for causing an outage.
The fix added a time-based eviction policy, clearing stale IP entries on a rolling window instead of letting them accumulate forever. Rate limiters need to bound their own memory usage, not just the request rate - a detail that's easy to skip when the feature works correctly in every manual test.
The Quieter Problem: 27 Silent Failures
None of the three bugs above would have been easy to catch through code review alone, and that pointed to a deeper issue: 27 places in the codebase caught exceptions and did nothing with them. A try/except: pass pattern, repeated across error-handling paths, meant that when something went wrong, the app kept running - silently, with no log entry, no alert, no trace.
This is a comfortable pattern to write and a dangerous one to ship. It hides exactly the kind of failure that matters most: the one that happens in production, once, under conditions you didn't test for.
Every one of those 27 handlers was rewritten to log the failure with enough context to debug it later - what operation failed, what input triggered it, and the original exception. None of them changed what the app does when something breaks. All of them changed whether you'd ever find out.
Proving It: Testing and Coverage
Fixes without tests are opinions. Each of the security changes above shipped with tests that reproduce the original failure mode - a forged Content-Length, an auth-service timeout, a rate-limit counter under sustained load - and assert the new, safe behavior.
Across the full audit cycle, the project went from 62% to 76% test coverage, with 654 out of 654 tests passing and 834 total tests in the suite. Coverage numbers alone don't prove correctness, but combined with targeted tests for each vulnerability, they gave me confidence to say the fixes actually work - not just that the code compiles.
What This Project Taught Me About Documentation
Fixing the bugs was half the work. The other half was writing it down in a way a future contributor - or an auditor, or a client - could actually use: an architecture diagram showing how webhooks flow through auth, queueing, and processing; a threat-model table mapping each attack surface to its mitigation; and a changelog that explains why each fix happened, not just what changed.
That last part turned out to matter most. Code shows what a system does. Documentation is the only place that shows what it's defending against, and why the defense looks the way it does. For any system that handles untrusted input - which is most systems - that record is worth as much as the fix itself.
GitHub Autopilot is open source. The full architecture docs, threat-model table, and audit history are available on GitHub: Shweta-Mishra-ai/github-autopilot.
I write about building and securing developer tools at TechNova World. If you're looking for someone who can both build the system and document it clearly for your team, let's talk.