DEV Community: Vivek Shetye

I Built an AI Team That Researches Stocks and Debates Investment Decisions 🤖

Vivek Shetye — Tue, 03 Nov 2026 15:02:54 +0000

What if you had an entire team of AI agents researching stocks for you?

Imagine one agent analyzing stock charts, another digging through financial reports, and another tracking the latest market news.

Now imagine two more AI agents arguing about whether you should buy or avoid a particular stock.

And instead of simply combining their opinions, the system makes them challenge each other’s conclusions before arriving at a final investment recommendation.

Sounds like something a hedge fund might build, right?

Well, there’s an open-source project that does exactly this.

It’s called TradingAgents, developed by I Built an AI Team That Researches Stocks and Debates Investment Decisions 🤖
, and it uses a multi-agent AI architecture to simulate how a financial research team collaborates.

I recently explored this project, studied its architecture, set it up, and put the agents to work analyzing one of the most talked-about companies in the stock market: NVIDIA (NVDA).

The result was interesting, but what fascinated me even more was how the agents worked together.

In this article, I’ll explain how TradingAgents works, walk through its architecture, show you how to get started, and share what happened during my experiment.

🎥 Full video walkthrough

💡 Why Use Multiple AI Agents for Stock Research?

If you’ve ever researched a stock before investing, you know how overwhelming the process can become.

You need to understand several things:

Financial performance: Revenue, earnings, profitability, and cash flow.
Valuation: Whether the stock is reasonably priced relative to its growth expectations.
Technical indicators: Price movements, moving averages, RSI, and MACD.
Market news: Earnings announcements, industry developments, and economic events.
Investor sentiment: How the broader market perceives the company.
Risk factors: Everything that could potentially go wrong.

And here’s something investors sometimes overlook:

A great company isn’t necessarily a great stock to buy at any price.

A business might be growing rapidly and dominating its industry, but if its stock price already reflects unrealistic growth expectations, investing could still be risky.

Now, you could ask ChatGPT or another LLM:

“Should I buy NVIDIA stock?”

But a single AI response might overlook important factors or lean too heavily toward one interpretation.

What if we approached this differently?

Instead of asking one AI model to handle everything, we could assign different research responsibilities to specialized AI agents.

One focuses on financial fundamentals. Another analyzes technical indicators. Another investigates risks.

Then we make them collaborate, debate, and challenge each other’s conclusions.

That’s the fundamental idea behind TradingAgents.

🏗️ What Is TradingAgents?

TradingAgents is an open-source multi-agent framework developed by Tauric Research.

It’s designed to simulate the collaborative decision-making process of a financial research and trading organization.

Rather than relying on a single LLM prompt, TradingAgents coordinates multiple specialized agents through a structured workflow.

The system includes:

Financial research analysts
Bullish and bearish researchers
A research manager
A trader agent
Risk management agents
A portfolio manager

Each agent has a specific responsibility, and information flows between them as the research progresses.

The interesting part is that the agents don’t simply generate independent reports.

They share findings, debate competing investment theses, evaluate risk, and eventually produce a consolidated recommendation.

Let’s understand how this works.

🔍 Step 1: Four AI Analysts Research the Stock

The workflow begins with four specialized research agents.

Each examines the company from a different perspective.

📊 1. Market Analyst

The Market Analyst focuses on technical analysis.

It examines information such as:

Historical stock prices
Moving averages
Relative Strength Index (RSI)
Moving Average Convergence Divergence (MACD)
Recent price trends

For example, the RSI can help identify whether a stock has experienced unusually strong buying or selling momentum.

The analyst uses available technical indicators to understand what’s happening with the stock’s price behavior.

💰 2. Fundamental Analyst

The Fundamental Analyst investigates the underlying business.

It examines financial information such as revenue, earnings, profitability, and other available company data.

Its goal is to answer questions like:

Is this company financially strong?

What do its fundamentals suggest about its business performance?

Are there financial weaknesses investors should consider?

For a company like NVIDIA, this analysis might involve examining revenue growth, margins, profitability, and the sustainability of demand for its products.

📰 3. News Analyst

Financial performance alone doesn’t tell the entire story.

A company’s outlook can change significantly because of new developments.

The News Analyst investigates relevant information such as:

Company announcements
Earnings-related developments
Industry news
Economic events
Broader market developments

For example, changes in semiconductor export restrictions or major AI infrastructure spending announcements could influence how investors evaluate NVIDIA.

🗣️ 4. Sentiment Analyst

The Sentiment Analyst examines available public discussions and sentiment-related information.

Its goal is to understand how investors and the broader market are discussing the company.

This can provide useful context about market expectations, enthusiasm, and concerns.

However, sentiment isn’t necessarily an indicator of whether a stock is fundamentally undervalued or overvalued.

It’s simply another perspective.

Why This Architecture Matters

The four analysts can gather information in parallel.

Rather than forcing a single AI agent to perform every research task, the system distributes responsibilities across specialized agents.

Their findings become the foundation for the next stage.

And this is where things get particularly interesting.

🐂 Step 2: Bull vs. Bear — AI Agents Debate the Investment

Once the analysts complete their research, their findings are passed to two additional agents.

The Bull Researcher and the Bear Researcher.

These agents have opposing responsibilities.

The Bull Researcher

The Bull Researcher builds the strongest possible argument for investing in the company.

For example, a bullish NVIDIA investment thesis might emphasize:

Strong demand for AI computing infrastructure
Revenue and earnings growth
Competitive advantages
Market leadership
Long-term industry opportunities

The Bear Researcher

The Bear Researcher takes the opposite position.

Its responsibility is to challenge the bullish argument and identify potential weaknesses.

For example:

Is the stock valuation too expensive?
Are growth expectations unrealistic?
Could competition increase?
Are there regulatory or geopolitical risks?
What happens if AI infrastructure spending slows?

But here’s the interesting part.

These agents actually debate each other.

The Bull presents an argument.

The Bear challenges it.

The Bull gets an opportunity to respond.

And the discussion can continue through multiple configurable rounds.

Instead of simply asking an LLM to list pros and cons, TradingAgents creates a structured exchange between competing investment perspectives.

After the debate, a Research Manager evaluates the arguments and produces an investment plan.

This approach doesn’t guarantee better investment decisions, but it provides a mechanism for examining competing interpretations before reaching a conclusion.

⚖️ Step 3: The Trader and Risk Management Agents

After the Research Manager produces an investment plan, the workflow moves to the Trader Agent.

Despite its name, the Trader Agent in my demonstration wasn’t executing any actual trades.

Instead, it translated the research into a more concrete investment proposal.

That proposal then went through another review process.

This time, three risk-focused agents evaluated it.

Aggressive Risk Analyst

Examines the investment proposal from a perspective that accepts greater risk in pursuit of potential returns.

Conservative Risk Analyst

Focuses on downside protection, uncertainty, and reasons to avoid excessive exposure.

Neutral Risk Analyst

Attempts to balance potential opportunities against the associated risks.

These perspectives are used to challenge the investment proposal.

Finally, the Portfolio Manager evaluates the research, proposed action, and risk discussion to produce the final recommendation.

So the overall workflow looks something like this:

              STOCK / COMPANY
                     |
                     v
            RESEARCH ANALYSTS
                     |
        +------------+------------+
        |            |            |
     Market      Fundamental     News
     Analyst       Analyst      Analyst
        |            |            |
        +------------+------------+
                     |
              Sentiment Analyst
                     |
                     v
             RESEARCH DEBATE
                     |
              +------+------+
              |             |
          Bull Agent     Bear Agent
              |             |
              +------+------+
                     |
                     v
              Research Manager
                     |
                     v
                Trader Agent
                     |
                     v
               RISK ANALYSIS
                     |
          +----------+----------+
          |          |          |
      Aggressive  Conservative  Neutral
          |          |          |
          +----------+----------+
                     |
                     v
              Portfolio Manager
                     |
                     v
            FINAL RECOMMENDATION

Simplified conceptual representation of the TradingAgents workflow. The analyst stage is designed to support parallel research.

This is what makes TradingAgents an interesting example of agentic AI and multi-agent orchestration.

It’s not just multiple LLM calls.

It’s a structured workflow in which different agents contribute information, challenge conclusions, and participate in a decision-making process.

🛠️ Step 4: Installing and Setting Up TradingAgents

Now that we understand the architecture, let’s look at how to run it.

TradingAgents is open source, so you can inspect the implementation, modify the agents, and experiment with different configurations.

1. Clone the GitHub Repository

Start by cloning the project:

git clone https://github.com/TauricResearch/TradingAgents.git
cd TradingAgents

Open the project in your preferred code editor.

I used VS Code for my demonstration.

2. Create a Python Virtual Environment

I used Python 3.13 and uv for environment management.

Create a virtual environment:

uv venv --python 3.13

Activate it on macOS or Linux:

source .venv/bin/activate

This keeps the project’s Python dependencies separate from other applications installed on your machine.

3. Install the Dependencies

Install the project using the dependency instructions provided in the repository.

For example, when installing from the project directory with uv:

uv pip install -e .

The exact installation steps may change between releases, so check the repository’s latest README before proceeding.

4. Configure Your AI Model Provider

TradingAgents supports configurable AI model providers.

Depending on the version and configuration, you can use supported providers such as OpenAI, Google, or Anthropic.

The repository includes an example environment configuration file.

You can copy it:

cp .env.example .env

Then configure the credentials required by your selected provider.

In my demonstration, I used the framework’s supported ChatGPT subscription authentication flow rather than manually configuring an API key.

For versions supporting this option, the login command is:

tradingagents login chatgpt

Follow the authentication instructions displayed in your terminal.

Note: Subscription-based authentication depends on the installed TradingAgents version and its supported integration. A ChatGPT subscription is not generally interchangeable with OpenAI API credits.

5. Launch TradingAgents

Once everything is configured, launch the CLI:

tradingagents

The interface guides you through several configuration options.

You can select:

The stock ticker
Analysis date
Output language
Research analysts
Research depth
AI model provider
LLM models
Reasoning effort

For my experiment, I selected NVIDIA (NVDA) and enabled all four research analysts.

I also chose the shallow research configuration to keep the demonstration relatively quick.

Research depth affects how many debate and discussion rounds the agents perform.

Once configured, the agents begin researching the company.

🚀 Step 5: I Asked the AI Team to Analyze NVIDIA

With everything running, I gave the system NVIDIA as the company to investigate.

The workflow started with the research analysts gathering information.

The Market Analyst examined technical indicators.

The Fundamental Analyst investigated the company’s financial position.

Meanwhile, the News and Sentiment Analysts collected information from their available data sources.

Once the initial research was complete, the system moved through the debate and risk evaluation stages.

Eventually, TradingAgents generated a detailed report containing the findings from the different agents.

The report included technical analysis, financial research, sentiment observations, news analysis, and the final investment thesis.

So, What Did the AI Agents Recommend?

After completing the analysis, the Portfolio Manager produced its final recommendation:

UNDERWEIGHT.

In portfolio terminology, an underweight position generally means maintaining less exposure to an asset than a relevant benchmark or reference allocation.

But I wouldn’t interpret this result as proof that NVIDIA is a bad investment.

The recommendation reflects the model-generated research, available information, and configuration used during this particular run.

I haven’t established whether this recommendation would outperform a benchmark or lead to better investment outcomes.

And that’s an important distinction.

Generating a convincing investment thesis is not the same as demonstrating predictive accuracy.

What I found useful was the structured research process.

Instead of receiving a short answer from a chatbot, I could examine the different research perspectives that contributed to the final decision.

That makes TradingAgents particularly interesting as an investment research assistant.

🧠 TradingAgents Also Has a Memory System

Another feature worth exploring is the framework’s built-in memory and historical evaluation capabilities.

After completing an analysis, TradingAgents can save information about the research decision in a local journal.

This can include:

Stock ticker
Analysis date
Generated recommendation
Evaluation status

Initially, the outcome is marked as pending.

That makes sense because the system doesn’t know what will happen to the stock after the recommendation.

Once sufficient trading sessions have passed, the framework can evaluate the stock’s subsequent performance against a benchmark.

In the configuration I explored, the default evaluation window was five trading days.

Based on the evaluation, an AI model can generate a reflection about the previous research.

For example:

What did the analysis get right?

What risks did it overlook?

Which assumptions turned out to be incorrect?

What lessons might be useful for future research?

Those reflections can then be retrieved as context for subsequent analyses.

This creates a basic feedback mechanism.

However, it’s important to understand that memory-based reflection is not the same as training or fine-tuning the underlying LLM.

The framework can reuse previous lessons as contextual information, but that doesn’t automatically mean its investment predictions improve over time.

Demonstrating that would require systematic evaluation.

🔮 What If We Took This Project Further?

After experimenting with TradingAgents, I started thinking about what else we could build using this architecture.

Because the project is open source, there are several interesting directions to explore.

1. Build an AI-Powered Stock Screener

Instead of analyzing one company at a time, imagine providing the system with a watchlist of 50 stocks.

The agents could research each company, evaluate its investment thesis, and produce a shortlist of opportunities worth investigating further.

You could then compare the companies using consistent research criteria.

2. Create a Financial Research Dashboard

Another possibility is building a dashboard that tracks the agents’ research over time.

For example, it could display:

Companies being monitored
Latest research recommendations
Bullish and bearish arguments
Risk assessments
Historical recommendations
Subsequent stock performance

This could make it easier to compare how investment theses evolve as new information becomes available.

3. Introduce Industry-Specific AI Agents

Different industries require different types of analysis.

For example, we could introduce specialist agents for:

Semiconductors: Manufacturing capacity, AI infrastructure demand, supply chains, and competitive positioning.

Healthcare: Clinical trials, regulatory approvals, product pipelines, and reimbursement risks.

Technology: Revenue growth, recurring revenue, customer retention, and competitive advantages.

These agents could supplement the general research team with more specialized domain knowledge.

4. Evaluate Whether Multi-Agent Research Actually Works

This is probably the most interesting direction from a research perspective.

We could systematically compare:

Single LLM vs. Multi-Agent Research vs. Traditional Financial Screening

And evaluate metrics such as:

Research completeness
Factual accuracy
Recommendation consistency
Risk identification
Token usage and cost
Latency
Performance against relevant benchmarks

We could also run historical evaluations while carefully controlling for look-ahead bias and survivorship bias.

That would help answer a much more meaningful question:

Do multi-agent AI systems actually improve investment research quality, or do they simply generate more elaborate reports?

🎬 I Stopped Making Motion Graphics Manually. Here’s How I Use AI Instead.

Vivek Shetye — Tue, 27 Oct 2026 15:05:51 +0000

What if you could create professional motion graphics for your videos without spending hours manually animating everything?

No After Effects. No manually setting keyframes. No endlessly adjusting animation timelines.

Just ChatGPT + Codex + Remotion.

I’ve been experimenting with a workflow that lets me turn ordinary video recordings into animated explainer videos using AI.

And here’s what makes it interesting:

The animations actually follow what I’m saying.

When I introduce a concept, the corresponding visual appears at exactly the right moment. When I move to the next idea, the animation transitions along with my narration.

All without manually creating every animation.

In this article, I’ll walk you through my entire workflow, from a simple video recording to a finished motion graphics video.

And the best part? You don’t need to be an experienced programmer or motion graphics designer to get started.

🎥 Full video walkthrough

🚀 The Problem with Traditional Motion Graphics

If you’ve ever tried creating motion graphics for YouTube videos, tutorials, or educational content, you probably know how time-consuming the process can be.

Traditionally, creating motion graphics involves:

Designing individual visual elements.
Positioning objects on the screen.
Creating and adjusting keyframes.
Managing animation timelines.
Synchronizing visuals with narration.
Exporting and reviewing the final video.

Tools like Adobe After Effects are incredibly powerful, but creating polished animations requires time, experience, and patience.

And when you’re producing educational videos regularly, spending hours animating a 20-second explanation isn’t always practical.

That’s what got me interested in AI-powered motion graphics generation.

Instead of manually animating everything, what if I could describe the visuals I wanted and let AI build them?

Turns out, we can.

🧠 The AI Motion Graphics Stack

My workflow uses three main tools:

1. ChatGPT — The Creative Director

I use ChatGPT to plan my animations.

It helps me transform a video transcript into a structured storyboard that defines:

What should appear on screen.
When each animation should start and end.
How elements should move.
What transitions should be used.
Which images and visual assets should appear.

Essentially, ChatGPT acts as my motion graphics designer.

2. Codex — The Developer

Once the storyboard is ready, I pass it to Codex.

Codex takes the instructions and generates the actual animation code using React and Remotion.

It creates scenes, positions elements, adds transitions, and synchronizes everything with the narration.

3. Remotion — The Video Engine

Remotion is an open-source framework that lets developers create videos programmatically using React.

Instead of building animations through a traditional editing timeline, you define scenes and animations through code.

Remotion then renders that code into a video.

This makes it particularly interesting when combined with AI coding agents.

The workflow looks like this:

         Original Video Recording
                    |
                    v
         Timestamped Transcript
                    |
                    v
           ChatGPT (Storyboard)
                    |
                    v
           Codex (React Code)
                    |
                    v
         Remotion (Preview + Render)
                    |
                    v
          Final Motion Graphics

ChatGPT plans → Codex builds → Remotion renders.

That’s the foundation of the entire workflow.

🛠️ Step 1: Set Up Your Remotion Project

Let’s start by creating a new Remotion project.

First, create an empty folder and open it in VS Code.

Then open your terminal and run:

npx create-video@latest .

This initializes a Remotion project in your current directory.

During setup, I usually select the Blank template because I prefer building animations from scratch.

Once the project is initialized, install the dependencies if they weren’t installed automatically.

npm install

Next, start the development server:

npm run dev

This launches Remotion Studio, where you can preview your animations, inspect individual frames, and render your final video.

At this point, we have a blank project.

Now we need to give AI something to work with.

Bonus: Install Remotion Agent Skills

I also install the Remotion agent skills for my coding agent.

These provide additional instructions and best practices for working with Remotion.

They help AI coding agents understand common animation patterns, composition structures, and recommended approaches.

📝 Step 2: Generate a Timestamped Transcript

Here’s something important about my workflow:

I don’t start by asking AI to generate random animations.

I start with the actual video recording.

For example, in one of my previous YouTube videos, I explained how AI agents work.

There’s a section where I introduce the concept of an agent loop.

I wanted to create an animation showing how an AI agent repeatedly:

Thinks about what to do.
Takes an action.
Observes the result.
Decides what to do next.

But for the animation to work, AI needs to understand two things:

WHAT I’m saying and WHEN I’m saying it.

That’s where timestamped transcripts come in.

I generate an SRT transcript of my recording.

Here’s a simplified example:

1
00:00:00,000 --> 00:00:04,000
At the center of an AI agent is an AI model.
2
00:00:04,000 --> 00:00:08,000
The model helps decide what to do next.
3
00:00:08,000 --> 00:00:13,000
The agent takes an action and observes the result.
4
00:00:13,000 --> 00:00:18,000
This process continues until the task is complete.

Now AI knows exactly when each concept is introduced.

For example:

At 0 seconds, display the AI model.
At 4 seconds, animate the decision-making process.
At 8 seconds, show the agent taking an action.
At 13 seconds, illustrate the repeating agent loop.

This is what makes the animations synchronize with the narration.

You can generate SRT transcripts using any transcription tool that supports timestamps.

Organizing Your Assets

I also place any required images, logos, and other visual assets inside my Remotion project’s public directory.

For example:

my-motion-graphics/
│
├── public/
│   ├── transcript/
│   │   └── transcript.srt
│   │
│   ├── image-assets/
│   │   ├── ai-model.png
│   │   ├── agent-icon.svg
│   │   └── background.png
│   │
│   └── audio/
│       └── narration.wav
│
├── src/
│
└── package.json

Now we have everything we need:

The original narration, a timestamped transcript, and our visual assets.

🎨 Step 3: Use ChatGPT to Create a Detailed Storyboard

This is probably the most important part of the entire process.

One mistake I initially made was giving AI vague instructions like:

“Create some cool motion graphics for this video.”

The problem?

AI might generate something visually interesting, but that doesn’t necessarily mean the animations help explain the content.

Instead, I use ChatGPT to create a detailed storyboard before generating any code.

I provide:

The specific transcript section I want to animate.
The full transcript for context.
My preferred visual style.
The location of my visual assets.
The output resolution and frame rate.

For example, here’s a simplified version of the prompt:

Write a prompt for codex to create a motion graphics video using Remotion. Give it a personality of an expert motion graphics designer.
I want to create professional motion graphics for
a YouTube explainer video using Remotion.
Your task is to analyze the provided transcript
and create a detailed animation storyboard.
Requirements:
1. Synchronize animations with the transcript timestamps.
2. Create visuals that help explain the narration.
3. Use a clean, minimal explainer-video style.
4. Avoid unnecessary or distracting animations.
5. Include scene-by-scene animation instructions.
6. Specify transitions, element positioning,
   and animation timing.
7. Use existing visual assets wherever appropriate.
8. Output resolution: 4K (3840x2160).
9. Frame rate: 30 FPS.
Transcript section:
[PASTE TRANSCRIPT]
Full video transcript:
[PROVIDE FULL TRANSCRIPT]
Visual assets:
public/image-assets/
Generate a detailed storyboard that can be
passed directly to an AI coding agent.

ChatGPT then generates a structured storyboard.

This is much more useful than simply asking AI to generate animations.

We now have a clear visual plan.

And that plan is what we give to Codex.

💻 Step 4: Let Codex Build the Motion Graphics

Now comes the fun part.

I open Codex inside VS Code and provide the storyboard generated by ChatGPT.

Codex can then:

Create React components for individual scenes.
Generate SVG graphics where needed.
Add transitions and animations.
Position elements on screen.
Synchronize scenes with the transcript.
Integrate existing images and assets.
Prepare the final Remotion composition.

Because Remotion uses React, Codex can generate the animation programmatically.

For example, here’s a simple Remotion animation:

import { useCurrentFrame, interpolate } from "remotion";
export const AnimatedTitle = () => {
  const frame = useCurrentFrame();
  const opacity = interpolate(frame, [0, 20], [0, 1], {
    extrapolateRight: "clamp",
  });
  const translateY = interpolate(
    frame,
    [0, 20],
    [40, 0],
    {
      extrapolateRight: "clamp",
    }
  );
  return (
    <div
      style={{
        opacity,
        transform: `translateY(${translateY}px)`,
        fontSize: 80,
        fontWeight: "bold",
        color: "white",
      }}
    >
      How AI Agents Work
    </div>
  );
};

This creates a simple title animation that fades in while moving upward.

Now imagine combining dozens of components like this with precise timing, transitions, and visual elements.

That’s essentially what Codex helps automate.

And instead of manually writing all this code, I describe the animation I want.

Let Codex Verify Its Own Work

Another useful part of my workflow is asking Codex to verify the generated animations.

I instruct it to:

Render screenshots of important frames.
Inspect the visual layout.
Identify overlapping or misplaced elements.
Fix any issues it detects.
Verify that animations follow the storyboard.

This doesn’t guarantee perfect results, but it helps catch common visual problems before rendering the entire video.

🎥 Step 5: Preview, Refine, and Render

Once Codex finishes generating the animations, I open Remotion Studio.

This is where I can:

Preview the complete animation.
Scrub through individual frames.
Verify scene transitions.
Check synchronization with narration.
Identify visual issues.

If something doesn’t look right, I don’t necessarily need to modify the React code myself.

I can simply tell Codex what needs to change.

For example:

The transition between scene 2 and scene 3
feels too abrupt.
Make the transition smoother and ensure
the elements don't overlap.
Also, keep the AI model icon visible
throughout the entire sequence.

Codex can update the animation accordingly.

Once everything looks good, I render the final video using Remotion Studio or ask Codex to handle the rendering.

And that’s it!

We now have a finished motion graphics video synchronized with the original narration.

💡 What I’ve Learned from Using AI for Motion Graphics

After experimenting with this workflow, a few things have become clear.

1. Good Storyboards Matter More Than Fancy Prompts

The biggest improvement came from separating the planning and implementation stages.

Instead of asking one AI agent to figure everything out, I use ChatGPT to design the animation and Codex to implement it.

This gives me much more control over the final result.

2. Timestamps Make a Huge Difference

Without timestamps, animations might look good but appear at the wrong moments.

Using SRT transcripts makes it easier to align visuals with the narration.

However, timestamps are only a starting point. Some animations still need timing adjustments during preview.

3. Simpler Animations Often Work Better

Not every sentence needs an elaborate animation.

Sometimes a simple diagram, highlighted keyword, or animated arrow communicates an idea better than an overly complicated sequence.

The goal should always be to make the explanation easier to understand.

4. AI Doesn’t Eliminate the Need for Review

AI can generate impressive animations, but the first version isn’t always perfect.

You may still need to adjust timing, fix layouts, or refine transitions.

The advantage is that you can make many of these changes through natural-language instructions instead of manually editing keyframes.

🔥 Who Can Benefit from This Workflow?

Although I primarily use this for YouTube videos, there are plenty of other applications.

YouTube creators: Create animated explanations, visual storytelling sequences, and educational B-roll.

Developers: Turn technical concepts, architecture diagrams, and system workflows into animations.

Educators: Create engaging visual explanations for complex topics.

Startup founders: Build product explainers, feature demonstrations, and promotional videos.

Content marketers: Generate motion graphics for social media and marketing campaigns.

And because Remotion is code-based, you can reuse animation components and visual styles across multiple videos.

That makes this workflow especially interesting for creators producing content regularly.

🚀 Final Thoughts: AI Is Changing Video Production

A few years ago, creating professional motion graphics required specialized animation skills and significant manual effort.

Today, AI coding agents and frameworks like Remotion are making programmatic video creation much more accessible.

And what excites me most isn’t just the ability to generate animations faster.

It’s the possibility of creating visual explanations that are directly connected to the content itself.

Instead of manually deciding when every element should appear, we can use transcripts, storyboards, and AI agents to automate much of that process.

Of course, AI-generated motion graphics still require creative direction and quality checks.

But the amount of manual work we can eliminate is already impressive.

The future of video editing might involve a lot less dragging keyframes and a lot more describing what you want to create.

And personally, I’m excited to see where this goes.

How I Built a Fully Local AI Coding Agent with Qwen 3.8, LM Studio, and Pi (No Paid APIs)

Vivek Shetye — Tue, 13 Oct 2026 15:33:47 +0000

What if you could have an AI coding agent that reads your files, writes code, executes commands, and fixes errors without sending a single model request to the cloud?

No OpenAI API. No Anthropic API. No per-token billing.

Just an open-weight AI model running on your own hardware.

I recently decided to test exactly how practical this setup has become.

Using Qwen 3.8 27B, LM Studio, and Pi Coding Agent, I built a fully local AI coding workflow and challenged it to create an entire web application from scratch.

And the results were surprisingly good.

The agent built a React + TypeScript application, executed commands, tested its work, identified errors, and fixed them without requiring my intervention.

All while running a 27-billion-parameter model locally on my MacBook Pro.

But there were some interesting trade-offs, especially around inference speed, hardware requirements, and context management.

In this article, I’ll walk through the complete setup, explain why I chose these tools, and share what actually happened when I put the agent to work.

🎥 Full video walkthrough

In the video, I cover:

Installing LM Studio and downloading Qwen 3.8 27B.
Understanding 4-bit quantization.
Starting the local inference server.
Installing Pi Coding Agent.
Connecting Pi to LM Studio.
Testing local tool calling.
Building the Mission Control dashboard.
Reviewing the results, performance, and limitations.

🚀 What Did I Build?

Before getting into the setup, here’s what I asked my local AI coding agent to create.

An AI Agent Mission Control dashboard for monitoring and managing a team of autonomous AI agents.

The application needed several features:

Agent Overview: Monitor active agents, running tasks, completed tasks, and failures.
Agent Management: View agents, their roles, current tasks, and statuses.
Kanban Task Board: Organize tasks across Backlog, In Progress, Review, and Completed.
Live Activity Feed: Track agent activity and task execution.
Analytics Dashboard: Visualize token usage, estimated costs, success rates, and task performance.
Modern UI: A responsive dark-themed interface with charts, animations, and polished interactions.

And I didn’t want a static HTML mockup.

I wanted a working React application with realistic sample data.

The interesting part?

The entire coding workflow ran locally.

The AI model wasn’t hosted by OpenAI, Anthropic, or another cloud inference provider.

It was running directly on my computer.

🧠 The Architecture: Three Tools, One Local AI Coding Agent

The architecture is relatively simple.

           Developer
               |
               v
       +----------------+
       | Pi Coding Agent|
       |                |
       | Read Files     |
       | Write Code     |
       | Execute Bash   |
       | Edit Files     |
       +----------------+
               |
               | OpenAI-Compatible API
               v
       +----------------+
       |   LM Studio    |
       |                |
       | Local Inference|
       | Server         |
       +----------------+
               |
               v
       +----------------+
       |  Qwen 3.8 27B  |
       |                |
       | 4-bit Quantized|
       | Local AI Model |
       +----------------+
               |
               v
        Your Computer

Each component has a specific responsibility.

1. Qwen3.8 27B — The Brain

Qwen 3.8 27B is the language model responsible for understanding instructions, reasoning about coding tasks, and generating code.

For this experiment, I used a 4-bit quantized version of the model, approximately 16GB in size.

This makes it much more practical to run a large model on consumer hardware.

2. LM Studio — The Local Inference Server

LM Studio makes running open-weight AI models locally much easier.

Instead of manually setting up inference frameworks, Python environments, or model-serving infrastructure, you get a desktop application where you can:

Discover and download models.
Load models into memory.
Chat with models directly.
Configure inference settings.
Expose models through an OpenAI-compatible API.

That last feature is particularly important because it allows external applications to communicate with your locally running model.

3. Pi Coding Agent — The Hands

A language model alone cannot automatically inspect your project files, modify code, or execute terminal commands.

That’s where Pi comes in.

Pi is a lightweight, extensible terminal-based AI coding agent.

It provides the tools and agent loop needed to interact with your development environment.

Instead of simply generating code in a chat window, Pi can use the model to work through actual coding tasks.

And because Pi is relatively minimal, it’s particularly interesting for local model experiments.

⚙️ Step 1: Install LM Studio and Download Qwen 3.8 27B

First, download LM Studio from its official website:

LM Studio: https://lmstudio.ai/

Install the application for your operating system and hardware.

Once installed, open LM Studio and navigate to the model search section.

Search for:

Qwen 3.8 27B

You’ll notice that multiple versions of the same model may be available.

For this experiment, I selected a 4-bit quantized version.

Why 4-bit Quantization?

Running a 27-billion-parameter model at full precision can require substantial memory.

Quantization reduces the precision used to store model weights.

For example:

Precision	Approximate Raw Weight Storage
FP16 (16-bit)	54 GB
INT8 (8-bit)	27 GB
4-bit	13.5 GB

These are theoretical weight-storage estimates. Actual model files and runtime memory requirements can be higher because of quantization metadata, model architecture, KV cache, and other overhead.

The 4-bit model I downloaded was approximately 16GB.

That’s a significant reduction compared with FP16.

Of course, quantization comes with trade-offs.

Reducing precision can affect model accuracy, especially on tasks involving complex reasoning or code generation.

However, modern quantization techniques can often preserve much of the original model’s capabilities while substantially reducing memory requirements.

For running larger models locally, this is extremely useful.

Load the Model

After downloading:

Open My Models in LM Studio.
Select your downloaded Qwen model.
Click Load Model.
Configure the context length according to your available memory.
Load the model.

For my experiment, I configured a context length of 200,000 tokens.

Keep in mind that configuring a large context window doesn’t mean your hardware can efficiently process that entire context. Actual memory usage and performance depend on the model, runtime, and KV-cache configuration.

Once the model is loaded, you can test it using LM Studio’s built-in chat interface.

Try sending a simple message to verify that inference is working.

Now we need to make this model accessible to our coding agent.

🔌 Step 2: Enable LM Studio’s Local API Server

LM Studio can expose your local model through an OpenAI-compatible API.

This is what allows Pi Coding Agent to communicate with Qwen.

Inside LM Studio:

Navigate to the Developer tab.
Enable the local server.
Verify that your model is available.

By default, LM Studio uses port 1234.

The local API endpoint is:

http://localhost:1234/v1

You can verify that the server is responding by running:

curl http://localhost:1234/v1/models

This should return information about the models exposed by your LM Studio server.

At this point, your computer is acting as its own AI inference server.

And the best part is that you don’t need to send inference requests to an external model provider.

Note: Keep the server bound to localhost unless you intentionally need remote access. Exposing an inference endpoint to your network introduces additional security considerations.

🛠️ Step 3: Install Pi Coding Agent

Now that Qwen is running locally, we need an agent that can actually work with our codebase.

For this, I’m using Pi Coding Agent.

Official project:

https://pi.dev/

Pi is designed around a relatively small set of core capabilities.

Out of the box, it provides four essential tools:

Tool	Purpose
`read`	Read files from your project
`write`	Create files and write content
`edit`	Modify existing files
`bash`	Execute terminal commands

That’s it.

And these four tools are enough to handle a surprising amount of coding work.

Pi also supports extensions, allowing you to add functionality when needed.

Install Pi

Follow the installation instructions on Pi’s official website.

After installation, open your terminal and run:

pi

If a new Pi session opens, the installation is working.

Now let’s connect it to our local model.

🔗 Step 4: Connect Pi to LM Studio

There are multiple ways to configure Pi to use a locally hosted model.

You can manually configure a custom provider, but I used a community extension called pi-lmstudio.

The extension discovers models exposed by LM Studio and makes them available inside Pi.

You can find it through Pi’s package directory.

Install the extension using the command provided on its package page.

Once installed, restart Pi:

pi

Then open the model selector:

/model

You should see the model currently exposed by LM Studio.

Select your Qwen 3.8 27B model.

And that’s it.

Pi is now using your locally running Qwen model instead of a hosted inference API.

The workflow looks like this:

Your Prompt
    |
    v
Pi Coding Agent
    |
    | Sends model request
    v
LM Studio API (localhost:1234)
    |
    v
Qwen 3.8 27B
    |
    | Returns response / tool calls
    v
Pi Coding Agent
    |
    | Executes tools
    v
Your Local Project

One important distinction: Pi executes tools locally, while LM Studio handles model inference.

The model generates the instructions and tool calls, but Pi is responsible for carrying them out.

Verify That Tool Calling Works

Before assigning a complicated task, I recommend testing the setup with something simple.

For example, create a Python file:

print("Hello, World!")

Then ask Pi:

Read the Python file in this directory, explain what it does, and create a document containing the explanation.

This tests whether the model can:

Understand the instruction.
Select the appropriate tools.
Read a local file.
Generate a summary.
Write a new file.

In my experiment, Pi successfully completed this test.

Now it was time for something more challenging.

💻 Step 5: Can a Local 27B Model Build an Entire Web Application?

This was the part I was most interested in.

Getting a model to answer questions is one thing.

Getting it to work through a multi-step software development task is something else entirely.

I started with an empty project directory.

Then I asked Pi to build an AI Agent Mission Control dashboard.

Here’s a condensed version of the prompt:

Build a modern, polished AI Agent Mission Control dashboard for managing and monitoring a team of autonomous AI agents.

The dashboard should include:

* Overview cards for Active Agents, Running Tasks, Completed Tasks, and Failed Tasks
* A list/grid of AI agents with names, roles, status, current task, and model being used
* A Kanban-style task board with Backlog, In Progress, Review, and Completed columns
* A Live Activity Feed showing agents starting tasks, using tools, completing work, and handing tasks to other agents
* System metrics such as token usage, estimated cost, success rate, and average task duration
* A sidebar with Overview, Agents, Tasks, Activity, Analytics, and Settings

Use realistic mock data for a small AI-agent team such as Research Agent, Coding Agent, Browser Agent, Content Agent, and QA Agent.

Make it feel like a premium SaaS mission-control product: clean dark theme, excellent typography, subtle animations, status indicators, charts, responsive layout, and polished interactions.

Build the complete working website, not just a static mockup.

Then I let the agent work.

What Happened?

The agent created a React + TypeScript project and started implementing the application.

It performed multiple tool calls, created files, wrote components, and executed terminal commands.

But what impressed me most wasn’t just the generated code.

It actually attempted to test and fix its own work.

During the process, the agent:

Created the application structure.
Generated React components.
Implemented the dashboard interface.
Ran the application locally.
Encountered errors.
Inspected the results using screenshots.
Made additional changes to address issues.

And it completed this workflow without requiring me to manually fix the errors it encountered.

The final result included the agent overview, task board, activity feed, analytics dashboard, and settings interface.

To be clear, this was a demonstration application using sample data, not a production-ready autonomous agent management platform.

But as an experiment in local AI-assisted software development, it was encouraging.

📊 The Results: Performance, Hardware, and Cost

Here’s the actual setup I used.

Component	Configuration
Computer	MacBook Pro
Chip	Apple M5 Max
Memory	48GB unified memory
AI Model	Qwen3.8 27B
Quantization	4-bit
Model Size	Approximately 16GB
Inference Runtime	LM Studio
Coding Agent	Pi
Observed Speed	10–15 tokens/second
Task Duration	A few hours
Paid Model API Cost	$0

The agent managed to build the application, but it wasn’t particularly fast.

At approximately 10–15 tokens per second, complex coding tasks can take quite a while.

And when the agent needs to perform multiple rounds of reasoning, tool execution, and error correction, that time adds up.

Still, the fact that this entire workflow ran on consumer hardware is interesting.

There was no per-token API bill.

However, zero API cost doesn’t mean zero cost.

You still need suitable hardware, electricity, and time.

And depending on your workload, a hosted model might still be more economical or productive.

🧩 Why I Chose Pi Instead of a Larger Coding Agent

This is one of the more interesting technical aspects of the experiment.

When running AI models locally, context management becomes especially important.

Every instruction, conversation message, tool definition, and tool result consumes tokens.

And those tokens aren’t free from a computational perspective.

Even when you’re not paying an API provider, your hardware still needs to process them.

For local models, large contexts can increase:

Memory consumption.
Time to process prompts.
KV-cache requirements.
Overall task execution time.

This is where Pi’s design becomes interesting.

A Minimal Agent Harness

Pi starts with four core tools:

read
write
edit
bash

Rather than beginning with a large collection of built-in capabilities, you can extend Pi as needed.

This gives you more control over the tools and functionality available to your local model.

For example, you might eventually add:

Browser automation.
Web search.
Custom development tools.
Project-specific workflows.
Additional integrations.

But you don’t necessarily need all of those capabilities loaded from the beginning.

A smaller tool surface can help keep the setup simpler.

Important: This doesn’t automatically mean Pi uses fewer tokens or performs better than Claude Code, Codex, or other coding agents in every scenario.

That would require a controlled benchmark using the same model, tasks, and evaluation criteria.

For my experiment, Pi was simply a good fit because I wanted a lightweight, extensible agent environment.

⚠️ What Are the Limitations of Running AI Coding Agents Locally?

As promising as the results were, there are some important limitations.

1. Hardware Requirements

A 27B model, even when quantized, requires a significant amount of memory.

And the model weights are only part of the equation.

You also need memory for inference overhead, context processing, and the operating system.

Not everyone has a machine capable of comfortably running models this large.

2. Inference Can Be Slow

My setup generated approximately 10–15 tokens per second.

That’s usable, but considerably slower than many hosted inference services.

For complicated tasks involving hundreds of tool calls or repeated debugging cycles, performance becomes a major consideration.

3. Tool-Calling Reliability Varies

An AI coding agent needs more than good code generation.

It must reliably decide when to use tools, generate valid tool calls, interpret results, and continue working.

A model might perform well on a single coding prompt but struggle with a long sequence of tool interactions.

This is something I want to evaluate more systematically in future experiments.

4. Local Models Aren’t Always Competitive With Frontier Models

Strong hosted models still have advantages on many complex software engineering tasks.

Smaller local models may struggle with:

Large codebases.
Complex architectural changes.
Long debugging sessions.
Multi-step reasoning.
Maintaining consistency across many files.

And quantization can introduce additional accuracy trade-offs.

So I wouldn’t consider this a universal replacement for cloud-based coding agents just yet.

5. Local Inference Doesn’t Automatically Make an Agent Safe

Running the model locally helps keep inference data under your control.

But an agent that can execute shell commands and modify files still has significant permissions.

A local model can make mistakes, delete files, or execute unintended commands.

For sensitive projects, consider using isolated development environments, restricted permissions, version control, and approval mechanisms.

Also remember that extensions or tools with network access may still communicate externally.

A fully local inference endpoint is not, by itself, a guarantee that every part of an agent workflow stays offline.

🔮 Are Fully Local AI Coding Agents Becoming Practical?

I think we’re getting closer.

Not because local models have suddenly become better than every cloud model.

But because the overall ecosystem has improved significantly.

We now have:

Open-weight models capable of increasingly sophisticated coding tasks.

Local inference tools like LM Studio that make downloading and running models accessible.

Lightweight agent frameworks like Pi that can turn those models into useful coding assistants.

And hardware capable of running models that would have been impractical for many developers just a few years ago.

What’s particularly interesting is that these components are independent.

You can swap the model without necessarily replacing your coding agent.

You can experiment with different quantization levels.

You can add or remove agent capabilities.

And you can choose how much of your development workflow runs locally.

For developers who care about privacy, experimentation, offline workflows, or avoiding recurring API costs, that’s a compelling direction.

The next challenge isn’t just making local models generate better code.

It’s making them reliable enough to complete long-running software engineering tasks with minimal human intervention.

And that requires evaluating the entire agent workflow, not just the model’s benchmark scores.

💬 Final Thoughts

My biggest takeaway from this experiment wasn’t that a local 27B model could replace Claude Code or Codex.

It was that a complete AI coding workflow can now run on a personal computer without relying on paid model APIs.

And that opens up some interesting possibilities.

Local-first development environments.

Private AI coding assistants.

Offline software development workflows.

And customizable agents that developers can adapt to their own hardware and requirements.

There’s still plenty of room for improvement, particularly around speed, reliability, and long-context performance.

But I’m excited to see how far this ecosystem progresses.

What if you could use Claude Code for FREE? 👀 I connected Claude Code OmniRoute free AI models from OpenRouter & NVIDIA. Here’s the full setup, trade-offs, and how it works 👇

Vivek Shetye — Tue, 06 Oct 2026 15:22:18 +0000

Vivek Shetye

Oct 6

How to Use Claude Code for Free with OmniRoute

#ai #opensource #agents #claude

6 min read

How to Use Claude Code for Free with OmniRoute

Vivek Shetye — Tue, 06 Oct 2026 15:01:32 +0000

Claude Code is one of my favorite AI coding agents.

It can inspect your codebase, edit files, execute terminal commands, use tools, and work through multi-step coding tasks directly from the terminal.

But normally, the intelligence behind that experience comes from Anthropic’s Claude models.

What if we kept Claude Code as the coding agent harness, but replaced the model underneath it?

That’s what I wanted to test.

The result looks like this:

Claude Code
     ↓
 OmniRoute
     ↓
Free AI Models
     ↓
OpenRouter / NVIDIA / Other Providers

Instead of sending model requests directly to Anthropic, Claude Code sends them to OmniRoute, which acts as an AI gateway and routes them to the models and providers you configure.

And yes — that means you can run Claude Code using free AI models. 🆓

But there’s an important distinction.

We are NOT getting Claude Sonnet or Claude Opus for free.

We’re using Claude Code as the agent harness while replacing the underlying model with free models from other providers.

Let’s set it up.

🎥 Full video walkthrough

I recorded the entire process step-by-step, including:

✅ Installing OmniRoute locally
✅ Connecting OpenRouter
✅ Connecting NVIDIA
✅ Importing free AI models
✅ Configuring Claude Code
✅ Fixing the authentication issue I encountered
✅ Launching Claude Code through OmniRoute
✅ Running Claude Code with free models

🧠 Claude Code Is More Than the Model

An important idea behind this experiment is separating the agent harness from the LLM powering it.

Claude Code provides the agent experience around the model.

It can:

inspect files
understand your project
modify code
execute terminal commands
use tools
iterate through coding tasks

Normally, Claude Code communicates with Anthropic’s models.

But if we place an AI gateway between Claude Code and the model, the architecture becomes much more flexible.

┌─────────────────┐
│   Claude Code   │
│  Coding Agent   │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│    OmniRoute    │
│   AI Gateway    │
└────────┬────────┘
         │
         ▼
┌─────────────────────────┐
│ OpenRouter │ NVIDIA │ … │
└─────────────────────────┘
         │
         ▼
┌─────────────────┐
│ Free AI Models  │
└─────────────────┘

Claude Code remains the interface and coding agent.

OmniRoute controls where the model requests go.

🔀 That separation is what makes this experiment interesting.

⚙️ Step 1: Install OmniRoute

The first thing we need is OmniRoute.

OmniRoute is an open-source AI gateway that can sit between applications such as Claude Code and different AI model providers.

Install OmniRoute using the installation instructions for your operating system.

Once installed, start it:

omniroute

After a few seconds, the OmniRoute server should start.

Open the dashboard URL shown in your terminal.

You now have a local AI gateway running on your machine.

🔌 Step 2: Connect an AI Provider

Next, we need models.

For my setup, I connected two providers:

OpenRouter
NVIDIA

Let’s start with OpenRouter.

Inside the OmniRoute dashboard, navigate to:

Providers → OpenRouter → Add Connection

Generate an API key from your OpenRouter account and add it to the connection.

If you only want free models, enable the option to:

Only import free models

Save the connection.

OmniRoute will import the available free models associated with that provider.

You can also run health checks from the dashboard to see which models are currently available. 🩺

🚀 Step 3: Add NVIDIA Models

I also connected NVIDIA to give OmniRoute another source of models.

The process is similar.

Navigate to:

Providers → NVIDIA → Add Connection

Generate an NVIDIA API key, paste it into OmniRoute, validate it, and save the connection.

Again, I configured the connection to import the free models available through the provider.

Now the architecture starts becoming more interesting:

                    ┌──→ OpenRouter → Free Models
Claude Code → OmniRoute
                    └──→ NVIDIA ───→ Free Models

Instead of Claude Code being tied directly to one model provider, the gateway can expose models from multiple providers.

💻 Step 4: Connect Claude Code to OmniRoute

Now for the important part.

If you don’t already have Claude Code installed, install it using Anthropic’s official installation instructions.

Once Claude Code is available, OmniRoute can configure it automatically.

Run:

omniroute setup-claude

OmniRoute creates profiles that allow Claude Code to communicate through the gateway.

However, I ran into an issue here.

🐛 The Authentication Problem I Encountered

My OmniRoute API was protected with authentication.

Normally, you should be able to create an access token and pass it while configuring Claude Code.

In my testing, however, the API key parameter wasn’t being respected correctly and Claude Code continued returning an authentication error.

So I used a temporary workaround.

I disabled API authentication in OmniRoute.

⚠️ Important Security Warning

I would only consider doing this when OmniRoute is:

running locally
bound appropriately
not exposed publicly
not accessible from an untrusted network

Do not expose an unauthenticated AI gateway to the internet.

After disabling authentication locally, I ran the Claude setup again and the connection succeeded.

🎯 Step 5: Launch Claude Code with a Free Model

Now we can actually launch Claude Code through OmniRoute.

I used OmniRoute’s coding profile:

omniroute launch --profile auto-coding

The auto-coding profile allows OmniRoute to select among the coding models available in the configuration.

And that’s it.

Claude Code starts normally.

Except now the architecture is:

Claude Code
     ↓
OmniRoute
     ↓
Alternative AI Model

Instead of:

Claude Code
     ↓
Anthropic Model

🎉 Claude Code is now running using the models routed through OmniRoute.

So… Is Claude Code Actually Free Now?

Kind of.

And this distinction is important.

You’re not getting Anthropic’s Claude Sonnet or Opus models for free.

You’re using the Claude Code agent experience with alternative models that may have free access tiers or free routes.

Think of it as separating two pieces:

Claude Code = Agent Harness
Free Model = Intelligence
OmniRoute = Routing Layer

This means the quality of the experience depends heavily on the model you choose.

And that brings us to the trade-offs.

⚡ Trade-Off #1: Free Models Have Limits

Free doesn’t mean unlimited.

Free model providers commonly impose things like:

request limits
token quotas
rate limits
temporary availability restrictions

A model that works today may also become unavailable or limited later.

For experimentation and personal projects, this can still be extremely useful.

For production workloads, you’ll want something more predictable.

🧩 Trade-Off #2: Not Every Model Works Well with Claude Code

This is probably the biggest issue.

Claude Code isn’t simply sending a prompt and displaying some text.

The model needs to participate in an agentic coding workflow.

That means capabilities such as:

Understand task
      ↓
Inspect files
      ↓
Call tools
      ↓
Modify code
      ↓
Run commands
      ↓
Observe results
      ↓
Continue reasoning

Some models handle these workflows surprisingly well.

Others don’t.

A model might be excellent at generating code but struggle with:

tool calling
instruction following
long coding tasks
maintaining context
structured outputs
multi-step agent workflows

So don’t evaluate a model purely by its coding benchmark score.

Agent compatibility matters too.

🔐 Trade-Off #3: Your Code Goes to the Upstream Provider

This one is important.

When you route Claude Code through another provider, your prompts and potentially parts of your codebase are being sent to that provider.

Before using this setup with private repositories or sensitive code, check the provider’s:

privacy policy
data retention policy
model training policy
API terms

I would be particularly careful with proprietary company code, credentials, customer information, and production secrets.

📜 Trade-Off #4: Check Provider Terms

You should also check the terms for every provider you connect.

Some providers may have restrictions around:

automation
proxies
redistribution
free API usage
rate limits
agent workloads

A few minutes reading the provider’s terms can save you from account problems later.

💡 Why This Is More Interesting Than Just “Claude Code for Free”

The cost savings are interesting.

But I think the architecture is the bigger story.

We’re increasingly separating the AI application from the AI model.

Instead of:

AI Application
      ↓
One Model
      ↓
One Provider

we can build:

                    ┌──→ Model A
                    │
AI Agent → Gateway ─┼──→ Model B
                    │
                    ├──→ Model C
                    │
                    └──→ Model D

Now the gateway becomes an abstraction layer between the agent and the models.

That opens up some interesting possibilities.

🧠 Use Different Models for Different Coding Tasks

Imagine routing:

Simple code edits
      ↓
Small / Free Model
Complex debugging
      ↓
Powerful Reasoning Model
Documentation
      ↓
Fast Cheap Model
Large Refactor
      ↓
Frontier Coding Model

Instead of asking:

“Which AI model should power my coding agent?”

the better question may eventually become:

“Which model should handle this particular task?”

That’s a very different architecture.

Final Thoughts

Claude Code doesn’t necessarily have to be coupled to a single model provider.

By introducing an AI gateway like OmniRoute, you can separate the coding agent experience from the model powering it.

That gives you room to experiment with:

🔀 multiple providers
🆓 free models
💰 cheaper models
🧠 stronger reasoning models
⚡ faster models
🛠️ models optimized for specific tasks

Free models won’t replace Claude Sonnet or Opus for every coding task.

But that’s not really the point.

The interesting part is being able to choose.

And as AI coding agents become more capable, I suspect the question will increasingly shift from:

“Which coding agent has the best model?”

to:

“Which models should my coding agent use for each task?”

JEV Explained: Why It Could Matter for AI Agents

Vivek Shetye — Wed, 30 Sep 2026 19:20:27 +0000

What if one of the most interesting new AI models doesn’t generate text at all?

No chatbot responses.

No code generation.

No essays.

Instead, it is designed to do something much narrower:

Make decisions for software.

That’s the idea behind JEV, a new model from TypeSafe AI.

And the numbers TypeSafe AI is reporting immediately caught my attention:

⚡ Decisions in milliseconds
💰 Around $0.042 per million input tokens
📤 No charge for output tokens
📊 On selected workflows, TypeSafe reports nearly 200× faster and 400× cheaper performance compared with the LLMs it tested

Those are TypeSafe AI’s own workflow comparisons, not universal benchmarks. Different workloads could produce very different results.

But the more interesting question isn’t whether JEV can beat an LLM on every benchmark.

It’s:

Why are we using generative LLMs for decisions that don’t require generating anything?

That question becomes especially interesting when you start building AI agents.

🎥 Full video walkthrough

🧠 What Is JEV?

The easiest way to understand JEV is to compare it with a traditional Large Language Model.

Imagine a customer sends this message:

“My package was supposed to arrive yesterday, but tracking hasn’t updated.”

You want your system to decide which team should handle it.

Possible options:

billing
shipping
other

Ask an LLM and it might generate something like:

This request should be routed to the shipping department because
the customer is asking about a delayed package.

That works.

But your application doesn’t necessarily need an explanation.

It needs:

shipping

Or, even better, probabilities associated with the available decisions.

Conceptually:

shipping: 0.94
billing: 0.02
other: 0.04

Your software can then decide what happens next.

That’s the fundamental difference.

LLMs generate tokens.

JEV evaluates predefined decisions.

The output is primarily intended for software to act on, rather than for a human to read as a conversation.

⚡ Why Remove Text Generation?

Modern LLMs are incredibly capable.

They can reason through complex problems, understand large codebases, write software, analyze documents, interact with tools, and orchestrate workflows.

But that flexibility comes with computational cost.

Now consider a typical AI agent.

The agent might need a powerful model to:

understand a repository
debug an application
reason through a complicated problem
generate a patch
plan a multi-step workflow

But between those large reasoning tasks, agents constantly make smaller decisions.

For example:

Which model should handle this task?
Did the previous step succeed?
Should I retry?
Does this command require approval?
Which tool should run next?
Should this result be escalated?

Today, we often solve these problems with…

another LLM call.

And that works.

But we’re invoking a generative model when the actual output we need might simply be:

retry

or:

needs_approval

or:

use_stronger_model

That is the category of problem TypeSafe AI is targeting with what it calls System One Models.

JEV is its first public model.

🔄 JEV vs LLMs: The Important Difference

Think about the two approaches like this:

Traditional LLM	JEV
Generates text	Evaluates decisions
Predicts the next token	Scores predefined options
Great for reasoning and generation	Designed for bounded judgments
Flexible output	Structured output
Human-readable responses	Software-actionable decisions
Useful for open-ended problems	Useful for constrained choices

This distinction matters.

JEV isn’t necessarily trying to become the model inside your AI coding agent that understands an entire repository and writes a feature.

Instead, it could potentially sit beside that model.

A powerful LLM handles:

reasoning
coding
planning
generation

JEV handles:

classification
routing
scoring
triage
bounded decisions

In other words:

The LLM generates. JEV decides.

🤖 Why JEV Could Be Interesting for AI Agents

This is where I think things get particularly interesting.

Consider an agent built using something like Hermes Agent, Claude Code, or another agent framework.

A simplified architecture today might look like:

User
 ↓
Agent
 ↓
LLM
 ↓
Tools
 ↓
LLM
 ↓
Tools
 ↓
LLM
 ↓
Result

The LLM becomes responsible for almost everything.

But imagine separating different responsibilities:

             ┌── Retrieval / Memory
             │
User → Agent ├── JEV → Fast decisions
             │
             ├── Powerful LLM → Reasoning + Coding
             │
             └── Tools → Actions

Now different components handle different types of work.

That’s closer to the architecture I would want to experiment with.

Here are three workflows I’d test first.

1. 🔀 Intelligent Model Routing

This might be the most obvious use case.

Suppose your AI agent has access to multiple models.

Maybe you have:

Fast/Cheap Model
        +
Powerful/Expensive Model

A simple task arrives:

Rename this variable across these files.

Do you really need your most expensive reasoning model?

Probably not.

But then another task arrives:

Debug this race condition across multiple distributed services.

That’s a very different problem.

Instead of sending everything to the strongest model, JEV could potentially classify the incoming task.

For example:

Task
 ↓
JEV
 ↓
┌─────────────────┐
│ SIMPLE          │ → Fast/Cheap Model
│ COMPLEX         │ → Powerful Model
│ UNCERTAIN       │ → Powerful Model
└─────────────────┘

This could make model routing much more deliberate.

And importantly, you’d want to measure whether the routing mistakes cost more than the model savings.

Cheap routing isn’t useful if important tasks constantly go to the wrong model.

That’s why I’d want to test:

Cost
Latency
Routing accuracy
Escalation rate
Task success rate

rather than looking only at token pricing.

2. 🛡️ Risk Detection Before Tool Execution

This one gets even more interesting.

AI agents increasingly interact with real tools.

They can:

execute shell commands
modify files
interact with APIs
deploy applications
access databases
manage infrastructure

Before executing an action, you might want another layer that evaluates what kind of action is being attempted.

Imagine an agent generates:

rm -rf ./build

versus:

rm -rf /

Those commands clearly shouldn’t be treated the same way.

A decision model could evaluate questions such as:

Does this affect production?
Does this modify sensitive files?
Could this expose credentials?
Is this destructive?
Should a human approve this?

The architecture might look like:

AI Agent
   ↓
Proposed Tool Action
   ↓
JEV
   ↓
Risk Classification
   ↓
Policy Engine
   ↓
ALLOW / BLOCK / REQUIRE APPROVAL

There’s an important distinction here:

JEV shouldn’t necessarily control permissions.

Your application policy should.

JEV could provide a classification or score, while deterministic code decides what actually happens.

That keeps the security boundary outside the model.

3. 🔎 Agent Output Triage

Imagine you have an autonomous agent processing hundreds or thousands of tasks.

Most results might be perfectly normal.

Some won’t be.

Today, you could send every result through another powerful LLM for verification.

But that potentially means:

1000 tasks
+
1000 verification LLM calls

Instead, a smaller decision model could potentially triage those outputs.

For example:

Agent Result
     ↓
    JEV
     ↓
 ┌───────────────┐
 │ Looks Normal  │ → Finish
 │ Uncertain     │ → Stronger LLM
 │ Suspicious    │ → Human Review
 └───────────────┘

Now your expensive reasoning model only gets involved when necessary.

Again, whether this actually improves the system depends on the accuracy of that triage.

But architecturally, it’s an interesting pattern.

🏗️ What Could the Future AI Agent Stack Look Like?

This leads to the bigger idea behind JEV.

Over the last few years, we’ve increasingly treated the LLM as the center of everything.

Need classification?

Use an LLM.

Need routing?

Use an LLM.

Need scoring?

Use an LLM.

Need validation?

Use another LLM.

Need to judge that LLM?

Use another LLM. 😅

That architecture works because modern models are incredibly flexible.

But flexibility doesn’t necessarily mean every task should use the same model.

A future AI agent stack could instead look something like:

┌─────────────────────────────┐
│      Agent Orchestration    │
├─────────────────────────────┤
│                             │
│ Retrieval → Context         │
│                             │
│ JEV → Fast Decisions        │
│                             │
│ LLM → Reasoning + Coding    │
│                             │
│ Tools → Actions             │
│                             │
├─────────────────────────────┤
│       Policy / Control      │
└─────────────────────────────┘

Each component has a more specialized responsibility.

The reasoning model doesn’t need to make every tiny routing decision.

The decision model doesn’t need to understand and rewrite an entire codebase.

And tools remain responsible for actually interacting with the outside world.

💰 What About JEV’s Cost and Speed Claims?

This is where it’s important to separate an interesting architecture from benchmark hype.

TypeSafe AI reports pricing around:

$0.042 per million input tokens, with no output-token charge.

It also reports latency in the millisecond range.

And on selected company workflows, TypeSafe says JEV achieved results approaching:

200× faster

and

400× cheaper

than the LLM approaches it compared against.

Those numbers sound impressive.

But they should be interpreted in context.

These aren’t universal statements that JEV is “400× better than LLMs.”

They’re results from specific workflows and comparisons.

The test I’d really like to see is much simpler.

Take the same real-world decision workload and run it through:

JEV
vs.
Small LLM
vs.
Frontier LLM

Then compare:

Accuracy
False positives
False negatives
Latency
Cost
Reliability

Because if JEV costs almost nothing but makes significantly more bad routing decisions, the token savings don’t matter.

On the other hand, if it can handle those decisions reliably at a fraction of the latency and cost…

then things get interesting very quickly.

🧩 Specialized Models vs One Giant Model

And this is the part of JEV that interests me most.

Not whether JEV itself becomes the dominant solution.

Not whether every AI agent suddenly needs it.

And definitely not whether it “replaces LLMs.”

It probably shouldn’t.

The interesting idea is that not every problem inside an AI agent needs to become another LLM generation.

We might eventually build agent systems where:

Retrieval handles context.
Decision models handle bounded judgments.
Reasoning models handle difficult problems.
Coding models handle software development.
Tools handle actions.
Policy code controls permissions.
Agents orchestrate everything.

Instead of asking one increasingly powerful model to do absolutely everything, we could build systems from specialized intelligence components.

And JEV is an interesting early example of what that architecture could look like.

Run Hermes Agent Inside Docker: A Safer Setup for Autonomous AI Agents 🐳

Vivek Shetye — Tue, 22 Sep 2026 16:35:22 +0000

Giving an AI agent unrestricted access to your entire computer is probably not a great idea.

Especially when that agent can:

Run terminal commands
Install packages
Create and modify files
Access external services
Work autonomously through multi-step tasks

Those capabilities are exactly what make Hermes Agent powerful.

But they also raise an important question:

Where should all of those actions actually run?

Instead of running Hermes Agent directly on my host machine, I decided to run it inside a Docker container and explicitly control what the agent can access.

The goal isn’t to make Hermes magically “secure” by putting it inside Docker.

The goal is much simpler:

Give the agent access to what it needs, not your entire computer.

In this tutorial, I’ll walk through the complete setup.

We’ll configure:

Hermes Agent inside Docker
Persistent storage for memory, sessions, and configuration
A limited filesystem boundary
Hermes Gateway
Hermes Dashboard
Dashboard authentication
Hermes Desktop connected to the container
Some important security considerations when containerizing autonomous AI agents

🎥 Prefer watching instead?

I walk through the complete setup step-by-step in the video:

Watch the full Hermes Agent Docker tutorial on YouTube →

🤖 Why Run Hermes Agent Inside Docker?

Hermes isn’t just a chatbot.

It can interact with your environment and perform actions on your behalf.

For example, depending on the tools you enable, Hermes can work with files, execute terminal commands, install dependencies, use external services, and autonomously work through tasks.

That’s incredibly useful.

But if the agent runs directly on your main machine, you need to think carefully about what files, credentials, and system resources it can reach.

That’s where Docker becomes useful.

Instead of letting Hermes operate directly across my host environment, I can put it inside a container and explicitly decide what crosses that boundary.

The architecture we’ll build looks roughly like this:

┌──────────────────── HOST MACHINE ────────────────────┐
│                                                      │
│  Docker                                              │
│                                                      │
│  ~/.hermes-docker  ───────────────┐                  │
│                                   │                  │
│                    ┌──────────────▼──────────────┐   │
│                    │      Docker Container       │   │
│                    │                             │   │
│                    │      Hermes Agent           │   │
│                    │                             │   │
│                    │  /opt/data                  │   │
│                    │  Gateway                    │   │
│                    │  Dashboard                  │   │
│                    └─────────────────────────────┘   │
│                                                      │
│  Personal files / SSH keys / other directories       │
│              NOT mounted into container              │
│                                                      │
└──────────────────────────────────────────────────────┘

Hermes gets a dedicated environment and a directory for its persistent data.

The rest of the machine stays outside that boundary unless I explicitly expose something.

🐳 Step 1: Install Docker

On macOS or Windows, install Docker Desktop and make sure the Docker engine is running.

On Linux, you can install Docker Engine using the instructions for your distribution.

You can verify Docker is available with:

docker --version

Once Docker is working, we can create an isolated environment for Hermes.

📁 Step 2: Create a Dedicated Directory for Hermes

First, create a directory specifically for the Dockerized Hermes instance:

mkdir -p ~/.hermes-docker

This directory is important for two reasons.

1. Persistence

Docker containers are disposable.

If we remove and recreate the Hermes container, we don’t want to lose things like:

Configuration
Sessions
Memory
Skills
Profiles
Logs
Other persistent agent data

We’ll store those outside the container in ~/.hermes-docker.

2. It becomes part of our trust boundary

Anything inside this directory is intentionally available to Hermes.

Instead of mounting my entire home directory, I’m giving the agent one dedicated location.

That distinction matters.

⚙️ Step 3: Run the Hermes Setup Inside Docker

Now we can start the official Hermes Agent image and launch the setup process:

docker run --rm -it \
  -v ~/.hermes-docker:/opt/data \
  nousresearch/hermes-agent setup

The most important part here is:

-v ~/.hermes-docker:/opt/data

The -v option creates a bind mount between a directory on the host and a directory inside the container.

Think of it like this:

HOST                           CONTAINER
~/.hermes-docker   ───────▶   /opt/data

The left side:

~/.hermes-docker

is the directory we just created on the host.

The right side:

/opt/data

is where that directory appears inside the Docker container.

So when Hermes saves configuration, sessions, memory, skills, or other persistent data under /opt/data, those files are actually stored in ~/.hermes-docker on the host.

That means we can destroy and recreate the container without losing the Hermes environment.

More importantly, we’re sharing one dedicated directory instead of the entire host filesystem.

🧠 Step 4: Configure Hermes Agent

The first run downloads the required Docker image and launches the Hermes setup wizard.

From there, follow the normal Hermes configuration process.

In my setup, I:

Use the quick setup
Authenticate with Nous Portal
Select a model
Keep the terminal backend local
Skip messaging platform configuration for now

Once setup finishes, the configuration is stored in our persistent directory.

That means future containers using the same mount can reuse it.

🌐 Step 5: Start the Hermes Gateway and Dashboard

Now we can start Hermes as a background container:

docker run -d \
  --name hermes-docker \
  --restart unless-stopped \
  -v ~/.hermes-docker:/opt/data \
  -p 18642:8642 \
  -p 19119:9119 \
  -e HERMES_DASHBOARD=1 \
  nousresearch/hermes-agent gateway run

There are a few important things happening here.

Run in the background

-d

starts the container in detached mode.

Give the container a name

--name hermes-docker

makes it easier to manage later:

docker logs hermes-docker
docker stop hermes-docker
docker start hermes-docker

Keep the persistent directory

We’re mounting the same directory again:

-v ~/.hermes-docker:/opt/data

So Hermes sees the configuration we created during setup.

Enable the dashboard

-e HERMES_DASHBOARD=1

enables the Hermes Dashboard.

🔌 Understanding the Port Mapping

You’ll notice I’m using:

-p 18642:8642
-p 19119:9119

Docker port mappings follow this pattern:

HOST_PORT:CONTAINER_PORT

So:

18642 → 8642
19119 → 9119

The ports on the right are the ones Hermes uses inside the container.

The ports on the left are the ones exposed on my Mac.

I’m deliberately using different host ports because I already have another Hermes instance running locally and don’t want the ports to clash.

If you’re running only the Docker instance, you can use the default ports 8642 and 9119 instead of 18642 and 19119.

✅ Step 6: Verify the Container Is Running

Run:

docker ps

You should see the hermes-docker container running.

If something isn’t working, one of the first places to look is the container logs:

docker logs hermes-docker

This becomes particularly useful when configuring the dashboard.

🔐 Step 7: Configure Dashboard Authentication

When I initially opened the dashboard, it didn’t start correctly.

Instead of hiding that part from the tutorial, I kept it in because it’s a useful troubleshooting example.

Checking:

docker logs hermes-docker

showed that the dashboard needed an authentication provider.

Hermes provides authentication options for securing dashboard access.

I configured mine using Nous Portal authentication.

Once authentication was configured, the dashboard started successfully.

With my port mapping, I can access it at:

localhost:19119

And now the Hermes Dashboard is running while Hermes itself remains inside the container.

💬 Step 8: Chat With Hermes Inside the Container

You can also enter the running container directly:

docker exec -it hermes-docker bash

Then start Hermes:

hermes

At this point you’re interacting with Hermes from a terminal inside the Docker container, rather than running Hermes directly on the host.

That’s useful for testing.

But I don’t necessarily want to enter the container every time I want to use the agent.

There’s a better option.

🖥️ Step 9: Connect Hermes Desktop to the Docker Instance

Hermes Desktop can connect to the Gateway running inside our container.

Open Hermes Desktop and navigate to:

Settings → Gateway → Remote Gateway

Then configure it to use the Gateway exposed by your Docker container.

In my setup, that’s my url:

http://localhost:19119

After authenticating, save the configuration and reconnect.

Now Hermes Desktop communicates with the Dockerized Hermes instance.

So from the user’s perspective, I still get the convenient desktop interface.

But the agent itself is operating inside the container.

🛡️ What Does Docker Actually Protect?

This is probably the most important part of the tutorial.

Running an AI agent inside Docker does not automatically make it safe.

Docker gives us a useful isolation boundary, but that boundary depends heavily on how the container is configured.

For example, imagine doing this:

Host machine
      │
      ▼
Entire home directory
      │
      ▼
Docker container
      │
      ▼
Hermes Agent

You’ve technically containerized Hermes.

But you’ve also exposed a huge portion of your machine to it.

That’s not the setup I want.

Instead:

Host machine
      │
      ├── Personal files       ❌
      ├── SSH keys             ❌
      ├── Other projects       ❌
      ├── Docker socket        ❌
      │
      └── ~/.hermes-docker     ✅
                │
                ▼
          Hermes Container

The container should get access only to the resources required for the task.

🚫 1. Don’t Mount Your Entire Home Directory

If you mount your complete home directory into the container, Hermes could potentially access whatever the container’s permissions allow within that mount.

That might include personal files, source code, configuration files, credentials, or other sensitive data.

That’s why I’m using:

~/.hermes-docker

as a dedicated location instead.

If Hermes doesn’t need something, don’t expose it.

🔑 2. Be Careful With Credentials

The same principle applies to API keys and tokens.

Only provide the credentials Hermes actually needs.

Avoid unnecessarily exposing things such as:

~/.ssh

or unrelated cloud credentials and secrets.

Containerization doesn’t help much if you put every sensitive credential inside the container anyway.

⚠️ 3. Don’t Expose the Docker Socket Without a Very Good Reason

Another thing I’m deliberately not mounting is:

/var/run/docker.sock

Giving a container access to the host Docker daemon can dramatically expand what that container is capable of doing.

For an autonomous agent, that’s an especially important boundary to think about.

If Hermes doesn’t need to control Docker on the host, don’t give it that capability.

🧩 4. Docker Doesn’t Solve Prompt Injection

There’s another important limitation.

An AI agent can still encounter malicious or adversarial content.

For example, content from a webpage, repository, document, or other external source could attempt to manipulate the agent into performing unintended actions.

Putting the agent inside Docker doesn’t eliminate that problem.

The difference is what happens after the agent attempts the action.

If Hermes only has access to:

/opt/data

then we’ve limited the environment available to it.

If we’ve mounted our entire computer, exposed sensitive credentials, and given it access to the Docker daemon, the potential impact is very different.

That’s why I think containerization is better understood as blast-radius reduction, not a magic AI security solution.

🎯 The Principle: Least Privilege for AI Agents

As AI agents become increasingly capable, I think one traditional security principle becomes even more important:

Least privilege.

An agent should receive the minimum access required to complete its task.

Not:

Agent
  │
  └── Everything on my computer

But:

Agent
  │
  ├── Required files
  ├── Required credentials
  ├── Required network services
  └── Required tools

Nothing more.

This becomes increasingly important as agents gain longer-running autonomy, terminal access, browser access, external integrations, scheduled execution, and the ability to coordinate multiple tools.

Docker vs VM for AI Agents

Docker is not the only way to isolate an autonomous agent.

You could also use:

A virtual machine
A dedicated VPS
A sandboxed execution environment
A separate physical machine
More restrictive container runtimes

A VM generally provides a stronger isolation boundary than a standard application container because it virtualizes a larger portion of the environment.

Docker, however, is lightweight and convenient for local development.

For my Hermes setup, it gives me a useful middle ground:

Direct Host Execution
        ↓
Docker Container
        ↓
Virtual Machine / Dedicated Environment

The appropriate boundary ultimately depends on what capabilities you’re giving the agent and what resources it can access.

Final Architecture

After everything is configured, my setup looks roughly like this:

                   ┌─────────────────────────────┐
                   │        Host Machine         │
                   │                             │
                   │      Hermes Desktop         │
                   └──────────────┬──────────────┘
                                  │
                           Remote Gateway
                                  │
                                  ▼
              ┌─────────────────────────────────┐
              │       Docker Container          │
              │                                 │
              │       Hermes Gateway            │
              │       Hermes Dashboard          │
              │       Hermes Agent              │
              │                                 │
              │       /opt/data                 │
              └───────────────┬─────────────────┘
                              │
                         Bind Mount
                              │
                              ▼
                    ~/.hermes-docker
                              │
              ┌───────────────┴──────────────┐
              │ Config • Memory • Sessions   │
              │ Skills • Profiles • Logs     │
              └──────────────────────────────┘
       Rest of host filesystem → NOT MOUNTED
       SSH keys                → NOT EXPOSED
       Docker socket           → NOT EXPOSED

Hermes still has the environment it needs to work.

But we’re being much more deliberate about what crosses the boundary.

Final Thoughts

Autonomous AI agents are becoming capable of doing much more than generating text.

They’re increasingly able to:

⚡ Execute commands
🗂️ Manipulate files
🌍 Interact with external systems
🧰 Use specialized tools
🔄 Run multi-step workflows autonomously

That’s exciting.

But the more capable the agent becomes, the more important its execution environment becomes.

Running Hermes Agent inside Docker doesn’t solve every security problem.

It does, however, give us a practical way to control what the agent can access and reduce the potential blast radius when something unexpected happens.

The rule I follow is simple:

Only give an AI agent access to what it actually needs.

Everything else should stay outside the boundary.

🚀 What if your AI agent wasn’t tied to a single model provider? I connected Hermes Agent to OmniRoute to route across free AI models from multiple providers. Here’s the full setup 👇 #AI #AIAgents #OpenSource

Vivek Shetye — Tue, 15 Sep 2026 15:14:45 +0000

Vivek Shetye

Sep 15

How I Run Hermes Agent on FREE AI Models with OmniRoute 🚀

#ai #hermes #agents #opensource

7 min read

How I Run Hermes Agent on FREE AI Models with OmniRoute 🚀

Vivek Shetye — Tue, 15 Sep 2026 15:05:20 +0000

Free AI models are great.

Until your agent hits a quota halfway through a task.

Then suddenly you’re:

Searching for another provider
Generating another API key
Changing endpoints
Switching models
Restarting your workflow

For a simple chat application, that might just be annoying.

For an AI agent executing multi-step tasks, it becomes a much bigger problem.

So instead of connecting Hermes Agent directly to a single AI provider, I tried a different architecture:

                 ┌── OpenRouter ──→ Free Models
                 │
Hermes Agent ─→ OmniRoute
                 │
                 └── NVIDIA ──────→ Available Models

Hermes talks to one endpoint.

OmniRoute sits between the agent and the model providers and handles the routing.

The result is a surprisingly useful setup for experimenting with Hermes Agent on free AI models without tightly coupling the agent to one provider.

In this tutorial, I’ll walk through exactly how I set it up.

🎥 Prefer the Video?

I created a complete step-by-step video showing the setup, configuration, and a real Hermes Agent task running through OmniRoute.

Watch: How I Run Hermes Agent on FREE AI Models

If you’d rather understand the architecture and follow the setup at your own pace, keep reading.

The Problem With Depending on One Free AI Provider

There are now plenty of ways to experiment with capable AI models without immediately paying for API usage.

That’s great for:

Learning agent development
Building prototypes
Testing AI workflows
Experimenting with different models
Running personal automation
Comparing model capabilities

But there’s an obvious limitation:

Free access is rarely unlimited or guaranteed.

A provider may impose request limits.

A particular model might become unavailable.

Traffic might affect access.

A model could disappear from the free catalog.

Or you might simply exhaust your quota.

If your architecture looks like this:

Hermes Agent
     │
     ▼
Provider A
     │
     ▼
Model

your agent is directly dependent on that provider.

Once Provider A stops serving your requests, your workflow stops too.

Instead, I wanted something closer to this:

                     ┌── Provider A
                     │
                     ├── Provider B
Hermes Agent ──→ Gateway
                     ├── Provider C
                     │
                     └── Provider D

That’s where OmniRoute comes in.

🧠 What Is Hermes Agent?

Hermes Agent is an open-source agent harness from Nous Research for building AI agents capable of doing much more than generating chat responses.

Depending on the tools and configuration available to it, Hermes can perform tasks involving things like:

User Request
     ↓
Hermes Agent
     ↓
Reasoning
     ↓
Tool Calls
     ↓
Files / Terminal / Web / Other Tools
     ↓
Tool Results
     ↓
More Reasoning
     ↓
Final Result

This makes model availability particularly important.

A multi-step agent workflow might require many model calls before the task is finished.

If your model becomes unavailable halfway through, that can interrupt the entire workflow.

🚦 What Is OmniRoute?

Think of OmniRoute as a traffic director for AI models.

Rather than forcing your application to manage every model provider individually, OmniRoute provides a gateway between your application and those providers.

Instead of:

Application → Provider

you get:

Application → OmniRoute → Providers

For this experiment, our application happens to be Hermes Agent.

So the architecture becomes:

                       ┌──────────────┐
                       │  OpenRouter  │
                       └──────┬───────┘
                              │
┌──────────────┐      ┌───────▼───────┐
│ Hermes Agent │ ───→ │   OmniRoute   │
└──────────────┘      └───────┬───────┘
                              │
                       ┌──────▼──────┐
                       │   NVIDIA    │
                       └─────────────┘

OmniRoute is open source, can run locally, and supports a large number of AI providers.

More importantly for this experiment, it allows us to expose a single endpoint to Hermes while managing multiple model/provider connections behind it.

⚡ Why Put an AI Gateway Between Hermes and the Models?

You could simply connect Hermes directly to OpenRouter, NVIDIA, or another provider.

So why add another layer?

Because it separates two responsibilities.

Hermes Agent handles the task.

OmniRoute handles model access and routing.

Conceptually:

Hermes:
"What should I do next?"

        ↓

OmniRoute:
"Which configured model should serve this request?"

That separation becomes increasingly useful as you add more providers.

Instead of reconfiguring the agent every time your model strategy changes, Hermes continues talking to the same gateway.

You manage providers behind it.

🛠️ Step 1: Install OmniRoute Locally

The first thing I did was install OmniRoute locally.

After following the installation instructions, OmniRoute can be started from the terminal.

Once the server starts, it provides access to its local dashboard.

From there you can manage things like:

Providers
Models
Endpoints
API Keys
Routing
Analytics

This becomes the control plane for our model infrastructure.

Important security note 🔐

If you’re running a gateway locally or exposing it beyond your machine, don’t leave default credentials unchanged.

Create proper API keys and configure authentication appropriate for your environment.

🔌 Step 2: Connect OpenRouter

The first provider I added was OpenRouter.

Inside OmniRoute:

Providers
   ↓
OpenRouter
   ↓
Add Connection

You’ll need an OpenRouter API key.

Once the connection is configured, OmniRoute can import the models available through that account.

For this experiment, I enabled the option to import only free models.

The result looks conceptually like:

OpenRouter
     │
     ├── Free Model A
     ├── Free Model B
     ├── Free Model C
     └── ...

You can also perform health checks to see whether configured models are currently available.

That’s already useful but we’re still relying on one provider.

So let’s add another.

🟢 Step 3: Add NVIDIA as Another Provider

I repeated essentially the same process with NVIDIA.

Generate an API key through NVIDIA’s model/API platform and add the connection inside OmniRoute.

Now our gateway has multiple provider connections:

                OmniRoute
                   │
          ┌────────┴────────┐
          ▼                 ▼
     OpenRouter           NVIDIA
          │                 │
      Free Models      Available Models

And this is where the architecture starts becoming interesting.

Hermes doesn’t need separate configurations for both providers.

As far as Hermes is concerned, there is still only:

One API endpoint

OmniRoute handles what happens behind it.

🤖 Step 4: Connect Hermes Agent to OmniRoute

Now we need to point Hermes toward the gateway.

Hermes supports configuring a custom model endpoint.

Using the Hermes model configuration flow, I selected the custom endpoint option and provided:

API Base URL → OmniRoute local endpoint

API Key → OmniRoute API key

Compatibility → Auto Detect

For the model configuration, I used OmniRoute’s:

auto/best-coding

This is where the model selection gets abstracted away from Hermes.

Instead of telling Hermes:

Always use Model X

we’re effectively saying:

Send the request to OmniRoute.

Let the routing layer determine the appropriate
configured model for the request.

Once that configuration is saved, OmniRoute becomes a model endpoint available to Hermes Agent.

🧪 Step 5: Give Hermes a Real Agent Task

A simple “Hello World” wouldn’t tell us much.

So I gave Hermes something closer to an actual agent workflow.

The task was essentially:

Find the latest and most important AI news from reliable sources, summarize the key developments, and create a polished single-page visual report presenting the major stories, trends, and sources.

That means Hermes needs to do much more than answer a single prompt.

Conceptually:

Research
   ↓
Collect Information
   ↓
Evaluate Sources
   ↓
Summarize
   ↓
Identify Trends
   ↓
Generate Web Page
   ↓
Write Files

And here’s the important part:

Hermes was sending its model requests through OmniRoute throughout the workflow.

Hermes Task
    │
    ▼
OmniRoute
    │
    ▼
Best Available Configured Model
    │
    ▼
Response
    │
    ▼
Hermes Continues Task

After a minute or two, Hermes completed the task and generated the visual AI news report.

📊 Step 6: See Which AI Models Actually Handled the Requests

This was one of my favorite parts of the experiment.

After Hermes finished, I went back into OmniRoute.

Under its usage analytics, I could inspect information about the requests being sent through the gateway.

That included things such as:

Requests
Input tokens
Output tokens
Model usage
Which models actually served requests

So instead of the model-routing layer being a complete black box, I could inspect what happened after the task.

The workflow effectively becomes:

             ┌───────────────┐
             │ Hermes Agent  │
             └───────┬───────┘
                     │
                     ▼
             ┌───────────────┐
             │   OmniRoute   │
             └───────┬───────┘
                     │
            Routing Decision
                     │
          ┌──────────┴──────────┐
          ▼                     ▼
      Model A                Model B
          │                     │
          └──────────┬──────────┘
                     ▼
                 Response

                     +

              Usage Analytics

For agent development, that observability is useful.

It gives you visibility into what your model-routing infrastructure is actually doing rather than only seeing the final agent output.

🧩 Why This Architecture Matters Beyond Free Models

The interesting part isn’t actually the word free.

It’s the abstraction.

Without a gateway:

Agent → Specific Provider → Specific Model

With a gateway:

Agent → Model Gateway → Routing Strategy → Provider/Model

That means the agent doesn’t necessarily need to care whether the underlying request eventually reaches one provider or another.

This becomes useful when experimenting with:

Multi-provider AI infrastructure

Different providers can sit behind one gateway.

Model fallback strategies

Your application doesn’t have to own every routing decision.

Cost optimization 💰

Different models can potentially be selected based on your infrastructure strategy.

Model experimentation

You can change the models behind the gateway without redesigning your entire agent integration.

Observability

A centralized gateway can give you a better view of model usage across your agent workflows.

And these patterns aren’t limited to Hermes.

The broader architecture is applicable to many AI applications:

AI Application
      ↓
AI Gateway
      ↓
Routing / Policies
      ↓
Multiple Models & Providers

As AI systems become more agentic, this layer becomes increasingly interesting.

⚠️ But There’s a Catch: “Free” Doesn’t Mean Unlimited

This is important.

Using multiple free-model providers does not magically create unlimited free inference.

Every provider has its own policies and constraints.

For example:

OpenRouter free models have limits.

NVIDIA access can vary depending on the model, account, availability, and traffic.

Free models may also:

Respond more slowly
Become temporarily unavailable
Change their limits
Disappear from catalogs
Have different context windows
Behave differently across tasks

And there’s another important consideration:

Always read the provider’s terms.

Some services have specific rules around:

Automation
API usage
Processing
Redistribution
Rate limits
Free-tier usage

A few minutes checking those requirements before building automation around a provider can save you problems later.

Think of this architecture as a way to use available resources more flexibly, not as a way to bypass provider limits.

🏗️ The Bigger Lesson: Decouple Your Agent From Your Model Provider

This experiment reinforced something I’ve increasingly found useful when working with AI agents:

The agent and the model infrastructure don’t have to be the same layer.

Your agent should focus on:

Reasoning
Tools
Memory
Tasks
Workflows

Your model gateway can focus on:

Providers
Models
Routing
Availability
Usage
Policies
Observability

Put those together and you get a much cleaner architecture:

┌──────────────────────────────┐
│          AI Agent            │
│                              │
│ Tools • Memory • Workflows   │
└──────────────┬───────────────┘
               │
               ▼
┌──────────────────────────────┐
│          AI Gateway          │
│                              │
│ Routing • Models • Analytics │
└──────────────┬───────────────┘
               │
       ┌───────┼───────┐
       ▼       ▼       ▼
   Provider Provider Provider
      A        B        C

For small experiments, this might feel like an extra layer.

But once you’re testing multiple models and providers—or building longer-running agent workflows—the separation starts making much more sense.

🎯 Final Thoughts

If you’re learning AI agent development and want to experiment without immediately spending heavily on API calls, Hermes Agent + OmniRoute + free AI model providers is an interesting setup to explore.

You get:

✅ One endpoint for Hermes Agent

🔀 Multiple model/provider options behind the gateway

🧠 Automatic model selection strategies

📈 Centralized usage analytics

🛠️ The flexibility to add or change providers later

Just remember that the free tiers still have their own quotas, availability constraints, and terms.

The real value here isn’t “unlimited free AI.”

It’s building an agent architecture that isn’t unnecessarily coupled to one model provider.

Hermes Agent goes far beyond basic prompting. 🤖 I broke down /loop, /goal, Cron, Delegation, Bot Mode, Kanban & Mixture of Agents and when each actually makes sense. A practical mental model for building more autonomous AI agents. 👇

Vivek Shetye — Tue, 08 Sep 2026 15:29:15 +0000

Hermes Agent Challenge Submission: Write About Hermes Agent

Vivek Shetye

Sep 8

🤖 Hermes Agent Advanced Features Explained: Loop, Goal, Cron, Delegation, Bot Mode, Kanban & Mixture of Agents

#ai #agents #hermesagentchallenge #opensource

6 min read

🤖 Hermes Agent Advanced Features Explained: Loop, Goal, Cron, Delegation, Bot Mode, Kanban & Mixture of Agents

Vivek Shetye — Tue, 08 Sep 2026 15:03:20 +0000

Using Hermes Agent as a normal AI assistant is pretty straightforward.

You give it a task. It reasons about the problem, uses tools when necessary, and gives you a result.

But Hermes gets much more interesting when you ask a different question:

How do I get an AI agent to keep working without constantly prompting it?

Hermes Agent gives you several ways to do this.

You can create recurring loops, give the agent a persistent goal, schedule Cron Jobs, delegate work to sub-agents, create persistent specialist bots, coordinate multiple agents through Kanban, or even use multiple LLMs through Mixture of Agents.

The confusing part?

These features all increase agent autonomy, but they solve very different problems.

I recently published a video breaking down how these advanced Hermes Agent features work under the hood and, more importantly, when you should use each one.

🎥 Watch the full video:

Here’s the mental model I use to distinguish them.

🔁 1. Loop: Repeat Something Inside the Current Session

The easiest way to understand Hermes Agent’s /loop command is:

“Do this again later while I’m working in this session.”

A loop takes a prompt and reruns it on a recurring cadence.

For example:

Check my deployment every 5 minutes and tell me when it goes live.

The key distinction is that Loop is time-driven and session-scoped.

Use it for monitoring, repeated checks, polling, or other temporary recurring work associated with your current Hermes session.

🎯 2. Goal: Keep Working Until the Objective Is Complete

/goal solves a completely different problem.

Instead of saying:

“Do this again later.”

You’re saying:

“Keep working until this is actually done.”

Some tasks can’t reliably be completed in a single agent turn.

Maybe Hermes needs to modify code, run tests, inspect failures, make another change, and repeat the process.

That’s where Goal becomes useful.

After each continuation, the result can be evaluated to determine whether the objective has actually been completed.

Hermes also supports a completion contract, which can describe:

The desired outcome
How success should be verified
Constraints that must remain true
What is inside or outside the task’s scope
When Hermes should stop and ask for help

Instead of:

Fix authentication.

You can define something closer to:

Fix authentication so all tests pass.

Keep the existing login API response unchanged.

Only modify the authentication service and its tests.

Stop and ask me if a database migration becomes necessary.

Now both execution and evaluation have a much clearer definition of “done.”

One important misconception:

Goal does not automatically create an agent team, Kanban workflow, or independent background process.

It keeps the current Hermes session focused on a standing objective.

So:

Loop → Do this again later.

Goal → Keep working until this is done.

⏰ 3. Cron Jobs: Scheduled, Unattended AI Agent Workflows

Both Loop and Goal are connected to an active Hermes session.

But what if you want Hermes to wake up every morning and perform some work even when you’re not interacting with it?

That’s where Cron Jobs come in.

For example:

Every morning at 9 AM:

- Check my news feeds
- Find important AI developments
- Summarize them
- Send me a report on Telegram

When the scheduled time arrives, Hermes can launch a fresh agent session specifically for that run.

This makes Cron useful for things like:

📊 Daily reports
🔍 Recurring research
🛡️ Security audits
🧪 CI health checks
📰 News monitoring
📨 Scheduled summaries

The fundamental distinction is:

Loop
↓
Keep checking while this session is active.

Cron
↓
Wake up independently on a schedule and run the task.

If I’m watching a deployment for the next 20 minutes, I’d probably use Loop.

If I want a report generated every morning whether I’m there or not, I’d use Cron.

🧩 4. Sub-Agent Delegation: Temporary Parallel Workers

Now we move from one autonomous agent to multiple agents working simultaneously.

Suppose I ask Hermes:

Compare three approaches for implementing authentication
and recommend the best one.

The main agent could investigate each approach sequentially.

Or it could delegate:

Parent Agent
     │
     ├── Sub-Agent A → Research approach A
     ├── Sub-Agent B → Research approach B
     └── Sub-Agent C → Research approach C

The sub-agents can perform those tasks independently and return their findings to the parent.

This provides two major advantages.

⚡ Parallelism

Independent work can happen simultaneously rather than forcing the main agent to perform everything sequentially.

🧱 Context Isolation

Large research, coding, refactoring, and analysis tasks can consume significant context.

Delegating them means much of the intermediate work stays inside the sub-agent’s context rather than flooding the parent’s conversation.

The important distinction is that delegation is temporary.

A sub-agent exists to perform a focused task, return its result, and finish.

If you want a specialist identity that persists over time, you need something else.

🤖 5. Profiles & Bot Mode: Persistent AI Specialists

Imagine having separate Hermes agents for:

👨‍💻 Coding
🔬 Research
✍️ Content
📈 Analysis

You probably don’t want your research agent’s accumulated context and workflows constantly mixed with your coding agent.

That’s what Hermes Profiles help solve.

A profile acts like a separate Hermes environment with its own configuration and state.

This allows specialist agents to develop independently.

Bot Mode builds a more intuitive multi-agent experience on top of these persistent profiles.

The key distinction from delegation is persistence:

Sub-Agent
↓
Temporary worker for a task

Bot / Profile
↓
Persistent specialist identity

Multiple bots can also communicate with one another, allowing you to create workflows involving specialist agents.

But once those workflows become complicated, messages alone aren’t enough.

📋 6. Kanban: Durable Multi-Agent Orchestration

Suppose you have three persistent Hermes specialists:

Researcher
    ↓
Analyst
    ↓
Writer

You want the researcher to collect evidence.

Only after the research is finished should the analyst evaluate it.

And only after the analysis is complete should the writer produce the final report.

You could coordinate everything through agent messages.

But what happens when the workflow introduces:

Dependencies
Failures
Retries
Artifacts
Structured handoffs
Human review
Work lasting hours or days

Messages alone become fragile.

That’s where Hermes Kanban becomes useful.

Instead of the workflow existing only inside agent conversations, the work becomes structured, persistent task state.

Conceptually:

Research Topic
      ↓
Analyze Evidence
      ↓
Human Review
      ↓
Write Final Report

If Analyze Evidence depends on Research Topic, it waits.

Once the research task is completed, the dependent task can proceed.

This makes Kanban useful for durable, long-running multi-agent workflows.

And Kanban and delegation aren’t competing ideas.

A persistent Kanban worker could own a research task and then delegate parts of that research to several temporary sub-agents.

Think of it this way:

Delegation → temporary help

Bot Mode → persistent specialists

Kanban → coordination between specialists

🧠 7. Mixture of Agents: Multi-Model Reasoning

This is probably the easiest concept to confuse with multi-agent orchestration.

Mixture of Agents (MoA) operates at the model layer.

Imagine you’re solving a difficult software architecture problem.

Instead of asking one LLM to immediately decide what to do, you could ask several models to independently analyze the problem.

Reference Model A ──┐
                    │
Reference Model B ──┼──→ Aggregator Model → Hermes Agent Loop
                    │
Reference Model C ──┘

One model might notice a security problem.

Another might suggest a much simpler architecture.

Another might identify an assumption everyone else missed.

Their perspectives are then provided to an aggregator model, which becomes the model responsible for acting.

This is why Mixture of Agents is better understood as a model strategy rather than a traditional multi-agent workflow.

You’re not necessarily creating:

Research Agent
+
Coding Agent
+
Review Agent

Instead, you’re giving the acting model multiple model perspectives before it makes decisions.

The Hermes session, tools, and underlying agent loop can remain the same.

What changes is the model strategy powering that loop.

🗺️ The Mental Model

If you remember only one thing from this article, make it this:

Hermes Feature	Mental Model	Best For
🔁 Loop	Repetition	Repeated checks in the current session
🎯 Goal	Completion	Working toward a clearly defined outcome
⏰ Cron	Scheduling	Unattended scheduled workflows
🧩 Delegation	Temporary parallelism	Independent subtasks
🤖 Profiles / Bot Mode	Persistent identity	Long-lived specialist agents
📋 Kanban	Durable orchestration	Dependencies, handoffs, retries, and review
🧠 Mixture of Agents	Multi-model reasoning	Multiple LLM perspectives on difficult problems

The features become much easier to reason about once you stop grouping all of them under “autonomous AI agents.”

They’re different primitives for controlling different dimensions of autonomy.

🚀 Putting It All Together

The interesting part is that these concepts can be combined.

Imagine a software development workflow where:

Cron
  ↓
Starts scheduled workflow
  ↓
Kanban coordinates persistent specialist profiles
  ↓
Research Agent delegates work to temporary sub-agents
  ↓
Architecture Agent uses Mixture of Agents
  ↓
Goal keeps implementation focused until verification passes

Now you’re moving beyond a chatbot that happens to call tools.

You’re designing an agentic system where scheduling, persistence, parallelism, identity, orchestration, and model reasoning are separate architectural decisions.

And that distinction becomes increasingly important as AI agents become more autonomous.

Choosing an enterprise MCP gateway isn’t about the longest feature list. 👀 I compared Bifrost, Docker, Kong, Lunar MCPX & Microsoft MCP Gateway across identity, security, tool governance, deployment, and failure handling. Here’s what I found 👇

Vivek Shetye — Mon, 31 Aug 2026 15:05:34 +0000

Compares Bifrost to LiteLLM

Vivek Shetye

Aug 31

Best Enterprise MCP Gateway for Your AI Agents in 2026

#ai #mcp #agents #llm

10 min read