Tamiz Uddin

Posted on Sep 5 Originally published at tamiz.pro

Why AI Agents Disappear in Production (and How to Keep Them Running for $5.70/Month)

#ai #machine #learning #agents

Originally published on tamiz.pro.

The Vanishing Act

AI agents vanish in production for three reasons: stateful sessions time out, dependencies bloat the runtime, and costs spiral silently. This guide fixes all three with minimal infra.

Prerequisites

Node.js 18+ or Python 3.9+
Docker (optional, for self-hosting)
A free-tier account (e.g., Railway, Render, or Fly.io)
Basic agent framework (e.g., LangChain, CrewAI, or custom LLM loop)

Step 1: Stateless Session Management

Store agent state outside the process using Redis or SQLite. This survives restarts and scales horizontally.

Python Example (Redis)

import redis, json

r = redis.Redis(host="localhost", port=6379, db=0)

def save_state(session_id: str, state: dict):
    r.setex(f"agent:{session_id}", 3600, json.dumps(state))

def load_state(session_id: str) -> dict:
    data = r.get(f"agent:{session_id}")
    return json.loads(data) if data else {}

Node.js Example (SQLite)

const Database = require("better-sqlite3");
const db = new Database("agent.db");

db.exec(`CREATE TABLE IF NOT EXISTS sessions (id TEXT PRIMARY KEY, state TEXT, updated_at DATETIME DEFAULT CURRENT_TIMESTAMP)`);

function saveState(sessionId, state) {
  const stmt = db.prepare(`INSERT OR REPLACE INTO sessions (id, state) VALUES (?, ?)`);
  stmt.run(sessionId, JSON.stringify(state));
}

function loadState(sessionId) {
  const row = db.prepare(`SELECT state FROM sessions WHERE id = ?`).get(sessionId);
  return row ? JSON.parse(row.state) : {};
}

Step 2: Lightweight Dependency Layer

Strip non-critical packages. Use llama-cpp-python or local LLMs for inference instead of external APIs when possible.

Minimal Agent Loop (Python)

import os
from langchain.llms import LlamaCpp
from langchain.agents import initialize_agent

llm = LlamaCpp(model_path="./models/phi-2.Q4_K_M.gguf", n_ctx=2048)

tools = [your_tool_here]

agent = initialize_agent(tools, llm, agent="zero-shot-react-description", verbose=True)

response = agent.run("Your task here")
print(response)

Step 3: Cost Control with Health Checks

Use a free-tier cron job to ping your agent every 5 minutes. If it’s down, restart it.

Bash Health Check Script

#!/bin/bash
curl -sSf https://your-agent-endpoint.com/health || docker restart ai-agent || systemctl restart ai-agent

Schedule via GitHub Actions or cron:

# .github/workflows/health-check.yml
name: Health Check
on:
  schedule:
    - cron: '*/5 * * * *'
jobs:
  check:
    runs-on: ubuntu-latest
    steps:
      - name: Ping Agent
        run: curl -sSf https://your-agent-endpoint.com/health || exit 1

Step 4: Deploy on Free Tier

Use Fly.io or Railway for persistent, low-cost hosting.

Fly.io Deployment (fly.toml)

app = "ai-agent"
kill_signal = "SIGINT"
kill_timeout = 5

[deploy]
  releases = false

[env]
  PORT = "8080"

[[services]]
  internal_port = 8080
  protocol = "tcp"

  [[services.ports]]
    handlers = ["http"]
    port = 80

Deploy:

flyctl launch --name ai-agent --image your-agent-image --region iad --hostname ai-agent.fly.dev

Final Architecture

Component	Cost	Persistence
SQLite/Redis	$0	Yes
Local LLM	$0	Yes
Fly.io Free Tier	$0–$5.70	Yes
Health Check Job	$0	Yes

Total: $5.70/month for a reliable, persistent AI agent.

Frequently Asked Questions

Q: What if my agent needs external APIs?

A: Cache responses locally and fall back to cached data on failure.

Q: How do I scale beyond one instance?

A: Add a load balancer and shared Redis for coordination.

Q: Can I use serverless?

A: Yes, but cold starts add latency. Stick to lightweight containers for always-on agents.

DEV Community