DEV Community: Srijan Paudel

The LLM Gateway & Router Index (2026)

Srijan Paudel — Thu, 02 Jul 2026 08:05:09 +0000

A gateway sits between your app and the model providers: one API, key management, caching, failover, cost control — and for routers, automatic model selection per request. Here is a neutral index by type (aggregator / proxy / router / API gateway), hosting, and license. No prices.

The matrix

Tool	Type	Hosting	License	Best for
OpenRouter	Aggregator / unified API	Managed	Proprietary	One API and billing across hundreds of models, pay-per-use
LiteLLM	Proxy gateway	Both (self-host or cloud)	Open-source	Self-hosted, OpenAI-compatible gateway + spend tracking across 100+ providers
Portkey	Proxy gateway + router	Both	Open-source core	Production gateway with routing, caching, guardrails, and observability
Cloudflare AI Gateway	Proxy gateway (edge)	Managed	Proprietary	Edge caching, rate-limiting, and analytics across providers with zero infra
Helicone	Proxy gateway + observability	Both	Open-source	Drop-in proxy logging, caching, and routing with minimal code
Kong AI Gateway	API gateway	Both	Open-source	Teams already on Kong or needing enterprise API governance
Vercel AI Gateway	Proxy gateway	Managed	Proprietary	Vercel / Next.js apps — one endpoint with failover and observability
Martian	Smart router	Managed	Proprietary	Dynamic routing to the best or cheapest model per request
Not Diamond	Smart router	Managed	Proprietary	Routing each prompt to the best-performing model automatically
Unify	Router + aggregator	Managed	Proprietary	Benchmarking and routing across providers on quality/cost/latency
Eden AI	Aggregator / unified API	Managed	Proprietary	One API across many AI services, not just LLMs
Apache APISIX AI Gateway	API gateway	Both	Open-source	Open-source API gateway with AI proxy and governance plugins

Quick picks

You want one API and billing across many models → OpenRouter
You want a self-hosted, OpenAI-compatible gateway → LiteLLM
You want routing + caching + guardrails in production → Portkey
You want edge caching/analytics with zero infra → Cloudflare AI Gateway
You want the cheapest/best model picked per request → Martian or Not Diamond
You already run Kong or need enterprise API governance → Kong AI Gateway
You're building on Vercel → Vercel AI Gateway
You want drop-in proxy logging that doubles as a gateway → Helicone

📚 More from The 2026 AI Stack Index: Automation Tools · Agent Frameworks · Vector Databases · LLM Observability · LLM Gateways

This is a neutral, no-affiliate reference — no prices (they go stale), no rankings-for-pay. The full, always-updated interactive version with FAQs and the rest of the AI-stack indexes lives at aiprosol.com/llm-gateways. Disclosure: I run Aiprosol, an automation consultancy — the index doesn't favour anyone.

The LLM Observability & Eval Index (2026)

Srijan Paudel — Thu, 02 Jul 2026 08:03:43 +0000

Once an agent hits production you need to see what it did and score whether it was any good. Here is a neutral index of the LLM observability + evaluation tools, by focus (tracing / evaluation / monitoring), hosting, and license. No prices — the tool is rarely the dominant cost.

The matrix

Tool	Focus	Hosting	License	Best for
LangSmith	All-in-one	Managed (enterprise self-host)	Proprietary	Teams building on LangChain / LangGraph — native graphs and replay
Langfuse	All-in-one	Both (self-host or cloud)	Open-source	Open-source, framework-agnostic tracing + eval with full data ownership (OTel)
Arize Phoenix	Tracing + evaluation	Both	Open-source	OTel-native tracing with rigorous, ML-grade evaluation primitives
Braintrust	Evaluation + tracing	Managed	Proprietary	Eval-first workflows — datasets, prompt iteration, and scoring
Confident AI (DeepEval)	Evaluation	Both	Open-source (DeepEval)	Pytest-style LLM evals and regression tests in CI
Weights & Biases Weave	Tracing + evaluation	Managed (enterprise self-host)	Proprietary	Teams already in the Weights & Biases ML ecosystem
Comet Opik	Tracing + evaluation	Both	Open-source	Open-source tracing + eval, optionally inside the Comet platform
Helicone	Tracing + monitoring	Both	Open-source	Drop-in proxy logging, cost tracking, and caching with minimal code
Langtrace	Tracing	Both	Open-source	Vendor-neutral, OpenTelemetry-native tracing
MLflow Tracing	Tracing + evaluation	Both	Open-source	Teams standardised on MLflow for the ML lifecycle
Latitude	Evaluation	Both	Open-source	Open-source prompt engineering with built-in evals
Maxim AI	All-in-one	Managed	Proprietary	End-to-end eval + observability across the agent lifecycle

Quick picks

You build on LangChain / LangGraph → LangSmith
You want open-source and full data ownership → Langfuse
You want OTel-native tracing with rigorous evals → Arize Phoenix
You want eval-first iteration (datasets, scoring) → Braintrust or Confident AI
You want CI regression tests for prompts → Confident AI (DeepEval)
You want drop-in proxy logging + cost control → Helicone
You already use W&B, MLflow, or Comet → Weave, MLflow, or Opik
You want vendor-neutral OpenTelemetry tracing → Langtrace

📚 More from The 2026 AI Stack Index: Automation Tools · Agent Frameworks · Vector Databases · LLM Observability · LLM Gateways

This is a neutral, no-affiliate reference — no prices (they go stale), no rankings-for-pay. The full, always-updated interactive version with FAQs and the rest of the AI-stack indexes lives at aiprosol.com/llm-observability. Disclosure: I run Aiprosol, an automation consultancy — the index doesn't favour anyone.

The Vector Database Index for RAG (2026)

Srijan Paudel — Thu, 02 Jul 2026 08:01:15 +0000

Vector-DB choice is a stack question, not a leaderboard. Already on Postgres? pgvector. Want zero-ops? Pinecone. Open-source + hybrid? Weaviate or Qdrant. Here is a neutral index of the RAG/memory layer by store type, hosting model, and license — no prices, no benchmarks that go stale.

The matrix

Database	Type	Hosting	License	Best for
Pinecone	Dedicated vector DB	Managed	Proprietary	Zero-ops, fully-managed production vector search
Weaviate	Dedicated vector DB	Both	Open-source	Open-source vector search with built-in hybrid search and modules
Qdrant	Dedicated vector DB	Both	Open-source	High-performance, Rust-based search with heavy metadata filtering
Chroma	Embedded	Both	Open-source	Lightweight, developer-friendly prototyping and small RAG apps
Milvus	Dedicated vector DB	Both (Zilliz managed)	Open-source	Billion-scale vector search
pgvector	Database extension	Both (any Postgres)	Open-source	Adding vectors to an existing Postgres stack without a new database
Redis	Document/cache DB + vectors	Both	Source-available	Low-latency vector search alongside caching/session data
Elasticsearch / OpenSearch	Search engine + vectors	Both	Open-source (OpenSearch) / mixed	Combining full-text and vector (hybrid) search at scale
MongoDB Atlas Vector Search	Document/cache DB + vectors	Managed	Proprietary (Atlas)	Vectors stored next to your operational documents
Azure AI Search	Search engine + vectors	Managed	Proprietary	Vector and hybrid search inside the Azure ecosystem
Vertex AI Vector Search	Dedicated vector DB	Managed	Proprietary	Google Cloud-scale ANN search (formerly Matching Engine)
LanceDB	Embedded	Both	Open-source	Embedded, multimodal, on-disk vector storage

Quick picks

You want zero-ops, fully managed vector search → Pinecone
You already run Postgres and want one database → pgvector
You want open-source with hybrid search → Weaviate or Qdrant
You're prototyping a RAG app fast → Chroma or LanceDB
You need billion-scale ANN → Milvus
You want vectors next to operational data → MongoDB Atlas or Redis
You need full-text + vector at scale → Elasticsearch / OpenSearch
You're committed to Azure or Google Cloud → Azure AI Search / Vertex AI Vector Search

📚 More from The 2026 AI Stack Index: Automation Tools · Agent Frameworks · Vector Databases · LLM Observability · LLM Gateways

This is a neutral, no-affiliate reference — no prices (they go stale), no rankings-for-pay. The full, always-updated interactive version with FAQs and the rest of the AI-stack indexes lives at aiprosol.com/vector-databases. Disclosure: I run Aiprosol, an automation consultancy — the index doesn't favour anyone.

The AI Agent Frameworks Index (2026)

Srijan Paudel — Sun, 28 Jun 2026 15:53:15 +0000

There are a dozen serious AI agent frameworks now, and the differences are real — chains vs graphs vs role-based crews vs SDKs. Here is a neutral index by language, design paradigm, license, and what each is genuinely best at. These are open-source libraries, so there are no prices — just a decision matrix.

The matrix

Framework	Languages	Paradigm	License	Best for
LangChain	Python · JS/TS	Chains / pipelines	Open-source	General-purpose LLM apps; the broadest integration ecosystem
LangGraph	Python · JS/TS	Graph / stateful	Open-source	Controllable, stateful, multi-step and multi-agent flows with human-in-the-loop
LlamaIndex	Python · TS	Data / RAG-centric	Open-source	Agents that reason over your own data (retrieval-augmented generation)
CrewAI	Python	Role-based crews	Open-source	Spinning up a team of role-playing agents quickly
Microsoft AutoGen (AG2)	Python	Conversational multi-agent	Open-source	Multi-agent conversation and research-style collaboration patterns
Microsoft Semantic Kernel	C# · Python · Java	SDK / plugins	Open-source	Embedding AI into enterprise and .NET applications
OpenAI Agents SDK	Python · JS/TS	Chains / pipelines	Open-source	Lightweight production agents with tools and handoffs (successor to Swarm)
Pydantic AI	Python	Type-safe	Open-source	Pythonic, type-checked agents with structured outputs
Haystack	Python	Data / RAG-centric	Open-source	Production search and RAG pipelines, with agent support (deepset)
Google ADK	Python · Java	SDK / plugins	Open-source	Building and deploying agents on Gemini / Vertex AI and Google Cloud
Vercel AI SDK	TypeScript	SDK / plugins	Open-source	AI features and agents in web apps, with streaming UI
n8n	No-code (+ JS)	No-code visual	Source-available · self-host	No-code agents (native AI / LangChain nodes) wired into real automations

Quick picks

You want the broadest ecosystem and maximum flexibility → LangChain
You need controllable, stateful, multi-step flows → LangGraph
The agent's value is RAG over your own data → LlamaIndex or Haystack
You want a team of role-playing agents fast → CrewAI or AutoGen
You're a .NET / enterprise shop → Semantic Kernel
You want type-safe, Pythonic agents → Pydantic AI
You're shipping agents inside a web / TypeScript app → Vercel AI SDK
You want no-code agents wired into automations → n8n

📚 More from The 2026 AI Stack Index: Automation Tools · Agent Frameworks · Vector Databases · LLM Observability · LLM Gateways

This is a neutral, no-affiliate reference — no prices (they go stale), no rankings-for-pay. The full, always-updated interactive version with FAQs and the rest of the AI-stack indexes lives at aiprosol.com/agent-frameworks. Disclosure: I run Aiprosol, an automation consultancy — the index doesn't favour anyone.

The AI Automation Tool Index (2026)

Srijan Paudel — Sun, 28 Jun 2026 14:32:40 +0000

Picking an automation platform in 2026 is mostly a pricing-model question — per-task vs per-operation vs per-execution decides your bill more than the logo does. Here is a neutral, vendor-independent index of the platforms worth shortlisting, by how they bill, whether you can self-host, and who each is for. No prices (vendor tiers move fast); pricing models only.

The matrix

Tool	Category	Pricing model	Self-host	AI	Best for
Zapier	No-code · cloud	Per-task	No (cloud-only)	AI actions, Agents, Chatbots, Copilot	Non-technical teams; the widest app-integration library; simple linear automations
Make	Visual no-code · cloud	Per-operation	No (cloud-only)	AI/LLM modules; built-in OpenAI and others	Visual builders wanting complex, multi-branch scenarios at lower cost than per-task
n8n	Source-available · self-host or cloud	Per-execution (cloud) / open-source (self-host)	Yes (self-host or managed cloud)	Native AI / LangChain nodes; build agents and RAG	Developers wanting control, data residency, and execution-based costs
Pipedream	Developer-first · code + no-code	Usage credits	No (cloud; bring your own code)	LLM building blocks; drop into code on any step	Developers who want a no-code surface but full code escape-hatches
Activepieces	Open-source · self-host or cloud	Open-source / self-host (usage tiers on cloud)	Yes (open-source core)	AI-native pieces; LLM steps and assistants	Teams wanting an open-source, self-hostable Zapier alternative
Microsoft Power Automate	Enterprise + RPA · cloud/desktop	Per-user / per-flow license	No (cloud; on-prem gateway + desktop flows)	AI Builder + Copilot	Microsoft 365 organisations; RPA and desktop automation
Workato	Enterprise iPaaS	Enterprise quote (recipe-based)	No (cloud; on-prem agents available)	Workato AI / Genie copilots	Large orgs needing governance, security, and IT-grade integration
Tray.io	Enterprise low-code iPaaS	Enterprise quote (platform/usage)	No (cloud)	Merlin AI agents	Ops and RevOps teams building complex internal integrations
Boomi	Enterprise iPaaS	Enterprise quote (connector-based)	Hybrid (cloud + on-prem runtimes)	Boomi AI	Enterprise data/app integration and EDI at scale
MuleSoft Anypoint	Enterprise iPaaS · API-led	Enterprise quote (subscription)	Hybrid (CloudHub + on-prem)	Einstein for Anypoint	API-led integration across large Salesforce and enterprise estates
Pabbly Connect	No-code · cloud	Flat-rate tiers	No (cloud)	AI app integrations	Cost-sensitive teams wanting predictable flat pricing
IFTTT	Consumer / prosumer · cloud	Flat-rate tiers	No (cloud)	Limited; some AI applets	Personal automations, smart home, and single-step applets

Quick picks

You're non-technical and want the broadest app coverage fast → Zapier
You want complex branching/visual logic at lower run cost → Make
You need self-hosting, data residency, or execution-based costs → n8n or Activepieces
You live in Microsoft 365 and need RPA/desktop automation → Power Automate
You're an enterprise needing governance and audited integration → Workato, Tray.io, Boomi, or MuleSoft
You're a developer who wants code on any step → Pipedream or n8n

📚 More from The 2026 AI Stack Index: Automation Tools · Agent Frameworks · Vector Databases · LLM Observability · LLM Gateways

This is a neutral, no-affiliate reference — no prices (they go stale), no rankings-for-pay. The full, always-updated interactive version with FAQs and the rest of the AI-stack indexes lives at aiprosol.com/automation-tools. Disclosure: I run Aiprosol, an automation consultancy — the index doesn't favour anyone.