Building a Production-Grade AI Agent Gateway in Node.js: RBAC, Gateway-Level HITL, and Dynamic Tools

When building AI agents for enterprise environments, relying solely on raw LLM calls is not enough. You need fine-grained control over execution boundaries, security guardrails, role-based access control (RBAC), and human-in-the-loop (HITL) intervention patterns.

Over the past few iterations, we designed and built a modular AI Agent Gateway in Node.js from scratch. Here is a breakdown of the core architecture and features built so far.

Architectural Overview
The gateway is structured following the Single Responsibility Principle, separating authentication, tool registries, semantic search caching, guardrails, and the core agent orchestrator into clean, decoupled modules.

agent.js: Core orchestrator managing the terminal CLI loop, safety guardrails, model fallback logic, semantic caching, and gateway-level decision routing.

tools/createTools.js: Dynamic tool factory that maps user permissions to authorized execution capabilities (GeoLocation, Weather, Currency Converter).

authentication/: JWT-based session verification tied to a mock database user profile.

timestampsWindow/: Rate-limiting and global request window management.

services/: Embedding generation and semantic search ranking for caching responses.

Key Features Implemented

1. Role-Based Access Control (RBAC)
   Every incoming request is verified via JWT. The user's ID is checked against a local user database to determine which tools they are explicitly authorized to use (e.g., user_123 has access to weather and geoLocation, but not currencyConverter). Unauthorized tools are excluded from the model payload to prevent unauthorized tool invocation and hallucination.

2. Gateway-Level Human-In-The-Loop (HITL) Escalation
   Instead of blocking execution completely when a user attempts a restricted action, the gateway intercepts the intent at the routing layer.

If a user requests an action requiring a restricted tool, the gateway pauses execution using node:readline/promises.

It prompts the operator interactively: Do you want to escalate permissions and temporarily unlock this tool? (y/n).

Upon a y confirmation, the tool is dynamically appended to the active session toolset, allowing the agent to proceed smoothly.

1. Real-World Tool Integration Integrated practical tools using Zod schemas and LangChain tool wrappers:

GeoLocation: Resolves city names to latitude and longitude using OpenStreetMap Nominatim APIs.

Weather: Fetches current weather metrics via Open-Meteo forecasts.

Currency Converter: Performs live currency exchange calculations using real-time open rates.

1. Guardrails & Fallback Mechanisms Input/Output Safety: A secondary guardrail agent evaluates user prompts and agent responses for malicious or sensitive data leaks.

Model Fallback Mapping: Automatically cycles through OpenRouter model limits using a fallback map when quotas are exhausted.

Semantic Caching: Stores previous query embeddings and responses to serve repetitive queries efficiently.

What is Next
With the core gateway, RBAC, and gateway-level HITL fully functional in a terminal environment, the next phase focuses on scaling to a production-ready system:

MongoDB: Persistent user management, chat histories, and audit logs.

Redis & BullMQ: Asynchronous background queue processing for heavy agent tasks.

Frontend Dashboard: A web-based interface to manage agent queries and monitor gateway logs directly from the client side.