Originally published on tamiz.pro.
The developer workflow is undergoing its most significant structural change since the introduction of the integrated development environment (IDE). For the past decade, AI assistance meant autocomplete: a model suggesting the next line of code as you type. That paradigm is now obsolete. We are entering the era of agentic AI—autonomous systems that don't just predict your next keystroke but execute entire engineering tasks, from writing a test suite to refactoring a legacy module to debugging a failing CI pipeline.
This shift isn't merely about smarter autocomplete. It's about a fundamental re-architecture of how software is built. At the center of this transformation are two distinct but interconnected technologies: AI coding agents (the workers) and AI gateways (the control plane). Understanding how they interact is critical for any engineer or architect looking to build, deploy, or manage modern software systems.
- 1. The Paradigm Shift: From Autocomplete to Agency
- 2. Anatomy of an AI Coding Agent
- 3. The AI Gateway: The New Control Plane
- 4. The Orchestration Layer: How Agents and Gateways Interact
- 5. Security and Governance in an Agentic World
- 6. Practical Implications for Engineering Teams
- 7. The Future: Multi-Agent Systems and Beyond
- Frequently Asked Questions
1. The Paradigm Shift: From Autocomplete to Agency
For years, the dominant AI interaction model in development was reactive. You wrote code, and the AI suggested completions. You encountered an error, and the AI explained it. The human remained the orchestrator of every action. The AI was a tool, not an actor.
Agentic AI flips this model. An agent is given a high-level objective—"fix the flaky test in payment-service"—and is empowered to take the necessary steps to achieve it. This includes:
- Reading and understanding the codebase context.
- Formulating a hypothesis about the root cause.
- Editing files across multiple modules.
- Running tests and linters to validate the changes.
- Iterating on failures until the objective is met.
This is not a single LLM call. It is a loop of perception, reasoning, and action. The agent doesn't just write code; it acts like a junior engineer who is tireless, context-aware, and capable of handling massive amounts of context.
The Two Pillars
This new workflow is powered by two distinct architectural components that are often confused:
| Component | Role | Analogy |
|---|---|---|
| AI Coding Agent | The executor. It plans, edits, and runs code. | The worker on the assembly line. |
| AI Gateway | The controller. It routes requests, manages tokens, enforces policies, and logs usage. | The quality control manager and supply chain. |
While the agent is the visible force driving the change, the gateway is the invisible infrastructure that makes it scalable, secure, and cost-effective. You cannot run agentic workflows in production without a robust gateway layer.
2. Anatomy of an AI Coding Agent
To understand how agents transform workflows, we need to understand how they are built. An AI coding agent is not a single monolithic program; it is a system composed of several interacting components.
2.1 The Core Loop
At its heart, an agent runs a simple but powerful loop:
# Simplified pseudo-code for an agentic loop
while not objective_is_complete():
# 1. Perception: Gather context from the environment
context = gather_context(repo, objective)
# 2. Reasoning: Decide on the next action
action = llm.think(objective, context, previous_attempts)
# 3. Action: Execute the decision
result = execute(action)
# 4. Evaluation: Check if the action moved us closer to the goal
if result.is_failure():
log_error(result)
adjust_strategy()
This loop is what separates an agent from a simple API call. The agent has a state (the objective, the history of actions, the current codebase state) and a policy (the instructions on how to choose actions).
2.2 Context Engineering
The biggest technical challenge for agents is context management. LLMs have finite context windows. A large codebase can easily exceed the token limits of even the most advanced models. Agents must therefore employ sophisticated context engineering techniques:
- Retrieval-Augmented Generation (RAG): Querying a vector database of the codebase to pull relevant files and symbols into the context window.
- Hierarchical Summarization: Summarizing large files into smaller chunks of metadata before passing them to the model.
- Graph-based Traversal: Treating the codebase as a dependency graph and traversing it intelligently rather than reading every file.
2.3 Tool Use and Function Calling
An agent is only as good as the tools it can wield. Modern agents are defined by their toolset: the set of functions they can call to interact with the world. For a coding agent, this includes:
read_file(path)edit_file(path, original, new)-
run_command(cmd)(e.g.,pytest,npm test) search_code(query)create_pull_request(branch, diff)
The LLM acts as the brain that selects which tool to use, but the agent framework provides the actual implementation. This is a critical distinction: the LLM doesn't execute code directly; it requests actions, and the framework executes them safely.
3. The AI Gateway: The New Control Plane
If the agent is the engine, the AI gateway is the dashboard, the fuel gauge, and the brake pedal. As teams scale from a single developer using a local agent to hundreds of developers using cloud-based agents, the gateway becomes non-negotiable.
3.1 What is an AI Gateway?
An AI gateway is a middleware layer that sits between the application (or the agent) and the LLM provider. It acts as a reverse proxy for AI requests. Instead of your agent calling api.openai.com directly, it calls your gateway, which then routes the request to the appropriate provider.
# Example: AI Gateway Configuration (e.g., LiteLLM, Portkey, Kong)
model_list:
- model_name: gpt-4o
litellm_params:
model: openai/gpt-4o
api_key: os.environ/OPENAI_API_KEY
- model_name: claude-3-5-sonnet
litellm_params:
model: anthropic/claude-3-5-sonnet
api_key: os.environ/ANTHROPIC_API_KEY
litellm_settings:
drop_params: true
num_retries: 3
3.2 Core Functions
The gateway provides several critical functions that are essential for agentic workflows:
| Function | Description | Why It Matters for Agents |
|---|---|---|
| Routing | Sends requests to the best model for the job. | Agents need to switch between fast/cheap models and slow/powerful models depending on the task. |
| Rate Limiting | Throttles requests to avoid hitting provider limits. | Agents make many rapid calls; without throttling, they'll get 429 errors. |
| Caching | Stores responses to identical prompts. | Agentic loops often re-ask the same question; caching saves money and latency. |
| Observability | Logs tokens, latency, cost, and errors. | You need to know how much your autonomous agents are spending. |
| Policy Enforcement | Blocks requests to disallowed models or domains. | Prevents agents from accidentally sending code to untrusted endpoints. |
3.3 The Multi-Provider Reality
One of the most compelling reasons for the gateway is model diversity. The best model for reasoning about architecture (e.g., Claude 3.5 Sonnet) might not be the best model for generating boilerplate tests (e.g., GPT-4o mini). A gateway allows you to define a routing strategy that automatically sends different types of requests to different providers.
// Example: Gateway routing strategy
{
"strategy": "semantic",
"rules": [
{
"match": {"task_type": "code_generation"},
"route_to": "gpt-4o-mini"
},
{
"match": {"task_type": "complex_refactor"},
"route_to": "claude-3-5-sonnet"
}
]
}
This capability is what makes agentic workflows economically viable. Without a gateway, you are forced to hard-code a single provider, which limits your flexibility and often leads to higher costs.
4. The Orchestration Layer: How Agents and Gateways Interact
Understanding the individual components is only half the battle. The real power emerges when you see how they interact in a production architecture. The agent is the client, and the gateway is the infrastructure. Here's how a typical request flows:
+-------------+ +-------------+ +-------------+ +-------------+
| Developer | --> | AI Agent | --> | AI Gateway | --> | LLM API |
| (Triggers) | | (Executor) | | (Controller)| | (Provider) |
+-------------+ +-------------+ +-------------+ +-------------+
| |
v v
+-------------+ +-------------+
| Code Editor | | Logging / |
| (Filesystem)| | Analytics |
+-------------+ +-------------+
4.1 The Loop in Practice
Let's trace a real-world scenario: "Fix the flaky test in the authentication module."
- Task Initiation: The developer assigns the task to the agent. The agent initializes its context with the repository.
-
Context Retrieval: The agent uses
search_codeto find the flaky test file. It reads the file and the associated module. - Reasoning: The agent calls the LLM via the gateway. The gateway routes this to a high-reasoning model (e.g., Claude 3.5 Sonnet) because it's a complex debugging task.
- Hypothesis Generation: The agent proposes a fix (e.g., "the test is flaky because of a race condition in the mock setup").
-
Action: The agent uses
edit_fileto modify the test. -
Validation: The agent runs
pytestviarun_command. The test passes. - Completion: The agent generates a diff and creates a pull request.
Throughout this process, the gateway is silently:
- Logging every LLM call, token count, and cost.
- Enforcing rate limits so the agent doesn't get throttled.
- Caching the context retrieval queries to speed up the loop.
4.2 The Cost of Autonomy
The biggest challenge in agentic workflows is runaway costs. An agent can easily make 50-100 LLM calls in a single task. Without a gateway, you have no visibility into this until the bill arrives. With a gateway, you can:
- Set a budget per task.
- Implement circuit breakers that halt the agent if costs exceed a threshold.
- Use semantic caching to avoid re-processing the same context.
# Example: Budget-aware agent loop with gateway integration
class BudgetAwareAgent:
def __init__(self, gateway_client, max_cost):
self.gateway = gateway_client
self.max_cost = max_cost
self.total_cost = 0
def run(self, task):
while not task.is_complete():
if self.total_cost > self.max_cost:
raise BudgetExceededError()
response = self.gateway.chat.completions.create(
model="auto", # Gateway decides based on routing strategy
messages=task.get_context()
)
self.total_cost += response.usage.cost
task.execute(response.content)
5. Security and Governance in an Agentic World
Autonomy introduces new security risks. An agent with write access to your repository is a powerful entity. If it's compromised or misdirected, it can do significant damage. This is where the gateway's policy enforcement capabilities become critical.
5.1 Data Leakage Prevention
The most common risk is data leakage. An agent might inadvertently send proprietary source code to a third-party API. The gateway can enforce policies that:
- Block requests containing sensitive patterns (e.g., API keys, private IP addresses).
- Redact PII before sending to the model.
-
Allowlist specific domains (e.g., only
api.openai.comandapi.anthropic.com).
5.2 Privilege Escalation
Agents must operate on the principle of least privilege. They should not have access to production secrets or infrastructure credentials. The agent framework should provide a sandboxed environment where:
- File system access is restricted to the working directory.
- Network access is limited to the gateway.
- Shell commands are validated against an allowlist.
# Example: Agent sandbox policy
sandbox:
file_access:
- path: "./src"
permission: "read_write"
- path: "./test"
permission: "read_write"
- path: "./secrets"
permission: "deny"
network:
allowed_domains:
- "gateway.internal"
deny_public: true
shell:
allowed_commands:
- "pytest"
- "npm test"
- "git diff"
5.3 Auditability
Because agents act autonomously, you need a complete audit trail. The gateway should log every request with:
- Agent ID (who initiated the request)
- Task ID (what the agent was trying to accomplish)
- Prompt hash (for security analysis)
- Timestamp and latency
- Cost
This allows you to answer the question: "Why did the agent modify auth.js at 3 AM?"
6. Practical Implications for Engineering Teams
For teams adopting agentic AI, the implications extend beyond just tooling. It changes team structure, code review practices, and performance metrics.
6.1 Code Review 2.0
When an agent writes 80% of a pull request, the human reviewer's role shifts. Instead of reviewing syntax, they must review intent and architecture. This requires new review strategies:
- Semantic Review: Focus on whether the code solves the problem correctly, not whether it's formatted correctly.
- Test Coverage Review: Ensure the agent wrote tests for its changes, not just the code.
- Context Review: Verify the agent didn't break unrelated functionality by changing shared modules.
6.2 The Rise of the "AI Ops" Engineer
Just as DevOps emerged to manage infrastructure, AI Ops is emerging to manage AI systems. This role is responsible for:
- Tuning gateway routing strategies.
- Monitoring token consumption and cost.
- Updating agent prompts and system instructions.
- Managing the vector database for code context.
6.3 Performance Metrics
Traditional metrics like lines of code per day are meaningless when an agent does the typing. New metrics are needed:
| Metric | Description |
|---|---|
| Task Completion Rate | Percentage of tasks the agent completes without human intervention. |
| Human Intervention Frequency | How often a developer must step in to correct the agent. |
| Cost per Task | Average gateway cost for a completed task. |
| Context Utilization | How efficiently the agent uses its context window. |
7. The Future: Multi-Agent Systems and Beyond
The current generation of agents is largely single-agent: one agent, one task. The next evolution is multi-agent systems where specialized agents collaborate.
7.1 The Agent Hierarchy
Imagine a system where:
- Orchestrator Agent breaks down a feature into subtasks.
- Frontend Agent implements the UI.
- Backend Agent implements the API.
- Test Agent writes and runs tests.
- Reviewer Agent performs a code review before handing off to a human.
This requires a sophisticated communication protocol between agents. The gateway will evolve to handle inter-agent communication, not just LLM calls.
# Example: Multi-agent orchestration
class Orchestrator:
def __init__(self, gateway):
self.gateway = gateway
self.agents = {
"frontend": FrontendAgent(gateway),
"backend": BackendAgent(gateway),
"tester": TestAgent(gateway)
}
def deploy_feature(self, feature_spec):
tasks = self.decompose(feature_spec)
results = {}
for task in tasks:
agent = self.agents[task.domain]
results[task.id] = agent.execute(task)
return self.integrate(results)
7.2 The Gateway as the Agent Mesh
In this future, the gateway becomes more than a proxy. It becomes the agent mesh: a service mesh for AI agents. It will handle:
- Service Discovery: Finding which agent can handle a specific task.
- Load Balancing: Distributing tasks across multiple agent instances.
- Retries and Timeouts: Handling agent failures gracefully.
- Circuit Breaking: Halting the entire system if one agent enters a bad state.
This is the logical endpoint of the current trend: a fully autonomous, self-healing software development ecosystem.
Frequently Asked Questions
What is the difference between an AI coding agent and an AI gateway?
An AI coding agent is the autonomous executor that plans and performs coding tasks (editing files, running tests). An AI gateway is the infrastructure layer that routes LLM requests, manages costs, enforces security policies, and provides observability. The agent is the "worker"; the gateway is the "control plane."
How do AI gateways help with cost control in agentic workflows?
Agentic workflows can make dozens of LLM calls per task. Gateways provide budget tracking, semantic caching (to avoid re-sending the same context), and circuit breakers that halt expensive tasks. They also allow routing to cheaper models for simple subtasks, significantly reducing overall spend.
Are AI coding agents safe for production codebases?
They can be, but only with proper guardrails. You must implement sandboxing (restricting file and network access), use an AI gateway for policy enforcement (blocking data leakage), and require human review for all agent-generated pull requests. Without these controls, autonomous agents pose significant security and quality risks.
This deep-dive was written for Tamiz's Insights. For more on modern development workflows, explore our technical articles.
...rols, autonomous agents pose significant security and quality risks. These risks are not hypothetical—they manifest in real incidents daily. A coding agent with overly broad filesystem permissions can silently overwrite production configuration. An agent with network access can exfiltrate secrets. An agent that "fixes" a failing test might delete the test instead of fixing the code. The industry is learning this the hard way.
The AI Gateway: Your New Control Plane
If coding agents are the new workforce, AI gateways are the new HR department, security guard, and compliance officer rolled into one. An AI gateway sits between your development environment and the LLM providers, intercepting every request and response. It's not a proxy—it's a policy enforcement point.
Why You Need One
Without a gateway, every developer's IDE plugin talks directly to OpenAI, Anthropic, or a local model. That means:
- No centralized observability — you can't answer "how many tokens did we burn this sprint?" without asking each developer.
- No consistent policy — one team might allow code suggestions in production files; another blocks them.
- No audit trail — when an agent makes a bad commit, you can't trace which prompt caused it.
- No cost control — a runaway agent loop can burn $500 in an hour.
A gateway changes all of that. It becomes the single choke point where you enforce rules, log everything, and meter usage.
Architecture Pattern
┌─────────────┐ ┌──────────────┐ ┌─────────────┐
│ IDE Plugin │───▶│ │───▶│ LLM │
│ CLI Agent │───▶│ AI Gateway │ │ Provider │
│ CI Pipeline│───▶│ │◀───│ (API) │
└─────────────┘ └──────────────┘ └─────────────┘
│
▼
┌─────────────┐
│ Policy Engine│
│ Audit Log │
│ Cost Meter │
└─────────────┘
Runnable Example: A Minimal AI Gateway
Here's a production-grade gateway implementation using Node.js and the OpenAI SDK. It supports rate limiting, prompt injection detection, and audit logging.
// gateway.js
import express from 'express';
import OpenAI from 'openai';
import rateLimit from 'express-rate-limit';
import { createHash } from 'crypto';
const app = express();
app.use(express.json());
// --- Configuration ---
const CONFIG = {
allowedModels: ['gpt-4o', 'gpt-4o-mini', 'claude-3-5-sonnet'],
maxTokensPerRequest: 4096,
maxRequestsPerMinute: 60,
blockPromptInjection: true,
auditLogPath: './audit.log',
};
// --- Rate Limiter ---
const limiter = rateLimit({
windowMs: 60 * 1000,
max: CONFIG.maxRequestsPerMinute,
standardHeaders: true,
legacyHeaders: false,
message: { error: 'Rate limit exceeded. Slow down, agent.' },
});
// --- Prompt Injection Detection ---
const INJECTION_PATTERNS = [
/ignore (all )?previous instructions/i,
/disregard (your )?(prior|earlier) (instructions|prompts)/i,
/you are now (dan|jailbreak|developer mode)/i,
/system:\s*override/i,
/<\|im_start\|>\s*system/i,
];
function detectInjection(prompt) {
if (!CONFIG.blockPromptInjection) return false;
return INJECTION_PATTERNS.some(pattern => pattern.test(prompt));
}
// --- Audit Logger ---
function auditLog(entry) {
const line = JSON.stringify({
timestamp: new Date().toISOString(),
...entry,
});
console.log(line);
// In production: append to file or ship to SIEM
}
// --- Token Metering (approximate) ---
function estimateTokens(text) {
return Math.ceil(text.length / 4);
}
// --- The Gateway Endpoint ---
app.post('/v1/chat/completions', limiter, async (req, res) => {
const { model, messages, max_tokens, user } = req.body;
// 1. Validate model
if (!CONFIG.allowedModels.includes(model)) {
auditLog({ event: 'MODEL_BLOCKED', model, user });
return res.status(403).json({
error: `Model '${model}' not allowed. Allowed: ${CONFIG.allowedModels.join(', ')}`,
});
}
// 2. Validate token budget
const requestedTokens = max_tokens || 1024;
if (requestedTokens > CONFIG.maxTokensPerRequest) {
auditLog({ event: 'TOKEN_BUDGET_EXCEEDED', requestedTokens, user });
return res.status(413).json({
error: `max_tokens ${requestedTokens} exceeds limit ${CONFIG.maxTokensPerRequest}`,
});
}
// 3. Scan for prompt injection
const fullPrompt = messages.map(m => m.content).join('\n');
if (detectInjection(fullPrompt)) {
auditLog({ event: 'INJECTION_DETECTED', user, model });
return res.status(400).json({
error: 'Prompt contains suspicious patterns. Request blocked.',
});
}
// 4. Forward to provider
const client = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
const startTime = Date.now();
try {
const completion = await client.chat.completions.create({
model,
messages,
max_tokens: requestedTokens,
});
const duration = Date.now() - startTime;
const usage = completion.usage;
// 5. Audit everything
auditLog({
event: 'COMPLETION_SUCCESS',
user,
model,
duration_ms: duration,
prompt_tokens: usage.prompt_tokens,
completion_tokens: usage.completion_tokens,
total_tokens: usage.total_tokens,
request_hash: createHash('sha256').update(fullPrompt).digest('hex').slice(0, 12),
});
res.json(completion);
} catch (error) {
auditLog({
event: 'COMPLETION_ERROR',
user,
model,
error: error.message,
duration_ms: Date.now() - startTime,
});
res.status(502).json({ error: 'Upstream provider error' });
}
});
// --- Health Check ---
app.get('/health', (req, res) => {
res.json({ status: 'ok', uptime: process.uptime() });
});
// --- Start Server ---
const PORT = process.env.PORT || 8080;
app.listen(PORT, () => {
console.log(`AI Gateway listening on port ${PORT}`);
});
Run it:
npm init -y
npm install express openai express-rate-limit
node gateway.js
Test it:
curl -X POST http://localhost:8080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-4o",
"messages": [{"role": "user", "content": "Refactor this function"}],
"max_tokens": 2048,
"user": "dev-42"
}'
Policy as Code: Defining Guardrails
A gateway without policies is just a pretty proxy. The real value comes from declarative policies that encode your organization's risk tolerance. Here's a policy engine that evaluates requests against rules before they reach the provider.
# policies.yaml
version: 1
policies:
- name: "block-secrets-in-context"
description: "Prevent agents from seeing secrets in their context window"
type: "content_scan"
action: "block"
patterns:
- "AKIA[0-9A-Z]{16}" # AWS Access Key
- "ghp_[a-zA-Z0-9]{36}" # GitHub PAT
- "sk-[a-zA-Z0-9]{32,}" # OpenAI Key
- "BEGIN (RSA|EC|OPENSSH) PRIVATE KEY"
- name: "restrict-file-writes"
description: "Agents can only write to src/ and tests/"
type: "filesystem"
action: "allow"
paths:
- "src/**"
- "tests/**"
deny:
- "src/config/production/**"
- ".env*"
- "*.pem"
- name: "require-human-approval"
description: "Any change touching 10+ files needs human sign-off"
type: "change_scope"
action: "require_approval"
threshold: 10
- name: "no-self-modification"
description: "Agents cannot modify their own configuration"
type: "filesystem"
action: "block"
paths:
- "agent.config.json"
- ".agent/**"
The gateway loads these policies, evaluates each request/response pair, and enforces actions. This turns security from a reactive afterthought into a proactive, code-reviewed artifact.
Deep Dive: The Agent Loop and the Gateway
The most interesting (and dangerous) part of agentic AI is the loop. An agent doesn't make one call—it makes dozens, iterating on its own output. Each iteration can compound errors. Here's what that looks like in practice:
Agent Loop:
┌─────────────────────────────────────────────────┐
│ 1. Analyze task │
│ 2. Generate plan │
│ 3. Execute step (write file, run test, etc.) │
│ 4. Observe result │
│ 5. If result != expected, adjust and retry │
│ 6. Repeat until done or max_iterations reached │
└─────────────────────────────────────────────────┘
Each iteration passes through the gateway. This means:
- Cost multiplies: 10 iterations × 2,000 tokens each = 20,000 tokens per task.
- Error propagation: a subtle misunderstanding in iteration 3 gets amplified by iteration 7.
- Context bloat: each iteration appends to the context window, degrading quality.
The Fix: Iteration Budgets and Checkpointing
// agent-loop-controller.js
export class AgentLoopController {
constructor({ gatewayUrl, maxIterations = 10, maxTokens = 50000 }) {
this.gatewayUrl = gatewayUrl;
this.maxIterations = maxIterations;
this.maxTokens = maxTokens;
this.totalTokensUsed = 0;
}
async runAgent(initialTask) {
let context = [{ role: 'user', content: initialTask }];
let iteration = 0;
let lastResult = null;
while (iteration < this.maxIterations) {
// Check token budget
if (this.totalTokensUsed >= this.maxTokens) {
console.warn(`Token budget exhausted at iteration ${iteration}`);
break;
}
// Call through the gateway
const response = await fetch(`${this.gatewayUrl}/v1/chat/completions`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
model: 'gpt-4o',
messages: context,
max_tokens: 2048,
user: 'agent-loop-controller',
}),
});
if (!response.ok) {
throw new Error(`Gateway error: ${response.status}`);
}
const data = await response.json();
const assistantMessage = data.choices[0].message;
const usage = data.usage;
this.totalTokensUsed += usage.total_tokens;
context.push(assistantMessage);
// Check if agent says it's done
if (assistantMessage.content.includes('[DONE]')) {
lastResult = assistantMessage.content;
break;
}
// Simulate agent executing a step (in real life, this runs tools)
const toolResult = await this.executeStep(assistantMessage.content);
context.push({ role: 'user', content: `Tool result: ${toolResult}` });
iteration++;
}
return {
finalResponse: lastResult,
iterations: iteration,
totalTokensUsed: this.totalTokensUsed,
contextSize: context.length,
};
}
async executeStep(agentOutput) {
// In production: parse agent output, run commands, capture results
return `Executed: ${agentOutput.slice(0, 50)}... (OK)`;
}
}
This controller enforces hard limits. An agent can't loop forever, can't blow the token budget, and every step is auditable through the gateway.
Real-World Integration: CI/CD Pipelines
The most impactful use case for coding agents isn't in the IDE—it's in CI/CD. Agents can triage failing tests, auto-fix lint errors, and even generate release notes. But they need guardrails.
Here's a GitHub Actions workflow that integrates an agent through a gateway:
# .github/workflows/agentic-fix.yml
name: Agentic Test Fixer
on:
workflow_dispatch:
pull_request:
types: [opened, synchronize]
jobs:
triage-and-fix:
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Run tests and capture failures
id: test
run: |
npm test 2>&1 | tee test-output.log
echo "exit_code=$?" >> $GITHUB_OUTPUT
- name: Invoke coding agent via gateway
if: steps.test.outputs.exit_code != '0'
env:
GATEWAY_URL: ${{ secrets.AI_GATEWAY_URL }}
GATEWAY_TOKEN: ${{ secrets.AI_GATEWAY_TOKEN }}
PR_NUMBER: ${{ github.event.pull_request.number }}
run: |
# Send failing test output to agent
curl -X POST "$GATEWAY_URL/v1/agent/triage" \
-H "Authorization: Bearer $GATEWAY_TOKEN" \
-H "Content-Type: application/json" \
-d "{
\"task\": \"Fix failing tests in this PR\",
\"context\": \"$(cat test-output.log | head -100)\",
\"pr_number\": $PR_NUMBER,
\"max_iterations\": 5,
\"allowed_paths\": [\"src/**\", \"tests/**\"]
}"
- name: Apply agent-generated patch
if: steps.test.outputs.exit_code != '0'
run: |
# Agent writes patch to a known location
if [ -f agent.patch ]; then
git apply agent.patch
git add -A
git commit -m "fix: auto-generated by coding agent [skip ci]"
git push
else
echo "No patch generated. Manual intervention required."
fi
The key insight: the agent never has direct push access. It can only propose changes. The CI pipeline applies them after validation.
Security Architecture: Defense in Depth
Let's map out the full security model. It's not enough to have a gateway—you need layers.
Layer 1: Input Sanitization
- Strip control characters
- Reject null bytes
- Normalize Unicode (to prevent homoglyph attacks)
Layer 2: Prompt Injection Defense
- Detect known injection patterns
- Use structured prompts (JSON schema) instead of raw text
- Never concatenate untrusted data into system prompts
Layer 3: Context Isolation
- Each agent gets a scoped context window
- Secrets are redacted before entering context
- PII is tokenized
Layer 4: Output Validation
- Verify agent output matches expected schema
- Check for malicious code patterns (eval, exec, child_process)
- Validate file paths against allowlist
Layer 5: Action Authorization
- Every filesystem write, network call, or shell command goes through a permission check
- Use least-privilege service accounts
- Require human approval for destructive actions
Layer 6: Audit and Observability
- Log every prompt, response, and action
- Generate trace IDs that correlate agent runs
- Alert on anomalous patterns (e.g., 50+ file writes in one run)
Runnable Example: Full Security Pipeline
Here's a complete middleware chain for a gateway that implements all six layers:
// security-pipeline.js
export function createSecurityPipeline(config) {
const middlewares = [];
// Layer 1: Input Sanitization
middlewares.push((req, res, next) => {
const body = req.body;
if (!body || typeof body !== 'object') {
return res.status(400).json({ error: 'Invalid request body' });
}
// Strip control characters from all string fields
const sanitize = (obj) => {
if (typeof obj === 'string') {
return obj.replace(/[\x00-\x1F\x7F]/g, '');
}
if (Array.isArray(obj)) return obj.map(sanitize);
if (typeof obj === 'object') {
return Object.fromEntries(Object.entries(obj).map(([k, v]) => [k, sanitize(v)]));
}
return obj;
};
req.body = sanitize(body);
next();
});
// Layer 2: Prompt Injection Defense
middlewares.push((req, res, next) => {
const prompt = JSON.stringify(req.body.messages || []);
const injectionPatterns = [
/ignore (all )?previous/i,
/you are now (dan|jailbreak)/i,
/system prompt override/i,
/<\|im_start\|>/i,
];
if (injectionPatterns.some(p => p.test(prompt))) {
req.injectionDetected = true;
}
next();
});
// Layer 3: Context Isolation
middlewares.push((req, res, next) => {
// Redact secrets from context
const redact = (text) => {
return text
.replace(/sk-[a-zA-Z0-9]{20,}/g, '[REDACTED_API_KEY]')
.replace(/AKIA[0-9A-Z]{16}/g, '[REDACTED_AWS_KEY]')
.replace(/ghp_[a-zA-Z0-9]{36}/g, '[REDACTED_GITHUB_TOKEN]');
};
if (req.body.messages) {
req.body.messages = req.body.messages.map(msg => ({
...msg,
content: redact(msg.content),
}));
}
next();
});
// Layer 4: Output Validation
middlewares.push((req, res, next) => {
const originalJson = res.json.bind(res);
res.json = (body) => {
// Check for dangerous patterns in output
const output = JSON.stringify(body);
const dangerousPatterns = [
/child_process/,
/exec\(/,
/eval\(/,
/process\.env/,
/fs\.writeFileSync/,
];
if (dangerousPatterns.some(p => p.test(output))) {
return res.status(400).json({
error: 'Model output contains potentially dangerous code patterns',
});
}
originalJson(body);
};
next();
});
// Layer 5: Action Authorization
middlewares.push((req, res, next) => {
// Check if the request is trying to perform an action
if (req.body.tool_calls) {
const allowedTools = config.allowedTools || [];
const blocked = req.body.tool_calls.filter(
tool => !allowedTools.includes(tool.name)
);
if (blocked.length > 0) {
return res.status(403).json({
error: `Tool(s) not authorized: ${blocked.map(t => t.name).join(', ')}`,
});
}
}
next();
});
// Layer 6: Audit Logging
middlewares.push((req, res, next) => {
const startTime = Date.now();
res.on('finish', () => {
const auditEntry = {
timestamp: new Date().toISOString(),
method: req.method,
path: req.path,
status: res.statusCode,
duration_ms: Date.now() - startTime,
user: req.body?.user || 'anonymous',
model: req.body?.model || 'unknown',
injection_detected: req.injectionDetected || false,
};
console.log(JSON.stringify(auditEntry));
});
next();
});
return middlewares;
}
The Human-in-the-Loop Pattern
Despite all the automation, the best agentic AI systems keep humans in the loop for high-stakes decisions. This isn't a limitation—it's a feature. The pattern is called "progressive automation":
- Level 0 (Manual): Human does everything.
- Level 1 (Assist): Agent suggests code, human applies.
- Level 2 (Supervised): Agent writes code, human reviews before merge.
- Level 3 (Conditional): Agent merges if tests pass and changes are below threshold.
- Level 4 (Full Auto): Agent operates independently within guardrails.
Most teams should target Level 2-3. Here's how to implement the approval gate:
// approval-gate.js
export class ApprovalGate {
constructor({ gatewayUrl, prService }) {
this.gatewayUrl = gatewayUrl;
this.prService = prService;
}
async processAgentChange(change) {
// Determine risk level based on change scope
const riskScore = this.calculateRisk(change);
if (riskScore >= 8) {
// High risk: require human approval
const approval = await this.requestHumanApproval(change);
if (!approval) {
return { status: 'rejected', reason: 'Human declined' };
}
return { status: 'approved', change };
}
if (riskScore >= 5) {
// Medium risk: auto-approve but notify
await this.notifyReviewers(change);
return { status: 'approved', change, notified: true };
}
// Low risk: auto-approve silently
return { status: 'approved', change };
}
calculateRisk(change) {
let score = 0;
// Score based on files touched
if (change.files.length > 10) score += 3;
if (change.files.length > 5) score += 2;
// Score based on sensitive paths
if (change.files.some(f => f.includes('config') || f.includes('env'))) {
score += 3;
}
// Score based on destructive operations
if (change.operations.includes('delete')) score += 2;
if (change.operations.includes('database_migration')) score += 3;
// Score based on test coverage impact
if (change.testsModified && !change.testsAdded) score += 1;
return score;
}
async requestHumanApproval(change) {
// In production: send Slack notification, wait for response
console.log(`[APPROVAL REQUIRED] ${change.description}`);
return new Promise(resolve => {
// Simulate human response after timeout
setTimeout(() => resolve(true), 3000);
});
}
async notifyReviewers(change) {
console.log(`[REVIEW NOTIFICATION] ${change.description}`);
}
}
Measuring the Impact: Metrics That Matter
If you're going to invest in agentic AI, you need to measure whether it's actually working. Here are the metrics that matter:
Productivity Metrics
- Time-to-merge: How long from PR creation to merge
- Cycle time: Time from first commit to merge
- PR throughput: PRs merged per developer per week
- Review turnaround: Time from PR ready to first review
Quality Metrics
- Defect escape rate: Bugs that reach production / total bugs
- Test coverage delta: Change in coverage after agent adoption
- Revert rate: Percentage of PRs that get reverted
- Incident count: Production incidents per month
Cost Metrics
- Cost per PR: Total AI spend / PRs merged
- Token efficiency: Tokens used / lines of code changed
- Agent iteration rate: Average iterations per task
- Wasted tokens: Tokens spent on rejected or reverted work
Security Metrics
- Injection attempts blocked: Count per week
- Policy violations: Number of blocked requests
- Secrets exposed: Any secret that reached a context window
- Unauthorized actions: Actions blocked by permission system
Here's a dashboard snippet:
// metrics-dashboard.js
export function renderMetrics(metrics) {
return `
<div class="metrics-grid">
<div class="metric-card">
<h3>Time-to-Merge</h3>
<p class="metric-value">${metrics.timeToMerge}h</p>
<p class="metric-trend ${metrics.timeToMergeTrend < 0 ? 'positive' : 'negative'}">
${metrics.timeToMergeTrend}% vs last sprint
</p>
</div>
<div class="metric-card">
<h3>Revert Rate</h3>
<p class="metric-value">${metrics.revertRate}%</p>
<p class="metric-trend ${metrics.revertRateTrend < 0 ? 'positive' : 'negative'}">
${metrics.revertRateTrend}% vs last sprint
</p>
</div>
<div class="metric-card">
<h3>Cost per PR</h3>
<p class="metric-value">$${metrics.costPerPR}</p>
<p class="metric-trend ${metrics.costPerPRTrend < 0 ? 'positive' : 'negative'}">
${metrics.costPerPRTrend}% vs last sprint
</p>
</div>
<div class="metric-card">
<h3>Injection Attempts</h3>
<p class="metric-value">${metrics.injectionAttempts}</p>
<p class="metric-trend">Blocked by gateway</p>
</div>
</div>
`;
}
The Organizational Shift
Adopting agentic AI isn't just a technical change—it's an organizational one. Here's what changes:
The Developer's Role Evolves
- From "writing code" to "specifying intent"
- From "debugging" to "reviewing agent output"
- From "implementing features" to "architecting solutions"
Team Structure Changes
- New role: AI Workflow Engineer — owns the gateway, policies, and agent configs
- New role: Prompt Engineer — designs the prompts and context strategies
- New role: AI Security Analyst — monitors for injection, data leakage, policy violations
Code Review Becomes Different
- Reviewing agent-generated code requires different skills
- You're not just checking syntax—you're checking intent
- Need to verify the agent didn't hallucinate APIs or introduce subtle bugs
Common Pitfalls and How to Avoid Them
Pitfall 1: Letting Agents Write to Production
The problem: An agent with access to production configs can cause outages.
The fix: Never give agents production credentials. Use a separate, read-only context for production data.
Pitfall 2: Not Setting Token Budgets
The problem: A runaway agent loop can burn thousands of dollars.
The fix: Set hard token limits per agent, per task, and per day. Enforce at the gateway.
Pitfall 3: Trusting Agent Output Without Validation
The problem: Agents hallucinate APIs, invent functions, and produce plausible-looking garbage.
The fix: Always validate agent output against a schema or type system. Run tests before merging.
Pitfall 4: Ignoring Context Window Management
The problem: As the context grows, the agent gets dumber. It loses track of earlier instructions.
The fix: Implement context summarization. Compress old messages into a summary that preserves key information.
Pitfall 5: No Rollback Plan
The problem: When an agent breaks something, you need to undo it fast.
The fix: Require agents to work in feature branches. Keep the main branch protected. Use atomic deployments.
The Road Ahead
We're in the early days of agentic AI. The next 12-18 months will bring:
- Standardized agent protocols — like MCP (Model Context Protocol), which will let agents discover and use tools across platforms.
- Self-improving agents — agents that learn from their mistakes and get better over time.
- Multi-agent orchestration — teams of agents working together, each with a specialized role.
- Local-first agents — running on-device models for privacy-sensitive work.
- Agent marketplaces — pre-built agents for common tasks (testing, refactoring, documentation).
The organizations that win won't be the ones with the most powerful models. They'll be the ones with the best infrastructure—the ones that build robust gateways, enforce sensible policies, and keep humans in the loop where it matters.
Conclusion
Agentic AI is rewriting developer workflows. Coding agents can triage tests, fix bugs, generate documentation, and refactor code at scale. But they're not magic—they're powerful tools that need guardrails.
The AI gateway is the control plane that makes agentic AI safe, observable, and cost-effective. It's where you enforce policy, detect injection attacks, meter usage, and audit every action. Without it, you're running an uncontrolled experiment in your production environment.
The developers who thrive in this new era won't be the ones who write the most code. They'll be the ones who can architect systems where AI does the heavy lifting—and where humans make the decisions that matter.
The future isn't AI replacing developers. It's developers who know how to harness AI, building systems that are more capable than anything we could have imagined.
This deep-dive was written for Tamiz's Insights. For more on modern development workflows, explore our technical articles.
Top comments (0)