DEV Community

Cover image for We Built 17 MCP Servers to Let AI Run Our Internal Operations

We Built 17 MCP Servers to Let AI Run Our Internal Operations

Ryosuke Tsuji on April 07, 2026

AI assistance disclosure: This article was drafted with the help of Claude. All technical content, design decisions, code references, and screensh...
Collapse
 
sidclaw profile image
SidClaw

17 MCP servers is a lot of surface area. curious how you're handling the "who can call what" question across all of them. when one agent has access to 17 different tool servers, the blast radius of a bad decision gets wide fast.

do you have any per-action controls, or is it more of a trust-the-agent-and-audit-after approach?

Collapse
 
ryantsuji profile image
Ryosuke Tsuji

@sidclaw
Great question — "who can call what" is exactly the right thing to worry about. The honest answer is that we use a different approach per server depending on the risk, and stack multiple layers. It's definitely not a "trust-the-agent-and-audit-after" setup.

First, a prerequisite: agents don't actually see all 17 at once

The headline says "17 MCP servers" but a single agent session almost never has all 17 loaded at the same time. For engineer-facing tools, we drop a .mcp.json at the root of each repository listing only the servers that make sense for that repo. A monorepo's .mcp.json might list code-graph, DB graph, CI, Grafana, git workspace, etc., while a standalone frontend repo gets just a single graph-rag entry. The result is that when you start Claude Code in a given workspace, only the relevant tools get loaded, naturally.

This is by far the simplest and most effective way to shrink blast radius: we solve the agent's "which of 17 tools should I pick?" problem upstream, at the developer-workflow level, before the model ever has to reason about it.

On top of that, each server internally stacks five more layers.

Layer 1: Enforce read-only at the cloud API level (gcloud / aws / gws)

The gcloud MCP is the clearest example. The user authenticates with the cloud-platform OAuth scope, but when the server actually runs a command, it impersonates a dedicated read-only Service Account and executes the gcloud CLI as that SA. That SA only has roles/viewer, so even if the agent tries to generate something destructive like gcloud run services delete, GCP's IAM itself rejects it with permission denied. Even if our server-side validation has a gap, the cloud API makes it physically impossible.

const readonlySa = new gcp.serviceaccount.Account(...);
new gcp.projects.IAMMember('readonly-viewer', {
  role: 'roles/viewer',  // ← the real safety net
  member: serviceAccountMember(readonlySa.email),
});
Enter fullscreen mode Exit fullscreen mode

aws-server does the same via Google OIDC → AWS STS AssumeRoleWithWebIdentity, swapping into a read-only IAM role. The key point is that the server's own SA is deliberately given no data permissions — all operations happen under the user's token, which has the nice side effect of getting individual email addresses into GCP Audit Log / CloudTrail automatically.

Layer 2: Defense in depth via input validation (DB access)

DB access MCPs do something more aggressive (SELECT against production DBs), so we add another layer in front. A SQL validator allows only SELECT / SHOW / DESCRIBE / EXPLAIN / WITH, and rejects DROP / TRUNCATE / DELETE / INSERT / UPDATE / ALTER / CREATE and multi-statement queries via regex before the query ever touches a driver. On top of that, the DB users themselves are split into view / edit / delete permission levels, and MCP traffic is pinned to view — a read-only account at the DB grant level.

So you get "MCP-side parser rejection × DB-side GRANT" as two independent layers. Slipping through one is plausible, slipping through both is essentially not.

Layer 3: Per-user ACLs in Firestore (write-capable tools for non-engineers)

For any MCP server that can write, we keep per-user ACLs in Firestore.

The permission doc for the non-engineer "edit and deploy from chat" workspace MCP looks like this:

{
  "email": "user@example.com",
  "allowedPaths": ["apps/some-service/", "packages/some-domain/"],
  "allowedStacks": ["api-some-service", "pages-some-service"],
  "role": "developer"
}
Enter fullscreen mode Exit fullscreen mode

write_file / commit tools do a prefix match against allowedPaths, and deploy tools additionally require allowedStacks membership plus role === 'developer'. Path traversal (..) is rejected by regex. Because both file paths and deployable stacks are whitelisted, a team member can't accidentally reach infra they don't own.

One step earlier in the stack, some servers skip individual ACLs and instead gate on users.division / groupName / teamName read from Firestore — a single org-level allow-list per server. When you don't need per-person granularity, it's much cheaper to operate.

Layer 4: PII anonymization at the data layer

This is the strongest layer because it doesn't rely on agent judgment at all. When the DB access MCP reads production data, a PII anonymization step strips the result before returning it: email becomes a***@example.com, personal names become 田***, phone numbers become ***-****-1234. It's two-tiered: a global rule set (matching column names like email, phone, password, etc.) plus DB-specific rules keyed by database.table.column.

Highly sensitive datasets (HR, etc.) go further — they're locked down at the BQ IAM level to a very small set of owners, so the MCP server's own SA can't even read them. We're not filtering these out in application code; BigQuery itself refuses the query.

Layer 5: Audit logging for every tool call

Every MCP server routes tool executions through a shared package (@cortex/mcp-tool-analytics) that writes to a BigQuery mcp_tool_calls table. Schema: timestamp, server_name, tool_name, user_email, nick_name, department, params, duration_ms, status, result_size, estimated_tokens.

One important detail: writes happen via impersonation of a dedicated writer SA, not from each server's own SA, which keeps the log-tampering surface tiny. We roll this up into dashboards — not just for post-hoc audit, but also for usage analysis (finding tools nobody actually calls, so we can delete them).

Summary

Risk Primary defense
Agent picks the wrong server out of 17 Layer 0: per-repository .mcp.json narrows the choice set upstream
Agent generates a destructive command Layer 1: IAM physically rejects it
Input sanitization gap Layer 2: SQL validator + DB GRANT
Out-of-scope resource modification Layer 3: Firestore ACL (paths / stacks / role / org)
PII leakage Layer 4: result masking + BQ IAM
Detection miss Layer 5: audit log through the shared package

The overall design principle is: start read-only, and if you need writes, explicitly whitelist exactly those operations. Because implementers get OAuth and audit logging for free from the shared packages (@cortex/mcp-oauth, @cortex/mcp-tool-analytics), new-server development only needs to focus on the ACL that's specific to that tool. That's a big part of how we got to 17 without the blast radius getting out of hand.

Collapse
 
renato_marinho profile image
Renato Marinho

Running 17 MCP servers across internal operations is a significant architectural achievement — most teams struggle to maintain even 3 or 4 reliably. The coordination overhead alone across Slack, GitHub, BigQuery, and custom internal tools must have been substantial to get right.

The question that becomes unavoidable at that scale is governance: when 17 agents are running with write access across your internal systems, how do you reconstruct exactly what each one did when something unexpected happens? How do you prevent PII from customer data flowing through to the LLM context during routine operations? And if one server starts misbehaving at 2am, how do you shut it down without taking down the others?

These are the questions Vinkius (vinkius.com) was built to answer. It runs pre-governed MCP servers inside V8 Isolate sandboxes — each call generates a SHA-256 cryptographic audit trail, PII is redacted at the protocol level before reaching the model, and there's a global kill switch per server. The SDK is Vurb.ts, which wraps MCP tool calls with these controls natively rather than as middleware.

Your architecture proves the technical feasibility of agent-driven internal operations. The governance layer is what makes it auditable and safe enough to trust at scale. Really impressive operational experiment — would love to see a follow-up on how you handle incidents across 17 concurrent servers.

Collapse
 
ryantsuji profile image
Ryosuke Tsuji • Edited

@renato_marinho
PII Protection:
PII redaction is handled at the data layer, not as middleware. Our DB Graph MCP server (detailed in this post) automatically anonymizes query results from production databases — emails become @.com, names become ***, phone numbers/addresses/card numbers are all masked before they ever reach the LLM context. This runs at both the MCP layer and the Lambda layer (dual validation), so even if one layer is compromised, PII doesn't leak. On the observability side, Grafana's log data is also protected — structured logs are designed to exclude PII fields, and Grafana access itself is scoped via Google OAuth with domain restrictions. Servers that don't touch customer data (infrastructure, CI/CD, documentation) simply don't have access to PII-containing databases in the first place — scope separation by design.

Observability & Incident Response:
Every MCP server is instrumented with OpenTelemetry, and all logs/traces/metrics are aggregated in Grafana. Grafana alerting rules are configured per server — latency spikes, error rate thresholds, and availability checks all trigger Slack notifications automatically. So if a server misbehaves at 2am, the on-call engineer gets a Slack alert immediately.

For investigation, we built a Grafana MCP server — meaning Claude Code itself can query logs and metrics. "Show me error logs from the DB Graph MCP in the last hour" returns
structured results directly in the AI context. This closes the loop: the same AI that uses the MCP servers can also diagnose issues with them.

Independent Deployment:
Each server is a separate Cloud Run service with its own Pulumi stack, service account, and IAM roles. Deploying, scaling, or shutting down one server has zero impact on the others. There's no shared runtime or process — they're fully isolated at the infrastructure level.