DEV Community

Cover image for Detect AI writing, then rewrite it human — Bedrock + Strands Agents + AgentCore Gateway
Yuuki Yamashita
Yuuki Yamashita

Posted on

Detect AI writing, then rewrite it human — Bedrock + Strands Agents + AgentCore Gateway

#aws #bedrock #ai #nextjs

Live demo: https://ai-text-checker-pi.vercel.app
Code: https://github.com/yama3133/ai-text-checker

I built AI Text Checker (Japanese name: 「AI文章判定くん」): paste some text and it scores how "AI-like" the writing is with reasons, then rewrites it to read as if a human wrote it. It works in Japanese and English, accepts paste / .txt / .md / .pdf, and is also exposed as an MCP tool so you can call it from Claude Code and other agents.

This post is less a feature tour and more a write-up of the decisions and gotchas — including one honest design choice that shaped the whole thing.

The honest premise: you cannot "detect AI" reliably

There is no technology today that definitively decides whether a text was written by a human or an AI. Dedicated detectors have high false-positive rates, especially on Japanese and on AI text that a human lightly edited. Asking an LLM to output a binary "AI or human" verdict is even shakier.

So I deliberately did not build an "AI detector." I built an AI-likeness editor:

  • ❌ No binary "AI/human" verdict.
  • ✅ An AI-likeness score (an estimate) plus the specific phrases that read AI-ish (mechanical connectives, safe generalities, formulaic closers, uniform rhythm, lack of specifics).
  • ✅ A rewrite that keeps the meaning but sounds human.

The UI says this in plain text: it is an estimate, it can be wrong, don't use it as proof of authorship. That framing plays to what an LLM is actually good at — editorial feedback and rewriting — instead of pretending it can do something it can't.

Stack

  • Next.js 16 (App Router) + Tailwind, deployed on Vercel
  • Amazon Bedrock, Claude Sonnet 4.6 via the Converse API (us-east-1)
  • Strands Agents (TypeScript SDK) for the "rewrite until it passes" loop
  • Amazon Bedrock AgentCore Gateway to expose the tool over MCP

Pick a model you can actually invoke

I wanted Opus 4.8, but on this account it returned AccessDenied (403). A model showing up in list-foundation-models / list-inference-profiles does not guarantee you can invoke it. I checked candidates with a one-line Converse "ping" and landed on Sonnet 4.6:

aws bedrock-runtime converse --region us-east-1 \
  --model-id us.anthropic.claude-sonnet-4-6 \
  --messages '[{"role":"user","content":[{"text":"ping"}]}]' \
  --inference-config '{"maxTokens":5}'
Enter fullscreen mode Exit fullscreen mode

The core analysis is a single Converse call with a system prompt that returns strict JSON (aiLikenessScore, label, summary, findings[], humanized). One detail that mattered for bilingual support: the commentary follows the UI language, but the rewrite must stay in the source language. The prompt says, in effect, "write the labels and reasons in English, but keep the rewrite in the same language as the input — never translate it." Without that line, English input sometimes came back rewritten into Japanese.

"Rewrite until it passes" with Strands Agents

A single rewrite might still read AI-ish. So there's a second mode: an agent that scores, rewrites, re-scores, and repeats until the AI-likeness drops below a threshold (or it hits a max number of rewrites). That is a genuine agent loop — a perfect fit for Strands.

The TypeScript SDK (@strands-agents/sdk) gives you an Agent, a tool() helper (Zod schemas), a BedrockModel, and structuredOutputSchema for typed results:

import { Agent, BedrockModel, tool } from "@strands-agents/sdk";
import { z } from "zod";

const scoreTool = tool({
  name: "score_ai_likeness",
  description: "Score how AI-like a text is (0-100). Call after every rewrite.",
  inputSchema: z.object({ text: z.string() }),
  callback: async (i) => JSON.stringify(await scoreOnce(i.text)),
});

const agent = new Agent({
  model: new BedrockModel({
    region: "us-east-1",
    modelId: "us.anthropic.claude-sonnet-4-6",
    maxTokens: 4096,
  }),
  tools: [scoreTool],
  structuredOutputSchema: z.object({
    finalScore: z.number(),
    iterations: z.number(),
    humanized: z.string(),
    notes: z.string(),
  }),
  systemPrompt: `Rewrite to read human. Score with score_ai_likeness,
    rewrite, re-score, repeat until below ${threshold} or ${maxRewrites} times.`,
});

const res = await agent.invoke(userPrompt);
return res.structuredOutput; // typed
Enter fullscreen mode Exit fullscreen mode

In practice the model often nails it in one rewrite — a 92/100 sample dropped to 12/100, and the structured output told me exactly what changed.

Gotcha — bundling. Strands pulls in many optional peer deps. In Next.js, mark it external so the bundler leaves it alone at runtime:

// next.config.ts
const nextConfig = {
  serverExternalPackages: ["@strands-agents/sdk", "pdf-parse-fork"],
};
Enter fullscreen mode Exit fullscreen mode

Gotcha — Vercel timeouts. The loop makes several Bedrock calls and can take 20–45s. Vercel Hobby caps functions at 60s, so a long auto-rewrite can time out. Fine locally; budget for it in production.

Exposing it over MCP with AgentCore Gateway

I wanted the tool callable from Claude Code, so I put it behind AgentCore Gateway as an MCP server. The shape:

MCP client (Claude Code) → AgentCore Gateway (MCP / JWT)
   → Lambda (detect_ai_style) → Bedrock (Sonnet 4.6)
Enter fullscreen mode Exit fullscreen mode

The Lambda is a single Node file (the Node 22 runtime already bundles the AWS SDK v3, so no dependencies to package). The Gateway target points at the Lambda and carries the tool schema.

The big surprise: AgentCore Gateway's authorizerType is CUSTOM_JWT — and that's the only option. There is no anonymous gateway. So an MCP client must present a bearer JWT. I set up a Cognito user pool with a client_credentials (machine-to-machine) app client, and pointed the gateway's authorizer at Cognito's discovery URL:

aws bedrock-agentcore-control create-gateway \
  --name ai-text-checker-gateway --protocol-type MCP \
  --authorizer-type CUSTOM_JWT \
  --authorizer-configuration '{"customJWTAuthorizer":{
     "discoveryUrl":"https://cognito-idp.us-east-1.amazonaws.com/<POOL_ID>/.well-known/openid-configuration",
     "allowedClients":["<CLIENT_ID>"]}}' \
  --role-arn <GATEWAY_ROLE_ARN>
Enter fullscreen mode Exit fullscreen mode

Two more sharp edges:

  • The Gateway invokes the Lambda with the tool arguments as the event, and passes the tool name via context as ${targetName}___${toolName}. With a single tool you can ignore the routing entirely.
  • The target toolSchema does not accept enum in property definitions — only type, properties, required, items, description. I moved the allowed values into the description.

Once it's up, the flow from a client is plain MCP JSON-RPC (initialize → tools/list → tools/call), with Authorization: Bearer <token>. The tool appears as detect___detect_ai_style. Tokens are short-lived, so I keep the client secret out of the repo and fetch a fresh JWT at runtime via the AWS CLI.

Takeaways

  • For "is this AI?", be honest: ship an estimate + editorial feedback, not a verdict.
  • Verify model access with a real Converse call before you build on a model id.
  • Strands makes an iterative, tool-using rewrite loop a few lines of typed code — but mark it external in Next.js.
  • AgentCore Gateway is a clean way to turn a Lambda into an MCP tool, but plan for mandatory JWT auth (Cognito), not an anonymous endpoint.

Try it: https://ai-text-checker-pi.vercel.app — and tell me where the score is wrong. It will be, sometimes. That's the point.

Top comments (0)