Give your AI agent three web research tools with one MCP server

#ai #mcp #javascript #webdev

Agents are good at reasoning about web pages and bad at fetching them reliably. Raw HTML is noisy, JS-heavy pages time out, and every agent project ends up rewriting the same "get title, meta, links, JSON-LD" helper.

I built a small MCP server that exposes three tools to Claude Desktop, Cursor or any MCP client. Disclosure: I'm the author and it's backed by my paid API on RapidAPI, which has a free plan (50 calls/month). The MCP wrapper itself is MIT.

The tools

Tool	Returns
`company_profile(url)`	name, description, logo, socials, emails, key pages, price mentions, tech stack
`site_audit(url, max_links?)`	0–100 score + SEO, performance, security-header and broken-link issues
`page_to_json(url, max_text?)`	meta, Open Graph, JSON-LD, headings, links, images, clean text

Setup

Subscribe to the Free plan of Web Research Agent API and copy your X-RapidAPI-Key.
Add to your MCP config (Claude Desktop claude_desktop_config.json, or Cursor .cursor/mcp.json):

{
  "mcpServers": {
    "web-research-agent": {
      "command": "npx",
      "args": ["-y", "github:elaysuu/web-research-agent-mcp"],
      "env": { "RAPIDAPI_KEY": "YOUR_KEY" }
    }
  }
}

Restart the client and ask something like: "Audit example.com and list the three most important fixes."

How an MCP tool wrapper works

The whole server is ~25 lines. Each tool is a zod schema plus a function that calls the API:

import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js';
import { z } from 'zod';

const HOST = 'web-research-agent-api.p.rapidapi.com';
async function call(path, params) {
  const qs = new URLSearchParams(params);
  const r = await fetch(`https://${HOST}${path}?${qs}`, {
    headers: { 'x-rapidapi-key': process.env.RAPIDAPI_KEY, 'x-rapidapi-host': HOST }
  });
  return { content: [{ type: 'text', text: await r.text() }], isError: !r.ok };
}

const s = new McpServer({ name: 'web-research-agent', version: '0.1.0' });
s.tool('site_audit', 'Audit a web page: SEO, performance, security headers, broken links.',
  { url: z.string(), max_links: z.number().int().min(0).max(25).optional() },
  a => call('/site-audit', a));
await s.connect(new StdioServerTransport());

Two lessons that apply to any MCP server you write:

Tool descriptions are the prompt. The model chooses tools from the description text, so say what comes back, not just what it does.
Return errors as isError: true with readable text (e.g. "RAPIDAPI_KEY is not set") so the agent can tell the user instead of looping.

Calling the API without MCP

curl -G "https://web-research-agent-api.p.rapidapi.com/page-to-json" \
  --data-urlencode "url=https://example.com" \
  -H "x-rapidapi-key: $RAPIDAPI_KEY" \
  -H "x-rapidapi-host: web-research-agent-api.p.rapidapi.com"

import os, requests
r = requests.get("https://web-research-agent-api.p.rapidapi.com/company-profile",
    params={"url": "stripe.com"},
    headers={"x-rapidapi-key": os.environ["RAPIDAPI_KEY"],
             "x-rapidapi-host": "web-research-agent-api.p.rapidapi.com"})
print(r.json())

Limits, honestly

It fetches server-rendered HTML; content that only appears after client-side JS may be missing.
It respects robots.txt and refuses private/internal addresses, so some URLs will be declined.
The audit is heuristic — a quick checklist, not a replacement for Lighthouse.

Repo: https://github.com/elaysuu/web-research-agent-mcp — issues and PRs welcome.

Update: the same server is also available hosted with pay-per-call pricing (no RapidAPI key, $0.01-$0.02 per tool call in USDC) on MCPize: https://mcpize.com/mcp/web-research-agent-mcp

DEV Community