DEV Community

Cover image for 🚀 MCP App Testing Framework!
Abe Wheeler
Abe Wheeler

Posted on

🚀 MCP App Testing Framework!

We just shipped sunpeak.ai as a standalone testing framework for MCP Apps!

If you're building MCP Apps for ChatGPT or Claude, you know the pain: deploy, open the host, start a conversation, trigger the tool, check the result. Repeat for both hosts, both themes, three display modes. That's 24 combinations per code change.

sunpeak replicates the ChatGPT and Claude runtimes locally. You write Playwright tests that call tools, render resources, and assert against the output. One test file runs against both hosts automatically.

What's included:

  • Unit tests (Vitest + happy-dom)
  • E2E tests against replicated ChatGPT and Claude runtimes
  • Visual regression testing with screenshot baselines
  • Live tests against real ChatGPT
  • Works with any MCP server in any language (Python, Go, TypeScript)

Add it to an existing project with one command:

pnpm add -g sunpeak && sunpeak test init

No paid host accounts. No AI credits. Runs in CI/CD.

MIT licensed and open source! https://sunpeak.ai/testing-framework/

Top comments (3)

Collapse
 
barissozen profile image
Baris Sozen

actually pulled sunpeak down today against our MCP (@hashlock-tech/mcp, atomic settlement layer for crypto trading agents, 6 tools,data-only). Boot was clean, Found 6 tool(s), 0 resource(s) came up in under 15s, browser auto-opened nicely.

Hit the "No tools with UI resources found on this server" state immediately, which is technically correct (our tools return JSON: tradeId, txHash,
settlement status) — but it made me realize sunpeak's true value prop deserves louder marketing. A few things that would have helped me reach for it sooner:

  1. A "Is sunpeak for me?" panel on the landing page — "you're building an MCP that returns rich UI " vs "you're building a data-only MCP ". Right now the positioning vs @modelcontextprotocol/inspector isn't loud, and the two tools serve genuinely different audiences.
  2. A "convert your data-only MCP to a UI-resource MCP" tutorial. Most published MCPs today are data-only — that's the upstream funnel for everything sunpeak wants to ship. Document the migration path and you onboard the entire existing ecosystem.
  3. A sample-apps gallery with 3–4 reference MCPs that do return UI resources, ideally open-source. Right now I had to imagine what "right fit" looks like.

The Playwright sim hooks + the inspector's chrome are all great — felt like a real product
the moment I clicked around. Bookmarking, will be back the day I add UI resources to our tools.

Nice work — looking forward to where this goes.

Collapse
 
abewheeler profile image
Abe Wheeler

Super grateful for the thoughtful feedback, Baris ❤️ Good point on differentiating against UI-less MCP, the "Apps" term doesn't do enough work (and it isn't widely known enough) for people to instantly recognize the difference. We might even fix by making the vanilla MCP use case first-class!

Collapse
 
hai_huang_f196ed9669351e0 profile image
Info Comment hidden by post author - thread only accessible via permalink
Shiplight

checkout shiplight.ai

Some comments have been hidden by the post's author - find out more