DEV Community: Michael F

ClaudeArena, one week after my first post: what the data showed, and what I changed

Michael F — Wed, 04 Nov 2026 17:58:38 +0000

On 28 October I posted about ClaudeArena, the free practice engine I built for Anthropic's Claude Certified Architect exams (CCAR-F and CCAR-P). Then people started using it, and real usage turned out to be a much better reviewer than I am.

In the first week: 67 sessions from visitors without an account, 14 new callnames, and a handful of people who used up all 120 free questions in a day. Small numbers, but enough to learn from.

The data pointed at the hardest topics

The site counts right and wrong answers per exam objective. These are anonymous daily totals with nothing about who answered. The lowest-scoring Foundations objectives right now:

Fixing truncated structured output: 62% right
Tool use and tool_choice for structured output: 65%
Parallel tool calls in one turn: 66%
Session resumption and forking: 66%
Agentic loop control and safety bounds: 68%
Iterative refinement workflows: 68%

It's a few hundred answers, so I wouldn't read too much into any single line. What stands out is the pattern: most of these are about constraining and sequencing an agent, not remembering a fact. That matches what people say about the real exam. The facts are the easy part, and the judgment calls are not.

At the other end, a few objectives score over 90%. That probably means their wrong options are too easy to spot, so those are next in line for rework.

"You chose B"

When you got a question wrong, the site showed the right answer and the explanation, but never your answer. And the trap you fell for is the thing you most need to understand.

Now the correction says "You chose B" above the right answer, and the results review shows both. It's a small change, and one I should have made on day one.

The site also counts which wrong option people pick, anonymously. Once there's a month of data, I want to publish a "where people struggle" page built from real answers.

Phone first, because that's where people study

A review at phone width was humbling. The header and menu took about 40% of the screen, and the start button sat at the bottom of a 3,000-pixel setup page.

What changed:

Focus mode during a session: the header folds into one slim bar with progress, timer and exit
Quick start: a 10-question practice button right under the exam choice
Your last settings are remembered on your device
A one-row menu on phones, and shorter intros

The leaderboard bug that wasn't a bug

The rule was: finish a full exam while signed in and you're on the board. On a quiet board, that put someone who answered 5 of 60 questions in first place, with a 0% run in second. Technically correct, and terrible.

Now only full exams that reach the 72% practice target go on the leaderboard. A run below that still counts for your own progress, XP and objective scores. It just doesn't take a place. Medals already worked this way, so the whole site now follows one rule: the leaderboard shows what you cleared, not that you showed up.

Exam guide vs today, rebuilt

This is the change I care most about.

The exam is built from its exam guide, and the CCAR-F guide is version 1.0 from July 2026. Claude Code, the Agent SDK and the API kept shipping after that. If you study from today's documentation, some of it won't match the answer the exam marks right. If you study only the guide, a few things you learn are already out of date at work.

So the site has two jobs: teach the answer the exam expects, and show where the real tools have moved on. That's what the Exam guide vs today page is for, and this week I rebuilt it:

Three columns: Topic, Exam guide (July 2026) and Today, so you see how the guide describes it next to how it works now
Every change is dated from Anthropic's own release notes: the Claude Code changelog, the Agent SDK changelog and the Claude Platform release notes
Changes that came before the guide are marked as such. The guide still teaches its own version, so that's still the exam answer
Only topics the guide actually teaches. Anything the guide never covers isn't an exam gap, so it moved to "Beyond the exam guide" notes on the study pages. Foundations is down to 16 rows, and every one matters on exam day

A few examples:

Forcing a tool. Guide: tool_choice "any" or a named tool forces a tool call. Today: Opus 5.5 (since 22 Sept) and Sonnet 5.5 (since 28 Sept) reject it with a 400 error, so you use auto with strict tool use or structured outputs.
The subagent tool. Guide: Task. Today: Agent, since Claude Code 2.1.69 in March 2026, which is before the guide.
Subagents spawning subagents. Guide: the coordinator spawns every subagent (hub-and-spoke). Today: up to 3 layers below the main agent by default, since Claude Code 2.1.219 (24 July 2026).
When subagents run. Guide: several Task calls in one response run in parallel. Today: subagents run in the background by default, since Claude Code 2.1.198 (1 July 2026).

The rule of thumb: answer the exam the guide's way, and know today's way for real work. The practice questions always follow the guide. Where today differs, the explanation adds one sentence starting with "Today, ..." so you learn both, and you're never marked wrong for answering the way the exam will mark you.

Studying, not just testing

The usage guide from my second post now lives on the site: read a domain, practise it straight away, drill your weakest objectives, then sit timed mocks until you're steady
2-week and 4-week study plans for Foundations, day by day. They're a suggested order, not a gate
Signed in, your account page lists your last sessions, so you can see whether you're actually improving
The site now says difficulty for Normal / Hard / Expert, instead of "mode"

If you're booking an exam

The cancel/reschedule window is 48 hours. The exam guides say 24, but Pearson VUE's page for Anthropic exams says 48, and late changes forfeit the fee
CCAR-P is changing. Anthropic publishes updated exam guides on 17 November, and the Professional exam changes on 7 December. I'll read the new guide the day it's out and update the Professional questions and study pages before 7 December. Exam guide vs today will get a Professional table built the same way

A few technical details

Page errors are now counted, not described. When something fails in a visitor's browser, only a fixed error code (like SESSION_CHECK_FAILED) is sent and counted per day. There's no message, no user and no IP, and the counts expire after 90 days. A console nobody reads isn't a log, but I didn't want a log that could hold personal data either.

The Cloudflare workers.dev address kept switching itself back on after every deploy, because Wrangler re-enables it unless wrangler.toml says workers_dev = false. Now it does, and a build check fails if anyone removes that line.

That check joins the others that run before every deploy, guarding question fairness, contrast, privacy and page titles. There are 70 of them now.

Try it

Still free and unofficial: 950+ questions, 120 free practice questions a day without an account, and an 8-question demo if you just want a look.

→ https://claudearena.dev

If you've sat the real exam, I'd love to know two things: how it compared to Hard and Expert here, and which topic surprised you most.

ClaudeArena Usage Guide

Michael F — Fri, 30 Oct 2026 16:40:37 +0000

(Optimized for CCAR-F and CCAR-P)

ClaudeArena is a free, focused practice platform for the Claude Certified Architect exams (Foundations and Professional). This guide shows the most effective way to use it.

1. Create a free callname (recommended)

Creating a callname is optional but strongly recommended.

It unlocks progress tracking, badges, medals, leaderboard ranking, and the ability to save results across devices — all of which help you train more efficiently and clearly see your improvement over time.

2. Work through the Study Guide domain by domain

Go to Study Guide and study one domain at a time.

For each domain:

Read the full domain page carefully (including any amber “Exam guide vs today” notes inside it).
Immediately click “Practise Domain X” at the bottom.
Do the practice in Normal mode.
After the session, review the Score by objective breakdown and click “Practice my weakest objectives”.
Drill the weak objectives until they improve.

When to move to the next domain:

Aim for roughly 75%+ on the domain and no major weak objectives left before continuing. Once you have completed all domains this way, proceed to step 3.

3. Review the full “Exam guide vs today” list

(Only after finishing all domains)

Use this as a final clarification reference — do not start here.

4. Move to mixed practice and higher difficulty

Do mixed-domain sessions.
Start in Normal mode.
Move to Hard mode once you can consistently score above 75% in Normal.
Use Expert mode only if you want extra challenge.

5. Full timed mock exams

Switch to Full exam mode (always timed, 120 minutes).
After every full mock, review domain + objective scores and immediately drill the weakest objectives again.
Repeat until you are consistently solid in Hard mode.

6. Final readiness check

When you are comfortably scoring well in Hard mode on full mocks and your weakest objectives are under control, you should be ready to book the real exam.

(No practice score can guarantee a pass on the actual scaled exam.)

Simple daily loop (after initial study):

Weak objectives → short drill → longer mixed/full session → review mistakes.

This approach keeps the focus on understanding first, then systematically closing gaps at both domain and objective level - the most efficient path to exam readiness on ClaudeArena - https://claudearena.dev

I built a free practice engine for Anthropic’s Claude Certified Architect exams

Michael F — Wed, 28 Oct 2026 20:21:57 +0000

I sat the Claude Certified Architect - Foundations exam (CCAR-F) in September and passed.

Dedicated prep time was only a few hours. Not because the exam is easy - there just wasn’t much realistic material to practice on. Anthropic provides a handful of sample questions and no real practice exam.

The exam is mostly judgment calls: when an agent should stop, how to isolate subagent context, which tool restrictions actually prevent a bad outcome, that kind of thing. Reading the docs helps, but it doesn’t give you the reps.

So I built ClaudeArena.

What it includes

950+ original scenario questions for both Foundations (CCAR-F) and Professional (CCAR-P)
Questions tagged to individual exam objectives (not just domains)
A “drill my weak spots” mode that builds short sessions from the objectives you’re weakest on
Full timed mock exams in the real format (60 or 63 questions, 120 minutes)
Three difficulty levels (Normal, Hard, Expert)

A few technical details

Early on I noticed the longest answer option was correct far too often. I added a build check that tries to beat chance using only the shape of the options (length, punctuation, the one that stands out). Getting that check to pass forced a lot of rewrites.

Grading happens on the server, so the browser never sees an answer key. No email is required, no analytics, and nothing is loaded from third parties.

The questions are written with Claude and I built the site with Claude Code. Each question records what its answer rests on (Anthropic’s docs, the exam guide, or established practice). They were re-checked against the current documentation on 14 October and also reviewed independently by a second model.

There’s also a short page that lists the places where the frozen exam guide and today’s documentation diverge.

Try it

You can start with an 8-question demo or practice without creating an account (120 questions a day). Everything is free and unofficial.

→ https://claudearena.dev

If you’ve taken the real exam, I’d be interested to hear how the difficulty compares.