ClaudeArena, one week after my first post: what the data showed, and what I changed

On 28 October I posted about ClaudeArena, the free practice engine I built for Anthropic's Claude Certified Architect exams (CCAR-F and CCAR-P). Then people started using it, and real usage turned out to be a much better reviewer than I am.

In the first week: 67 sessions from visitors without an account, 14 new callnames, and a handful of people who used up all 120 free questions in a day. Small numbers, but enough to learn from.

The data pointed at the hardest topics

The site counts right and wrong answers per exam objective. These are anonymous daily totals with nothing about who answered. The lowest-scoring Foundations objectives right now:

• Fixing truncated structured output: 62% right
• Tool use and tool_choice for structured output: 65%
• Parallel tool calls in one turn: 66%
• Session resumption and forking: 66%
• Agentic loop control and safety bounds: 68%
• Iterative refinement workflows: 68%

It's a few hundred answers, so I wouldn't read too much into any single line. What stands out is the pattern: most of these are about constraining and sequencing an agent, not remembering a fact. That matches what people say about the real exam. The facts are the easy part, and the judgment calls are not.

At the other end, a few objectives score over 90%. That probably means their wrong options are too easy to spot, so those are next in line for rework.

"You chose B"

When you got a question wrong, the site showed the right answer and the explanation, but never your answer. And the trap you fell for is the thing you most need to understand.

Now the correction says "You chose B" above the right answer, and the results review shows both. It's a small change, and one I should have made on day one.

The site also counts which wrong option people pick, anonymously. Once there's a month of data, I want to publish a "where people struggle" page built from real answers.

Phone first, because that's where people study

A review at phone width was humbling. The header and menu took about 40% of the screen, and the start button sat at the bottom of a 3,000-pixel setup page.

What changed:

• Focus mode during a session: the header folds into one slim bar with progress, timer and exit
• Quick start: a 10-question practice button right under the exam choice
• Your last settings are remembered on your device
• A one-row menu on phones, and shorter intros

The leaderboard bug that wasn't a bug

The rule was: finish a full exam while signed in and you're on the board. On a quiet board, that put someone who answered 5 of 60 questions in first place, with a 0% run in second. Technically correct, and terrible.

Now only full exams that reach the 72% practice target go on the leaderboard. A run below that still counts for your own progress, XP and objective scores. It just doesn't take a place. Medals already worked this way, so the whole site now follows one rule: the leaderboard shows what you cleared, not that you showed up.

Exam guide vs today, rebuilt

This is the change I care most about.

The exam is built from its exam guide, and the CCAR-F guide is version 1.0 from July 2026. Claude Code, the Agent SDK and the API kept shipping after that. If you study from today's documentation, some of it won't match the answer the exam marks right. If you study only the guide, a few things you learn are already out of date at work.

So the site has two jobs: teach the answer the exam expects, and show where the real tools have moved on. That's what the Exam guide vs today page is for, and this week I rebuilt it:

• Three columns: Topic, Exam guide (July 2026) and Today, so you see how the guide describes it next to how it works now
• Every change is dated from Anthropic's own release notes: the Claude Code changelog, the Agent SDK changelog and the Claude Platform release notes
• Changes that came before the guide are marked as such. The guide still teaches its own version, so that's still the exam answer
• Only topics the guide actually teaches. Anything the guide never covers isn't an exam gap, so it moved to "Beyond the exam guide" notes on the study pages. Foundations is down to 16 rows, and every one matters on exam day

A few examples:

• Forcing a tool. Guide: tool_choice "any" or a named tool forces a tool call. Today: Opus 5.5 (since 22 Sept) and Sonnet 5.5 (since 28 Sept) reject it with a 400 error, so you use auto with strict tool use or structured outputs.
• The subagent tool. Guide: Task. Today: Agent, since Claude Code 2.1.69 in March 2026, which is before the guide.
• Subagents spawning subagents. Guide: the coordinator spawns every subagent (hub-and-spoke). Today: up to 3 layers below the main agent by default, since Claude Code 2.1.219 (24 July 2026).
• When subagents run. Guide: several Task calls in one response run in parallel. Today: subagents run in the background by default, since Claude Code 2.1.198 (1 July 2026).

The rule of thumb: answer the exam the guide's way, and know today's way for real work. The practice questions always follow the guide. Where today differs, the explanation adds one sentence starting with "Today, ..." so you learn both, and you're never marked wrong for answering the way the exam will mark you.

Studying, not just testing

• The usage guide from my second post now lives on the site: read a domain, practise it straight away, drill your weakest objectives, then sit timed mocks until you're steady
• 2-week and 4-week study plans for Foundations, day by day. They're a suggested order, not a gate
• Signed in, your account page lists your last sessions, so you can see whether you're actually improving
• The site now says difficulty for Normal / Hard / Expert, instead of "mode"

If you're booking an exam

• The cancel/reschedule window is 48 hours. The exam guides say 24, but Pearson VUE's page for Anthropic exams says 48, and late changes forfeit the fee
• CCAR-P is changing. Anthropic publishes updated exam guides on 17 November, and the Professional exam changes on 7 December. I'll read the new guide the day it's out and update the Professional questions and study pages before 7 December. Exam guide vs today will get a Professional table built the same way

A few technical details

Page errors are now counted, not described. When something fails in a visitor's browser, only a fixed error code (like SESSION_CHECK_FAILED) is sent and counted per day. There's no message, no user and no IP, and the counts expire after 90 days. A console nobody reads isn't a log, but I didn't want a log that could hold personal data either.

The Cloudflare workers.dev address kept switching itself back on after every deploy, because Wrangler re-enables it unless wrangler.toml says workers_dev = false. Now it does, and a build check fails if anyone removes that line.

That check joins the others that run before every deploy, guarding question fairness, contrast, privacy and page titles. There are 70 of them now.

Try it

Still free and unofficial: 950+ questions, 120 free practice questions a day without an account, and an 8-question demo if you just want a look.

→ https://claudearena.dev

If you've sat the real exam, I'd love to know two things: how it compared to Hard and Expert here, and which topic surprised you most.