On 19 November 2026 two changes went live together on the certifications page. Four free Claude Certified practice exams went in, 1,165 questions in all, and 2,995 of the 3,101 published Atlassian questions got new wording.
I'll give you my take before the changelog. I think the stronger model we brought in for the wording writes better sentences than our own pipeline ever did. It also changed what some questions claimed, in sentences that read perfectly well. Scripts check every rewrite. So does a second reader, a separate AI agent whose only job is to prove a rewrite wrong. Those checks are why I'm comfortable putting the result in front of you.
Why would you care how a practice question is worded? You read hundreds of them in a sitting, often in a second language, and you're tired. A clipped sentence costs attention you need for the question itself.
Claude Certified Architect, Developer and Associate practice exams, free on the same access key
There's now a practice exam here for four of Anthropic's proctored Claude certifications.
| Exam | Code | Questions in our pool | Anthropic's exam through 6 December 2026 | Anthropic's exam from 7 December 2026 |
|---|---|---|---|---|
| Claude Certified Associate – Foundations | CCAO-F | 293 | 60 questions, 120 minutes | 70 questions, 120 minutes |
| Claude Certified Developer – Foundations | CCDV-F | 319 | 53 questions, 120 minutes | 70 questions, 120 minutes |
| Claude Certified Architect – Foundations | CCAR-F | 287 | 60 questions, 120 minutes | no December edition listed |
| Claude Certified Architect – Professional | CCAR-P | 266 | 63 questions, 120 minutes | 70 questions, 120 minutes |
The exam lengths are from Anthropic's certification page as it read on 18 November 2026. Three of the four exams get an updated edition on 7 December 2026, and the page says the date you test decides which version you take.
A quick mock here draws 20 questions. For the three exams that change, a full paper follows the December edition in its length and its domain weights, so it has 70 questions. For Architect – Foundations it has 60.
Each question shows the sentence from the Claude documentation that decides the answer, with a link to the page. Each of the four options gets its own explanation. If you think we're wrong, you can check the page. The access key we emailed you for the Atlassian exams opens these too, because an access key belongs to your email address and to no single exam.
These are our own questions. Anthropic has no part in them and doesn't endorse them. None of them is copied from an exam guide or from another site.
Before you trust a score, you should know how we judge it. Anthropic's exams pass on a scaled score of 720 on a scale of 100 to 1,000, and a practice percentage can't be converted to that. So the result screen measures you against a 75% practice target and says in plain words that the 75% is LeanZero's line.
You should also know who can sit the exam itself. Anthropic's page says the exams are open to people at Claude Partner Network organizations. Our practice exams are free for anybody.
I should name one limit. Our Claude questions all have one correct answer out of four, while the Architect – Foundations guide says its paper also has questions where you select more than one. We don't have those yet.
In September I wrote that the Claude Architect syllabus exposed the gaps of someone self-taught. The four exams above cover that syllabus and three more.
Our own pipeline made the Atlassian questions choppy
We had checked the Atlassian questions several times. They were still tiring to read. We did that to them ourselves.
In September we measured the options in my screenshots of mock exam questions for three of the five exams. The receipt, dated 2026-09-07 in tools/option_shape.py, gives a median of 4, 5 and 6 words an option there. The options we had written had a median of 14 to 16 words. So our pipeline squeezed them down to between 5 and 8. Our written rules and automatic checks all pushed toward fewer words, and not one rule asked for a readable sentence.
I think the explanations under each answer took the worst of it. In the 2,995 reworded questions, 10,436 of 12,438 option explanations opened with an internal label or a verdict word such as "Wrong artifact" or "Correct --". A double hyphen stood in for punctuation in 10,084 of them. Those labels meant something to the pipeline and nothing to a candidate.
Before and after: six ACP practice exam questions, from ACP-120 to ACP-620
A script copied these out of the bank, the published text first and the reworded text second. The stem is the question text above the options. All six rewordings were accepted by the second reader I describe further down.
ACP-120, Jira administration, the pool behind our ACP-120 practice exam. A stem written as four clipped sentences:
Before: A project key changed from OPS to RUN. Old OPS links still resolve. Another team wants OPS for a new project. What should you do before deleting the retained previous key?
After: A project key was changed from OPS to RUN. The old OPS links still resolve. Now another team wants to use OPS for a new project. What should you do before deleting the previous key that was kept?
The explanation for one wrong option, "Rebuild its notification scheme":
Before: Wrong object -- notification schemes do not replace old project-key references.
After: Deleting a previous project key affects links, filters, gadgets, and queries that still use OPS. A notification scheme has nothing to do with references to the old project key.
ACP-120 again. Double hyphens doing the work of commas:
Before: Work item CNF-88 -- carrying security level 'Executives Only' -- is moved from project 'Confidential' into 'General', which has no security scheme at all. Which statement about CNF-88 after the move is correct?
After: Work item CNF-88, which has the security level 'Executives Only', is moved from the project 'Confidential' into 'General', which has no security scheme at all. Which statement about CNF-88 after the move is correct?
The explanation for the wrong option "Reverts within 30 days". Keep an eye on the word "documented", because it comes back later:
Before: Does not exist -- no delayed reversion of a moved work item's security level is documented anywhere on the page.
After: A moved work item takes the destination project's security settings when it moves, and nothing describes a moved work item's security level reverting after a 30-day delay.
ACP-220, Confluence administration.
Before: Your flow fans into three branches, each taking 1 minute, and finishes after 1 minute. How much processing time do the branches contribute?
After: Your flow has three branches that each take 1 minute to run, and it finishes after 1 minute. How much processing time do the branches contribute?
The explanation for the wrong option "0 minutes.":
Before: Wrong -- partial/insufficient: Delay and Send web request are named as processing-time contributors, but branches are named as a further contributor on the same page, so excluding them entirely misstates the rule.
After: Branches count toward processing time, with each branch counted separately, so their contribution is not 0 minutes.
ACP-420, Jira Service Management. Capitals for emphasis, and a double hyphen:
Before: You need a dashboard filter that returns only your own PENDING approval steps -- including ones you already voted on while other approvers decide, but no completed steps. Which function should it use?
After: You need a dashboard filter that returns only your own pending approval steps, including ones you already voted on while other approvers decide, but leaves out completed steps. Which function should it use?
The explanation for the wrong option "myPendingApproval()":
Before: Wrong -- myPendingApproval() excludes steps the user already voted on, the opposite of what is needed here.
After: The filter must include pending approval steps the current user has already voted on, but myPendingApproval() finds only pending steps the current user is yet to approve or decline and leaves out the ones already voted on.
ACP-520, Atlassian Cloud organization administration.
Before: On a Standard Cloud plan, a site admin -- not an org admin -- opens Connected Apps. Which statement about installing and managing apps there is correct?
After: On a Standard Cloud plan, a site admin who is not an org admin opens the Connected Apps page. Which statement about installing and managing apps there is correct?
The old explanation for the wrong option "Site admins can't install public Marketplace apps" quoted the page at you. The page names both roles for both kinds of app, and the new explanation says that in plain words:
Before: Wrong -- the page opens 'As an administrator, you can install and manage both Marketplace and private apps through the Connected Apps page' and its panel names 'Role: Organization admin, Site admin', with no public/private split by role.
After: Both organization admins and site admins can install Marketplace apps and private apps through the Connected Apps page. There is no split by role between public Marketplace apps and private apps.
ACP-620, Jira project administration. A passive stem with nobody in it:
Before: The Sprint Report burndown for MERIDIAN, a company-managed Scrum project, is opened on a board that has never been configured to track remaining estimates and time spent. Three lines -- grey, red, and green -- are expected. What actually shows?
After: An admin opens the Sprint Report burndown for Meridian, a company-managed Scrum project, on a board that has never been configured to track remaining estimates and time spent. The admin expects to see the grey, red, and green lines. What actually shows?
This one comes from the first passes on 18 November 2026, the day the rewording ran, when the model was still allowed to reword the options. Its correct option changed like this:
Before: Only grey and red; green needs tracking on.
After: Only the grey and red lines show because the green line requires tracking remaining estimates and time spent.
And the explanation for a wrong option:
Before: Partial/insufficient -- wrongly extends the time-tracking dependency to the red line, which is documented as always present alongside the grey guideline.
After: The red line represents the actual work done during the sprint and is shown together with the grey line. It does not depend on configuring the board to track remaining estimates and time spent.
What a stronger model did well, and where it moved facts
We run bulk work on cheap models. For this pass, on 18 November 2026, we paid for a stronger one and used it only for the prose. Each question was its own job in Elzi, our agent app.
That model knows nothing about Atlassian and sees one short briefing. In it are the question as published, the fact behind each option, the words that must survive character for character, and what reads badly in this particular question.
You can see what it did well in the six pairs. It says who does what, in the order it happens, and it puts the articles and the verbs back. Where an explanation opened with "Wrong -- partial/insufficient:", it now starts by saying what's true.
It went wrong mostly in one place. A published explanation often rests on what the documentation doesn't say, for example that no such limit is documented. The model liked to turn that into a flat product fact such as "there is no limit".
More than half of the second reader's refusals that day were of this kind. "Not documented" and "cannot" are different claims, and only the first one is ours to make.
It moved other things too. Options gained a clause that stated the very rule the candidate is meant to know. A Confluence question that says "flow", the current name, came back saying "automation rule". "PDF export" came back as "pdf export".
Our second reader refused about a third of the early rewrites, most often for an option that had gained, lost or changed a word. So that evening we locked the options.
Since then the runner puts the published options back character for character, so in a new rewrite only the stem and the explanations can differ. 1,292 accepted rewordings still carry options reworded before the lock, each compared claim by claim with the published option. The sixth pair above is one of them.
How the rewording runs, script by script
The repository is private. So I'll give you the file names as they are in it, and what each script refuses.
- Notes. An agent reads each published question whole and writes a note with what the question tests, the fact that makes each option right or wrong, and the terms to keep. The note is a fact check as well. If the quote doesn't carry the correct option, or a second option can be defended, the question gets no note and is parked as a doubt.
-
rewrite_brief.py buildturns a note into the briefing. It refuses to build when a term to keep isn't an exact piece of the published stem or options. -
rewrite_run.py runsends one briefing per job and collects one answer. When the automatic check refuses the answer, the model gets one retry with the check's own words. -
The automatic check (
check_oneinrewrite_brief.py, whichrewrite_run.pycalls) compares the answer with the published question. Option letters and order must match. Digits in the stem and in each option must be unchanged. Terms to keep must be there character for character. An acronym in the published stem, such as PDF, API or JQL, must stay in the stem in capitals. The stem keeps its number of sentences, and a direct question keeps its opening word. No explanation may open with a label. The correct option may not start to stand out by length. -
rewrite_silence.pylooks for the failure described above. When a published line rests on what the documentation doesn't say, the new line must still point at the documentation. - The second reader is another AI agent in a separate session, and its one job is to prove the rewrite wrong. It never sees the notes, because the notes could carry the same mistake as the rewrite. Its ground truth is the published question. It compares old and new option by option, and writes down the exact old words, the exact new words and why the change matters.
- A refusal goes back to the model with the reader's words. A question refused twice keeps its published wording, and so does one that fails the automatic check in two whole runs.
-
rewrite_run.py applywrites the stem, the options, the one-line rule and the explanations of accepted questions into the bank. The answer key, the documentation quote and the link stay as they were.
python3 tools/rewrite_brief.py build --notes rewrite/notes/acp-520/<batch>.json --out rewrite/briefs/acp-520
python3 tools/rewrite_run.py run --notes rewrite/notes/acp-520/<batch>.json --jobs 6
python3 tools/rewrite_silence.py --apply
python3 tools/rewrite_run.py apply --notes rewrite/notes/acp-520/<batch>.json
Of the 3,101 questions that went in, 1,159 were sent back at least once after an objection, from the reader or from the stricter checks we added during the day.
One mistake in this was ours and had nothing to do with the model. Two of our own checks contradicted each other: one refused any explanation that mentioned the documentation, and the other demanded that mention whenever the published line had it. Questions caught between them could never pass. Once we fixed the check, 73 failed rewrites passed without a new model call.
236 doubts about questions that were already published
236 published questions were doubted along the way. The notes step parked 166 of them before any rewording, and the second reader raised the other 70 while it compared a rewrite with the published question. We settled them against the live Atlassian pages, one by one.
165 were repaired from the documentation. 66 were sound as published. That left 5. For 4 of them the documentation couldn't settle the doubt, and for 1 the page contradicted our answer key. We checked those 5 in the live product on a test site of our own. There, 2 were repaired, 1 was sound, and 2 were withdrawn.
That makes 167 repairs. Every repaired question kept its answer key. 67 of the repairs changed the documentation quote under the question. In 46 the old quote was kept and extended, and in 21 it was replaced.
Both withdrawn questions are from ACP-120. In the first, Atlassian's own recipient table backs one of our wrong options almost word for word, so no answer key could stand. The second describes a custom field whose context is limited to two work types. Jira now refuses to build that, and the API answered "The global context cannot be made non-global".
How the Claude certification practice questions were made and checked
We built the Claude exams the other way round, facts first.
- Facts. Before any question existed we collected fact rows from the Claude documentation. Each quote is one to three sentences copied by script from a saved copy of the page. The four exams rest on 3,614 fact rows from 678 saved documentation pages.
- Questions only from those rows. A question may use nothing that a fact row doesn't state. Facts marked volatile (model names, prices, token limits) are never used, so no answer goes stale with the next release. Where an exam guide and the documentation disagree, we publish nothing on that point.
-
claude_check.pyis the automatic gate. The quote must be found, word for word, in the saved page. The correct option may not be more than 10% longer than the next one or more than 15% shorter. No letter may be the answer in more than 40% of a batch. A stem may not share more than 25% of its six-word runs with the exam guide or with another of our stems. -
claude_blind.py pageprints each stem and its options with no answer. A separate agent answers and commits its answers before it may open anything else. Only then does it read the saved page around the quote and try to prove the question wrong. Its verdict is release, fix or withdraw. -
claude_export.py --liveexports released questions only. It fetches each cited page as served that day and searches for each quote word for word. On the evening of 18 November 2026 all 1,165 exported questions were confirmed on the live pages.
Was the blind reader any good? The same kind of reader answered an exam guide's twelve official sample questions blind and got 12 of 12, marking 2 of them hard. On its first blind check of our own drafts it answered 30 of 30 as well, and named the wording that gave each answer away. So a wrong answer was never the main worry.
About half of our first drafts went back for the same reason. The answer was right, but somebody who had never used the product could reach it by matching words.
We wrote 1,380 Claude questions and published 1,165. Blind readers withdrew 201, and 14 more are held back. Our hardest exam cost the most. For Architect – Professional we wrote 388 and published 266, with 122 withdrawn.
What is not done
106 Atlassian questions keep their published wording. 16 of them ended the run with an open objection from the second reader, and 8 failed the automatic check. The other 82 are settled as "the published wording stays". Among those 82 are 18 whose accepted rewording we put back, because it failed one of the last checks on the finished bank. No further run is scheduled, and I'm fine with that. A question that can't be reworded without moving a fact stays the way it was.
I also want to be plain about who the readers are. In both pipelines they're AI agents with a strict brief, and none of them has sat your exam form. You have, or you will, so I'd like to hear from you.
Changelog, 19 November 2026
Certifications page
- Vendor switch between Atlassian and Claude, with Atlassian as the default, and four Claude practice exams with 293, 319, 287 and 266 questions.
- One access key opens both vendors. Existing access keys open the Claude exams.
- Claude results are shown against LeanZero's 75% practice target, with Anthropic's scaled pass score of 720 stated next to it.
Atlassian question pools
- Explanations reworded in 2,995 questions, the stem in 2,332 of them and the options in 1,292.
- The rewording changed no answer key, documentation quote or link.
- 167 published questions repaired after a doubt, 165 from the documentation and 2 in the live product, all with the same answer key. 67 of the repairs changed the documentation quote.
- 2 ACP-120 questions withdrawn.
- Pool sizes are 890, 784, 509, 488 and 430 questions for ACP-120, 220, 420, 520 and 620.
We also fixed two small things on the way. A quote taken from documentation published as Markdown no longer shows its formatting marks in the review screen, and a hover border on cards in dark mode that never appeared now does.
Over to you
I'm left with something I can't answer from our own numbers. A stronger model made thousands of explanations easier to read in one day. It also changed "not documented" into "cannot" often enough that we had to write a script against it. We found those because each question had a published version to compare with.
Suppose you let a model reword your runbooks or your admin documentation tonight and they read better in the morning. How would you find the sentences where the meaning moved?
If you've done a rewrite like that at work, tell me what you checked, in the comments under this article.
And if you sit one of the mocks and a question reads wrong, the quote and the link are right under it. Send the question and the sentence that proves it to office@leanzero.net. That is how four bad Atlassian questions got fixed in September.
Originally published on leanzero.net. More Atlassian, Forge and local-AI write-ups at leanzero.net/blog, and if you're planning a migration or a Forge app, that's what we do: leanzero.net/services.
Top comments (0)