<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Szymon Paluch</title>
    <description>The latest articles on DEV Community by Szymon Paluch (@szymonpaluch).</description>
    <link>https://dev.to/szymonpaluch</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4033858%2F83ca0373-4fe9-4f36-ad35-29e23f9a3315.png</url>
      <title>DEV Community: Szymon Paluch</title>
      <link>https://dev.to/szymonpaluch</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/szymonpaluch"/>
    <language>en</language>
    <item>
      <title>60 Claude Certified Architect Exam Questions &amp; Answers (CCAR-F)</title>
      <dc:creator>Szymon Paluch</dc:creator>
      <pubDate>Fri, 16 Oct 2026 07:30:05 +0000</pubDate>
      <link>https://dev.to/szymonpaluch/60-claude-certified-architect-exam-questions-answers-ccar-f-4e8l</link>
      <guid>https://dev.to/szymonpaluch/60-claude-certified-architect-exam-questions-answers-ccar-f-4e8l</guid>
      <description>&lt;p&gt;Which exam are you preparing for?&lt;/p&gt;

&lt;p&gt;&lt;a href="https://szymonpaluch.com/claude-certified-architect-practice-exam" rel="noopener noreferrer"&gt;Architect Foundations (CCAR-F)&lt;/a&gt; &lt;a href="https://szymonpaluch.com/claude-certified-architect-professional-practice-exam" rel="noopener noreferrer"&gt;Architect Professional (CCAR-P)&lt;/a&gt; &lt;a href="https://szymonpaluch.com/claude-certified-associate-practice-exam" rel="noopener noreferrer"&gt;Associate (CCAO-F)&lt;/a&gt; &lt;a href="https://szymonpaluch.com/claude-certified-developer-practice-exam" rel="noopener noreferrer"&gt;Developer (CCDV-F)&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;All 60 original questions from my free Claude Certified Architect Foundations (CCAR-F) practice set are on this page, each with its answer and explanation. I walk through three of them step by step: the right answers, the traps and where to study. The same questions also run as an online test with a study mode, a timed 120-minute attempt and results by domain.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://szymonpaluch.com/claude-certified-architect-practice-exam" rel="noopener noreferrer"&gt;Take the free practice exam →&lt;/a&gt; 60 questions · no account needed&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fby8v3zvpxm93gdugedkb.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fby8v3zvpxm93gdugedkb.webp" alt="Three original CCAR-F questions explained: blocking a refund with a hook, representing missing data as null and compacting session context" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Studying for Claude Certified Architect Foundations and stuck between two plausible answers? Remembering the letter will only get you so far. These three examples work through the requirement, the mechanism that meets it and the reason to reject each alternative.&lt;/p&gt;

&lt;h2&gt;
  
  
  What exactly you get
&lt;/h2&gt;

&lt;p&gt;I wrote these questions during preparation, using the public exam objectives. They do not come from the live exam. Each asks you to pick one of four options. The $500 threshold and 11% rate below are assumptions in the exercises, not measurements from my client projects.&lt;/p&gt;

&lt;p&gt;I passed CCAR-F on July 15 with 833/1000. My &lt;a href="https://szymonpaluch.com/blog/posts/claude-certified-architect-exam" rel="noopener noreferrer"&gt;account of taking the exam&lt;/a&gt; covers the preparation and the sitting itself. That score is personal experience, not evidence that the question set produces the same result for other people.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the real exam covers
&lt;/h2&gt;

&lt;p&gt;The official CCAR-F has 60 questions, a 120-minute limit and a passing score of 720 on a 100 to 1000 scale. It includes both single-answer and multiple-response items and selects four of six scenarios. My mock covers all six scenarios with single-answer questions only. This comparison follows &lt;a href="https://everpath-course-content.s3-accelerate.amazonaws.com/instructor%2F6nizmqk8tpzpfjvt6qmmav7rh%2Fpublic%2F1783542750%2FClaude+Certified+Architect+%E2%80%93+Foundations+Exam+Guide.pdf" rel="noopener noreferrer"&gt;Anthropic’s July 2026 exam guide, version 1.0&lt;/a&gt;, checked on September 29. Here are the domain weights and the question counts in my set:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Domain&lt;/th&gt;
&lt;th&gt;Weight&lt;/th&gt;
&lt;th&gt;Questions in my mock&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;D1 · Agentic Architecture &amp;amp; Orchestration&lt;/td&gt;
&lt;td&gt;27%&lt;/td&gt;
&lt;td&gt;16&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;D2 · Tool Design &amp;amp; MCP Integration&lt;/td&gt;
&lt;td&gt;18%&lt;/td&gt;
&lt;td&gt;11&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;D3 · Claude Code Configuration &amp;amp; Workflows&lt;/td&gt;
&lt;td&gt;20%&lt;/td&gt;
&lt;td&gt;12&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;D4 · Prompt Engineering &amp;amp; Structured Output&lt;/td&gt;
&lt;td&gt;20%&lt;/td&gt;
&lt;td&gt;12&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;D5 · Context Management &amp;amp; Reliability&lt;/td&gt;
&lt;td&gt;15%&lt;/td&gt;
&lt;td&gt;9&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;On the exam, I found decisions with several plausible solutions the hardest. These exercises therefore start with a concrete constraint: who can approve a refund, how to represent a missing value, or when to condense a session. Your choice should follow from that constraint.&lt;/p&gt;

&lt;p&gt;This set covers Foundations. The &lt;a href="https://szymonpaluch.com/blog/posts/claude-certified-architect-professional" rel="noopener noreferrer"&gt;Architect Professional (CCAR-P) guide&lt;/a&gt; describes that separate scope and its own examples. A CCAR-F mock score does not assess readiness for Professional.&lt;/p&gt;

&lt;p&gt;If Professional is your exam, take the &lt;a href="https://szymonpaluch.com/claude-certified-architect-professional-practice-exam" rel="noopener noreferrer"&gt;63-question CCAR-P practice exam&lt;/a&gt; instead. It follows the Professional guide and, like the real exam, includes multiple-response questions.&lt;/p&gt;

&lt;p&gt;Associate and Developer have &lt;a href="https://szymonpaluch.com/claude-certification-practice-exams" rel="noopener noreferrer"&gt;their own free full-length mocks&lt;/a&gt; too, each written from its own exam guide.&lt;/p&gt;

&lt;h2&gt;
  
  
  Three questions to try
&lt;/h2&gt;

&lt;p&gt;Pick an answer and write one sentence explaining your choice before reading the explanation. You can then check whether you understood the mechanism or merely recognized a familiar term. These are questions 3, 55 and 20 from the full set below, lightly edited for this walkthrough.&lt;/p&gt;

&lt;h3&gt;
  
  
  QUESTION 3 · D1 · AGENTIC ARCHITECTURE
&lt;/h3&gt;

&lt;p&gt;Company policy states that any refund above $500 must be approved by a manager. An audit shows the support agent has autonomously called &lt;code&gt;process_refund&lt;/code&gt; for higher amounts several times. What is the best way to guarantee the policy holds?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Add a PreToolUse hook on &lt;code&gt;process_refund&lt;/code&gt; that blocks any call above $500 and redirects the flow to human escalation.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Add a system-prompt line instructing the agent to always escalate refunds above $500.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Remove &lt;code&gt;process_refund&lt;/code&gt; from the agent's toolset so every refund is handled by a human.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Fine-tune the model on historical tickets so it learns the $500 threshold.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Correct: A. A &lt;code&gt;PreToolUse&lt;/code&gt; hook can deny a tool call before the refund executes. Here it checks the amount and routes larger refunds for approval. B and D influence model behaviour but do not enforce the limit. C also blocks permitted refunds up to $500. In production, enforce the rule in the refund service too, so other paths cannot bypass the check. See &lt;a href="https://code.claude.com/docs/en/agent-sdk/hooks" rel="noopener noreferrer"&gt;tool blocking with Agent SDK hooks&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;To test the hook, prepare two calls: $500 and $501. The first should proceed; the second should wait for a person. Then explicitly ask the agent to ignore the limit and try again. Check that the code still stops the refund. You do not have to guess whether the model understood the request. If the amount is absent or cannot be parsed, stop the call and report an error too.&lt;/p&gt;

&lt;h3&gt;
  
  
  QUESTION 55 · D4 · STRUCTURED OUTPUT
&lt;/h3&gt;

&lt;p&gt;For documents that lack a purchase-order number, Claude invents a plausible-looking number about 11% of the time instead of signaling absence. Downstream systems treat the fabricated values as real. What is the best schema-level fix?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Add a required confidence float to every field and discard low values downstream.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Switch &lt;code&gt;tool_choice&lt;/code&gt; to "auto" so the model can skip extraction when the number is missing.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Add a few-shot example with a correctly formatted purchase-order number.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Allow null in the &lt;code&gt;po_number&lt;/code&gt; field and instruct the model to return null when the value is not present in the source.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Correct: D. Allowing &lt;code&gt;null&lt;/code&gt; gives the model a way to represent a missing number without supplying a string. Nullable and optional mean different things: a field can be required while its value may be null. A adds a confidence estimate, B changes tool selection, and C illustrates the format; none defines absence in the schema. This fix does not guarantee factual accuracy, so still test extraction on documents without a number. The &lt;a href="https://platform.claude.com/docs/en/build-with-claude/structured-outputs" rel="noopener noreferrer"&gt;structured outputs documentation&lt;/a&gt; lists supported types, including &lt;code&gt;null&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Use a document with no purchase-order number and expect &lt;code&gt;"po_number": null&lt;/code&gt;. Then supply a document with a number and check that the model copies it unchanged. Both cases need to pass. Valid JSON alone is insufficient: a fabricated number can fit the schema too. If the model still adds one, compare the result with the source and inspect the field’s instructions.&lt;/p&gt;

&lt;h3&gt;
  
  
  QUESTION 20 · D5 · CONTEXT MANAGEMENT
&lt;/h3&gt;

&lt;p&gt;Midway through a coherent multi-hour refactor, the session approaches its context limit. The history is full of superseded intermediate diffs, but the task and the goal are unchanged and the latest file states are still valid. What should you do to keep going without losing the thread?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Start a brand-new session with no prior context and re-describe the task.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Fork the session to branch off a divergent line of work.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Request a larger context window and continue as-is.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Run &lt;code&gt;/compact&lt;/code&gt; to condense the accumulated history into a summary and continue the same task.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Correct: D. &lt;code&gt;/compact&lt;/code&gt; summarizes the history and frees context for continued work. Tell it which goal and decisions to preserve, then check the summary: compression can omit details. A requires rebuilding context from scratch, B creates a branch, and C leaves the redundant history in place. The &lt;a href="https://code.claude.com/docs/en/best-practices#manage-context-aggressively" rel="noopener noreferrer"&gt;Claude Code context recommendations&lt;/a&gt; show how to give &lt;code&gt;/compact&lt;/code&gt; a specific focus.&lt;/p&gt;

&lt;p&gt;Before compaction, record what should work after the refactor and which tests demonstrate it. You can ask: &lt;code&gt;/compact Preserve the refactor goal, changed files and errors we have not fixed yet&lt;/code&gt;. Read the resulting summary. If an important decision is missing, add it before the agent resumes editing. Then run the same tests. This exercise continues one existing task.&lt;/p&gt;

&lt;p&gt;If you chose another option, identify the requirement it misses. D1 is about preventing an operation, D4 about representing missing data, and D5 about continuing the same task. That mistake tells you what to revisit in the documentation.&lt;/p&gt;

&lt;h2&gt;
  
  
  All 60 questions with answers
&lt;/h2&gt;

&lt;p&gt;Here is the full set, in the same order as the online test and the PDF. Each question names its domain and scenario. The answer and its explanation stay folded under “Show the answer”, so choose your option first.&lt;/p&gt;

&lt;p&gt;Keep count of your correct answers. The next section turns that count into the mock’s score. To match the real exam’s time limit, give yourself 120 minutes for all 60.&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 1 · D1 · Agentic Architecture &amp;amp; Orchestration
&lt;/h3&gt;

&lt;p&gt;Scenario: Customer Support Resolution Agent&lt;/p&gt;

&lt;p&gt;A customer support resolution agent built with the Claude Agent SDK reaches backend systems through custom MCP tools (get_customer, lookup_order, process_refund, escalate_to_human). The engineering team wrote a wrapper loop that ends the conversation whenever the assistant's text matches a closing phrase like "let me know if there's anything else." Production logs show that in 11% of sessions the loop terminates while a tool_use block is still pending, so the refund is never actually executed. What is the correct way to drive the agentic loop?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Terminate only when stop_reason is end_turn; whenever stop_reason is tool_use, execute the requested tool, append the tool_result to the message list, and continue the loop.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Expand the regex allowlist of closing phrases so it recognizes every polite sign-off the model might emit before stopping.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Add a max-iteration cap and stop the loop once the assistant produces two consecutive text-only responses.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Run a lightweight intent classifier on each assistant message to decide whether the turn represents a genuine completion.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: A. The agentic loop must be driven by the API's stop_reason signal, not by parsing natural-language content. tool_use means the model wants a tool run, so you execute it, append the tool_result, and iterate; end_turn is the only correct termination signal. Matching closing phrases (B) is exactly the brittle text-parsing that caused the bug and will still misfire. An iteration cap plus consecutive text heuristic (C) can truncate legitimate multi-step work and still ignores stop_reason. A classifier (D) is over-engineering that reintroduces natural-language inference where a deterministic protocol field already gives the answer.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 2 · D1 · Agentic Architecture &amp;amp; Orchestration
&lt;/h3&gt;

&lt;p&gt;Scenario: Customer Support Resolution Agent&lt;/p&gt;

&lt;p&gt;When the support agent invokes escalate_to_human, human agents report they must re-read the entire raw conversation to understand each case, and average handle time for escalated tickets has risen. What should the agent do at the moment of escalation to fix this?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Attach the full untrimmed transcript of the conversation so the human has complete information.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Enable extended thinking on the escalation turn so the model reasons longer before handing off.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Escalate earlier in ambiguous cases so humans receive tickets before much conversation has accumulated.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Produce a structured handoff summary — the customer's issue, verified identifiers, tool results and account state gathered, actions already attempted, and the specific recommended next step.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: D. Effective human handoff requires a structured summary that front-loads exactly what the human needs: the problem, confirmed identifiers, what was tried, and a recommendation. That is what removes the re-reading burden and shortens handle time. Dumping the full transcript (A) is the current failure mode — it forces the human to reconstruct the case themselves. Extended thinking (B) improves the model's own reasoning but produces no artifact for the human. Escalating earlier (C) changes when tickets arrive, not the missing information problem, and would likely lower first-contact resolution.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 3 · D1 · Agentic Architecture &amp;amp; Orchestration
&lt;/h3&gt;

&lt;p&gt;Scenario: Customer Support Resolution Agent&lt;/p&gt;

&lt;p&gt;Company policy states that any refund above $500 must be approved by a manager rather than auto-issued. The support agent, however, has autonomously called process_refund for amounts over $500 in several audited cases. What is the best way to guarantee the policy holds?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Add a PreToolUse interception hook on process_refund that blocks any call with an amount over $500 and redirects the flow to escalate_to_human.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Add a line to the system prompt instructing the agent to always escalate refunds over $500 instead of processing them.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Remove process_refund from the agent's toolset entirely so every refund is handled by a human.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Fine-tune the model on historical tickets so it learns the $500 threshold from past manager approvals.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: A. A policy violation that must never occur calls for an interception (PreToolUse) hook that deterministically blocks the disallowed process_refund call and redirects to escalation. The hook enforces the threshold regardless of what the model decides. A prompt instruction (B) is advisory and is precisely what the audited violations bypassed. Removing the tool (C) throws away all legitimate sub-$500 automation and tanks first-contact resolution. Fine-tuning (D) is heavy, slow, and still probabilistic — it cannot promise the hard $500 boundary is never crossed.

&lt;p&gt;Worked through step by step above&lt;br&gt;
&lt;/p&gt;

&lt;br&gt;
&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 4 · D1 · Agentic Architecture &amp;amp; Orchestration
&lt;/h3&gt;

&lt;p&gt;Scenario: Customer Support Resolution Agent&lt;/p&gt;

&lt;p&gt;get_customer and lookup_order call two different backend systems that return dates in different formats (ISO vs. US) and amounts with inconsistent currency notation. Logs show the model occasionally misreads these heterogeneous fields, e.g., treating a refund amount in the wrong units. What is the most robust remedy?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Instruct the model in the system prompt to normalize all dates and currency values before reasoning about them.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Add a PostToolUse hook that normalizes each tool's raw output into one canonical schema before it is appended to the context.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Increase the context window so the model has more room to keep the varied formats straight.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Spin up a dedicated formatting subagent that the coordinator calls after every backend lookup.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: B. A PostToolUse hook is the designed mechanism for normalizing heterogeneous tool output into a consistent schema before it ever reaches the model, eliminating the ambiguity deterministically. Asking the model to normalize in-prompt (A) leaves the same error-prone inference in place — the source of the current misreads. A larger context window (C) addresses volume, not format inconsistency, and does nothing for unit confusion. A separate formatting subagent (D) adds a full round-trip and orchestration overhead for a transformation a hook performs inline and reliably.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 5 · D1 · Agentic Architecture &amp;amp; Orchestration
&lt;/h3&gt;

&lt;p&gt;Scenario: Customer Support Resolution Agent&lt;/p&gt;

&lt;p&gt;The coordinator has already run get_customer and confirmed the account, then spawns subagents to check order history and refund eligibility in parallel. Those subagents return results that contradict the confirmed account — some even re-run get_customer and pick a different customer match. What is the correct fix?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Enlarge the shared context window so the subagents can see the coordinator's earlier turns.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Store the confirmed customer record in a global variable that the SDK automatically injects into every subagent.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Instruct each subagent to call get_customer itself at the start so it independently establishes the account.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Include the coordinator's already-verified findings — the confirmed customer record and identifiers — explicitly in each subagent's Task prompt, since subagents do not inherit the coordinator's conversation context.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: D. Subagents run with their own fresh context and do not inherit the coordinator's conversation, so any prior findings they must use have to be passed explicitly in the Task prompt. Handing each subagent the confirmed customer record eliminates the contradictions and the redundant lookups. Enlarging a "shared" context window (A) misdescribes the architecture — there is no shared window to grow across the hub-and-spoke boundary. There is no SDK mechanism that auto-injects a global variable into subagents (B); it is a plausible-sounding non-feature. Having each subagent re-run get_customer (C) is exactly what produced the divergent matches and duplicates work.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 6 · D2 · Tool Design &amp;amp; MCP Integration
&lt;/h3&gt;

&lt;p&gt;Scenario: Customer Support Resolution Agent&lt;/p&gt;

&lt;p&gt;process_refund handles both full order refunds and small partial goodwill credits. Logs show the agent issues full refunds when only a partial credit was warranted in 17% of cases because it cannot tell from the single tool which behavior it is invoking. What is the best fix?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Raise the agent's context window so the entire refund policy document fits alongside the conversation.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Split process_refund into two tools, process_full_refund and process_partial_credit, each with a distinct description and explicit boundary on when to use it.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Add three few-shot examples of partial-credit conversations to the system prompt while keeping the single tool.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Set tool_choice to "any" so the model is compelled to select a tool on every refund turn.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: B. When one tool conflates two distinct behaviors, the reliable fix is to split it into separately named tools whose descriptions encode clear boundaries, making the choice explicit at selection time. Option A adds context but leaves the overloaded tool ambiguous, so the same misclassification persists. Option C helps at the margins but few-shot prompt examples cannot substitute for a tool interface that still exposes one undifferentiated action. Option D only guarantees that some tool is called, not that the correct refund behavior is chosen, and does nothing about the underlying overloading.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 7 · D2 · Tool Design &amp;amp; MCP Integration
&lt;/h3&gt;

&lt;p&gt;Scenario: Customer Support Resolution Agent&lt;/p&gt;

&lt;p&gt;When the customer database is briefly unreachable, get_customer returns a plain empty result, and the agent tells the customer "I can't find any account with that information" and closes the case. Real accounts are being turned away during transient outages. What should the MCP tool return instead?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Automatically retry get_customer in a tight loop until the database responds, however long that takes.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Increase the get_customer connection timeout so slow database responses are no longer cut off.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; A structured error with an errorCategory (e.g. transient) and isRetryable metadata that lets the agent distinguish an access failure from a valid empty result and respond accordingly.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Treat every non-successful get_customer response as an escalation trigger and route all of them to escalate_to_human.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: C. The root cause is that an access failure is being conveyed as a valid empty result, so the agent draws the wrong conclusion; the tool must emit structured error metadata (errorCategory, isRetryable) that separates 'lookup failed' from 'no such customer.' Option A retries blindly with no bound and still lacks the semantics to tell the two conditions apart. Option B may reduce some timeouts but does not fix the core ambiguity when a failure does occur. Option D over-escalates by treating validation and business outcomes identically to infrastructure faults, defeating the resolution target and burying real transient errors.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 8 · D2 · Tool Design &amp;amp; MCP Integration
&lt;/h3&gt;

&lt;p&gt;Scenario: Customer Support Resolution Agent&lt;/p&gt;

&lt;p&gt;For refunds above the agent's $500 authority, process_refund correctly declines, but it returns a generic isError response. Logs show the agent then retries the same $640 refund three times before eventually giving a confusing reply. How should the tool signal this condition?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Have the agent retry with exponential backoff, since repeated attempts eventually succeed for most declined refunds.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Raise the agent's refund authority limit above $640 so the call no longer fails.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Instruct the agent in the system prompt to never retry a refund more than once.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Return a structured error with errorCategory "business" and isRetryable set to false, so the agent stops retrying and routes the request to escalate_to_human.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: D. A policy-limit rejection is a business error that is deterministically non-retryable; encoding errorCategory "business" with isRetryable false tells the agent that retrying is pointless and that escalation is the correct next step. Option A applies retry logic meant for transient faults to a permanent business rule, wasting calls and never succeeding. Option B changes policy to dodge the control rather than handling the boundary the business intends to enforce. Option C is a prompt-only patch that leaves the error semantically indistinguishable from a retryable one, so behavior stays fragile compared with programmatic metadata the agent can act on reliably.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 9 · D5 · Context Management &amp;amp; Reliability
&lt;/h3&gt;

&lt;p&gt;Scenario: Customer Support Resolution Agent&lt;/p&gt;

&lt;p&gt;In longer billing-dispute conversations, the agent uses progressive summarization to keep the history compact. Testers find that after several turns it quotes the wrong refund amount and an incorrect order date, even though those values appeared earlier. What is the best remedy?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Maintain a persistent 'case facts' block (order id, amount, dates, customer expectation) stored outside the summarized history and re-injected verbatim each turn.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Switch to a model with a larger context window so no summarization is needed.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Call get_customer and lookup_order again on every turn to refetch the numbers.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Enable extended thinking so the model reasons more carefully before quoting figures.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: A. Progressive summarization compresses away precise numbers, dates, and stated expectations, so the fix is to keep those exact values in a persistent case-facts block that lives outside the summarized transcript and is re-supplied unchanged. Option B only delays the same information loss once conversations grow, and does not protect the specific fields. Option C is wasteful and can still be re-summarized away, plus it re-incurs latency and error surface each turn. Option D improves reasoning but cannot recover facts that summarization has already discarded from context.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 10 · D5 · Context Management &amp;amp; Reliability
&lt;/h3&gt;

&lt;p&gt;Scenario: Customer Support Resolution Agent&lt;/p&gt;

&lt;p&gt;A customer gives only the name "Maria Garcia." get_customer returns three distinct matching accounts. Logs show the agent proceeds with the most recently active match, and in several cases it has processed refunds against the wrong customer's order. What is the correct behavior?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Continue with the most recently active of the three accounts, since it is the most likely current customer.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Pick the match with the highest fuzzy name-similarity score and proceed.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Escalate every multiple-match case straight to a human agent.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Ask the customer for an additional identifier (order number, email, or ZIP) to disambiguate before taking any account action.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: D. When multiple customer records match, the agent must not guess; requesting an additional identifier resolves the ambiguity and prevents acting on the wrong account. Option A guesses on recency and is exactly the behavior producing misdirected refunds. Option B guesses on name similarity, which cannot distinguish two people with the same name. Option C needlessly hands off a case the agent can resolve itself with one clarifying question, undermining the first-contact-resolution target when disambiguation is straightforward.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 11 · D3 · Claude Code Configuration &amp;amp; Workflows
&lt;/h3&gt;

&lt;p&gt;Scenario: Code Generation with Claude Code&lt;/p&gt;

&lt;p&gt;Your team standardized naming and error-handling conventions by adding them to ~/.claude/CLAUDE.md on the tech lead's machine. Everything works for the lead, but three newly onboarded engineers who cloned the repo report that Claude Code ignores the conventions entirely, even though &lt;code&gt;git status&lt;/code&gt; shows a clean checkout. What is the most effective fix?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Add an &lt;a class="mentioned-user" href="https://dev.to/import"&gt;@import&lt;/a&gt; to the repo-root CLAUDE.md pointing at ~/.claude/CLAUDE.md so the personal file is pulled in for everyone&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Have each engineer run /memory after cloning to force-reload the user-scoped memory files&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Move the conventions into the project-scoped .claude/CLAUDE.md (or repo root) so they are committed to version control and load for every clone&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Convert the conventions into a skill under .claude/skills/ so they activate on demand for all users&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: C. The CLAUDE.md hierarchy loads user-scoped instructions from ~/.claude/CLAUDE.md, which is personal to a machine and is NOT shared through version control — the classic 'new teammate didn't get the instructions' symptom. Committing the conventions to the project-scoped .claude/CLAUDE.md (or repo root) makes them load for every clone, so C is correct. An &lt;a class="mentioned-user" href="https://dev.to/import"&gt;@import&lt;/a&gt; to ~/.claude/CLAUDE.md (A) fails because that file doesn't exist on the teammates' machines. Running /memory (B) only inspects already-loaded memory files; it cannot pull in instructions that were never in a shared scope. A skill (D) is on-demand rather than always-loaded, so it wouldn't provide the persistent conventions the team wants.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 12 · D3 · Claude Code Configuration &amp;amp; Workflows
&lt;/h3&gt;

&lt;p&gt;Scenario: Code Generation with Claude Code&lt;/p&gt;

&lt;p&gt;Your team's /investigate-flaky skill greps the entire test suite, reads dozens of files, and prints verbose exploration before it answers. Developers notice that after running it, the main session is cluttered with raw output and Claude's subsequent answers in that session degrade in quality. What is the best fix?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Add a step to CLAUDE.md instructing Claude to summarize the skill's output before continuing&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Set context:fork in the skill's SKILL.md frontmatter so its verbose exploration runs in an isolated context and only the result returns to the main session&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Increase the context window so the accumulated exploration output no longer causes degradation&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Restrict the skill with an allowed-tools list so it reads fewer files during exploration&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: B. Skills support context:fork in their SKILL.md frontmatter precisely to isolate verbose, exploratory work in a separate context and return only the distilled result, keeping the main session clean — so B is correct. A CLAUDE.md instruction to summarize (A) is a prompt-only measure that runs after the pollution already occurred and is unreliable. A larger context window (C) does not fix attention dilution from irrelevant accumulated output; it just delays it. An allowed-tools list (D) restricts which tools the skill may use but does not stop verbose output from landing in the main context, and it would also cripple the investigation the skill exists to perform.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 13 · D3 · Claude Code Configuration &amp;amp; Workflows
&lt;/h3&gt;

&lt;p&gt;Scenario: Code Generation with Claude Code&lt;/p&gt;

&lt;p&gt;A team's &lt;code&gt;.claude/skills/dep-audit/&lt;/code&gt; skill is meant to be run manually before releases, but Claude keeps auto-invoking it mid-session whenever dependency files come up, dumping roughly 4,000 lines of resolution output into unrelated refactoring sessions. The team still wants engineers to be able to run &lt;code&gt;/dep-audit&lt;/code&gt; explicitly. What is the most effective fix?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Call &lt;code&gt;/compact&lt;/code&gt; automatically at the end of the skill so the audit output is condensed before work continues&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Set &lt;code&gt;disable-model-invocation: true&lt;/code&gt; in the skill's SKILL.md frontmatter so the skill runs only when a user explicitly invokes &lt;code&gt;/dep-audit&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Delete the skill's &lt;code&gt;description&lt;/code&gt; frontmatter so Claude can no longer match it to conversation topics&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Move the audit instructions into CLAUDE.md so they load once at startup instead of triggering mid-session&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: B. &lt;code&gt;disable-model-invocation: true&lt;/code&gt; is the frontmatter switch designed for exactly this: it removes the skill from Claude's automatic invocation while keeping explicit &lt;code&gt;/dep-audit&lt;/code&gt; slash invocation available, so release engineers keep the workflow and refactoring sessions stay clean. &lt;code&gt;/compact&lt;/code&gt; (A) reacts only after the 4,000 lines have already polluted and degraded the session, and compaction can discard needed detail. Deleting the description (C) is a hack — the description also documents the skill for humans and tooling, and gutting metadata to suppress triggering is not the supported mechanism. Moving the instructions into CLAUDE.md (D) changes where guidance loads but makes the verbose audit part of every session's baseline context, worsening rather than fixing the problem.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 14 · D3 · Claude Code Configuration &amp;amp; Workflows
&lt;/h3&gt;

&lt;p&gt;Scenario: Code Generation with Claude Code&lt;/p&gt;

&lt;p&gt;A shared &lt;code&gt;generate-docs&lt;/code&gt; skill is supposed to read source and write Markdown into &lt;code&gt;docs/&lt;/code&gt;, but twice it has run &lt;code&gt;Edit&lt;/code&gt; and shell &lt;code&gt;rm&lt;/code&gt; and overwritten source files. Adding the reminder "only write to docs/, never edit source" to the skill body has not reliably stopped it. What is the best way to guarantee the skill cannot touch source?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Add a stronger system-prompt instruction forbidding edits to any file outside &lt;code&gt;docs/&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Require the skill to run in plan mode every time so a human approves each action&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Declare &lt;code&gt;allowed-tools&lt;/code&gt; in the skill's SKILL.md frontmatter, restricting it to Read/Glob/Write so the harness blocks Edit and Bash entirely&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Add a global PreToolUse hook that inspects every Edit across all sessions and rejects ones outside &lt;code&gt;docs/&lt;/code&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: C. When deterministic tool restriction is required, prompt text is insufficient because the model can still choose a forbidden tool; the &lt;code&gt;allowed-tools&lt;/code&gt; frontmatter enforces the restriction at the harness level, scoped exactly to this skill. Stronger prose (option A) is still an instruction the model may ignore, which is the observed failure. Plan mode (option B) adds friction to every run and relies on the human catching the bad action rather than preventing it. A global PreToolUse hook (option D) is the right idea at the wrong scope — it would police every session's edits when the problem is confined to one skill, and &lt;code&gt;allowed-tools&lt;/code&gt; is the purpose-built, skill-scoped mechanism.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 15 · D3 · Claude Code Configuration &amp;amp; Workflows
&lt;/h3&gt;

&lt;p&gt;Scenario: Code Generation with Claude Code&lt;/p&gt;

&lt;p&gt;A shared &lt;code&gt;/scaffold-component&lt;/code&gt; skill expects a component-name argument, but telemetry shows teammates invoke it with no argument or a malformed one about a third of the time, producing empty or misnamed scaffolds. What is the most effective way to reduce these malformed invocations?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Add an &lt;code&gt;argument-hint&lt;/code&gt; field to the skill's SKILL.md frontmatter so the expected argument and format are shown at the invocation prompt&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Document the correct usage in the project CLAUDE.md so it is always loaded into context&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Rename the skill to &lt;code&gt;/scaffold-component-with-name&lt;/code&gt; so the required input is implied by the name&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Have the skill body instruct Claude to ask the user for the name whenever it is missing&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: A. &lt;code&gt;argument-hint&lt;/code&gt; is the frontmatter field designed precisely to surface the expected argument and format at the moment of invocation, guiding the caller before a malformed run happens. Documenting usage in CLAUDE.md (option B) helps the model but does not present the hint to the human at invocation time, where the mistake occurs. Renaming (option C) is a cosmetic nudge that neither shows the format nor prevents empty calls. A body instruction to ask when missing (option D) is a runtime fallback that adds a round-trip and still does not communicate the expected format up front — the direct, purpose-built fix is &lt;code&gt;argument-hint&lt;/code&gt;.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 16 · D3 · Claude Code Configuration &amp;amp; Workflows
&lt;/h3&gt;

&lt;p&gt;Scenario: Code Generation with Claude Code&lt;/p&gt;

&lt;p&gt;A developer needs three changes that all touch the shared &lt;code&gt;PaymentStatus&lt;/code&gt; enum and the several functions that switch on it. They sent each change as a separate sequential message; after the third message the project no longer compiles because the earlier edits did not anticipate the later ones. What is the best way to structure the request?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Run each of the three changes in its own fresh session so they stay cleanly isolated&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Enable plan mode and approve the three changes one at a time in sequence&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Keep them sequential but ask Claude to re-read the enum before each message&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Batch the three interacting changes into a single message so Claude edits the enum and all dependent switch sites together with full mutual context&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: D. Fixes that interact — here all three touch one enum and its dependent switches — should be batched into a single message so the model reasons about them jointly and keeps the code internally consistent; only independent changes benefit from being sequenced separately. Separate fresh sessions (option A) make the interaction worse by discarding shared context each time. Plan mode approvals (option B) still process the edits one at a time, reproducing the same ordering problem. Re-reading the enum each message (option C) treats a coordination problem as a memory problem; the real fix is to let the model see all three interacting edits at once.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 17 · D4 · Prompt Engineering &amp;amp; Structured Output
&lt;/h3&gt;

&lt;p&gt;Scenario: Code Generation with Claude Code&lt;/p&gt;

&lt;p&gt;Asking Claude to "make the input validation more robust and idiomatic" yields inconsistent output across runs: sometimes it adds zod, sometimes hand-written checks, sometimes nothing meaningful. The team wants the validation style to come out consistent. What is the most effective refinement?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Expand the request into a longer, more detailed prose description of what "robust" and "idiomatic" mean&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Provide 2-3 concrete input-to-output examples showing the exact validation style and library you want&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Fine-tune a model on the team's existing validation code so it internalizes the house style&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Add a standing CLAUDE.md rule stating that validation should always be robust and idiomatic&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: B. For ambiguous, style-sensitive tasks, concrete input/output examples communicate the target far more reliably than prose and generalize across cases — a few examples pin down the exact pattern the vague adjectives left open. More detailed prose (option A) still leaves "robust" and "idiomatic" open to interpretation, which is the source of the variance. Fine-tuning (option C) is heavy over-engineering for what a few examples in the prompt solve. A CLAUDE.md rule (option D) restates the same vague adjectives as an always-loaded instruction without demonstrating the concrete format, so the inconsistency persists.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 18 · D5 · Context Management &amp;amp; Reliability
&lt;/h3&gt;

&lt;p&gt;Scenario: Code Generation with Claude Code&lt;/p&gt;

&lt;p&gt;During a 200-file framework migration carried out in one long session, logs show Claude re-migrating files it already converted and skipping others as the conversation grows. What technique best keeps the migration on track?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Switch to a larger-context model so the whole file list stays in the window&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Run &lt;code&gt;/compact&lt;/code&gt; frequently to keep the history short&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Maintain a scratchpad manifest file that records migrated vs pending files and update it after each file, so progress survives context growth&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Do the entire migration in a single large message to avoid state loss between turns&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: C. For large, long-running tasks the durable pattern is an external scratchpad or manifest that tracks state on disk, so progress is authoritative regardless of what falls out of the conversation window. A larger window (option A) still degrades over a 200-file span and does not give a reliable record of what is done. &lt;code&gt;/compact&lt;/code&gt; (option B) shrinks history but can summarize away exactly the per-file progress you need, and it does not create a checkable record. A single massive message (option D) is impractical at this scale and still offers no crash-safe record of completed work — the manifest provides both tracking and recovery.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 19 · D5 · Context Management &amp;amp; Reliability
&lt;/h3&gt;

&lt;p&gt;Scenario: Code Generation with Claude Code&lt;/p&gt;

&lt;p&gt;To rename a widely-used internal API, Claude must first locate every call site across a large monorepo. Doing this discovery inline reads dozens of files and floods the main session with search output before any edits begin, and the later edits then start to wander. What is the best approach?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Delegate the discovery to an Explore subagent that searches the monorepo and returns only a summary of call sites, keeping the main context focused on the edits&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Read every candidate file into the main session first so all call sites are visible before editing&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Run &lt;code&gt;/compact&lt;/code&gt; after each search so the accumulated output is repeatedly condensed&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Move to a larger-context model so the exploration output no longer competes with the edits&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: A. Verbose discovery over a large codebase should be delegated to a subagent so the exploration happens in an isolated context and only a distilled summary of call sites returns — the main session stays focused on performing the rename. Reading every file into the main session (option B) is exactly the flooding behavior causing the edits to wander. Repeated &lt;code&gt;/compact&lt;/code&gt; (option C) is a lossy patch applied after the noise already accumulated and interrupts the flow. A bigger window (option D) still suffers attention dilution from the raw search output; isolating the discovery is what preserves the working context.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 20 · D5 · Context Management &amp;amp; Reliability
&lt;/h3&gt;

&lt;p&gt;Scenario: Code Generation with Claude Code&lt;/p&gt;

&lt;p&gt;Midway through a coherent multi-hour refactor, the session approaches its context limit. The history is full of superseded intermediate diffs, but the current task and goal are unchanged and the latest file states are still valid. What should the developer do to keep going without losing the thread?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Start a brand-new session with no prior context and re-describe the task&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Fork the session to branch off a divergent line of work&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Request a larger context window and continue as-is&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Run &lt;code&gt;/compact&lt;/code&gt; to condense the accumulated history into a summary while continuing the same task&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: D. When the task is a single coherent thread and only the accumulated intermediate history is bloating the context, &lt;code&gt;/compact&lt;/code&gt; condenses that history into a summary so work can continue seamlessly on the same task. Starting fresh (option A) is the move when prior tool results are stale or the task changes — neither applies here, and it discards still-valid state. Forking (option B) is for pursuing a divergent branch, not for continuing one coherent line. A larger window (option C) postpones the limit without addressing the superseded-diff clutter; compaction removes the dead weight while preserving the goal.

&lt;p&gt;Worked through step by step above&lt;br&gt;
&lt;/p&gt;

&lt;br&gt;
&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 21 · D5 · Context Management &amp;amp; Reliability
&lt;/h3&gt;

&lt;p&gt;Scenario: Code Generation with Claude Code&lt;/p&gt;

&lt;p&gt;In a long debugging session Claude repeatedly runs the full test suite; each run dumps roughly 2,000 lines of passing-test noise into the context. Over time Claude starts overlooking the failing-test details and the constraints stated at the start of the session. What is the best fix?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Switch to a larger-context model so all the test output fits comfortably&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Trim each test run's output to the relevant failing-test lines before it accumulates, and keep the key constraints in a summary placed at the top of the context&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Enable extended thinking so Claude reasons harder over the full accumulated output&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Instruct Claude to re-read the entire session history at the start of every turn&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: B. The failure is context degradation from unbounded verbose tool output plus lost-in-the-middle placement of the key constraints; the fix is to trim tool outputs to the relevant fields before they accumulate and to keep the critical summary up front where attention is strongest. A larger window (option A) still lets 2,000 lines per run bury the failing details and dilute attention — capacity is not the problem, signal-to-noise is. Extended thinking (option C) makes the model reason harder over the same noisy context without removing the noise. Re-reading the whole history each turn (option D) amplifies the very clutter that is drowning the failing-test details.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 22 · D1 · Agentic Architecture &amp;amp; Orchestration
&lt;/h3&gt;

&lt;p&gt;Scenario: Multi-Agent Research System&lt;/p&gt;

&lt;p&gt;The synthesis subagent's reports read as generic overviews: in 30% of runs they omit the specific figures and named sources the web-search subagent surfaced. Tracing shows the Task prompt sent to the synthesis subagent contains only the report topic and formatting instructions. What is the best fix?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Increase the coordinator's context window so more findings stay resident across delegations&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Have the coordinator embed the web-search and document-analysis findings verbatim in the synthesis subagent's Task prompt&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Add a 'recall the earlier research' instruction to the synthesis subagent's system prompt&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Give the synthesis subagent its own web-search tool so it can re-gather the missing figures&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: B. Subagents do not inherit the coordinator's conversation context; in a hub-and-spoke topology the coordinator must pass prior findings explicitly in the Task prompt, so embedding the web-search and document-analysis results verbatim is the fix. A larger context window (A) does not cross the process boundary into a freshly spawned subagent. Instructing it to 'recall' earlier research (C) fails because the subagent starts with no prior turns to recall. Giving it its own search tool (D) is wrong scope and over-engineering — it duplicates work and diverges from the evidence already gathered.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 23 · D1 · Agentic Architecture &amp;amp; Orchestration
&lt;/h3&gt;

&lt;p&gt;Scenario: Multi-Agent Research System&lt;/p&gt;

&lt;p&gt;The coordinator dispatches three independent regional web-search subagents (EU, US, APAC), but issues each Task call only after the previous subagent returns. End-to-end research latency is about 3x the slowest single search. What change most reduces latency?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Emit all three Task calls in a single assistant response so the subagents execute concurrently&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Merge the three regional searches into one subagent that loops over regions internally&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Add a fixed iteration cap so the coordinator stops waiting after a set number of rounds&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Set tool_choice to 'any' so the coordinator selects tools faster&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: A. Parallel subagents are launched by issuing multiple Task calls in one assistant response; because the three regional searches are independent, doing so collapses their latency to roughly the slowest single search. Merging into one looping subagent (B) removes the parallelism and the per-region differentiation. An iteration cap (C) does not address latency and risks dropping regions entirely. tool_choice 'any' (D) governs which tool the model picks, not whether subagents run concurrently.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 24 · D1 · Agentic Architecture &amp;amp; Orchestration
&lt;/h3&gt;

&lt;p&gt;Scenario: Multi-Agent Research System&lt;/p&gt;

&lt;p&gt;A newly configured coordinator answers research questions directly from its own knowledge instead of delegating; its attempts to invoke subagents never appear in the trace. The subagent definitions exist and work when tested standalone. What is the root cause?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; The subagents were registered at user scope while the coordinator runs at project scope&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; The coordinator's model is too small to plan multi-agent delegation&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; The coordinator's allowedTools list omits 'Task', so it cannot spawn subagents&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; tool_choice is forced to a specific tool, blocking delegation&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: C. To spawn subagents the coordinator's allowedTools must include 'Task'; when it is absent the coordinator has no delegation mechanism and silently answers on its own, exactly as observed. Scope mismatch (A) is not the gating mechanism for spawning and the subagents work standalone. Model size (B) does not explain a total absence of delegation attempts — the capability is simply not exposed. A forced tool_choice (D) would produce visible forced tool calls, not silent non-delegation.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 25 · D1 · Agentic Architecture &amp;amp; Orchestration
&lt;/h3&gt;

&lt;p&gt;Scenario: Multi-Agent Research System&lt;/p&gt;

&lt;p&gt;For open-ended briefs like 'assess the competitive landscape for X', the coordinator runs a fixed three-step chain (search then summarize then format). Reviewers note the reports consistently miss emerging subtopics that only surface mid-investigation. What orchestration change best fits this workload?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Add few-shot examples of good reports to the formatting step&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Raise the temperature of the synthesis subagent to encourage broader coverage&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Keep the fixed chain but add a second formatting pass&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Switch to dynamic decomposition so the coordinator spawns follow-up subtasks based on what earlier findings reveal&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: D. Open-ended investigation calls for dynamic decomposition, where the coordinator spawns follow-up subtasks in response to what earlier findings reveal; a fixed chain suits only predictable, well-defined multi-aspect work and structurally cannot chase subtopics it did not pre-plan. Few-shot formatting examples (A) and a second formatting pass (C) improve presentation, not investigative coverage. Raising temperature (B) adds randomness, not the adaptive depth that surfaces emerging subtopics.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 26 · D1 · Agentic Architecture &amp;amp; Orchestration
&lt;/h3&gt;

&lt;p&gt;Scenario: Multi-Agent Research System&lt;/p&gt;

&lt;p&gt;After the shared search and document-analysis phase completes, stakeholders want two report variants from the same evidence — a one-page executive brief and a technical deep-dive — without re-running the costly searches. What is the best way to produce both?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Continue the same session, generate the brief, then ask for the deep-dive in the next turn&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Fork the session at the end of the research phase into two branches, one per report variant&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Start two fresh sessions and re-run the search subagents for each&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Increase the context window so both reports fit in one generation&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: B. fork_session creates divergent branches that share all prior state, so forking after the research phase lets each branch produce one variant from the same gathered evidence without re-searching. Continuing one session (A) entangles both outputs in a single history and biases the deep-dive on the already-generated brief. Two fresh sessions (C) wastefully re-run the expensive searches. A bigger context window (D) is unrelated to branching divergent outputs.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 27 · D1 · Agentic Architecture &amp;amp; Orchestration
&lt;/h3&gt;

&lt;p&gt;Scenario: Multi-Agent Research System&lt;/p&gt;

&lt;p&gt;Logs show the document-analysis subagent attempting to invoke the web-search subagent directly to fill gaps, producing routing errors and duplicated searches. How should the topology be corrected?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Route all requests through the coordinator: subagents return gaps to it, and it dispatches any follow-up web searches&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Let subagents call each other but add a lock to prevent duplicate searches&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Merge the document-analysis and web-search subagents into one general subagent&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Give every subagent the full tool set so none needs to call another&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: A. The correct topology is hub-and-spoke: subagents report findings and gaps back to the coordinator, which dispatches any follow-up work — peer-to-peer subagent calls are the defect. Adding a lock (B) legitimizes and entrenches the wrong topology. Merging the two subagents (C) collapses the intended specialization and differentiation. Giving every subagent the full tool set (D) over-provisions tools, degrades tool selection, and still bypasses the coordinator's orchestration.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 28 · D1 · Agentic Architecture &amp;amp; Orchestration
&lt;/h3&gt;

&lt;p&gt;Scenario: Multi-Agent Research System&lt;/p&gt;

&lt;p&gt;A weekly market report is produced by --resume of a named coordinator session. This week the report cites last week's prices as current, because the resumed session replays cached web-search tool results. What is the best fix?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Fork the prior session so the stale results are copied into a new branch&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Keep using --resume but instruct the coordinator to ignore old tool outputs&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Increase the context window so both old and new results coexist&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Start a fresh session, inject a concise summary of the prior report's conclusions, and re-run the searches&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: D. When prior tool results are stale, the right move is to start fresh with an injected summary of the earlier conclusions and re-run the searches, so the report is grounded in current data rather than replayed cached results. Forking (A) copies the stale tool outputs into the new branch, preserving the problem. Prompt-based suppression under --resume (B) is unreliable and the outdated results still occupy context. A larger window (C) keeps the stale data resident, compounding the confusion.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 29 · D1 · Agentic Architecture &amp;amp; Orchestration
&lt;/h3&gt;

&lt;p&gt;Scenario: Multi-Agent Research System&lt;/p&gt;

&lt;p&gt;The coordinator's agentic loop sometimes stops before the report subagent has run: it ends whenever the latest assistant text contains the phrase 'research complete'. Intermittently the synthesis subagent uses that phrase mid-analysis, halting the workflow. How should loop termination be driven?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Add a minimum iteration count before the coordinator is allowed to stop&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Use a stricter regex that also requires the word 'final'&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Continue the loop while stop_reason is tool_use (appending each tool result) and stop only on end_turn&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Have a classifier judge whether the report is truly finished&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: C. The agentic loop must be driven by the API's stop_reason — continue and append the tool result while stop_reason is tool_use, and terminate only on end_turn — never by parsing the model's natural-language text. A minimum iteration count (A) masks the bug with an arbitrary floor. A stricter regex (B) still parses assistant text and remains brittle. A classifier (D) is over-engineering for a deterministic termination signal the API already provides.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 30 · D2 · Tool Design &amp;amp; MCP Integration
&lt;/h3&gt;

&lt;p&gt;Scenario: Multi-Agent Research System&lt;/p&gt;

&lt;p&gt;The synthesis subagent is initialized with the full registry of 18 tools shared across the system (web_search, fetch_url, parse_pdf, extract_tables, generate_chart, render_report, and others). Traces show that in 22% of runs it issues a fresh web_search or parse_pdf call instead of combining the findings the coordinator already passed in its prompt, adding 6-9 s of latency per run. What is the most effective fix?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Restrict the synthesis subagent to the small set of tools its role actually needs, removing the search and parsing tools from its configuration&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Increase the synthesis subagent's context window so all prior findings fit and it stops re-fetching them&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Add a system-prompt line instructing the synthesis subagent to never call web_search or parse_pdf&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Force tool_choice to "any" so the model commits to exactly one deliberate tool call per turn&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: A. Over-provisioning tools degrades selection: a role with 18 tools available will occasionally pick search or parse tools it should not touch. Scoping the synthesis subagent to only the tools its role needs removes the wrong options entirely and is the governing fix. A larger context window (B) is irrelevant — the findings already fit; the model is choosing to re-fetch, not overflowing. A prompt prohibition (C) is non-deterministic (the 22% proves prompt-only guardrails leak). Forcing tool_choice "any" (D) makes it worse by mandating a tool call every turn when synthesis often needs none.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 31 · D2 · Tool Design &amp;amp; MCP Integration
&lt;/h3&gt;

&lt;p&gt;Scenario: Multi-Agent Research System&lt;/p&gt;

&lt;p&gt;The document-analysis subagent must call classify_document before running any field extraction, because the extraction schema depends on the detected type. Relying on a prompt instruction, it skips classification in 9% of runs and extracts against the wrong schema. The coordinator invokes it once per document, and the required first step is always the same. What is the best fix?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Set tool_choice to "any" so the subagent is required to use some tool on the opening turn&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; On the first turn, set tool_choice to the forced form {"type":"tool","name":"classify_document"} so classification always runs before extraction&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Strengthen the system prompt with an emphatic "ALWAYS classify before extracting" plus two worked examples&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Add an interception hook that inspects each extraction call and blocks it whenever no classification has run&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: B. When a specific tool must run first deterministically, forcing tool_choice to that named tool on the opening turn is the direct API mechanism and guarantees classify_document executes before any extraction. Setting tool_choice "any" (A) forces a tool but not which one, so the model could still call an extraction tool. A stronger prompt (C) is exactly the non-deterministic approach that already failed 9% of the time. An interception hook (D) would work but is heavier cross-tool policy machinery for a single fixed first-step ordering that the forced tool_choice already guarantees at the API level.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 32 · D2 · Tool Design &amp;amp; MCP Integration
&lt;/h3&gt;

&lt;p&gt;Scenario: Multi-Agent Research System&lt;/p&gt;

&lt;p&gt;The document-analysis subagent handles mixed inputs: some are PDFs requiring parse_pdf, and some are already plain-text snippets that need no tool at all. It is currently configured with tool_choice "any". Logs show it calling parse_pdf on plain-text inputs — returning garbage — in 18% of cases. What is the best fix?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Force tool_choice to the parse_pdf tool so parsing behavior is deterministic&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Keep "any" but add a preprocessing stage that strips plain-text inputs out before dispatch&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Set tool_choice to "auto" so the model can respond directly when no tool is required&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Add more few-shot examples of plain-text inputs to the subagent's prompt&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: C. tool_choice "any" forces at least one tool call on every input, so plain-text snippets get needlessly routed through parse_pdf. "auto" lets the model choose to use no tool when none applies, which is exactly the flexibility this mixed-input role needs. Forcing parse_pdf (A) makes the wrong behavior mandatory. A preprocessing stage (B) addresses the symptom at the wrong layer and adds a component when a one-line config change suffices. Few-shot examples (D) cannot override the hard constraint that "any" mandates a tool call regardless.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 33 · D2 · Tool Design &amp;amp; MCP Integration
&lt;/h3&gt;

&lt;p&gt;Scenario: Multi-Agent Research System&lt;/p&gt;

&lt;p&gt;The web-search subagent is equipped with two tools, search_web and search_academic, each carrying a one-line description ("search the web" / "search academic sources"). It routes queries to the wrong tool in 31% of cases — for example, sending peer-reviewed-literature queries to search_web. What is the best first step?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Consolidate the two tools into a single unified search tool&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Add a separate router model that classifies each query and selects the tool programmatically&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Set a forced tool_choice so the subagent always uses one designated search tool&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Expand and differentiate both tool descriptions — specifying input formats, example queries, source coverage, and explicit boundaries for when to use each&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: D. Tool descriptions are the primary tool-selection mechanism, so the first move is to enrich them: state input formats, give example queries, spell out source coverage, and draw explicit boundaries between the two. Consolidating (A) destroys the useful distinction the two tools provide — here differentiating is the fix, not merging. A dedicated router model (B) over-engineers a problem that a description edit resolves. A static forced tool_choice (C) cannot work when the correct tool legitimately varies per query.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 34 · D5 · Context Management &amp;amp; Reliability
&lt;/h3&gt;

&lt;p&gt;Scenario: Multi-Agent Research System&lt;/p&gt;

&lt;p&gt;The synthesis subagent merges findings from the web-search and document-analysis subagents into a unified narrative but drops the record of which source each claim came from. The report-generation subagent can then no longer attach citations, and 40% of the report's sentences ship uncited. What is the best fix?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Have the report-generation subagent re-run web_search to find a supporting source for each sentence&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Add a citation post-processor that guesses the most likely source for each uncited sentence&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Preserve claim-to-source mappings through the synthesis step so each synthesized claim carries its originating source(s) into report generation&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Instruct synthesis to insert "(source)" placeholders that a human fills in after the report is generated&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: C. Provenance must be preserved through synthesis, not reconstructed afterward: each synthesized claim should carry its originating source(s) forward so report generation can cite the actual origin. Re-running web_search (A) invents new sources that may not be what the claim was actually based on. A guessing post-processor (B) fabricates provenance. Placeholder markers (D) defer the problem to humans and leave the report uncited by default. Carrying claim-source mappings end to end is the governing principle.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 35 · D5 · Context Management &amp;amp; Reliability
&lt;/h3&gt;

&lt;p&gt;Scenario: Multi-Agent Research System&lt;/p&gt;

&lt;p&gt;Two sources report different market-size figures for the same market — $4.2B (2021 data) versus $6.8B (2023 data). The synthesis subagent silently picks $6.8B and presents it as the single authoritative fact. Reviewers, seeing the discarded figure elsewhere, flag the report as unreliable. What is the best fix?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Always keep the larger or more recent figure and drop the other before synthesis&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Average the two figures into a single blended number&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Route every conflicting statistic to a human reviewer before synthesis can continue&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Annotate both statistics with their source and collection/publication date, so the temporal difference is surfaced rather than misread as a contradiction&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: D. These figures are not actually contradictory — they are different years — so the fix is to annotate both with their source and date, which requires the pipeline to preserve publication/collection dates precisely so temporal change is not mistaken for conflict. Silently keeping one figure, even the more recent one (A), or averaging them (B) destroys provenance and can be flatly wrong. Human routing (C) is unnecessary overhead when the discrepancy is fully explained by date; escalation should be reserved for genuinely ambiguous conflicts, not every differing number.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 36 · D5 · Context Management &amp;amp; Reliability
&lt;/h3&gt;

&lt;p&gt;Scenario: Multi-Agent Research System&lt;/p&gt;

&lt;p&gt;The coordinator passes each subagent's output forward verbatim: the web-search subagent's full scraped page bodies — roughly 38k tokens — flow straight into the synthesis subagent's prompt. Synthesis quality has dropped and it now misses findings buried in the middle of the passed-in content (lost-in-the-middle). What is the best fix?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Trim each subagent's output to its relevant extracted findings before passing it downstream, and lead the synthesis prompt with a key-facts summary under section headers&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Move synthesis to a larger-context model so all 38k tokens fit comfortably&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Keep passing the raw pages but add a prompt instruction telling synthesis to focus on the important parts&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Have synthesis re-fetch and re-summarize each page itself before combining them&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: A. Downstream context should be budgeted: trim verbose tool outputs to the relevant extracted fields before they accumulate, and counter lost-in-the-middle by placing the key summary first under clear section headers. A larger-context model (B) does not fix attention dilution — lost-in-the-middle persists at any window size when the prompt is full of low-signal bulk. A "focus on important parts" instruction (C) does not reduce the noise the model must still attend to. Re-fetching (D) duplicates work and re-introduces the same raw volume the trimming was meant to remove.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 37 · D1 · Agentic Architecture &amp;amp; Orchestration
&lt;/h3&gt;

&lt;p&gt;Scenario: Developer Productivity with Claude&lt;/p&gt;

&lt;p&gt;A coordinator agent is asked to produce a migration plan for a 400,000-line legacy billing monolith. The engineer hard-codes an upfront decomposition into five subagent tasks (billing, invoicing, tax, reporting, auth). Production runs show the final plan omits an entire 'dunning' subsystem and a currency-conversion layer that no subagent was ever assigned. What is the most effective decomposition strategy?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Raise the coordinator's maximum iteration count so it has more turns to notice the missing subsystems on its own&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Run an initial mapping pass that enumerates the repository's actual modules, then dynamically spawn one subagent per discovered component&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Expand the fixed list from five to twelve subagent tasks based on the engineer's prior knowledge of the billing domain&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Give the coordinator a larger context window so the entire monolith fits inside a single prompt&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: B. For unfamiliar or legacy code you cannot know the true structure in advance, so any static decomposition guarantees blind spots — exactly the omitted 'dunning' and currency layers. The governing pattern is mapping-then-adaptive decomposition: discover the real module structure first, then decompose against what actually exists, which is why B is correct. A is wrong because more iterations do not change the coverage of a fixed assignment list. C is still a static decomposition bounded by incomplete prior knowledge — the missing subsystems were unknown, so listing more names from memory does not surface them. D confuses context size with decomposition; the failure is missed coverage, not insufficient window.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 38 · D1 · Agentic Architecture &amp;amp; Orchestration
&lt;/h3&gt;

&lt;p&gt;Scenario: Developer Productivity with Claude&lt;/p&gt;

&lt;p&gt;The coordinator spawns a subagent with a single Task call to trace how a payment webhook propagates through the &lt;code&gt;orders&lt;/code&gt; module. Logs show the subagent re-runs Grep to rediscover the entrypoint the coordinator had already located two turns earlier, and reaches a conclusion that contradicts the coordinator's established finding. What is the root cause?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; The subagent's allowedTools list is missing the Task tool, so it cannot delegate the trace further&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; The Message Batches API strips prior turns, so the multi-turn context was silently dropped&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Subagents do not inherit the coordinator's conversation, so prior findings must be passed explicitly in the Task prompt&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; tool_choice was set to "auto" instead of forcing the Grep tool for the trace&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: C. In a hub-and-spoke design each subagent starts with a clean context and sees only what the coordinator writes into its Task prompt, so unless prior findings are passed in, the subagent rediscovers and can contradict them — making C correct. A is wrong because the Task tool governs spawning further subagents, not inheriting context. B is irrelevant: interactive delegation does not run through the Batches API. D is wrong because tool_choice controls which tool is invoked, not whether earlier conclusions are carried over.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 39 · D1 · Agentic Architecture &amp;amp; Orchestration
&lt;/h3&gt;

&lt;p&gt;Scenario: Developer Productivity with Claude&lt;/p&gt;

&lt;p&gt;An engineer runs &lt;code&gt;--resume&lt;/code&gt; on a named exploration session from three days ago to continue a refactor. Since then a teammate moved several files and renamed a package. The resumed agent replays cached Read and Grep results from the old history and proposes edits against paths that no longer exist. What is the best way to continue the work?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Use &lt;code&gt;--resume&lt;/code&gt; but manually enlarge the context window so more of the stale history is retained&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; &lt;code&gt;fork_session&lt;/code&gt; from the three-day-old session to branch the work onto a new line&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Re-run the same session with &lt;code&gt;/compact&lt;/code&gt; to shrink the stale tool outputs before editing&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Start a fresh session and inject a short summary of the prior conclusions, discarding the stale tool results&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: D. When prior tool results are stale, replaying them drives edits against a state of the world that no longer holds; the fix is to keep the conclusions but re-ground on the current tree — start fresh with an injected summary, so D is correct. A is wrong because retaining more stale history is the problem, not a lack of window. B still carries the stale Read/Grep results into the fork. C compacts (summarizes) the stale outputs rather than replacing them with fresh reads of the moved files, so the wrong paths persist.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 40 · D2 · Tool Design &amp;amp; MCP Integration
&lt;/h3&gt;

&lt;p&gt;Scenario: Developer Productivity with Claude&lt;/p&gt;

&lt;p&gt;A security scan flags that the team's committed .mcp.json contains a hardcoded GitHub personal-access token used by an MCP server. The team wants the server config to stay shared and version-controlled, but each engineer (and CI) must supply their own credential without leaking it. What should they do?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Encrypt the .mcp.json file and commit the encrypted blob, decrypting it at session start.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Reference the credential as ${GITHUB_TOKEN} in .mcp.json so each environment supplies its own value from the environment at load time.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Move the server to user-scoped ~/.claude.json so the token lives on a single trusted machine.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Store the token in the project CLAUDE.md so it loads with memory and is easy to rotate.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: B. MCP configuration supports ${ENV_VAR} expansion precisely so secrets stay out of version control while the config itself remains shared: writing ${GITHUB_TOKEN} lets each machine and CI job inject its own value at load time (B). Committing an encrypted blob (A) still ships a secret in the repo and adds key-management burden. Moving to user scope (C) sacrifices the requirement that the config stay shared and version-controlled. Putting the token in CLAUDE.md (D) commits the plaintext secret into VCS — the same leak, in a different file.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 41 · D2 · Tool Design &amp;amp; MCP Integration
&lt;/h3&gt;

&lt;p&gt;Scenario: Developer Productivity with Claude&lt;/p&gt;

&lt;p&gt;The agent connects to a database MCP server to help engineers query a legacy schema. Traces show that before every real query it fires 6-10 exploratory tool calls (list_tables, describe_table, sample_rows) to rediscover what exists, adding seconds of latency and thousands of tokens per session. The schema changes rarely. What is the best way to reduce this overhead?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Force tool_choice to "any" so the model commits to a productive tool sooner.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Raise the context window so more exploratory results fit without truncation.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Add a short-lived cache in front of the query tool so repeated probes return faster.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Expose the schema catalog as an MCP resource the agent reads once upfront, instead of probing with repeated exploratory tool calls.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: D. MCP resources exist to expose content catalogs (schemas, indexes, file listings) that the agent can read directly, eliminating the exploratory tool calls used just to discover what is available (D). Because the schema is stable, a resource is read once and reused. tool_choice "any" (A) forces a tool but does nothing to remove the discovery calls. A larger window (B) accommodates the bloat rather than preventing it. A cache (C) speeds up the redundant probes but still incurs the round trips and token cost — the discovery pattern itself is the problem.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 42 · D2 · Tool Design &amp;amp; MCP Integration
&lt;/h3&gt;

&lt;p&gt;Scenario: Developer Productivity with Claude&lt;/p&gt;

&lt;p&gt;The team added a code-intelligence MCP server that does semantic, cross-file symbol search over an indexed monorepo. In practice the agent almost always reaches for the built-in Grep tool instead, so it misses semantically related call sites that don't share literal text. The MCP tool's description currently reads only "Search the code index." What is the most effective first step?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Enrich the MCP tool's description with example queries and a note that it performs semantic, cross-file search that literal text matching misses, so the agent prefers it over Grep.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Remove Grep from the agent's allowedTools so the MCP search tool is the only option.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Add a CLAUDE.md rule instructing the agent to never use Grep for code search.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Force tool_choice to the MCP search tool on every turn.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: A. Tool descriptions are the primary tool-selection mechanism, so a thin description ("Search the code index") loses to a well-understood built-in like Grep; enriching it with example queries and an explicit boundary (semantic vs literal matching) makes the agent prefer it where appropriate (A). Removing Grep entirely (B) is over-scoped — literal search is still the right tool for many tasks. A CLAUDE.md prohibition (C) fights good behavior with a blanket rule instead of clarifying when each tool wins. Forcing tool_choice every turn (D) breaks legitimate Grep and Read use and does not teach correct selection.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 43 · D2 · Tool Design &amp;amp; MCP Integration
&lt;/h3&gt;

&lt;p&gt;Scenario: Developer Productivity with Claude&lt;/p&gt;

&lt;p&gt;The agent is asked to rename a shared constant throughout a generated client file. Its Edit call fails with a non-unique-match error because the target string appears on many lines that must all be updated consistently, and no single surrounding context makes one occurrence unique. What is the correct way to complete the change?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Switch to a Bash sed global substitution to bypass the Edit tool.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Retry the identical Edit call with tool_choice forced so it applies despite the ambiguity.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Delete the file and have the agent regenerate it from scratch with the new constant.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Read the whole file, then Write it back with the modification, since Edit requires a unique match and this change spans repeated occurrences.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: D. Edit performs a targeted replacement on a unique match; when the anchor is inherently non-unique and the change spans many identical occurrences, the correct fallback is to Read the full file and Write it back with the edits applied (D). A sed substitution (A) sidesteps the built-in file tools and forfeits the reviewability and safety of Read/Write. Forcing tool_choice (B) does not resolve the ambiguity — Edit still cannot pick which occurrence to change. Regenerating the file (C) risks discarding unrelated hand-made changes and is far more destructive than needed.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 44 · D3 · Claude Code Configuration &amp;amp; Workflows
&lt;/h3&gt;

&lt;p&gt;Scenario: Developer Productivity with Claude&lt;/p&gt;

&lt;p&gt;An engineer asks the tool to determine which of 30 services emits a particular Kafka event. Answering requires grepping and reading across dozens of files, producing thousands of lines of intermediate tool output. Done inline, this verbose discovery floods the main session's context and later steps degrade from lost-in-the-middle effects. What is the best approach?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Increase the main session's context window so all of the intermediate tool output fits&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Delegate the discovery to an Explore subagent that returns only the concise finding, keeping the verbose output out of the main context&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Turn on plan mode so the tool plans the searches before it runs them&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Disable Grep and instead Read each candidate file directly in the main session&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: B. Verbose, exploratory discovery belongs in an isolated Explore subagent that reports back only the concise answer, so the main context stays clean and free of attention dilution — making B correct. A is wrong because a larger window still accumulates and dilutes attention rather than preventing the flood. C plans the searches but does not reduce the volume of intermediate output they generate. D is slower and still pours every file's contents into the main context, worsening the problem.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 45 · D3 · Claude Code Configuration &amp;amp; Workflows
&lt;/h3&gt;

&lt;p&gt;Scenario: Developer Productivity with Claude&lt;/p&gt;

&lt;p&gt;A senior engineer added the team's coding-convention instructions to &lt;code&gt;~/.claude/CLAUDE.md&lt;/code&gt;, and the tool follows them perfectly on her machine. A newly onboarded teammate clones the same repository, but the tool ignores those conventions for him. What is the correct fix?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Move the conventions into the project's &lt;code&gt;.claude/CLAUDE.md&lt;/code&gt; (or root CLAUDE.md) checked into version control, since user-level memory is personal and not shared via VCS&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Add the conventions to the teammate's &lt;code&gt;~/.claude.json&lt;/code&gt; MCP configuration so they load for him&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Convert the conventions into a Skill so they load on demand whenever they are relevant&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Register the conventions in a &lt;code&gt;.claude/config.json&lt;/code&gt; commands array so they sync with the repository&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: A. &lt;code&gt;~/.claude/CLAUDE.md&lt;/code&gt; is user-scoped memory that never travels with the repository, which is exactly why a fresh clone does not pick it up; project-level CLAUDE.md is shared through VCS, so A is correct. B misuses MCP config, which defines tool servers rather than conventions and is itself personal. C makes the always-on conventions on-demand and still requires committing something to be shared. D references a &lt;code&gt;.claude/config.json&lt;/code&gt; commands array that does not exist.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 46 · D3 · Claude Code Configuration &amp;amp; Workflows
&lt;/h3&gt;

&lt;p&gt;Scenario: Claude Code for Continuous Integration&lt;/p&gt;

&lt;p&gt;A CI pipeline calls Claude Code to review each pull request, and a shell gate script greps stdout for the word "CRITICAL" to fail the build. The gate is flaky because Claude phrases severe findings differently each run — "severe issue," "major bug," "critical defect." What is the most effective fix?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Add a post-processing library with a synonym dictionary that normalizes severity wording before the grep runs&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Invoke Claude with --output-format json plus a --json-schema so findings are emitted as machine-parseable structured fields the gate reads deterministically&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Instruct the prompt to "always use the exact word CRITICAL for severe issues" and rely on that wording&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Set temperature to 0 so the phrasing of findings stays stable between runs&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: B. CI consumers must read structured, machine-parseable output rather than parsing free-form natural language. Running with --output-format json and a --json-schema forces findings into typed fields (severity, file, line) the gate can evaluate deterministically, so key B is correct. A builds a brittle synonym-normalization layer that will always lag new phrasings and is unneeded once output is structured. C is a prompt-only instruction that offers no deterministic guarantee — the model can still deviate. D misunderstands temperature: even at 0 the output is prose, not a stable schema, and the gate remains fragile.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 47 · D3 · Claude Code Configuration &amp;amp; Workflows
&lt;/h3&gt;

&lt;p&gt;Scenario: Claude Code for Continuous Integration&lt;/p&gt;

&lt;p&gt;In the pipeline, the same Claude Code session that generates an implementation also performs the automated review step. It consistently rates its own output clean, yet human reviewers keep finding real bugs it missed. What is the most effective change?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Run the review in a separate, independent Claude Code instance that shares no context with the generating session&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Enable extended thinking on the generating session before it reviews its own code&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Instruct the generating session to "review critically and assume the code contains bugs"&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Lower the pass threshold so more borderline findings surface in the same session&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: A. A session that produced code carries the same assumptions that caused its defects, so self-review systematically misses them. An independent instance with no shared context reviews the diff fresh and catches issues the author blind spots hid, so key A is correct. B adds reasoning depth but keeps the identical context and biases, so the blind spots persist. C is a prompt nudge that cannot overcome the structural problem of reviewing one's own reasoning. D changes sensitivity, not perspective — the same session still fails to see the bugs it never flagged.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 48 · D3 · Claude Code Configuration &amp;amp; Workflows
&lt;/h3&gt;

&lt;p&gt;Scenario: Claude Code for Continuous Integration&lt;/p&gt;

&lt;p&gt;Locally, the author's Claude generates tests that follow team conventions (naming, arrange-act-assert structure). In CI on a clean checkout, the generated tests ignore those conventions. The conventions are documented only in the author's ~/.claude/CLAUDE.md. What is the best fix?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Pass the conventions as a --system-prompt argument in the CI command on every run&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Add the conventions to the CI build server's ~/.claude/CLAUDE.md so the runner picks them up&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Reproduce the author's local machine environment inside the CI job&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Move the conventions into the project's committed .claude/CLAUDE.md so every checkout — local and CI — loads them from version control&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: D. User-level ~/.claude/CLAUDE.md is personal and not shared through VCS, which is exactly why it is absent on a clean CI checkout. Moving the conventions into the project's committed .claude/CLAUDE.md makes them load for every clone, in CI and for every teammate, so key D is correct. A wires the guidance into a fragile command flag that lives outside the repo and drifts. B stashes instructions on one build server, still unversioned and invisible to teammates. C is disproportionate and does not address the real cause — the instructions simply are not in the shared project scope.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 49 · D3 · Claude Code Configuration &amp;amp; Workflows
&lt;/h3&gt;

&lt;p&gt;Scenario: Claude Code for Continuous Integration&lt;/p&gt;

&lt;p&gt;The project's always-loaded CLAUDE.md has grown to embed a long, detailed code-review rubric. That rubric is needed only by the review job, but it is now injected into every CI job — test generation, doc updates — inflating token cost and diluting the model's focus on those tasks. What is the best fix?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Delete the rubric from CLAUDE.md and inline its full text into each review command invocation&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Split CLAUDE.md into two files and symlink the right one into place per job&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Keep the rubric in CLAUDE.md but wrap it in comment markers so it is visually separated&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Move the review rubric into an on-demand Skill under .claude/skills so only the review job loads it, keeping CLAUDE.md lean for always-on context&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: D. CLAUDE.md is always loaded, so anything job-specific placed there taxes every unrelated run. A Skill loads on demand only when its job invokes it, so moving the rubric into .claude/skills keeps it available to the review job while leaving CLAUDE.md lean for the rest — key D is correct. A loses versioned reuse and bloats the command definition. B is a fragile symlink hack that duplicates and desynchronizes the guidance. C changes formatting but the commented text is still loaded into context for every job, so the cost and dilution remain.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 50 · D4 · Prompt Engineering &amp;amp; Structured Output
&lt;/h3&gt;

&lt;p&gt;Scenario: Claude Code for Continuous Integration&lt;/p&gt;

&lt;p&gt;The CI reviewer posts naming-convention and style comments, but developers report that roughly 60% of these particular comments are wrong or trivial. They have started ignoring the bot entirely — including its genuine security findings. What is the most effective response?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Add a single global confidence threshold so only comments the model rates above 0.8 are posted&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Temporarily disable the naming-convention category, then rewrite its prompt with explicit categorical rules before re-enabling it&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Route every comment through a human moderation queue before it is posted to the pull request&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Train a sentiment classifier on developer reactions and suppress whichever categories draw negative responses&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: B. A single high-false-positive category erodes trust in the entire reviewer, so the governing principle is to remove the offending category from production and fix its prompt rather than paper over it. Temporarily disabling naming-convention comments stops the trust bleed immediately, and rewriting the category with explicit categorical criteria addresses the root cause before re-enabling it. A global confidence threshold (A) is the wrong instrument — self-reported confidence is poorly calibrated and would also suppress good findings from healthy categories. A human moderation queue (C) defeats the point of automation and does not fix the noisy prompt. A sentiment classifier (D) is heavy over-engineering that treats symptoms (developer annoyance) rather than the miscalibrated category definition.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 51 · D4 · Prompt Engineering &amp;amp; Structured Output
&lt;/h3&gt;

&lt;p&gt;Scenario: Claude Code for Continuous Integration&lt;/p&gt;

&lt;p&gt;The CI reviewer handles clear-cut defects well but is inconsistent on genuinely ambiguous cases — for example, whether a bare &lt;code&gt;except:&lt;/code&gt; is acceptable in a one-off script versus a shared library. Enumerating every possible scenario in the prompt is impractical. What is the best approach?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Add two to four few-shot examples of representative ambiguous cases, each with the reasoning behind its verdict&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Write an exhaustive list in CLAUDE.md of every code pattern and its correct verdict&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Fine-tune a dedicated model on the team's full history of past review decisions&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Lower the bar so every borderline case is flagged and pushed to a human reviewer&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: A. Few-shot examples are the right tool for ambiguous judgment because a small set of well-chosen cases with their reasoning lets the model generalize the underlying principle rather than memorize a lookup table. Two to four exemplars covering the representative ambiguous patterns teaches the script-versus-library distinction without enumerating everything. An exhaustive list (B) is the enumeration approach the stem already rejects as impractical and it still will not cover unseen cases. Fine-tuning (C) is disproportionate engineering for a prompt-level problem and is slow to iterate. Flagging every borderline case to humans (D) abandons the goal of useful automated feedback and floods reviewers with noise.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 52 · D4 · Prompt Engineering &amp;amp; Structured Output
&lt;/h3&gt;

&lt;p&gt;Scenario: Claude Code for Continuous Integration&lt;/p&gt;

&lt;p&gt;In the pipeline, Claude Code generates test cases and then, within the same session, reviews those tests for quality. The review almost never finds problems, yet audits show 20% of the generated tests have weak or tautological assertions. What explains this and what is the fix?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Enable extended thinking in the same session so the self-review reasons more carefully before approving&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Have the same session review the tests three times and take a majority vote on each&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Add more detailed quality criteria to the existing session's review prompt&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; A session reviewing its own output is biased toward confirming it, so have a separate, independent Claude instance review the generated tests&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: D. A session evaluating work it just produced is anchored to its own reasoning and tends to confirm rather than challenge it, which is why the self-review misses weak assertions the audit catches. An independent instance with no stake in the original generation reviews far more critically, so routing the tests to a fresh reviewer is the correct fix. Extended thinking (A) makes the same biased session think longer about the same conclusion; it does not remove the conflict of interest. Voting across three passes of the same session (B) compounds the shared bias instead of countering it. More detailed criteria (C) help at the margin but cannot overcome a reviewer predisposed to approve its own output.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 53 · D4 · Prompt Engineering &amp;amp; Structured Output
&lt;/h3&gt;

&lt;p&gt;Scenario: Structured Data Extraction&lt;/p&gt;

&lt;p&gt;Production logs show that 8% of extraction responses fail the downstream JSON.parse() step because Claude wraps its output in markdown code fences or prepends a sentence like "Here is the extracted data:". Prompt tweaks reduced but did not eliminate the failures. What is the most reliable fix?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Add a stern system-prompt line ("Return only valid JSON, no markdown, no commentary") and set temperature to 0 to make formatting deterministic.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Define the target structure as a tool input_schema, force that tool via tool_choice, and read the structured JSON directly from the tool_use block.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Post-process every response with a regex that strips markdown fences and leading prose before calling JSON.parse().&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Lower max_tokens so the model has no room to add explanatory prose around the JSON object.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: B. The governing principle is that tool_use with a JSON schema is the most reliable way to obtain structured output because the model returns a validated JSON object in the tool_use block, eliminating syntax and formatting errors entirely. Forcing the tool guarantees the structured path is taken. Prompt instructions plus temperature 0 (A) still rely on the model formatting free text and leave residual failures. Regex stripping (C) is a brittle patch that breaks on unexpected wrappers and does not address the root cause. Capping max_tokens (D) can truncate valid JSON and does not stop fence-wrapping.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 54 · D4 · Prompt Engineering &amp;amp; Structured Output
&lt;/h3&gt;

&lt;p&gt;Scenario: Structured Data Extraction&lt;/p&gt;

&lt;p&gt;After migrating to tool_use with a strict JSON schema, JSON parse failures dropped to zero. However, reconciliation shows 6% of extracted invoices have line-item amounts that do not sum to the stated invoice total. What does this reveal about schema-based structured output?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; The schema is misconfigured; adding "additionalProperties": false will force the line items to reconcile with the total.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; tool_choice should be "any" rather than forced so the model can self-correct the arithmetic before returning.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Schema enforcement removes syntax and type errors but not semantic errors, so a downstream arithmetic validation check must flag or trigger a retry on mismatches.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; max_tokens is too low, so the model omits line items; raising it will make the totals add up.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: C. The principle is that a JSON schema guarantees structural and type validity but cannot enforce semantic correctness such as line items summing to a total. The correct response is a downstream business-rule validation step that recomputes the sum and flags or retries mismatches. Option A confuses a structural constraint (additionalProperties) with an arithmetic invariant it cannot express. Option B misuses tool_choice, which controls tool selection, not numeric correctness. Option D misattributes a semantic discrepancy to a token limit; truncation would cause parse or completeness failures, not silent miscalculation.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 55 · D4 · Prompt Engineering &amp;amp; Structured Output
&lt;/h3&gt;

&lt;p&gt;Scenario: Structured Data Extraction&lt;/p&gt;

&lt;p&gt;For documents that lack a purchase-order number, audit logs show Claude inventing plausible-looking PO numbers about 11% of the time instead of signaling absence. Downstream systems then treat the fabricated values as real. What is the best schema-level fix?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Add a required confidence float to every field so downstream code can discard PO numbers below a threshold.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Switch tool_choice to "auto" so the model can decline to call the extraction tool when the PO number is missing.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Add a few-shot example containing a correctly formatted PO number to anchor the expected pattern.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Make the po_number field nullable in the schema and instruct the model to return null when the value is not present in the source.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: D. The principle is that making the field nullable gives the model a sanctioned way to represent absence, so it returns null instead of fabricating a value to satisfy a required field. The key can stay required while its value may be null; an optional field would also stop the fabrication, but the key would then be omitted rather than set to null. Pairing this with an explicit instruction to emit null when the data is absent directly addresses the hallucination. A confidence score (A) is poorly calibrated and does not stop the fabrication, only tries to filter it afterward. Setting tool_choice to "auto" (B) risks skipping extraction entirely for otherwise-complete documents. A few-shot example of a valid PO (C) reinforces the format but gives no path for the missing case, which is the actual problem.

&lt;p&gt;Worked through step by step above&lt;br&gt;
&lt;/p&gt;

&lt;br&gt;
&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 56 · D4 · Prompt Engineering &amp;amp; Structured Output
&lt;/h3&gt;

&lt;p&gt;Scenario: Structured Data Extraction&lt;/p&gt;

&lt;p&gt;The extraction schema uses an enum with six known values for document_type. A new vendor begins sending "credit memo" documents, which are not in the enum, and the model force-fits them to "invoice", corrupting downstream routing. What is the best design change?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Add an "other" value to the enum plus an optional document_type_detail string so novel types are captured verbatim instead of misclassified.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Remove the enum constraint entirely and let the model emit any free-text string for document_type.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Fine-tune a dedicated document-type classifier and run it before extraction to assign the type.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Brainstorm and add every conceivable document type to the enum so no future document falls outside it.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: A. The principle is designing enums for extensibility with an "other" (or "unclear") escape value plus a free-text detail field, so unanticipated categories are surfaced rather than silently forced into a wrong bucket. This keeps the controlled vocabulary intact while capturing novel cases for review. Dropping the enum (B) sacrifices the validation and normalization that make the field useful downstream. A fine-tuned classifier (C) is heavy over-engineering when a schema change solves the problem. Enumerating every possible type (D) is impossible to complete and still breaks on the next truly novel document.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 57 · D4 · Prompt Engineering &amp;amp; Structured Output
&lt;/h3&gt;

&lt;p&gt;Scenario: Structured Data Extraction&lt;/p&gt;

&lt;p&gt;Extractions pass the JSON schema but sometimes violate a business rule (invoice_date later than due_date). The current recovery discards the result and re-runs the identical prompt from scratch; 40% of retries still fail the rule. What is the most effective retry strategy?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Re-run the identical prompt up to five times and accept the first result that happens to pass the rule.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Append the previous failed extraction and the specific validation error messages to a follow-up message, asking the model to correct only the offending fields.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Lower temperature to 0 and re-run the original prompt exactly once.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Move all rule violations to the Message Batches API and reprocess them overnight.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: B. The validation-retry principle is to feed the model its own failed output plus the exact validation errors, so it can correct the specific fields rather than re-deriving everything blindly. This targeted feedback is far more likely to resolve a rule violation than repetition. Re-running the same prompt (A, C) provides no new information and mostly reproduces the same error. The Batch API (D) is for high-volume latency-tolerant work, not a corrective loop, and it does nothing to give the model the failure context it needs to fix the fields.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 58 · D4 · Prompt Engineering &amp;amp; Structured Output
&lt;/h3&gt;

&lt;p&gt;Scenario: Structured Data Extraction&lt;/p&gt;

&lt;p&gt;For scanned documents where the tax ID is physically cut off at the page edge and simply absent from the source text, the validation-retry loop retries five times and still fails, burning tokens on every attempt. What is the correct understanding of this situation?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Raise the retry ceiling to ten; with enough attempts the model will eventually recover the tax ID.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Enable extended thinking so the model reasons more deeply and reconstructs the missing tax ID.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Retries can fix format and structural errors but cannot recover information absent from the source, so detect the absence, return null, and route the document to human review instead of retrying.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Switch tool_choice to "any" so the model can select a different extraction tool that will locate the tax ID.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: C. The principle is that a validation-retry loop repairs formatting and structural mistakes but cannot conjure information that is not present in the source document. Recognizing absence, returning null, and escalating to a human is the correct terminal behavior, and it stops the wasteful token spend. Increasing retries (A) cannot recover data that was never in the input. Extended thinking (B) improves reasoning over available evidence but cannot invent an absent value without fabricating it. Changing tool_choice (D) alters which tool runs, not what text exists on the truncated page.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 59 · D4 · Prompt Engineering &amp;amp; Structured Output
&lt;/h3&gt;

&lt;p&gt;Scenario: Structured Data Extraction&lt;/p&gt;

&lt;p&gt;A 50,000-document extraction batch completes and 300 requests returned errors. The team needs to reprocess only the failures and reliably match every result back to its source document. What is the correct approach?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Assign each request a custom_id mapping to its source document, correlate results by custom_id, and submit a new batch containing only the 300 failed custom_ids.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; Resubmit the entire 50,000-document batch, since batches are atomic and cannot be reprocessed partially.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Rely on result ordering, since the Batch API guarantees that output order matches input order.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Convert the 300 failures to synchronous calls and use response timestamps to align each result with its source document.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: A. The principle is that each Batch API request carries a caller-supplied custom_id, which is the correlation key for matching results back to sources and for selectively resubmitting only the failed items. Reprocessing the 300 failed custom_ids in a fresh batch is both cheap and precise. Resubmitting all 50,000 (B) wastes cost and time; batches are not atomic in that sense. Result ordering is not guaranteed (C), which is exactly why custom_id exists. Timestamp matching (D) is fragile and unnecessary when a deterministic custom_id mapping is available.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Question 60 · D4 · Prompt Engineering &amp;amp; Structured Output
&lt;/h3&gt;

&lt;p&gt;Scenario: Structured Data Extraction&lt;/p&gt;

&lt;p&gt;The pipeline handles six document types, each with its own extraction tool. Even when the type is already known from the intake folder that routed the document, the caller still sends all six tools with tool_choice "auto", and in 9% of cases the model invokes the wrong extractor. What is the best fix?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A.&lt;/strong&gt; Keep tool_choice "auto" but add sentiment analysis of the document to help the model decide which extractor fits.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B.&lt;/strong&gt; When the document type is already known from routing, force the specific extractor with tool_choice {"type": "tool", "name": "extract_invoice"} rather than leaving selection to "auto".&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;C.&lt;/strong&gt; Set tool_choice to "any" so the model is at least required to call some extraction tool.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;D.&lt;/strong&gt; Merge all six extraction tools into one large tool with a combined schema covering every document type.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;/p&gt;
  Show the answer
  &lt;br&gt;
Correct answer: B. The principle is that when the correct tool is already determined by upstream logic, you should force it with tool_choice {"type":"tool","name":...} instead of asking the model to re-decide, which removes the selection error entirely. "auto" (A) reintroduces the very choice that is failing, and sentiment analysis is irrelevant over-engineering. "any" (C) still lets the model pick the wrong extractor among the six. Merging into one mega-tool (D) creates a bloated schema and does not use the routing information you already have, likely degrading extraction quality.&lt;br&gt;


&lt;p&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  How to read your score
&lt;/h2&gt;

&lt;p&gt;The mock reports a score from 100 to 1000, the same range as the official exam, but it uses my own formula: 100 + 900 × the fraction answered correctly. With 60 questions, each correct answer adds 15 points. The threshold is 720, which takes 42 correct answers. It is a reference within this tool. It is neither Anthropic’s scoring formula nor a validated prediction of passing the official exam.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Correct answers&lt;/th&gt;
&lt;th&gt;Mock score&lt;/th&gt;
&lt;th&gt;What to do next&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;0 to 26&lt;/td&gt;
&lt;td&gt;100 to 490&lt;/td&gt;
&lt;td&gt;Redo the set in study mode and read every explanation.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;27 to 34&lt;/td&gt;
&lt;td&gt;505 to 610&lt;/td&gt;
&lt;td&gt;Go back to the documentation for your two weakest domains.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;35 to 41&lt;/td&gt;
&lt;td&gt;625 to 715&lt;/td&gt;
&lt;td&gt;Just under 720. For each miss, name the requirement your choice ignored.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;42 to 49&lt;/td&gt;
&lt;td&gt;730 to 835&lt;/td&gt;
&lt;td&gt;At or above 720 in this tool. Explain why each wrong option is wrong.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;50 to 60&lt;/td&gt;
&lt;td&gt;850 to 1000&lt;/td&gt;
&lt;td&gt;After several attempts you may be recalling the key. Change one assumption in a scenario and explain the answer again.&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h2&gt;
  
  
  Study mode and exam mode
&lt;/h2&gt;

&lt;p&gt;For a timed run, open the &lt;a href="https://szymonpaluch.com/claude-certified-architect-practice-exam" rel="noopener noreferrer"&gt;free 60-question CCAR-F practice exam&lt;/a&gt;. Study mode shows the explanation after “Check answer” and pauses the timer while you read. Exam mode gives you 120 minutes, with flagging and answer changes until submission. The tool’s questions are in English.&lt;/p&gt;

&lt;p&gt;Results show your correct-answer count, the score from the table above and a breakdown across five domains.&lt;/p&gt;

&lt;p&gt;Progress saves locally in this browser, so you can close the tab and resume later. The test page also explains how to practise in chat through MCP. You do not need an account or an email address to answer the questions.&lt;/p&gt;

&lt;p&gt;If you would rather work on paper, the same 60 questions are in &lt;a href="https://szymonpaluch.com/downloads/claude-certified-architect-foundations-practice-questions.pdf" rel="noopener noreferrer"&gt;a free PDF with the answer key at the end&lt;/a&gt;. The key explains every option, as the online test does. You can download it without an email address.&lt;/p&gt;

&lt;h2&gt;
  
  
  How I would use it
&lt;/h2&gt;

&lt;p&gt;On your next attempt, focus on what you can explain without the answer key:&lt;/p&gt;

&lt;h3&gt;
  
  
  PREP PLAN
&lt;/h3&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;First pass in study mode.&lt;/strong&gt; No time pressure, question by question. Read every explanation, even when you got it right. The goal is understanding why the distractors are wrong.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;A full sitting in exam mode.&lt;/strong&gt; One session, 120 minutes, no breaks. Practise working within a time limit.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Study the domain breakdown.&lt;/strong&gt; The per-domain score tells you where to read up. Go back to the docs for your two weakest areas.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Explain the decision without the key.&lt;/strong&gt; A high score after several attempts may reflect memorized answers. Change one assumption in the scenario and explain whether the same answer still fits.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;If you struggle with the responsibilities of an agent and its tools, read my &lt;a href="https://szymonpaluch.com/blog/posts/agentic-design-rules" rel="noopener noreferrer"&gt;12 rules for building agents&lt;/a&gt;. I wrote them while preparing for the same exam; they expand on topics from D1 and D2.&lt;/p&gt;

&lt;p&gt;Check booking eligibility separately from your mock score. I cover the current account requirements and &lt;a href="https://szymonpaluch.com/blog/posts/claude-certification-without-partner" rel="noopener noreferrer"&gt;exam access without a partner employer&lt;/a&gt; in a separate article. A high practice score does not change the organizer’s requirements.&lt;/p&gt;

&lt;p&gt;Have you identified a weak area? The &lt;a href="https://szymonpaluch.com/blog/posts/claude-certified-architect-certification-guide" rel="noopener noreferrer"&gt;Foundations study plan&lt;/a&gt; maps each domain to resources and exercises. Return to the questions after doing an exercise and check whether you can reject the wrong options without looking at the key.&lt;/p&gt;

&lt;p&gt;Ready for 60 questions?&lt;/p&gt;

&lt;p&gt;Free, in the browser, no signup. Progress is saved locally.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://szymonpaluch.com/claude-certified-architect-practice-exam" rel="noopener noreferrer"&gt;Take the practice exam →&lt;/a&gt; &lt;a href="https://szymonpaluch.com/downloads/claude-certified-architect-foundations-practice-questions.pdf" rel="noopener noreferrer"&gt;Download the PDF&lt;/a&gt; &lt;a href="https://github.com/hculap/claude-certified-architect-practice-exam" rel="noopener noreferrer"&gt;Code on GitHub&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Frequently asked questions
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Are these real exam questions?
&lt;/h3&gt;

&lt;p&gt;No. I wrote all 60 questions myself from Anthropic’s public exam guide, and none is taken from the exam. Each has one correct answer, while the official exam also has multiple-response items. The set is not affiliated with or endorsed by Anthropic.&lt;/p&gt;

&lt;h3&gt;
  
  
  Is there a PDF?
&lt;/h3&gt;

&lt;p&gt;Yes. The same 60 questions are in a free PDF with the answer key at the end, and the key explains every option. The download link is on this page, and you do not need an email address.&lt;/p&gt;

&lt;h3&gt;
  
  
  What score should I aim for before booking?
&lt;/h3&gt;

&lt;p&gt;The official exam’s passing score is 720 on a 100 to 1000 scale. This mock uses 720 as its reference too, which takes 42 of 60 correct answers, but the formula is mine, not Anthropic’s, so no mock score predicts your result. Before booking, make sure you can explain why each wrong option is wrong in your two weakest domains.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Important disclaimer.&lt;/strong&gt; This is unofficial study material. It is not affiliated with, authorized or endorsed by Anthropic. I wrote all 60 questions myself, modeled solely on the publicly available exam guide. Claude and Claude Certified Architect are names owned by Anthropic.&lt;/p&gt;




&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;No partner employer?&lt;/strong&gt; Anthropic admits to its exams only people from Claude Partner Network organizations. Join our collective of AI consultants and sit all four exams through our partner organization. &lt;a href="https://szymonpaluch.com/certified-ai-consultant-program" rel="noopener noreferrer"&gt;See the program →&lt;/a&gt; · &lt;a href="https://szymonpaluch.com/claude-certified-architect-practice-exam" rel="noopener noreferrer"&gt;Free practice exam →&lt;/a&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;&lt;em&gt;Originally published at &lt;a href="https://szymonpaluch.com/blog/posts/claude-certified-architect-practice-questions" rel="noopener noreferrer"&gt;szymonpaluch.com&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>claude</category>
      <category>ai</category>
      <category>certification</category>
      <category>career</category>
    </item>
    <item>
      <title>Mutation testing: can you trust AI-generated tests?</title>
      <dc:creator>Szymon Paluch</dc:creator>
      <pubDate>Wed, 14 Oct 2026 07:30:03 +0000</pubDate>
      <link>https://dev.to/szymonpaluch/mutation-testing-can-you-trust-ai-generated-tests-jcg</link>
      <guid>https://dev.to/szymonpaluch/mutation-testing-can-you-trust-ai-generated-tests-jcg</guid>
      <description>&lt;p&gt;Mutation testing breaks your code in one place and checks whether any test noticed. Code coverage measures execution. Mutation score measures assertion. Since a model started writing the tests, that stopped being an academic distinction.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fzmr8b5iawu05mhgbiox6.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fzmr8b5iawu05mhgbiox6.webp" alt="Mutation testing: an original line of code with three mutants below it, two marked killed and one marked survived, next to a green test suite" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Coverage lost its meaning the moment it became free. An agent adds two hundred cases in a minute. Coverage jumps to ninety percent, CI goes green. None of those three things answers the only question worth asking: if this code were wrong, would any assertion break. Mutation testing answers it mechanically. It is the one measurement of test quality that did not get cheaper when writing tests did.&lt;/p&gt;

&lt;h2&gt;
  
  
  How mutation testing works
&lt;/h2&gt;

&lt;p&gt;Mutation testing is a technique for judging the quality of a test suite: a tool makes one small, deliberate change to production code and checks whether any test catches it. You take one production file and change one thing in it. Greater-or-equal becomes greater. A conjunction becomes a disjunction. A condition becomes true. A loop step runs the other way. A whole branch disappears. The result is a &lt;strong&gt;mutant&lt;/strong&gt;. You run the tests that were supposed to guard that line.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;A test failed, so the mutant is &lt;strong&gt;killed&lt;/strong&gt;. Your tests really do enforce that decision.&lt;/li&gt;
&lt;li&gt;Everything passed green with broken code, so the mutant &lt;strong&gt;survived&lt;/strong&gt;. That is a finding. You have a line your tests execute and say nothing about.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Repeat a few hundred times per file. The &lt;strong&gt;mutation score&lt;/strong&gt; is the share of killed mutants among the ones that could be judged at all.&lt;/p&gt;

&lt;p&gt;The whole idea fits in one sentence. A test checks your code. A mutation test checks your test.&lt;/p&gt;

&lt;p&gt;The idea is half a century old and rests on two assumptions. Both are worth knowing before you believe any number. First: programmers write nearly correct code, so real bugs are small deviations from a good version rather than random noise. Second: a suite that catches those small, single-point faults also catches the compound ones. What is new is only that the technique finally pays off, and the reason has nothing to do with the technique.&lt;/p&gt;

&lt;h2&gt;
  
  
  What coverage will not tell you
&lt;/h2&gt;

&lt;p&gt;Coverage tells you whether a line ran. Mutation tells you whether anyone asserts anything about it. Two different quantities, and only the second one matters.&lt;/p&gt;

&lt;p&gt;A test that calls a function and never checks the result gives you one hundred percent coverage. A test that stubs every dependency and asserts on the stub gives you one hundred percent coverage. A test that recorded the current output as the expected one gives you one hundred percent coverage. That last one stays green when the output is wrong.&lt;/p&gt;

&lt;p&gt;This is not a niche observation. In April 2026 Thoughtworks put mutation testing into the Trial ring of its Technology Radar for the first time. The stated reason is exactly this: high coverage percentages can mask logically hollow tests or generated code that has never been meaningfully asserted.&lt;/p&gt;

&lt;h2&gt;
  
  
  AI in software testing: what changes when a model writes the tests
&lt;/h2&gt;

&lt;p&gt;Four things at once, and only the first is obvious.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Writing tests got cheap, judging them did not.&lt;/strong&gt; Two hundred generated cases land in a pull request. The same person reads them as before, only now there is ten times more to read. So they stop reading. Mutation score keeps pace with how fast tests now appear, because a machine computes it.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The failure mode of a model-written test is precisely the one mutation catches.&lt;/strong&gt; A model writing a test against finished code has that code in context. The cheapest route to green runs through a test that repeats the implementation. It stubs the dependencies and asserts on the stub. It copies the current result into the expected one. It checks that something got called instead of checking what came out. Such a test executes, passes and raises coverage. It would also pass with the condition inverted. That is the definition of a mutant that survived.&lt;/p&gt;

&lt;p&gt;This is not an opinion. A team in Luxembourg, the same group that has worked on mutation for years, went through twenty-four projects. Models write assertions that describe actual behaviour rather than expected behaviour. Random generators had the same problem before them. When the model judges for itself whether an assertion is right, it lands somewhere in the forties. On buggy code it falls another eight to nine points. So the model is strong as an author and weak as a reviewer, which is the same trap behind &lt;a href="https://szymonpaluch.com/blog/posts/agent-self-evaluation-trap" rel="noopener noreferrer"&gt;why a model cannot be its own reviewer&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;That matters, because we hand reviews to agents more and more often. Since DevDay 2026 Codex can take a first pass at code review in the cloud before you open a pull request. I described &lt;a href="https://szymonpaluch.com/blog/posts/openai-codex-cloud" rel="noopener noreferrer"&gt;what a team sets up before an agent works in a repository unsupervised&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;You can see it in what agents actually commit. A study from this year went through more than 1.2 million commits from 2025 across more than two thousand repositories. Of the agent commits that touch test files at all, thirty-six percent add mocks. For the same kind of commit written without an agent, twenty-six. The authors say it plainly: put your mocking rules in the agent's config file.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;When an agent can edit the tests, the tests stop being an independent measure.&lt;/strong&gt; The task reads: make CI green. Weakening an assertion satisfies that task and is a legal move. A benchmark published late last year measured which route models take to a green run. Editing the test file itself turned out to be rare: under one percent on well-specified problems, and zero for all three agents on the ones whose oracle was arbitrary. One of the three did delete the test file outright, in three percent of the well-specified runs. Something else rises instead. Satisfying the visible test cases without solving the problem jumps from about two percent to twenty-two, and to forty-four for the worst agent. That is nine problems and the authors call the sample noisy, but the direction is not in doubt. A different route, the same lesson: a check the agent can see is a check it can satisfy without solving anything. So it is worth settling in advance &lt;a href="https://szymonpaluch.com/blog/posts/bounded-autonomy" rel="noopener noreferrer"&gt;whether the test file belongs to what an agent changes without asking&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Mutation score has a property here that coverage lacks. Delete an assertion and coverage stays exactly where it was. Mutation score drops. A measure has to sit beyond the reach of whatever it measures, which is the same reason to iterate an agent only against &lt;a href="https://szymonpaluch.com/blog/posts/agentic-ratchet-immutable-yardstick" rel="noopener noreferrer"&gt;a measure it cannot move itself&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;A growing share of your code exists to police the model.&lt;/strong&gt; Trust rules, response validation, parsing, fallbacks, retries, limits. That code runs on the rare path. Tests get written for the common one. So it is exactly the part of the system a green suite says nothing about.&lt;/p&gt;

&lt;h2&gt;
  
  
  What it found in production code
&lt;/h2&gt;

&lt;p&gt;That is as much context as I can give. A production B2B application, where a model extracts values out of inbound documents and the code around it decides whether to trust them. The suite: over a thousand examples, zero failures, green for a long time.&lt;/p&gt;

&lt;p&gt;I wrote the harness from scratch. The off-the-shelf one failed on three independent counts at once. It needs a newer version of the language than the app runs on. It bumps a parser library shared with the linter, so a testing tool would cost us a diff across every lint rule. And it is commercially licensed for private repositories. Sixteen operators, chosen for the bug classes that cost the most in this kind of code. Boundaries and off-by-one. Swallowed exceptions. In-place mutation of shared state.&lt;/p&gt;

&lt;p&gt;First audit of one file: &lt;strong&gt;486 mutants. 235 killed, 204 survived, 44 on lines no selected test executes.&lt;/strong&gt; I am holding back the percentage, because the percentage is not the finding. Three things came out of it, and only one of them reads "write a test".&lt;/p&gt;

&lt;h3&gt;
  
  
  A real bug
&lt;/h3&gt;

&lt;p&gt;One function decides which value to accept: the one from the model, or the one extracted from the document. It had two trust rules and both looked only at the shape and length of the text. Neither asked whether that text had anything to do with the place it supposedly came from. A short, well-formed, entirely invented value won. Nothing downstream re-checked it. The neighbouring test avoided that path only because its two strings happened to be the same length.&lt;/p&gt;

&lt;p&gt;That is a finding about the design, not about the tests. The trust rule checked the format of a value and never its provenance, and those are two different things. The guard existed, somebody wrote it, and it passed review. Nobody ever checked it, which is exactly what is meant by &lt;a href="https://szymonpaluch.com/blog/posts/silent-failures-agentic-systems" rel="noopener noreferrer"&gt;a failure that lights no lamp&lt;/a&gt;. The fix holds to one rule. If a value shares no token with the source text our own extraction reconstructs, it cannot have come from that source. I checked the rule against the full historical set, not a sample.&lt;/p&gt;

&lt;h3&gt;
  
  
  Dead code
&lt;/h3&gt;

&lt;p&gt;Across the whole first audit, no longer just that one file, 106 mutants sat on lines no test executes. All of them in four methods. None of those four methods has a caller anywhere in the repository. They arrived together in a single import and nobody touched them since. Result: 214 deleted lines and one decision instead of 106 tickets.&lt;/p&gt;

&lt;h3&gt;
  
  
  Fresh code with fresh tests
&lt;/h3&gt;

&lt;p&gt;The third one is the most uncomfortable. I ran the harness on its own operators file. Hours earlier that file had gained four new operators and 101 lines of new tests, in the same commit. &lt;strong&gt;97 mutants, 87 killed, 90.6 percent. Five of the nine survivors sat in code written that same day, alongside those tests.&lt;/strong&gt; One exposed a dead null guard where null cannot occur. Another hit a condition no example ever reached.&lt;/p&gt;

&lt;p&gt;Ninety percent is a good score. Nine unguarded decisions in a file that had just passed full review and received a fresh set of tests is also a fact.&lt;/p&gt;

&lt;h2&gt;
  
  
  The mutation score lies without an "I don't know" bucket
&lt;/h2&gt;

&lt;p&gt;This is where measurement parts ways with a dashboard. This is the part that gets skipped most often.&lt;/p&gt;

&lt;p&gt;My harness distinguishes eight outcomes per mutant. Only two of them say anything about your tests: killed and survived. The other six are the harness admitting it has no verdict. The mutant hung. The mutant blew up the process. No selected test executes that line. The file cannot be audited at all. The mutant did not parse. The child process died before the harness installed the mutant.&lt;/p&gt;

&lt;p&gt;Put those six in the denominator and you get a higher, useless number, which is one more &lt;a href="https://szymonpaluch.com/blog/posts/llm-confidence-score-trap" rel="noopener noreferrer"&gt;figure that looks like evidence and is not&lt;/a&gt;. Separate them and you get something you can work with. On one file, 47 of 54 entries that looked like survivors turned out to be lines nobody executes, in three methods nobody calls. Glued together they read as 54 tickets and a score of 27 percent. Separated they read as three findings and a score of 74 percent of what the tests actually reach. The second version is true. The first sounds scarier.&lt;/p&gt;

&lt;p&gt;The same split separates two different jobs. A surviving mutant is a test that executes something and stays silent about it, so the fix is a stronger assertion in a test that already exists. A mutant on an uncovered line means there is no test, so the fix is a test that does not exist yet. Two different jobs, two different budgets.&lt;/p&gt;

&lt;p&gt;And one thing without which the rest is guesswork. Before you count anything, run the tests against the unmutated file. If that is not green, no number for that file means anything. Here, that step caught a file that resets its own mutex when reloaded. Without the check the report would have shown a wall of killed mutants and a beautiful score. A reload side effect would have killed every one of them, and no test would have killed any. The same shape of defect turns up anywhere you measure with a model: &lt;a href="https://szymonpaluch.com/blog/posts/non-generative-ai-vs-llm" rel="noopener noreferrer"&gt;a question whose answer is always yes returns a number indistinguishable from a measurement&lt;/a&gt;, and the report built on it looks clean.&lt;/p&gt;

&lt;p&gt;When a model stands on both sides, that stops being hygiene. It becomes the thing every number after it rests on. An agent that writes the code, the tests and the mutants has closed the loop around itself. The measure survives that arrangement only if it can say "I don't know". And only if it refuses to credit a defence it cannot prove.&lt;/p&gt;

&lt;h2&gt;
  
  
  How to run mutation testing without killing CI
&lt;/h2&gt;

&lt;p&gt;Mutation is expensive by construction, because one mutant means one test run. A few hundred mutants per file times your suite's startup time turns a coffee break into an overnight job. Google has been doing this for over a decade and published the bill. Almost 17 million mutants generated across 776,000 changes. 2.1 million of them shown to humans in code review, across more than 24,000 developers.&lt;/p&gt;

&lt;p&gt;The most important number in that bill has nothing to do with performance. &lt;strong&gt;At the start, developers judged 85 percent of the mutants shown to them as useless.&lt;/strong&gt; It took six years of collected feedback and rules that suppress whole classes of mutants to lift the productive share from 15 to 89 percent. The median number of mutants per change fell from 820 to seven along the way.&lt;/p&gt;

&lt;p&gt;The conclusion is inconvenient for anyone who just wants to switch a tool on. A raw audit produces mostly noise, and that noise costs the time of the most expensive people in the company. Four things bring the cost back to something sane.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Mutate the diff, not the repository.&lt;/strong&gt; The only mode that survives contact with a team is "only the lines this branch changed". The agent version is simpler: mutate what the model just wrote. Google measured what that costs: on medium and large changes, commit-level mutation slows review no more than computing coverage does, and a visible overhead shows up only on the smallest changes.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Run only the tests that could kill.&lt;/strong&gt; A mutant on a line a given test file never executes is wasted budget. Line-level coverage tells you which tests stand a chance.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Start the process once.&lt;/strong&gt; A fresh suite start per mutant costs me 1.98 seconds for framework-free tests and 17.24 seconds with a full application boot. Of those 17.24, fully 16.15 is loading that does not depend on which mutant is about to run. A fork of an already-booted session costs about half a second.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Pick the files by hand.&lt;/strong&gt; I declare the list of audited files and the list of tests meant to be their coverage explicitly. Naming conventions do not work: in a real codebase some tests are named after a scenario rather than a class, and one will cover a file no naming rule would ever associate with it.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The tooling is ready and mature. Java has PIT. JavaScript and TypeScript have Stryker, C# has Stryker.NET. PHP has Infection. Python has mutmut and cosmic-ray. Rust has cargo-mutants. Ruby has the mutant gem. Before you pick one, check the two things I tripped over: the minimum language version and the licence for a private repository.&lt;/p&gt;

&lt;h2&gt;
  
  
  What mutation testing will not catch
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;It will not replace tests you never wrote.&lt;/strong&gt; At twenty percent coverage, mutation tells you what you already know, only at a higher price. It is an instrument for judging tests that exist.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;It does not reach one hundred percent, and that is not the goal.&lt;/strong&gt; Some survivors are not findings at all. The mutated code is genuinely equivalent to the original and no test could tell them apart. Two shapes recur. A comparison whose operands are already an epsilon apart. A redundant early return whose fall-through computes the same answer. Equivalence is undecidable in general, so a nonzero survivor count is normal even with excellent tests. Google’s team put it plainly in an earlier paper: full mutation adequacy is neither practical nor desirable.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Mutation score is a terrible target.&lt;/strong&gt; Set a threshold in CI and you get exactly what you asked for and nothing else: tests written to kill mutants. Treat the number as a thermometer. The work is in the survivor list read by method, not in a bar on a dashboard.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Not every "defended" means defended.&lt;/strong&gt; In my harness a mutant that hangs or blows up the process counts as defended. Broken code that hangs or explodes will not reach production either. But neither outcome proves that any assertion stated anything. A file with many of those has an inflated score and the report has to show it.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Not everything can be measured.&lt;/strong&gt; My harness refuses ActiveRecord models by name. It installs a mutant by re-evaluating the whole file, and on a model that does not replace the definitions, it appends them. Validations go 9, then 18, then 27. The unmutated baseline run doubles the same way, so it stays green. Then a duplicated validation message kills every mutant, and no test kills any. A wall of false kills and a great score. Refusing is the only honest answer there.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;And the most important limit, the one the AI pieces leave out.&lt;/strong&gt; Mutation is a unit-level instrument. The most common failure in agent-written code does not sit in one line. It sits in two correct pieces that do not compose. Mutation will not see that, and is not meant to.&lt;/p&gt;

&lt;h2&gt;
  
  
  Where to start
&lt;/h2&gt;

&lt;p&gt;Take one file. Preferably the one you are afraid to touch, and the one holding your trust rules for the model. Run mutation on it. Ignore the percentage and read the survivor list grouped by method. The densest group is one story and one test to write, not twenty tickets.&lt;/p&gt;

&lt;p&gt;Then wire it into the diff. Exactly where the agent hands work back.&lt;/p&gt;

&lt;h2&gt;
  
  
  Frequently asked questions
&lt;/h2&gt;

&lt;h3&gt;
  
  
  What is mutation testing?
&lt;/h3&gt;

&lt;p&gt;Mutation testing is a technique for judging the quality of a test suite. A tool introduces a small, deliberate change into production code, for example turning greater-or-equal into greater or deleting a statement, and runs the tests. If a test fails, the mutant is killed and the tests guard that decision. If they all pass, the mutant survived and you have a line your tests assert nothing about. The mutation score is the share of killed mutants among the ones that could be judged.&lt;/p&gt;

&lt;h3&gt;
  
  
  How is mutation score different from code coverage?
&lt;/h3&gt;

&lt;p&gt;Coverage answers whether a given line ran during the tests. Mutation score answers whether any assertion would break if that line were wrong. Those are two different quantities. A test with no assertion, a test asserting on a stub, and a test that recorded the current output as the expected one all give full coverage and zero diagnostic value.&lt;/p&gt;

&lt;h3&gt;
  
  
  Why do AI-generated tests have high coverage and a weak mutation score?
&lt;/h3&gt;

&lt;p&gt;A model writing a test against finished code has that code in context, so the cheapest route to green is a test that repeats the implementation instead of checking it. Research shows that models write assertions describing actual rather than expected behaviour, and that they stub dependencies more often than non-agent commits do instead of testing the real interaction. Such a test raises coverage and would pass with the condition inverted.&lt;/p&gt;

&lt;h3&gt;
  
  
  Is mutation testing worth the cost?
&lt;/h3&gt;

&lt;p&gt;Yes, provided you do not run it across the whole repository. One mutant means one test run, so a raw audit is measured in hours. The version that pays off mutates only the lines changed on a branch, runs only the tests that could kill, and starts the process once instead of once per mutant. In that shape the cost is comparable to computing coverage inside the same code review process.&lt;/p&gt;

&lt;h3&gt;
  
  
  What is a good mutation score?
&lt;/h3&gt;

&lt;p&gt;There is no single threshold and one hundred percent is not the goal, because some mutants are equivalent to the original and no test can tell them apart. It works better to set the expectation per file, according to what a bug there costs. High for code that decides money, permission boundaries and how far a model's answer is trusted. Low for layers that only move data around. The value sits in the survivor list read by method, not in the percentage.&lt;/p&gt;

&lt;h3&gt;
  
  
  Will mutation testing replace code review?
&lt;/h3&gt;

&lt;p&gt;No. Mutation works at the level of a single unit and answers one question: do the tests guard this decision. It will not see a bug that consists of two correct pieces failing to compose, and that is the most common failure in agent-written code. Mutation is an input to review rather than a replacement for it.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;Originally published at &lt;a href="https://szymonpaluch.com/blog/posts/mutation-testing-ai-generated-tests" rel="noopener noreferrer"&gt;szymonpaluch.com&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>testing</category>
      <category>ai</category>
      <category>programming</category>
      <category>tdd</category>
    </item>
    <item>
      <title>Claude Certified Architect Foundations: my exam experience</title>
      <dc:creator>Szymon Paluch</dc:creator>
      <pubDate>Mon, 12 Oct 2026 11:06:59 +0000</pubDate>
      <link>https://dev.to/szymonpaluch/claude-certified-architect-foundations-my-exam-experience-1fdn</link>
      <guid>https://dev.to/szymonpaluch/claude-certified-architect-foundations-my-exam-experience-1fdn</guid>
      <description>&lt;p&gt;Two hours with a proctor and decisions between several plausible answers. This is my account of taking Claude Certified Architect Foundations on July 15, 2026 and what helped me prepare.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fqk7u20tanjl7l9m5mybk.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fqk7u20tanjl7l9m5mybk.webp" alt="Notebook sketch: a stressed architect at a keyboard during an exam, a robot behind him, code fragments and graphs all around" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;I passed Claude Certified Architect Foundations on July 15, 2026. The questions that took most time had several answers that seemed reasonable. A detail in the scenario decided the choice. I spent almost the full two hours answering and reviewing the questions.&lt;/p&gt;

&lt;h2&gt;
  
  
  What this certification actually is
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Claude Certified Architect Foundations&lt;/strong&gt; is Anthropic’s exam for solution architects. It assesses decisions when building with Claude Code, the Agent SDK, the API and MCP. For me it was a test of knowledge; client delivery still needs code, tests and work within real constraints.&lt;/p&gt;

&lt;p&gt;The exam lasts two hours and is proctored. Before starting, I had to prepare my computer and show the room. During my sitting, I worked in one exam window.&lt;/p&gt;

&lt;p&gt;Not everyone can sit it, either. Anthropic admits only people from Claude Partner Network companies and rejects personal email addresses. If your company is not a partner, here is &lt;a href="https://szymonpaluch.com/blog/posts/claude-certification-without-partner" rel="noopener noreferrer"&gt;what to do when the registration page rejects your email&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Proctoring: two hours under watch
&lt;/h2&gt;

&lt;p&gt;This is how my online sitting worked, before the questions and while I answered them:&lt;/p&gt;

&lt;h3&gt;
  
  
  WHAT PROCTORING LOOKS LIKE
&lt;/h3&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;The software restricted me to the exam window.&lt;/strong&gt; I could not switch to other tabs or applications.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;I showed the room and desk on camera.&lt;/strong&gt; I could not leave notes on the desk or use a second screen.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The camera and microphone stayed on throughout the exam.&lt;/strong&gt; The screen, camera image and audio were recorded.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;I worked in the same question window for two hours.&lt;/strong&gt;&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The procedure required preparing my workspace and equipment. Read the instructions before your appointment so that you can resolve computer problems before check-in.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the questions were like
&lt;/h2&gt;

&lt;p&gt;The questions asked me to choose a solution under specific constraints. My form had four options and one correct answer. The differences could be small: which step to take first, or which solution met the cost and response-time requirements.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;You have situation X. Which step do you take &lt;strong&gt;first&lt;/strong&gt;?&lt;/li&gt;
&lt;li&gt;You have three technically correct solutions. Which is &lt;strong&gt;most optimal for cost and latency&lt;/strong&gt;?&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Sometimes I hesitated between two answers that could both work. I had to return to the scenario’s constraints. A detail in the prompt or configuration could change the choice, so I reviewed every question.&lt;/p&gt;

&lt;h2&gt;
  
  
  The sample questions lull you into a false sense of ease
&lt;/h2&gt;

&lt;p&gt;The sample questions I studied were easier than those on my exam form. They helped me understand how the questions were asked, but did not establish the difficulty of the whole sitting. Even after a correct answer, check why the other options do not fit.&lt;/p&gt;

&lt;p&gt;This describes my own sitting on July 15, 2026, not every possible exam form. The current guide also allows multiple-response questions. I collected the format and exercises for all five domains in the &lt;a href="https://szymonpaluch.com/blog/posts/claude-certified-architect-certification-guide" rel="noopener noreferrer"&gt;current CCAR-F study plan&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  How to prepare
&lt;/h2&gt;

&lt;p&gt;Practise choosing a solution as you prepare. Add an example to each definition: when would you use the tool, and what error could occur? Check what changes with a different configuration.&lt;/p&gt;

&lt;p&gt;Work through agent architecture, tools and MCP, Claude Code, prompts and context. Compare the options for each task. Which reduces cost, and which reduces response time? Where can an agent act alone, and where must code enforce approval? Explain the choice against the requirements.&lt;/p&gt;

&lt;p&gt;I later wrote up the cheat sheet I made for myself while prepping as &lt;a href="https://szymonpaluch.com/blog/posts/agentic-design-rules" rel="noopener noreferrer"&gt;twelve rules for building agents&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  How I chose solutions for the scenarios
&lt;/h2&gt;

&lt;p&gt;An architect often chooses among several technically valid solutions. Budget, the user’s work and responsibility for maintenance all matter. That is how I read the exam questions: I looked for the solution that met the conditions of the task.&lt;/p&gt;

&lt;p&gt;Partner firms also count active people with eligible credentials when assessing their Claude Partner Network tier. I examined &lt;a href="https://szymonpaluch.com/blog/posts/claude-certified-architect-jobs" rel="noopener noreferrer"&gt;what that rule means for candidates and how to separate it from actual job openings&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;I passed above the threshold. My preparation also left me with original questions that you can use to study.&lt;/p&gt;

&lt;p&gt;I released the 60 questions I wrote during preparation as a free &lt;a href="https://szymonpaluch.com/claude-certified-architect-practice-exam" rel="noopener noreferrer"&gt;practice exam in the browser&lt;/a&gt;. You can choose study or exam mode and inspect results by domain.&lt;/p&gt;

&lt;p&gt;A separate article works through &lt;a href="https://szymonpaluch.com/blog/posts/claude-certified-architect-practice-questions" rel="noopener noreferrer"&gt;three examples with explanations&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fpl0h06i1mnutrf4s1a9z.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fpl0h06i1mnutrf4s1a9z.webp" alt="Claude Certified Architect Foundations, certification issued by Anthropic" width="600" height="600"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Claude Certified Architect Foundations&lt;/p&gt;

&lt;p&gt;Certification issued by Anthropic · verified on Credly&lt;/p&gt;

&lt;p&gt;&lt;a href="https://www.credly.com/badges/4dc5dab3-1196-410e-956d-5c316d9123eb/public_url" rel="noopener noreferrer"&gt;See the verification →&lt;/a&gt;&lt;/p&gt;




&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;No partner employer?&lt;/strong&gt; Anthropic admits to its exams only people from Claude Partner Network organizations. Join our collective of AI consultants and sit all four exams through our partner organization. &lt;a href="https://szymonpaluch.com/certified-ai-consultant-program" rel="noopener noreferrer"&gt;See the program →&lt;/a&gt; · &lt;a href="https://szymonpaluch.com/claude-certified-architect-practice-exam" rel="noopener noreferrer"&gt;Free practice exam →&lt;/a&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;&lt;em&gt;Originally published at &lt;a href="https://szymonpaluch.com/blog/posts/claude-certified-architect-exam" rel="noopener noreferrer"&gt;szymonpaluch.com&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>claude</category>
      <category>ai</category>
      <category>certification</category>
      <category>career</category>
    </item>
    <item>
      <title>Workflow Studio: observe and design Claude Code workflows</title>
      <dc:creator>Szymon Paluch</dc:creator>
      <pubDate>Fri, 24 Jul 2026 12:56:41 +0000</pubDate>
      <link>https://dev.to/szymonpaluch/workflow-studio-observe-and-design-claude-code-workflows-59k7</link>
      <guid>https://dev.to/szymonpaluch/workflow-studio-observe-and-design-claude-code-workflows-59k7</guid>
      <description>&lt;p&gt;Claude Code's Workflow tool fans out subagents in parallel and leaves the result on disk as raw logs. I built a local dashboard that reads those artifacts and renders the run, plus a block canvas that compiles back to a runnable script. Here is what it does, what it deliberately does not do, and how to install it.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The short version:&lt;/strong&gt; Workflow Studio is a local companion for Claude Code's built-in Workflow tool. It has two halves — &lt;strong&gt;Observe&lt;/strong&gt;, a dashboard that reads Claude Code's own on-disk run artifacts and draws each multi-agent run as a phase graph and a wall-clock timeline, and &lt;strong&gt;Author&lt;/strong&gt;, a 12-block canvas that compiles to a real, runnable Workflow script. Both are exposed to the agent through an MCP server with nine tools. It runs on &lt;code&gt;127.0.0.1&lt;/code&gt;, it is MIT-licensed, and you install it with one paste. The current release is v0.2.0.&lt;/p&gt;

&lt;h2&gt;
  
  
  The problem: a multi-agent run is a black box
&lt;/h2&gt;

&lt;p&gt;Claude Code's Workflow tool is its deterministic multi-agent primitive: you write a small JS script that fans agents out in parallel, pipes a list through stages, gates on a value, loops until a condition holds. It works. The problem starts the moment the run finishes.&lt;/p&gt;

&lt;p&gt;A finished run leaves real artifacts on disk. Under &lt;code&gt;~/.claude/projects/&amp;lt;project&amp;gt;/&amp;lt;session&amp;gt;/subagents/workflows/wf_*/&lt;/code&gt; you get a &lt;code&gt;journal.jsonl&lt;/code&gt;, a per-agent transcript for every subagent, and a small metadata file each. That is a lot of ground truth. Nothing renders it. So you scroll. You cannot see the phases at a glance, how wide the fan-out actually went, how many tokens each agent burned, how long each one took, or which branch a gate picked when it mattered.&lt;/p&gt;

&lt;p&gt;That is the same shape of failure I wrote about in &lt;em&gt;silent failures in agentic systems&lt;/em&gt;: nothing crashes, so nothing tells you that eleven of your twelve verifiers returned an empty result. And it compounds on the authoring side — the only way to build one of these is to hand-write the script, and the only channel between the script you wrote and the agent that runs it is a &lt;code&gt;.js&lt;/code&gt; file dropped on disk.&lt;/p&gt;

&lt;h2&gt;
  
  
  What Workflow Studio is
&lt;/h2&gt;

&lt;p&gt;A locally-run companion layer, not a runtime. It ships no agents of its own and defines no execution engine; it sits on top of the one you already have. Two surfaces plus a channel:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Observe&lt;/strong&gt; — a dashboard that passively reads Claude Code's run artifacts and renders each run as a phase/agent graph and a timeline.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Author&lt;/strong&gt; — a no-code block canvas that compiles down to a real Workflow script.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;MCP&lt;/strong&gt; — a stdio server with nine tools, so the same Claude Code agent that runs your workflows can also read your designs, inspect the observations, and write new designs back into your builder.&lt;/li&gt;
&lt;/ul&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;12&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Block kinds&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;9&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;MCP tools&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;127.0.0.1&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Default bind&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;MIT&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;License&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;One thing to be clear about up front, because it shapes everything else: &lt;strong&gt;Workflow Studio cannot start a run.&lt;/strong&gt; Not over MCP, not from the builder. It reads what Claude Code already produced and it hands the agent a script the agent runs with its own Workflow tool. That is a design decision, not a gap, and I explain why in the authoring section.&lt;/p&gt;

&lt;h2&gt;
  
  
  Observe: see what actually ran
&lt;/h2&gt;

&lt;p&gt;The dashboard turns a run into a graph you can read. Phases, the fan-out, tokens per agent, the branch taken, each agent's output. Click a node and the inspector shows what that agent was given and what it returned — its prompt, its thinking, its tool calls, its output, its model, its state and how long it took, and its spawn depth.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fte3e8c1qs5rq7yco3amz.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fte3e8c1qs5rq7yco3amz.webp" alt="Observe — phase graph of a real deep-research run" width="800" height="500"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Observe · a real deep-research run of mine — 111 agents across Scope → Search → Fetch → Verify → Synthesize, 3.7M tokens, about 22 minutes. One run on one machine, not a benchmark.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;The timeline view answers the other question the graph does not: what overlapped with what. A fan-out that looks like one wide row in the graph is usually a staircase in wall-clock time, and seeing that is how you find out that your second fan-out was waiting on a single straggler from the first.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F3ztgfl45179drnlc8j3k.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F3ztgfl45179drnlc8j3k.webp" alt="Observe — the wall-clock timeline" width="800" height="500"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Observe · the wall-clock timeline — the Review fan-out runs first, then Verify staggers in as each agent lands.&lt;/em&gt;&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Honest by construction.&lt;/strong&gt; The numbers come from two different places and the dashboard says which. When Claude Code's own progress overlay is still on disk, labels, phases, states and token counts are read straight from it and the run is authoritative. When that overlay has already been garbage-collected, phases and labels are derived heuristically from the journal, and the run is flagged as such — tokens or timings that could not be measured are marked, never invented. A run still in flight is labelled &lt;code&gt;live&lt;/code&gt; and its elapsed times are lower bounds. You are never shown a number that looks measured but isn't.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Observing is also where the next workflow usually starts. When a run turns out well, you promote it to a reusable workflow in one click — the graph you were just reading opens in the builder as a starting point, so a good pipeline becomes a template instead of something you reconstruct from memory next month.&lt;/p&gt;

&lt;p&gt;Two practical notes. The dashboard refreshes by polling with a short cache, not by pushing — it catches up within a couple of seconds, it is not a socket. And it reads across all your projects and sessions, so old runs stay browsable long after the session that produced them is gone.&lt;/p&gt;

&lt;h2&gt;
  
  
  Author: build the recipe, not the boilerplate
&lt;/h2&gt;

&lt;p&gt;The other half is a canvas. You drop blocks, wire the ports, and Workflow Studio compiles the graph to a real Claude Code Workflow script — the native calls (&lt;code&gt;agent()&lt;/code&gt;, &lt;code&gt;phase()&lt;/code&gt;, &lt;code&gt;parallel()&lt;/code&gt;, &lt;code&gt;pipeline()&lt;/code&gt;, &lt;code&gt;log()&lt;/code&gt;) plus a sidecar comment holding the design. The round-trip is lossless: a compiled script reopens as the exact graph you drew, so you can edit it on the canvas, in the editor, or both, without either side losing work.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fo14mp0dpzltz3wummroa.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fo14mp0dpzltz3wummroa.webp" alt="Author — the block builder" width="800" height="500"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Author · the block builder — typed blocks compile to a runnable script.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;There are twelve block kinds. Each one declares the shape of its output, which is what lets a downstream block branch or map on a real field instead of on a string you hope is there:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Block&lt;/th&gt;
&lt;th&gt;What it does&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;start&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;The entry anchor. Two or more children run in parallel.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;agent&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;One agent with a prompt.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;fan-out&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;N agents running the same prompt in parallel.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;pipeline&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Each list item flows through the stages independently.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;loop&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Refine a result in place over N rounds, or until a condition holds.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;gate&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Run a branch only if a declared field meets a condition.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;switch&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Branch on a declared enum value — a k-way router.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;filter&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Keep only list items matching a field. No agent involved.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;rank&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Sort a list by a field and keep the top N. No agent involved.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;sub-workflow&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;A nested workflow reused as a single block.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;input&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;A fixed list of items — a source for fan-out, pipeline or rank.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;param&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;A named value you can @-mention inside prompts.&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;You never start from an empty canvas. Every new workflow begins from one of seven built-in patterns that already exercise the dynamic primitives: &lt;strong&gt;Basic&lt;/strong&gt; (agent plus in-place iteration), &lt;strong&gt;Classify and act&lt;/strong&gt; (a classifier enum routing to one of several branches), &lt;strong&gt;Fan-out and synthesis&lt;/strong&gt;, &lt;strong&gt;Adversarial verification&lt;/strong&gt; (independent verifiers trying to refute the worker's result), &lt;strong&gt;Generate and filter&lt;/strong&gt;, &lt;strong&gt;Tournament&lt;/strong&gt;, and &lt;strong&gt;Loop until done&lt;/strong&gt;. Pick one, then edit it.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;The Run button does not run anything.&lt;/strong&gt; The builder has a Run action, and it does not execute your workflow. It opens a dialog that says so in plain words and hands you a phrase to paste into Claude Code, then watches for the run to show up in Observe. The reason is boring and load-bearing: a server-side shell-out would turn a loopback dashboard into an unauthenticated local exec surface. I rejected that in design review and I would reject it again. Execution stays where the permissions live — in your Claude Code session.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2&gt;
  
  
  Install it in one paste
&lt;/h2&gt;

&lt;p&gt;You need &lt;code&gt;uv&lt;/code&gt; / &lt;code&gt;uvx&lt;/code&gt; on your &lt;code&gt;PATH&lt;/code&gt; and Python 3.9 or newer. Nothing else — the dashboard ships pre-built inside the package, so there is no Node step.&lt;/p&gt;

&lt;p&gt;You are probably already sitting in a coding agent. Hand it this and walk away:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Read https://github.com/hculap/workflow-studio/blob/main/AGENT_INSTALL.md
and set up Workflow Studio for me — run the steps, verify it,
and tell me whether to restart Claude Code.
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Prefer to do it yourself? These two are slash commands you type in a Claude Code session — an agent cannot run them for you:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;/plugin marketplace add hculap/workflow-studio
/plugin install workflow-studio@workflow-studio
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;On Claude Code 2.1 and newer there are shell equivalents (&lt;code&gt;claude plugin marketplace add …&lt;/code&gt;, &lt;code&gt;claude plugin install …&lt;/code&gt;), which is what the agent path uses. Either way, &lt;strong&gt;restart Claude Code afterwards&lt;/strong&gt; — the MCP tools and the &lt;code&gt;/workflow-studio:dashboard&lt;/code&gt; command only load on the next session. Verify with &lt;code&gt;claude mcp list&lt;/code&gt;; you want to see the server as connected. If it shows as failed, it is almost always &lt;code&gt;uvx&lt;/code&gt; missing from the &lt;code&gt;PATH&lt;/code&gt; a GUI-launched Claude Code inherited.&lt;/p&gt;

&lt;p&gt;You can also skip the plugin entirely:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# dashboard on http://127.0.0.1:8787/ (opens a browser)&lt;/span&gt;
uvx workflow-studio

&lt;span class="c"&gt;# MCP server on stdio — register it by hand&lt;/span&gt;
claude mcp add workflow-studio &lt;span class="nt"&gt;-s&lt;/span&gt; user &lt;span class="nt"&gt;--&lt;/span&gt; uvx workflow-studio mcp
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;One caveat worth knowing: &lt;code&gt;uvx&lt;/code&gt; pulls the package from PyPI and the plugin pins no version, so you pick up new releases silently.&lt;/p&gt;

&lt;h2&gt;
  
  
  Where it fits next to LangGraph, CrewAI and n8n
&lt;/h2&gt;

&lt;p&gt;Workflow Studio is not competing with those. They are a different layer of the stack. LangGraph, CrewAI, AutoGen, the OpenAI Agents SDK, n8n and Flowise each hand you a runtime to adopt, and each is portable across model providers — which is a genuine strength Workflow Studio does not have and is not trying to have. Workflow Studio assumes you already run Claude Code, and gives you a way to see what a run did and to author the next one visually.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Tool&lt;/th&gt;
&lt;th&gt;What it is&lt;/th&gt;
&lt;th&gt;Authoring&lt;/th&gt;
&lt;th&gt;Observes real runs&lt;/th&gt;
&lt;th&gt;Local · no account&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Workflow Studio&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Observe + author layer for Claude Code's Workflow tool. Not a runtime.&lt;/td&gt;
&lt;td&gt;No-code canvas (12 blocks) compiling to a real Workflow script&lt;/td&gt;
&lt;td&gt;Yes — reads Claude Code's on-disk artifacts, no SDK, heuristics flagged&lt;/td&gt;
&lt;td&gt;Yes — loopback, no account, no telemetry&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;LangGraph + Studio&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Provider-agnostic framework for stateful graph agents (Py/JS)&lt;/td&gt;
&lt;td&gt;Code-first — you write the graph; Studio visualizes and debugs it&lt;/td&gt;
&lt;td&gt;Yes, and strongly — step and time-travel debugging, deep tracing via LangSmith&lt;/td&gt;
&lt;td&gt;Partial — library runs local; Studio wants a (free) LangSmith account&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;CrewAI / AutoGen&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Code-first multi-agent frameworks&lt;/td&gt;
&lt;td&gt;Code (Python / YAML); AutoGen Studio adds a capable no-code GUI&lt;/td&gt;
&lt;td&gt;Yes or partial — you instrument the runtime (AutoGen via OpenTelemetry)&lt;/td&gt;
&lt;td&gt;Partial — libraries local; CrewAI observability hosted-first&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;OpenAI Agents SDK&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Lightweight code-first framework&lt;/td&gt;
&lt;td&gt;Code only (Python / TS)&lt;/td&gt;
&lt;td&gt;Yes — built-in tracing, on by default&lt;/td&gt;
&lt;td&gt;Partial — traces default to a hosted dashboard; no bundled local UI&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;n8n / Flowise&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Visual low-code platforms that are themselves the runtime&lt;/td&gt;
&lt;td&gt;Visual no-code / low-code canvas&lt;/td&gt;
&lt;td&gt;Yes — but of runs inside their own platform, not Claude Code runs&lt;/td&gt;
&lt;td&gt;Yes — self-hostable, usually via Docker&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;That table reflects each tool's own positioning as of mid-2026, and licenses and project status change. Every one of them is capable in its niche — several have a more full-featured canvas or stronger tracing than a v0.2.0 side project, and all of them are portable across model providers, which Workflow Studio is not. The axis here is fit, not ranking. If you are not on Claude Code, none of this is for you.&lt;/p&gt;

&lt;h2&gt;
  
  
  FAQ
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Does Workflow Studio run my workflows?&lt;/strong&gt;&lt;br&gt;
No. It cannot start a run, from the builder or over MCP. It compiles a script and hands it to your Claude Code agent, which runs it with its own Workflow tool. A server-side shell-out would turn a loopback dashboard into an unauthenticated local execution surface, so execution stays inside your Claude Code session where the permission model lives.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;What do I need to install it?&lt;/strong&gt;&lt;br&gt;
&lt;code&gt;uv&lt;/code&gt; / &lt;code&gt;uvx&lt;/code&gt; on your &lt;code&gt;PATH&lt;/code&gt; and Python 3.9 or newer. Nothing else — the dashboard ships pre-built inside the package, so there is no Node step. Install it as a Claude Code plugin from the GitHub marketplace, or run it standalone with &lt;code&gt;uvx workflow-studio&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Does any of my data leave the machine?&lt;/strong&gt;&lt;br&gt;
The application makes no outbound calls and collects no telemetry. It reads local run artifacts and binds to &lt;code&gt;127.0.0.1&lt;/code&gt; by default. Two caveats: &lt;code&gt;uvx&lt;/code&gt; downloads the package from PyPI on first run and on version bumps, and the local server has no authentication, so keep it on loopback unless you add your own auth in front of it.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Are the dashboard's token and timing numbers real?&lt;/strong&gt;&lt;br&gt;
When Claude Code's progress overlay is still on disk, yes — they are read straight from it. When that overlay has been garbage-collected, phases and labels are derived heuristically from the run journal and the run is flagged as such. A run still in flight is labelled &lt;code&gt;live&lt;/code&gt; and its elapsed times are lower bounds. Nothing estimated is presented as measured.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Is it an alternative to LangGraph, CrewAI or n8n?&lt;/strong&gt;&lt;br&gt;
No — it's a different layer. Those are runtimes you adopt, and they are portable across model providers. Workflow Studio ships no runtime and defines no agents; it sits on top of Claude Code's Workflow tool, which it assumes you already use. If you are not on Claude Code, it has nothing to offer you.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Is Workflow Studio open source?&lt;/strong&gt;&lt;br&gt;
It is MIT-licensed, with one nuance worth knowing: the GitHub repository is the Claude Code plugin marketplace — manifests, a skill, a command and documentation. The application source is distributed in the PyPI sdist rather than published as a browsable repository, so to read the code you unpack the package.&lt;/p&gt;




&lt;p&gt;&lt;strong&gt;Links&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Landing page with interactive demos: &lt;a href="https://szymonpaluch.com/workflow-studio/" rel="noopener noreferrer"&gt;https://szymonpaluch.com/workflow-studio/&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;Plugin on GitHub (MIT): &lt;a href="https://github.com/hculap/workflow-studio" rel="noopener noreferrer"&gt;https://github.com/hculap/workflow-studio&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;PyPI: &lt;a href="https://pypi.org/project/workflow-studio/" rel="noopener noreferrer"&gt;https://pypi.org/project/workflow-studio/&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If you try it, tell me what your run graph looked like — I want to know how wide people are actually fanning out.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>programming</category>
      <category>productivity</category>
      <category>opensource</category>
    </item>
    <item>
      <title>12 Rules for Building AI Agents That Survive Production</title>
      <dc:creator>Szymon Paluch</dc:creator>
      <pubDate>Fri, 17 Jul 2026 12:30:14 +0000</pubDate>
      <link>https://dev.to/szymonpaluch/12-rules-for-building-ai-agents-that-survive-production-4j9h</link>
      <guid>https://dev.to/szymonpaluch/12-rules-for-building-ai-agents-that-survive-production-4j9h</guid>
      <description>&lt;p&gt;I sat the Claude Certified Architect exam expecting questions about model parameters, context limits, and API flags. I got something else. The exam barely tests trivia. It tests judgment: given a broken agent and four plausible fixes, which one actually addresses the root cause?&lt;/p&gt;

&lt;p&gt;The interesting part was how few ideas the whole thing rests on. The same handful of rules kept deciding the "right" answer, and they are the same rules that decide whether an agent holds up once real users touch it. Below are the twelve I kept running into, plus the four traps that look like solutions and are not.&lt;/p&gt;

&lt;p&gt;This is my own study material, derived from publicly available exam guidance. It reflects how I build, not an official Anthropic position.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The twelve rules&lt;/strong&gt;&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;&lt;p&gt;Enforce determinism in code, not in prompts&lt;br&gt;
If a rule has to fire every single time, it is not a job for a prompt. A prompt is a suggestion the model usually follows. "Usually" is not a guarantee. When you need a guarantee, put it in a hook, a gate, or an allowlist. Code enforces. Prose requests.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Pick the cheapest fix that hits the root cause&lt;br&gt;
Before you build a subsystem, try the levers that cost minutes: a sharper tool description, an explicit acceptance criterion, a config change. Most "we need to build X" moments dissolve once you test the cheap fix first. Reach for the classifier only after the one-line change fails.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Bad tool selection? Start with the descriptions&lt;br&gt;
When an agent keeps picking the wrong tool, the description is almost always the culprit, not the model. Tool descriptions are the primary signal the model uses to choose. Rewrite them to say exactly when to use the tool and when not to, before you go anywhere near few-shot examples.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Over-engineering is almost always the wrong answer&lt;br&gt;
Narrowing scope and improving the prompt beat a new subsystem far more often than engineers expect. Every subsystem you add is one more thing to debug, monitor, and keep in sync. Complexity is a cost you pay forever, not once.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;A bigger context window does not fix attention&lt;br&gt;
Stuffing more into the window does not make the model pay better attention to what matters. It often does the opposite. Decomposition and structure fix attention: smaller tasks, cleaner inputs, explicit boundaries. "Just use the model with the larger context" is a non-answer.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Model confidence and tone are not signals&lt;br&gt;
A confident-sounding answer is not a correct one. Self-assessed confidence is poorly calibrated, and on the hard cases the model is often most sure precisely when it is wrong. Never gate a decision on how certain the model claims to be. Verify with something external.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;An independent instance beats self-review&lt;br&gt;
A model grading its own work is biased toward its own work. A fresh instance, with no memory of having produced the output, catches far more errors. If you want a real review step, spin up a separate reviewer, do not ask the author to check itself.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Least privilege for tools&lt;br&gt;
Give each agent role only the tools it needs, roughly four to five. This is not only a security point. More tools measurably degrade selection quality: the model gets worse at choosing when the menu is longer. Fewer, sharper tools beat a giant toolbox.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Structure beats free text, everywhere&lt;br&gt;
Structured errors, structured outputs, structured handoffs between agents, all with explicit provenance. Free-text prose between components is where information quietly rots. When agent A hands to agent B, pass a schema, not a paragraph.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Do not suppress errors, and do not panic&lt;br&gt;
Handle transient failures locally: retry the flaky network call, back off, move on. For everything else, escalate with context so the coordinator can decide. The two failure modes to avoid are swallowing the error silently and blowing up the whole run over something recoverable.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Match the API to your latency requirement&lt;br&gt;
The Batch API costs about half as much but can take up to 24 hours. That is great for an overnight job and useless for anything a human is waiting on. Pick synchronous calls for human-blocking tasks and batch for the rest. Cost and latency are a trade you make on purpose.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;A retry fixes format, not missing data&lt;br&gt;
Retrying a call can correct a malformed structure. It cannot conjure information that was never in the source. If the data is not there, running it again just wastes tokens and time. Know which problem you have before you hit retry.&lt;/p&gt;&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The four traps&lt;br&gt;
These are the answers that feel smart in the moment and cost you later. Every one of them showed up as a tempting-but-wrong option.&lt;/p&gt;

&lt;p&gt;"Just write in the prompt that it is mandatory." Shows up exactly when you need determinism, which is the one thing a prompt cannot give you. See rule 1.&lt;br&gt;
"Add few-shot examples." Genuinely useful for ambiguous judgment calls. Useless for hard rules, where it creates the illusion of a guarantee without the guarantee.&lt;br&gt;
"Build a classifier." A heavy solution reached for before anyone tested the cheap levers. Sometimes correct, usually premature. See rule 2.&lt;br&gt;
"Make one tool that does everything." Merging tools feels like simplification. It makes selection worse, because now the model has to guess intent inside a single overloaded tool. See rule 8.&lt;br&gt;
It collapses into three ideas&lt;br&gt;
Strip away the specifics and the twelve rules are really three:&lt;/p&gt;

&lt;p&gt;What must always work goes in code, not in a request. Determinism is an architecture decision, not a wording decision.&lt;br&gt;
Match the complexity of the fix to the size of the problem. Cheap lever first, subsystem last.&lt;br&gt;
Pass structure, not prose. Between components, schemas beat sentences.&lt;br&gt;
None of this is exotic. It is the difference between an agent that demos well and one that is still running in three months. The exam rewards it because production rewards it.&lt;/p&gt;

&lt;p&gt;Szymon Paluch. I work with teams on agentic system strategy, taking Claude Code and agents from demo to production.&lt;/p&gt;


&lt;div class="crayons-card c-embed text-styles text-styles--secondary"&gt;
    &lt;div class="c-embed__content"&gt;
        &lt;div class="c-embed__cover"&gt;
          &lt;a href="https://szymonpaluch.com/blog/posts/agentic-design-rules" class="c-link align-middle" rel="noopener noreferrer"&gt;
            &lt;img alt="" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fszymonpaluch.com%2Fimages%2Fog%2Fblog%2Fposts%2Fagentic-design-rules.jpg" height="420" class="m-0" width="800"&gt;
          &lt;/a&gt;
        &lt;/div&gt;
      &lt;div class="c-embed__body"&gt;
        &lt;h2 class="fs-xl lh-tight"&gt;
          &lt;a href="https://szymonpaluch.com/blog/posts/agentic-design-rules" rel="noopener noreferrer" class="c-link"&gt;
            12 rules for building AI agents | Szymon Paluch
          &lt;/a&gt;
        &lt;/h2&gt;
          &lt;p class="truncate-at-3"&gt;
            The full cheat-sheet from prepping for the Claude Certified Architect exam: 12 rules, 7 recurring traps, the specifics of all 5 domains, and a facts table. The rules that decide whether an agent survives production.
          &lt;/p&gt;
        &lt;div class="color-secondary fs-s flex items-center"&gt;
            &lt;img alt="favicon" class="c-embed__favicon m-0 mr-2 radius-0" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fszymonpaluch.com%2Ffavicon.svg" width="64" height="64"&gt;
          szymonpaluch.com
        &lt;/div&gt;
      &lt;/div&gt;
    &lt;/div&gt;
&lt;/div&gt;


</description>
      <category>agents</category>
      <category>ai</category>
      <category>architecture</category>
      <category>claude</category>
    </item>
  </channel>
</rss>
