DEV Community

Cover image for Can we actually build the thing now?
Chigozirim Eke
Chigozirim Eke

Posted on

Can we actually build the thing now?

Part 1 ended with "Now we can actually build it." So in Part 2 we build it on both frameworks, and give each one the same request to compare the courses they make.

The first iteration built by yours truly Claude Opus 5.5 had 235 tests, and every single one of them passed. But when I started poking (ok, so this has been my huge limitation with Claude. It doesn't surface gaps until I start poking), I realized not one of those tests had ever talked to a real model 🙃

When I pointed it at a real model, it failed on the very first call. And when it finally did produce a Terraform course, the quiz answers sat right under the questions and every exercise came with the finished files.

So I deleted the entire folder and started from scratch. The rebuild came out much better, and it's the one we'll walk through in this post.

Rant Aside. Because accuracy is super matters to me, whenever I'm learning something new, I search for "the most up to date resource", and more than once I've bought a course only to realise a few lessons in that it's already out of date. Case in point: I started Part 1 a couple of weeks ago, and Strands has already shipped a whole agent harness.

We'll start with what using it looks like, then build it up side by side the way we did in Part 1.

Everything here ran between 27 and 29 September 2026, on the same versions as Part 1 (strands-agents 1.55.1 and google-adk 2.9.0), and the code is in the companion repo. Both frameworks are actively developed, and strands-agents 1.61.1 and google-adk 2.12.0 were out by the time this went up, so some APIs may have changed since.

What should a learning system actually do?

When I started over, I wrote down what I actually want from this thing:

  1. I say what I want to learn, and it asks a couple of questions to get the course right.
  2. It covers what people actually do with it at work. For example, pinning a shared module's version.
  3. It's as current as the docs on the day it's built.
  4. I learn by doing, and there are tests that verify my work.
  5. I have to try before I see the answer.
  6. It doesn't cost a fortune to build, and it tells me what it spent.

What does using it look like?

You'll need uv. Then install it once, straight from the repo:

uv tool install "git+https://github.com/coolchigi/strands-vs-adk#subdirectory=part2-learning-system"
Enter fullscreen mode Exit fullscreen mode

and start a course from anywhere:

learn "I want to learn TypeScript so I can build a small web app"
Enter fullscreen mode Exit fullscreen mode

The first time, it asks for a Gemini API key and saves it, so you never set it up again. That key is for the ADK build, which runs on Gemini and is the default. To build with Strands, which runs Claude on Amazon Bedrock, add --engine strands. You'll need AWS credentials and access to Claude in Bedrock, and the Strands Python quickstart walks through setting both up:

learn --engine strands "I want to learn TypeScript so I can build a small web app"
Enter fullscreen mode Exit fullscreen mode

If you already have AWS credentials and no Gemini key, it picks Strands on its own and doesn't ask for a key.

Every course has a spending cap, $5 on Gemini and $10 on Claude by default, and we'll get to how it works in the bill section.

Then it asks what it needs to know, and you answer right there:

A few questions first, so I build the right course:
  What is your background with programming, particularly JavaScript?
> I know JavaScript well but have never used TypeScript
  What kind of web app are you looking to build, and do you have a specific framework in mind (e.g., React, Next.js, Node.js)?
> a small web app with a backend API

Got it. Building your TypeScript course in ~/courses/typescript
This usually takes 30 to 90 minutes. You can stop it any time, and running the same command
again picks up where it left off.

  Looking for the official TypeScript documentation
  ✓ Found it: TypeScript, version 7.0
  ✓ Worked out 15 things you'll need to be able to do
  ✓ Checked 126 facts, word for word, against 17 official pages
  ✓ Planned 5 units, 12 lessons
  Writing lesson 1 of 12: Compiler CLI and Strict Type Validation
      ✓ passed its checks
  ...
Your course is ready: ~/courses/typescript
12 lessons, and it cost about $2.94 to build.
Enter fullscreen mode Exit fullscreen mode

That's a trimmed output of the real run. Once the build finishes, you get a folder with a README.md for each lesson. Each one has the explanation, where every fact links to the page it came from, then a worked example, then an exercise with files for you to change.

You work through it with learn:

cd ~/courses/typescript
learn next      # where you are, and what's next
learn check     # run the exercise's tests
learn hint      # one hint at a time
learn solution  # the answer, when you want it
learn quiz      # answer first, then see the answer
learn review    # questions you missed come back after 1, 3, 7 and 21 days
Enter fullscreen mode Exit fullscreen mode

Lesson 1's exercise gives you a formatter.ts with two TODO comments, which mark the code you have to write. One of them is to give a parameter a type. Here's what learn check says if you fix the code but skip the type:

$ learn check
Checking units/u1-strict-compiler-configuration-and-compilation/u1-l1-compiler-cli-and-strict-type-validation/exercise ...

formatter.ts(1,28): error TS7006: Parameter 'input' implicitly has an 'any' type.

Not yet. Read the messages above, change your files, and check again.
Stuck? learn hint
Enter fullscreen mode Exit fullscreen mode

Add the type and it passes, then points you to learn quiz.

That error comes from the TypeScript compiler itself, because learn check runs each subject's real tools. So what you need installed depends on what you're learning. A Terraform course needs Terraform, a Python course needs Python 3.9 or newer, and a JavaScript or TypeScript course needs Node. Anything else has no tests, so you compare your work with the solution.

How is it built?

Behind those learn commands is the system we sketched in Part 1, with one new agent at the front:

learner ─▶ intake ─▶ researcher ─▶ curriculum builder ─▶ teacher ─▶ your course
                         â–²                  â–²                â–²
                         └──── the feedback agent reviews each stage, sends it back with reasons
Enter fullscreen mode Exit fullscreen mode
  • The intake asks what it needs to know, like where you're starting from and what you want to build.
  • The researcher reads the official docs and makes a list of what the course has to teach (i.e. its objectives). It only keeps facts from the live pages.
  • The curriculum builder turns that list into units and lessons, and picks an exercise or a quiz to check each one.
  • The teacher writes each lesson, including the tests that check your exercise.
  • The feedback agent checks each stage's work and sends it back if something's wrong. Code checks what it can, like running every exercise, and a model reviews the rest.

Most of the system, like fetching pages and running tests, is plain Python that both sides share. The frameworks handle the parts this post compares: reaching the docs, getting typed answers back, running the feedback loop and keeping the bill down.

Both frameworks plug in through one method:

class Brain(Protocol):
    def ask(self, name: str, instruction: str, prompt: str, schema: type[T], research: bool = False) -> T:
        """One agent call, returning `schema`. With research=True the agent can search and read."""
Enter fullscreen mode Exit fullscreen mode

Each framework implements ask its own way and runs the stages in its own graph. The Strands side uses Claude Sonnet 4.6 on Amazon Bedrock, which is Strands' default, and the ADK side uses Gemini 3.8 Flash. So when the two sides behave differently, it can be the framework or the model, and we'll say which one it was each time.

How does the researcher reach the docs?

The researcher needs tools to find the right pages and read them. Some of those tools come from MCP servers. MCP (Model Context Protocol) is an open source standard for connecting AI apps to outside tools and data, so the same server works with both Strands and ADK.

Here's how they both do it:

Strands ADK
No search in the package google_search, Google Search built into Gemini only
MCP servers go straight into tools MCP servers come in through McpToolset
web_fetch reads a page live load_web_page, which I couldn't use

Both sides also get the same two extra tools. One is AWS's documentation server, which searches and reads AWS's docs with no key. The other is list_docs_pages, a small tool we wrote that lists every page a docs site has, from the site's own sitemap.

Strands

Strands ships tools for fetching pages and none for searching, so I used an MCP server. Here we create an MCPClient and pass it straight into tools, and Strands opens and closes the connection for us. This is what connecting to the AWS docs server looks like:

from strands import Agent
from strands.tools.mcp import MCPClient

aws_docs = MCPClient(url="https://knowledge-mcp.global.api.aws")
researcher = Agent(system_prompt="Answer from the AWS documentation, in one sentence.",
                   tools=[aws_docs], callback_handler=None)

print(researcher("How big can a single S3 object be?"))
Enter fullscreen mode Exit fullscreen mode
A single Amazon S3 object can be a maximum of **5 terabytes (TB)** in size, though the largest
object that can be uploaded in a single PUT request is **5 GB** ...
Enter fullscreen mode Exit fullscreen mode

Keep that 5 TB in mind, because we'll ask the ADK side the same question.

For general web search I used Exa's hosted MCP server because it didn't need an API key. Its free tier ran out partway through a single course build, so on Strands, web search stays off unless you add an Exa or Tavily key. For now, you don't need one, because list_docs_pages finds the pages on the docs site itself.

To read a page, web_fetch has two modes. The default, agentic, hands the page to a second agent and returns that agent's summary, and markdown returns the page itself. We use markdown mode because, for every fact, the researcher copies the sentence on the page that backs it up, and our code checks that sentence is really there.

from strands.vended_tools.web_fetch import make_web_fetch

fetch = make_web_fetch(mode="markdown", max_content_chars=50_000)
Enter fullscreen mode Exit fullscreen mode

It needs the web-fetch extra, so pip install "strands-agents[web-fetch]".

ADK

ADK ships search as google_search, and it only works on Gemini. On any other model, like Claude, the agent still builds, but the first call fails with ValueError: Google search tool is not supported for model ....

MCP servers come in through McpToolset. This is the same AWS docs server, connected next to google_search:

import asyncio

from google.adk.agents import Agent
from google.adk.runners import InMemoryRunner
from google.adk.tools import google_search
from google.adk.tools.mcp_tool import McpToolset, StreamableHTTPConnectionParams
from google.genai import types

aws_docs = McpToolset(connection_params=StreamableHTTPConnectionParams(url="https://knowledge-mcp.global.api.aws"))
researcher = Agent(
    name="researcher",
    model="gemini-3.8-flash",
    instruction="Answer from the AWS documentation, in one sentence.",
    tools=[google_search, aws_docs],
    generate_content_config=types.GenerateContentConfig(
        tool_config=types.ToolConfig(include_server_side_tool_invocations=True)
    ),
)


async def main() -> None:
    runner = InMemoryRunner(agent=researcher, app_name="docs")
    session = await runner.session_service.create_session(app_name="docs", user_id="me")
    message = types.Content(role="user", parts=[types.Part(text="How big can a single S3 object be?")])
    async for event in runner.run_async(user_id="me", session_id=session.id, new_message=message):
        if event.is_final_response() and event.content and event.content.parts:
            print("".join(part.text or "" for part in event.content.parts))
    await aws_docs.close()


asyncio.run(main())
Enter fullscreen mode Exit fullscreen mode
According to AWS documentation, a single Amazon S3 object can be up to 50 TB in size ...
Enter fullscreen mode Exit fullscreen mode

The Strands agent said 5 TB and the ADK agent said 50 TB, and AWS's own page says "each object can be up to 50 TB in size". Both agents had the same AWS docs server. Claude searched once, got a result that mentioned both 5 TB and 50 TB, and answered 5 TB. Gemini kept going, with more searches, and in both runs I traced it read the S3 FAQ page itself before answering. That's why nothing a model says goes into a course unless it comes with a quote from the live page.

The same S3 question asked on both sides, against the same AWS docs server. Claude searches once and answers 5 TB. Gemini searches, reads the S3 FAQ page, searches again, and answers 50 TB

If you use McpToolset yourself, run all your agent calls on one event loop. The toolset keeps its MCP session on the loop it first ran on. My first version called asyncio.run for every agent call, and asyncio.run closes its loop when it's done, so every call after the first found its session on a closed loop.

ADK has its own page loader, load_web_page, but it kept losing the pages we needed. Terraform's docs redirect a lot, and it won't follow a redirect, so it came back with Failed to fetch url: ... instead of the page. When a page did load, it threw away every line of 3 words or fewer, which took out 84 of the 181 lines in one tutorial's code examples, $ terraform init and required_providers { included. So on the ADK side, the agents find the pages, and our own code fetches them in full.

The ADK example also had a generate_content_config line we haven't explained yet. It's there because of the very first real call.

What happened on the first real call?

The researcher's first job is to find the official docs and the current version, and hand them back as a typed plan through output_schema, which is how ADK asks a model for a typed answer. The first real call came back with this:

400 INVALID_ARGUMENT: Please enable tool_config.include_server_side_tool_invocations
to use Built-in tools with Function calling.
Enter fullscreen mode Exit fullscreen mode

Hmm, function calling. That's weird because I hadn't given it a function.

ADK had. When an agent has both output_schema and tools, ADK quietly adds a function tool called set_model_response, and the model hands back its typed answer by calling it. So my agent was sending Google Search, a built in tool, together with a function tool, and the Gemini API wants you to say that's on purpose.

I tried GoogleSearchTool(bypass_multi_tools_limit=True) first and got the same 400, because ADK adds set_model_response after that setting is checked.

The fix was what the error asked for, turning on include_server_side_tool_invocations:

import asyncio

from google.adk.agents import Agent
from google.adk.runners import InMemoryRunner
from google.adk.tools import google_search
from google.genai import types
from pydantic import BaseModel


class Plan(BaseModel):
    official_domains: list[str]
    current_version: str | None = None


planner = Agent(
    name="plan",
    model="gemini-3.8-flash",
    instruction="Find the official docs domain and the current release of the subject. Use google_search.",
    tools=[google_search],
    output_schema=Plan,
    output_key="plan",
    generate_content_config=types.GenerateContentConfig(
        tool_config=types.ToolConfig(include_server_side_tool_invocations=True)
    ),
)


async def main() -> None:
    runner = InMemoryRunner(agent=planner, app_name="plan")
    session = await runner.session_service.create_session(app_name="plan", user_id="me")
    message = types.Content(role="user", parts=[types.Part(text="Terraform")])
    async for _ in runner.run_async(user_id="me", session_id=session.id, new_message=message):
        pass
    done = await runner.session_service.get_session(app_name="plan", user_id="me", session_id=session.id)
    print(done.state["plan"])


if __name__ == "__main__":
    asyncio.run(main())
Enter fullscreen mode Exit fullscreen mode
{'official_domains': ['developer.hashicorp.com', 'terraform.io'], 'current_version': '1.16.4'}
Enter fullscreen mode Exit fullscreen mode

Without the generate_content_config line, the 400 comes back. None of my 235 tests could have caught this, because they ran against a fake model, and a fake model accepts any request.

Which one found the current version?

The plan step asks for the subject's current version. On 27 September Terraform's was 1.16.4, per HashiCorp's own release API. Here's every plan step I ran while the Strands side searched through Exa:

Answer Strands (Claude + Exa) ADK (Gemini + Google Search)
1.16.4 1 8
1.16.3 5 0
1.16.2 3 0
"1.9.x (1.9 series, latest stable as of mid-2024)" 1 0

Twice Claude quoted the Terraform install page as saying "Version: 1.16.3" when the live page said 1.16.4. Exa's copy of the page was a few days behind, and Claude was quoting the copy. ADK's search is Google's index, and Strands' search is whatever server you plug in, so checking how fresh it is falls to you.

So now the version gets checked like every other fact. The plan has to name an official page that states it, with the exact words, and our code fetches that page and checks the words are there. Once Strands had web_fetch to read the live page, it got 1.16.4 too.

There's one bug left here. Every course's README says which version it was written against. If the feedback agent sends research back for more facts, a version our code couldn't find on the live page, like Claude's 1.16.3, can still end up in that README.

How do we get typed answers back?

Every agent in the system hands back a typed object, like a plan, a list of facts or a lesson. Both frameworks take a Pydantic model and give you something shaped like it, and they differ in where you pick it up.

Here's how they both do it:

Strands ADK
structured_output_model= on the call or the agent output_schema= on the agent
The object comes back on result.structured_output A dict lands in session state under output_key
Works next to any tools Adds a function tool, so built in tools need include_server_side_tool_invocations

Strands

We pass the model on the call, and get the object back on the result:

from pydantic import BaseModel
from strands import Agent


class Question(BaseModel):
    stem: str
    options: list[str]
    answer: int  # index into options
    explanation: str


agent = Agent(system_prompt="You write quiz questions about Terraform.", callback_handler=None)
result = agent("One question on what terraform init does.", structured_output_model=Question)

question = result.structured_output
print(type(question).__name__)
print(question.stem)
print(question.options[question.answer])
Enter fullscreen mode Exit fullscreen mode
Question
What is the primary purpose of the `terraform init` command?
It initializes a working directory by downloading providers, modules, and setting up the backend
Enter fullscreen mode Exit fullscreen mode

result.structured_output is a real Question. Strands gives the model a tool shaped like our schema and makes it call it, and if the model never does, you get StructuredOutputException. So the Strands side catches that and asks again, with the reason.

ADK

We put the schema on the agent, and the answer lands in session state:

import asyncio

from google.adk.agents import Agent
from google.adk.runners import InMemoryRunner
from google.genai import types
from pydantic import BaseModel


class Question(BaseModel):
    stem: str
    options: list[str]
    answer: int  # index into options
    explanation: str


agent = Agent(
    name="quiz",
    model="gemini-3.8-flash",
    instruction="You write quiz questions about Terraform.",
    output_schema=Question,
    output_key="question",
)


async def main() -> None:
    runner = InMemoryRunner(agent=agent, app_name="quiz")
    session = await runner.session_service.create_session(app_name="quiz", user_id="me")
    message = types.Content(role="user", parts=[types.Part(text="One question on what terraform init does.")])
    async for _ in runner.run_async(user_id="me", session_id=session.id, new_message=message):
        pass
    done = await runner.session_service.get_session(app_name="quiz", user_id="me", session_id=session.id)
    raw = done.state["question"]
    print(type(raw).__name__)
    question = Question.model_validate(raw)
    print(question.stem)
    print(question.options[question.answer])


asyncio.run(main())
Enter fullscreen mode Exit fullscreen mode
dict
Which of the following actions is performed by running the 'terraform init' command?
Initializes the backend configuration, downloads provider plugins, and installs referenced modules
Enter fullscreen mode Exit fullscreen mode

It comes back as a plain dict, so turning it into a Question is our job. And if the agent ends its turn without writing an answer, state stays empty. That's what stopped the first TypeScript build, with nothing was written to session state under 'answer', so the ADK side also asks again with the reason.

Where does the feedback loop live?

As we saw in Part 1, both frameworks can loop inside a graph. Where they differ is what a node can be, and what stops the loop.

Here's how they both do it:

Strands ADK
GraphBuilder, conditional edges Workflow, routed edges
set_max_node_executions() caps the loop Your router stops the loop
A node has to be an agent or a MultiAgentBase A node can be any function
Hitting the cap ends the graph as FAILED Nothing to hit

The first difference matters here, because our stages aren't agents. The research stage, for example, is Python code that calls several agents and checks every quote in between.

Strands

In Strands, a graph node has to be an agent or a MultiAgentBase, and you only find that out when the graph runs. add_node accepts a plain function, so does build(), and then the graph reaches that node and stops with ValueError: Node 'stage' of type '<class 'function'>' is not supported.

So each stage gets wrapped in a small MultiAgentBase subclass. Here's the loop with stand-in stages that just print, so it runs without a model or any keys:

import asyncio

from strands.multiagent import GraphBuilder
from strands.multiagent.base import MultiAgentBase, MultiAgentResult, Status

board = {"reviews": 0, "verdict": None}


class Stage(MultiAgentBase):
    def __init__(self, name, work):
        super().__init__()
        self.name = name
        self.work = work

    async def invoke_async(self, task, invocation_state=None, **kwargs):
        # the graph runs nodes on its event loop, and real stages block, so off it they go
        await asyncio.to_thread(self.work)
        # results stays empty: Strands walks into it to build the next node's input,
        # so it has to be real results or nothing
        return MultiAgentResult(status=Status.COMPLETED, results={})


def teach():
    print("teaching")


def review():
    board["reviews"] += 1
    board["verdict"] = "revise" if board["reviews"] < 3 else "approved"
    print(f"review {board['reviews']}: {board['verdict']}")


builder = GraphBuilder()
builder.add_node(Stage("teaching", teach), "teaching")
builder.add_node(Stage("feedback", review), "feedback")
builder.add_edge("teaching", "feedback")
builder.add_edge("feedback", "teaching", condition=lambda state: board["verdict"] == "revise")
builder.set_entry_point("teaching")
builder.set_max_node_executions(10)
builder.reset_on_revisit(True)
graph = builder.build()

if __name__ == "__main__":
    result = graph("build the course")
    print(result.status)
Enter fullscreen mode Exit fullscreen mode
teaching
review 1: revise
teaching
review 2: revise
teaching
review 3: approved
Status.COMPLETED
Enter fullscreen mode Exit fullscreen mode

In Stage, results={} has to stay empty. If you put the stage's output there, Strands tries to read it as the next node's input and crashes with AttributeError: 'str' object has no attribute 'results'. So the stages share their work through board, a plain dict, and the graph only decides what runs next.

If the reviewer never approves, the same file stops after 10 node executions and ends Status.FAILED. It doesn't raise, so check result.status.

Without set_max_node_executions, build() only logs a warning, "Graph without execution limits may run indefinitely if cycles exist".

ADK

In ADK, a node can be a plain function, no subclass needed:

import asyncio

from google.adk import Event, Workflow
from google.adk.runners import InMemoryRunner
from google.genai import types

board = {"reviews": 0, "verdict": None}
MAX_REVIEWS = 4


async def teaching() -> None:
    # nodes run on the workflow's event loop, and real stages block, so off it they go
    await asyncio.to_thread(print, "teaching")


def feedback() -> None:
    board["reviews"] += 1
    board["verdict"] = "revise" if board["reviews"] < 3 else "approved"
    print(f"review {board['reviews']}: {board['verdict']}")


def route() -> Event:
    if board["verdict"] == "approved" or board["reviews"] >= MAX_REVIEWS:
        return Event(route="stop")
    return Event(route="revise")


def stop() -> None:
    return None


root_agent = Workflow(name="revision_loop", edges=[
    ("START", teaching, feedback, route),
    (route, {"revise": teaching, "stop": stop}),
])


async def main() -> None:
    runner = InMemoryRunner(agent=root_agent, app_name="loop")
    session = await runner.session_service.create_session(app_name="loop", user_id="me")
    message = types.Content(role="user", parts=[types.Part(text="build the course")])
    async for _ in runner.run_async(user_id="me", session_id=session.id, new_message=message):
        pass


if __name__ == "__main__":
    asyncio.run(main())
Enter fullscreen mode Exit fullscreen mode

It prints the same thing, without the status line. In ADK, route is the only thing that ends this loop. Without the board["reviews"] >= MAX_REVIEWS check, a reviewer that never approves keeps it looping, because nothing in ADK limits how many times a node runs.

What the real system does with it

The learning system's graphs are those two shapes, with each review as a node of its own:

A run through the learning system. The research review sends research round again with its reasons, then forward. The curriculum review finds the problem started in research and sends the run back upstream, and research keeps what it verified before the run goes forward to done

After each stage, a review decides what happens next. The run moves on, the stage runs again with the reviewer's reasons, or the run goes back to research if that's where the problem started. A stage gets 3 tries, and after that the run moves on and the course lists what's still wrong.

Going back to research used to start it again from nothing. On the Strands side, a reviewer found a few gaps, the rerun threw away 306 facts that had already been checked, and that one trip cost $11.81. Now research that comes back keeps everything it already checked and only looks for what's missing.

How do we keep the bill down?

A course takes hundreds of model calls, from picking pages to writing and reviewing every lesson, and more when a review sends something back. The Strands side runs on Claude Sonnet 4.6, which costs 4 times what Gemini 3.8 Flash does per token.

Part 1 covered the limits each framework has built in. Strands takes limits= on a call, and ADK takes RunConfig(max_llm_calls=...) on a run. But those only count within one call or one run, and a course is hundreds of them, so I wanted one number for the whole course, in dollars.

I added a spending cap to every course, and a meter that counts what every agent has spent. Before each model call, the cap checks the meter and refuses the call if we've exceeded the cap. Both frameworks give us a hook in exactly the right place, and they attach differently.

Here's how they both do it:

Strands ADK
add_hook on an agent you already have before_model_callback on the agent
Usage is on agent.event_loop_metrics Usage is on each llm_response
Raise, and it comes out as is, or wrapped once a tool has run Raise, and it comes back as an error event first

Strands

Strands already counts each agent's tokens in agent.event_loop_metrics, so the hook just reads them:

from strands import Agent
from strands.hooks import BeforeModelCallEvent

PRICE = (3.00, 15.00)  # USD per million tokens, in and out, for Claude Sonnet 4.6
CAP = 0.001


class BudgetExceeded(RuntimeError):
    pass


def spent(agent: Agent) -> float:
    usage = agent.event_loop_metrics.accumulated_usage
    return (usage["inputTokens"] * PRICE[0] + usage["outputTokens"] * PRICE[1]) / 1_000_000


def enforce_cap(event: BeforeModelCallEvent) -> None:
    if spent(event.agent) >= CAP:
        raise BudgetExceeded(f"spent ${spent(event.agent):.4f}, the cap is ${CAP}")


agent = Agent(system_prompt="Answer in one short paragraph.", callback_handler=None)
agent.add_hook(enforce_cap)

print(agent("What does terraform init do?"))
print(f"spent ${spent(agent):.4f}")
try:
    print(agent("And terraform plan?"))
except BudgetExceeded as e:
    print("stopped:", e)
Enter fullscreen mode Exit fullscreen mode
`terraform init` initializes a Terraform working directory by downloading ...
spent $0.0018
stopped: spent $0.0018, the cap is $0.001
Enter fullscreen mode Exit fullscreen mode

BeforeModelCallEvent also has a cancel field, which looks like the obvious tool. Cancelling hands the loop a fake assistant message, "model call denied by hook", so a plain call returns that as if the model had said it, and a structured call fails blaming a model that never got the call. Raising says what actually happened.

There's one more catch, and it only shows up in a real run. On the first model call of an invocation, your exception comes out exactly as you raised it, which is all the example above ever does. On any call after a tool has run, Strands wraps it in its own EventLoopException. My research agents use tools, so the first time the cap fired for real, except BudgetExceeded didn't match and the run died with a traceback. The fix is to look through the exception's causes.

ADK

Here we pass ADK the callbacks when we build the agent, and count the tokens as each response comes back:

import asyncio

from google.adk.agents import Agent
from google.adk.runners import InMemoryRunner
from google.genai import types

PRICE = (0.75, 3.75)  # USD per million tokens, in and out, for Gemini 3.8 Flash
CAP = 0.0001
tokens = {"in": 0, "out": 0}


class BudgetExceeded(RuntimeError):
    pass


def spent() -> float:
    return (tokens["in"] * PRICE[0] + tokens["out"] * PRICE[1]) / 1_000_000


def enforce_cap(callback_context, llm_request):
    if spent() >= CAP:
        raise BudgetExceeded(f"spent ${spent():.6f}, the cap is ${CAP}")


def count(callback_context, llm_response):
    usage = llm_response.usage_metadata
    if usage:
        tokens["in"] += usage.prompt_token_count or 0
        # thinking tokens are billed as output and reported separately
        tokens["out"] += (usage.candidates_token_count or 0) + (usage.thoughts_token_count or 0)


agent = Agent(
    name="tutor",
    model="gemini-3.8-flash",
    instruction="Answer in one short paragraph.",
    before_model_callback=enforce_cap,
    after_model_callback=count,
)


async def ask(runner, session_id, text):
    message = types.Content(role="user", parts=[types.Part(text=text)])
    async for event in runner.run_async(user_id="me", session_id=session_id, new_message=message):
        if event.error_code:
            print("error event:", event.error_code)
        elif event.is_final_response():
            print(event.content.parts[0].text)


async def main():
    runner = InMemoryRunner(agent=agent, app_name="tutor")
    session = await runner.session_service.create_session(app_name="tutor", user_id="me")
    await ask(runner, session.id, "What does terraform init do?")
    print(f"spent ${spent():.6f}")
    try:
        await ask(runner, session.id, "And terraform plan?")
    except BudgetExceeded as e:
        print("stopped:", e)


asyncio.run(main())
Enter fullscreen mode Exit fullscreen mode
The `terraform init` command initializes a working directory containing Terraform ...
spent $0.001424
error event: BudgetExceeded
stopped: spent $0.001424, the cap is $0.0001
Enter fullscreen mode Exit fullscreen mode

Two things are different here. Gemini 3.8 Flash thinks, and thinking is billed as output but reported separately as thoughts_token_count, so a meter that only adds candidates_token_count undercounts every call. And the exception comes back twice, first as an error event with error_code set to its class name, then raised again when the stream ends. So every loop over ADK's events checks for an error event and stops straight away.

The bigger difference is that ADK can put the cap on every agent at once. In both frameworks we can add enforce_cap to an agent after it's built, with add_hook on Strands, or by setting the agent's before_model_callback on ADK. But ADK also has plugins, which we register once for the whole app. A plugin's before_model_callback runs before every model call, for every agent. Strands has nothing like that, so each Strands agent needs its own hook.

What did it cost?

Here's what each course cost, from the meters, at list prices:

Course Built with Cost
Terraform ADK, Gemini 3.8 Flash $3.35 for the lessons (research and planning ran before the meter saved anything)
Terraform Strands, Claude Sonnet 4.6 $18.36: research $7.06, planning $0.87, 23 lessons $10.43
TypeScript ADK, Gemini 3.8 Flash, through learn $2.94, everything included

Across the 2 days of building, with the lost runs, the restarts and the bugs, Claude on my Bedrock account came to about $59, going by AWS's Cost Explorer.

Two bugs let it get that high. The Strands meter remembered each agent's usage by id(agent), and Python hands a new object the id of one that's been thrown away, so fresh agents inherited dead ones' totals and the meter undercounted. One run reported -834,706 input tokens for a single role. And the cap lived in the process, so every restart got a fresh budget. Both are fixed now. Each agent's count lives in its own hooks, and spend is saved to disk the moment it's counted, so a restarted course carries on from what it already spent.

A cap that just stops leaves you with half a course, though. So now, halfway to the cap, the course switches to a cheaper model to finish, and tells you:

  $5.00 spent of your $10.00 cap, so the rest of this course is written by a cheaper model
  (global.anthropic.claude-haiku-4-5-20251001-v1:0).
Enter fullscreen mode Exit fullscreen mode

On Strands that's Claude Haiku 4.5, at $1 in and $5 out per million tokens against Sonnet 4.6's $3 and $15. On ADK it's Gemini 3.5 Flash-Lite, at $0.30 and $2.50. Before trusting either, I asked each one to find Terraform's current version through the system's own Brain, with its tools and typed answers. Both found 1.16.4 on the live install page.

Swapping the model mid-course is easy on both sides, because each call builds a fresh agent anyway. Strands takes a different BedrockModel, and ADK takes a different model name. The cap, the switch point and the cheaper model are all settings in one file.

Bedrock's US-only inference profile costs 10% more than the global one, which I only found by asking AWS's price list. And Gemini 3.8 Flash's price doubles on 1 January 2027, to $1.50 in and $7.50 out.

Newer, cheaper Claude models came out while I was writing this. Sonnet 5.5 came to Bedrock on 28 September at $2 in and $10 out, and Haiku 5.5 on 7 October, from $0.10 in and $0.50 out.

Is the course any good?

Here's what each side produced from the same request, "I'd like to deeply understand Terraform and build projects along the way."

Strands (Claude) ADK (Gemini)
Facts checked against a live page 546, from 39 sources 162, from 19 sources
Lessons 23 in 5 units 12 in 4 units
Exercises that pass every check 12 of 23 12 of 12
Lessons that ship with a known problem 23 1
Cost $18.36 $3.35, plus research and planning

The TypeScript course from earlier was built through learn. It has 12 lessons, all 12 exercises pass every check, and nothing is left on its known problems list, for $2.94.

I planned to judge the courses by their review scores. Then I did the lessons myself, and found more than the reviewers had.

The exercises

The first finished ADK course had 5 exercises out of 12 that failed their own checks, and 4 of them hit the same error, Unknown condition value ... This means you have executed a run block with command = plan. That one was my fault.

I'd told the teacher to write the tests with command = plan, and at plan time things like IDs don't exist yet. The tests run against a mock provider (i.e. a fake AWS), so command = apply doesn't create anything real or need an AWS account. It also makes up values like IDs, so the tests have something to check.

After that fix, I rebuilt the ADK course and worked through lesson 1 myself. I left one of its 3 TODOs alone, filled in the other two, and learn check still said "Passed". The system's rule was that an untouched exercise (i.e. every TODO left alone) has to fail its tests before the course ships. If the tests check just one TODO, that rule passes, so the TODO I left alone was never tested.

So now the system also checks each TODO on its own. It fills in every TODO with the answer except one, and the tests have to fail. It does that for each TODO in turn, so you can't skip any of them and still pass. 3 of the 12 lessons in that course had a TODO you could skip.

Even with every TODO checked on its own, a wrong answer can still pass. In unit 3, lesson 3 of the same course, a TODO asks you to output the ID of the EC2 instance the lesson builds. If you output its ARN instead, learn check still says "Passed". The mock provider makes up both values as random 8-character strings, so the test can't tell them apart.

TypeScript had a sneakier version. Node runs TypeScript by stripping the types out without checking them, so an exercise about types could pass with no types at all. So TypeScript exercises also go through the real compiler in strict mode, which is the TS7006 you saw earlier.

The quizzes

Here's where the right answer sat:

Right answer is option 1 2 3 4
ADK (Gemini) 30 5 1 0
Strands (Claude) 14 40 15 0

Gemini put the right answer first 83% of the time, and Claude put it second 58% of the time, so a learner who always picked 1 would pass most of ADK's quizzes. The course writer shuffles the options now, and after shuffling, the ADK course's 36 right answers are spread 10, 7, 9 and 10 across the 4 positions.

The reviewers

Gemini's reviewer signed off on all 12 of ADK's final lessons. Claude's reviewer, with the same prompt, didn't sign off on any of the Strands lessons it reviewed. So I swapped them. Claude reviewed ADK's 12 lessons and found 29 problems, at least one in every lesson, and most were real. One was a worked example that says terraform fmt scans the .terraform folder (that's where providers are cached), and another was a lesson about terraform state mv whose exercise never has you run it. Gemini reviewed Strands' 23 and passed 13 of them.

So Claude's course looks worse on paper because its reviewer is harder to please, and Gemini's looks clean partly because its reviewer isn't.

What the courses missed

Remember item 2 on my list, a course that covers what people actually do at work? Where I work, we publish Terraform modules, and every team that uses one pins a version, with ?ref= on a Git source or version on a registry module, and bumps it when there's a new release. We do that all the time.

Neither course taught it as something you do. The ADK course mentions it in one lesson's worked example and quiz, and the Strands course mentions it once in passing. Both researchers built their plans from how the docs are organised, and the docs don't put "publish a module and pin it downstream" on the front page.

That's a gap in the research, so that's where the fix went. The researcher now writes a list of what someone who uses the subject at work does routinely, and names the objective that teaches each item. A plain check flags any item with no objective, and the research reviewer reads each pairing and says when an objective only teaches part of an item.

Then I ran the same Terraform research twice more. The check did its job, and in both runs the reviewer caught items that were only half taught, and the research added objectives for them. Module pinning still didn't make it in, because neither run put it on the list in the first place. Of the 6 research runs that wrote this list, 2 put module pinning on it. The check can only hold the objectives to the list, and the list is still the model's idea of what people do at work.

What I think fixes it is asking. The intake already asks where you're starting from. If it also asked how your team uses the subject, I'd have told it we pin shared modules, and the research would have had to cover that.

Both courses say their exercises run against LocalStack, an AWS emulator (i.e. it pretends to be AWS on your own machine), but neither shows you how to set LocalStack up. In the ADK course, that just means the LocalStack steps fail. The Strands course could cost you money, though, because its last 6 lessons set Terraform up to talk to real AWS. Two of those lessons tell you to run terraform apply, so if you have AWS credentials on your machine, you'd create real VPCs and EC2 instances in your account.

learn check is safe in every lesson, because its tests run against the mock provider and never touch AWS. The Strands course's README warns about this at the top now.

What I think fixes it is running AWS exercises against an emulator by default, and letting you switch to your own AWS account if you want the real thing.

So which one?

Here's the whole comparison in one place:

Strands ADK
Web search Bring an MCP server google_search, Gemini only
Reading a page web_fetch, markdown or agentic load_web_page, which I couldn't use
Typed answers structured_output_model= on the call or the agent, the object on the result output_schema on the agent, a dict in session state, plus a flag next to built in tools
The feedback loop GraphBuilder, capped by set_max_node_executions Workflow, stopped by your router
A stage that isn't an agent Subclass MultiAgentBase Any function
A spending cap add_hook on each agent A callback on each agent, or one plugin for every agent
Errors from your hooks Wrapped in EventLoopException once a tool has run An error event first, raised at the end

Both frameworks ran the same stages through the same kind of graph, with the same typed answers.

How good the courses came out was down to the models and to my checks. Claude found 3 times as many facts and wrote twice as many lessons, at 4 times Gemini's price per token, and its reviewer was much harder to please. Gemini was cheaper, and its reviewer passed its own side's lessons. Most of the problems I found doing the lessons came from a check I hadn't written yet, or one I'd written wrong.

So which one would I pick? For myself, Strands. But ADK did some things better, and if those matter more for what you're building, it's the better pick.

Strands was easier to start with. Once AWS credentials and access to the model in Bedrock are set up, an agent is one Python file and three lines of code, and as it runs it prints each tool it calls. It also had set_max_node_executions, which stops the graph once it has run a set number of nodes. ADK has nothing like it, and it was my backstop in case the feedback loop ever went around forever.

ADK is lighter to set up, because a Gemini API key is enough to run it on your machine. adk web gives you a chat in the browser with a trace of every call next to it. Any plain function can be a stage in the workflow, where Strands needs it wrapped in a class. And its google_search tool found the current Terraform version 8 times out of 8, though it only works with Gemini.

And whichever framework you pick, get a second model to review the first.

Both Terraform courses and the TypeScript course are in the repo, and each lesson lists its known problems at the top. If you work through one and find something they missed, open an issue and tell me.

Top comments (0)