What happens to a workflow if generation becomes 10× faster but the tools keep you waiting? I tested that question with scripted delays, then ran a separate experiment with real HTTPS requests. No model was timed.
So your agent gets ten times faster, right? 🚀
Let's find out with a small Python lab. Then let's do it again with real network calls, because that's where it got interesting.
Round 1: a fake agent with honest timers 🧪
An agent request usually looks like this: the model writes for a while, then it calls some tools (search, a database, an API), then it continues. I used timers for both parts so you can see exactly where every second goes.
- Simulated generation: a delay calculated as if producing 60 tokens. At 100 tokens/second that's 0.6 s. At 1,000 tokens/second (10× faster) it's 0.06 s.
- Three tools: they take 0.2, 0.4 and 0.6 seconds. They don't depend on each other.
I tried two upgrades, alone and together: a 10× shorter simulated generation delay, and running the three tools at the same time instead of one after another.
Save this as wait_lab.py:
import asyncio
import statistics
import time
conditions = [(100, False), (1000, False), (100, True), (1000, True)]
async def trial(rate, parallel):
start = time.perf_counter()
await asyncio.sleep(60 / rate) # pretend the model is writing
if parallel:
await asyncio.gather(*(asyncio.sleep(t) for t in (0.2, 0.4, 0.6)))
else:
for delay in (0.2, 0.4, 0.6):
await asyncio.sleep(delay)
return time.perf_counter() - start
async def main():
samples = {condition: [] for condition in conditions}
for repeat in range(5):
order = conditions if repeat % 2 == 0 else list(reversed(conditions))
for rate, parallel in order:
samples[(rate, parallel)].append(await trial(rate, parallel))
for condition in conditions:
values = samples[condition]
print(condition, 'median', round(statistics.median(values), 4))
asyncio.run(main())
Run python3 wait_lab.py. No packages needed. asyncio.gather is what runs things at the same time [1].
Here's what my run gave (median of 5 tries each, Python 3.14.7):
| Setup | Time | Speedup |
|---|---|---|
| Base generation delay, tools one by one | 1.805 s | 1× |
| 10× shorter delay, tools one by one | 1.264 s | 1.43× |
| Base generation delay, tools together | 1.203 s | 1.50× |
| 10× shorter delay + tools together | 0.663 s | 2.72× |
Read that again. The 10× shorter generation delay made this scripted workflow 1.43× faster. Overlapping the independent tool delays produced a larger speedup in this scenario.
My measured run. Timers, not a real model: this is about where time goes, not about Mercury itself.
Original meme. Emoji: Twemoji, CC-BY 4.0.
The math, in one line 💡
This has a name: Amdahl's law [3]. It says speeding up one part only helps as much as that part's share of the total.
If the model's writing takes a fraction p of the time, and you make it s times faster:
speedup = 1 / ((1 - p) + p / s)
In the nominal timer budgets, simulated generation takes 0.6 of 1.8 seconds, so p = 1/3. With s = 10: 1 / (2/3 + 1/30) = 1.43×. That matches the measured speedup to rounding.
Three sanity checks that make it click:
- If the model was 0% of the time, a faster model changes nothing.
- If it was 100% of the time, you get the full 10×.
- Even an infinitely fast model can't beat
1 / (1 - p). Here that's 1.5×.
Calculated from the formula, not measured. Notice how flat the curves are until the model is most of the wait.
Round 2: real network calls. Plot twist. 🌐
Timers are polite: my scripted delays mostly add a little scheduler overhead. Real requests have more moving parts.
So I replaced the timers with three real requests to PyPI's JSON API [2]: package info for numpy, pandas and requests. One after another, then all together. Five rounds each, alternating which went first. Then I did it again with HEAD requests, which ask the same URLs but skip downloading the body.
Save this as network_lab.py and run python3 network_lab.py. It uses the standard library, makes 66 requests including warm-ups, and saves every timed request to network-results.json. Run it once, rather than hammering a public service. Your medians are a new observation, not a reproduction of the exact numbers below.
import asyncio
import json
import statistics
import time
import urllib.request
from pathlib import Path
PACKAGES = ("numpy", "pandas", "requests")
def fetch(package, method):
start = time.perf_counter()
request = urllib.request.Request(
f"https://pypi.org/pypi/{package}/json", method=method)
with urllib.request.urlopen(request, timeout=20) as response:
body = response.read()
status = response.status
return dict(package=package, status=status, bytes=len(body),
seconds=time.perf_counter() - start)
async def trial(method, parallel):
start = time.perf_counter()
if parallel:
rows = await asyncio.gather(*(
asyncio.to_thread(fetch, p, method) for p in PACKAGES))
else:
rows = [await asyncio.to_thread(fetch, p, method) for p in PACKAGES]
return dict(seconds=time.perf_counter() - start, requests=rows)
async def main():
results = {}
for method in ("GET", "HEAD"):
await trial(method, False) # unrecorded warm-up; no connection reuse assumed
samples = {"sequential": [], "parallel": []}
for repeat in range(5):
order = ("sequential", "parallel")
if repeat % 2:
order = tuple(reversed(order))
for name in order:
samples[name].append(await trial(method, name == "parallel"))
medians = {name: statistics.median(r["seconds"] for r in rows)
for name, rows in samples.items()}
results[method] = dict(samples=samples, medians=medians)
print(method, medians, "speedup",
medians["sequential"] / medians["parallel"])
Path("network-results.json").write_text(json.dumps(results, indent=2))
asyncio.run(main())
Real HTTPS requests from one home connection. Your numbers will differ.
In the retained five-round run, the full downloads did not get faster together: 0.95×. The concurrent median was slightly slower. 🫠
The tiny HEAD requests got 1.66× faster.
I also reran the literal code above during this revision. That separate five-round run gave 1.22× for GET and 1.50× for HEAD. I kept both sets of raw samples. The full-download result changed direction; the original chart records its own run, not a stable prediction for your connection.
Look at each download alone versus together in that run:
-
numpy(3.7 MB): 0.27 s alone, 0.53 s together. -
pandas(2.2 MB): 0.16 s alone, 0.42 s together. -
requests(0.2 MB): 0.07 s alone, 0.10 s together.
Together, each download got slower. Shared bandwidth is one possible explanation, but I didn't measure saturation. DNS/TLS setup, CDN behavior, caching, server load and thread overhead can also affect these timings. Changing GET to HEAD changes both the request method and the transferred body; it doesn't isolate a single cause.
Original meme with my measured numbers. Emoji: Twemoji, CC-BY 4.0.
⚠️ Common mistake: wrapping every tool call in
gatherand expecting a speedup. Calls must be independent to overlap safely. Independent, latency-bound calls often benefit most; big downloads, a shared rate limit or a busy database can reduce or reverse the gain.⚠️ Common mistake: judging a model by tokens per second alone. Time the whole thing, from request to useful answer, and write down how much was model and how much was tools. A fast wrong answer is still wrong.
Fine print
Round 1 uses timers, not a model. Round 2 uses real requests, but from one connection, in one observation session, with five rounds each. None of this measures Mercury or any other model's real speed. A real agent also has first-token delay, retries and tools that depend on each other, and a new model can change how many steps it takes.
What it does show: before paying for speed, find out where your time actually goes. Sometimes the fix is a faster model. Sometimes it's one gather. And sometimes it's neither. 🐢
References
Asset credits: charts and memes are original, made from my saved results. Emoji graphics: Twemoji, CC-BY 4.0.
AI disclosure
AI was used to draft the article.
Top comments (0)