mech.app

Posted on Sep 19 Originally published at mech.app

Scry: Congestion Pricing as Agent Rate-Limiting Infrastructure

#agents #api #infrastructure #mcp

Scry is a programmable search API for agents that replaces hard rate limits with congestion pricing. Instead of getting a 429 when you hit a quota, you get a price signal. The agent (or its orchestrator) decides whether to pay more or back off. This shifts rate-limiting from a binary wall into an economic feedback loop.

The infrastructure question is not whether congestion pricing works in theory. It is how agents read and react to dynamic cost signals, how multi-agent systems arbitrate shared budgets, and whether retry logic amplifies congestion instead of damping it.

What Scry Actually Does

Scry exposes an MCP server that lets agents run SQL-like queries over 164 billion rows of indexed internet documents: Reddit, Hacker News, arXiv, Stack Exchange, Wikipedia, prediction markets. Agents query the underlying records instead of scraping search results.

The pricing model is congestion-based. When load spikes, the cost per query rises. When load drops, cost falls. The agent sees the price before it commits the query and can decide to proceed, defer, or cancel.

Key primitives:

/v1/scry/schema returns the live schema and current pricing for each relation.
Queries run under a deadline, memory ceiling, and row cap.
Each query returns a cost in the response metadata.
Agents can inspect cost before execution if the SDK supports dry-run or estimate endpoints.

This is not a token bucket. It is a spot market for query capacity.

Congestion Pricing as a Rate-Limiting Primitive

Traditional rate limits are binary. You get 100 requests per minute. Request 101 gets a 429. The agent retries with exponential backoff or queues the request.

Congestion pricing replaces the binary with a gradient. The agent sees:

Current price per query.
Historical price distribution (if the API exposes it).
Remaining budget (if the orchestrator tracks it).

The agent can:

Execute immediately if the price is acceptable.
Defer the query if the price is high and the task is not urgent.
Cancel the query if the price exceeds the value of the answer.

Trade-offs:

Approach	Predictability	Burst Handling	Agent Complexity	Cost Control
Hard rate limit	High	Poor (429 storms)	Low (retry logic)	Fixed ceiling
Token bucket	Medium	Good (burst allowance)	Medium (token awareness)	Fixed ceiling
Congestion pricing	Low	Excellent (price signal)	High (cost-aware logic)	Dynamic, requires budget arbiter

The agent needs cost-aware decision logic. If the orchestrator does not expose pricing signals to the agent, the agent cannot react. If the agent does not have a budget, it will pay any price.

Multi-Agent Budget Arbitration

When multiple agents share a single billing account, congestion pricing creates a coordination problem. Who gets to spend the budget when prices spike?

Three arbitration models:

Centralized arbiter: The orchestrator tracks cumulative spend and allocates budget to agents. Agents request permission before executing expensive queries. This adds latency but prevents budget exhaustion.
Agent bidding: Each agent declares a willingness-to-pay for each query. The orchestrator sorts by bid and executes queries until the budget is exhausted. This is efficient but requires agents to estimate query value.
No arbiter: Agents execute queries until the account balance hits zero. First-come-first-served. This is simple but can starve low-priority agents or blow the budget on low-value queries.

Scry does not enforce an arbitration model. The orchestrator must implement it. If you run ten agents on one account with no arbiter, the first agent to wake up can drain the budget.

Implementation sketch for centralized arbiter:

class BudgetArbiter:
    def __init__(self, total_budget: float):
        self.remaining = total_budget
        self.lock = asyncio.Lock()

    async def request_query(self, agent_id: str, estimated_cost: float) -> bool:
        async with self.lock:
            if estimated_cost > self.remaining:
                return False  # Deny
            self.remaining -= estimated_cost
            return True  # Approve

    async def refund(self, actual_cost: float, estimated_cost: float):
        async with self.lock:
            self.remaining += (estimated_cost - actual_cost)

The arbiter needs cost estimates before execution. If Scry exposes a /v1/scry/estimate endpoint, the agent can call it before requesting approval. If not, the agent must use historical cost data or a conservative upper bound.

Retry Loops and Congestion Amplification

Standard retry logic uses exponential backoff tied to time. If a request fails, wait 1 second, then 2, then 4. This works for transient failures but not for congestion pricing.

If the price spikes and the agent retries immediately, it pays the high price. If the agent waits and the price is still high, it retries again. If many agents do this, retries amplify congestion instead of damping it.

Cost-aware backoff:

If the query fails due to high cost (not a 429, but a cost threshold), the agent should back off exponentially based on price, not time.
If the price is 10x the baseline, wait longer than if the price is 2x.
If the price is falling, retry sooner.

The SDK needs to expose price history or a price trend signal. Without it, the agent is blind.

Example backoff logic:

async def query_with_backoff(query: str, max_cost: float, max_retries: int = 5):
    for attempt in range(max_retries):
        price = await scry.get_current_price(query)
        if price <= max_cost:
            return await scry.execute(query)

        # Exponential backoff scaled by price ratio
        price_ratio = price / max_cost
        wait_time = min(2 ** attempt * price_ratio, 300)  # Cap at 5 minutes
        await asyncio.sleep(wait_time)

    raise Exception("Query cost exceeded budget after retries")

This prevents the agent from hammering the API during price spikes. But it requires the SDK to expose real-time pricing.

Observability and Cost Attribution

Congestion pricing makes cost attribution critical. If an agent runs 1,000 queries and the bill is $500, which queries were expensive? Which agents are burning budget?

Required observability:

Per-query cost in response metadata.
Per-agent cumulative spend.
Price history over time (to detect spikes).
Query latency vs. cost correlation (to detect whether high cost buys speed).

If the orchestrator does not log per-query cost, you cannot debug budget overruns. If you cannot correlate cost with query type, you cannot optimize agent behavior.

Scry returns cost in the response, but the orchestrator must log it. If you use LangChain or a similar framework, you need a custom callback to capture cost metadata.

Deployment Shape

Scry runs as an MCP server. The agent (ChatGPT, Claude, Cursor, any MCP client) connects to https://mcp.scry.io. No local infrastructure. Sign-in creates the account.

Failure modes:

Price spike during critical query: The agent defers or cancels, and the task fails. Mitigation: set a high max_cost for critical queries.
Budget exhaustion mid-task: The orchestrator runs out of money, and all agents stop. Mitigation: reserve budget for high-priority agents.
Cost estimation error: The agent approves a query based on an estimate, but the actual cost is higher. Mitigation: refund logic in the arbiter.
Retry amplification: Agents retry during price spikes, driving the price higher. Mitigation: cost-aware backoff.

Security boundaries:

API key per account. If an agent leaks the key, any process can drain the budget.
No per-agent sub-keys. The orchestrator must enforce budget limits in-process.
Query deadlines and row caps prevent runaway queries, but not runaway cost.

When Congestion Pricing Makes Sense

Congestion pricing works when:

Query value varies widely (some answers are worth $1, others $0.01).
Load is bursty (price signals smooth demand).
Agents can defer non-urgent queries (elastic demand).

It does not work when:

All queries are equally urgent (no deferral strategy).
Agents cannot estimate query value (cannot decide whether to pay).
The orchestrator cannot track spend (budget overruns are invisible).

Technical Verdict

Use Scry if you are building multi-agent systems that need large-scale internet search and you can implement cost-aware orchestration. The congestion pricing model is a better fit for agent workloads than hard rate limits, but only if your orchestrator can track spend, arbitrate budgets, and expose pricing signals to agents.

Avoid it if your agents cannot defer queries, you need predictable costs, or you are running a single agent with simple retry logic. In those cases, a fixed-rate API with token buckets is simpler.

The infrastructure gap is observability. Scry returns cost per query, but you need to log it, aggregate it, and feed it back into agent decision logic. If your orchestration layer does not do this, you will burn budget without knowing why.

DEV Community