Market mechanisms like double auctions rely on assumptions about participant behavior. Humans converge toward equilibrium prices through repeated rounds of bidding. A new paper from Struski et al. replaces human traders with LLM agents and finds those assumptions break. Markets populated by GPT-4, Claude, and Llama agents exhibit slower convergence or none at all, delivering less efficient resource allocation than human-run markets.
This matters if you are building procurement bots, automated negotiation systems, or any agent that participates in price discovery. The plumbing question is not whether LLMs can generate plausible bids. It is whether the market rules you inherited from human-designed mechanisms still produce the outcomes you expect when agents replace people.
What the Experiment Tested
The researchers replicated classic double auction experiments. In a double auction:
- Buyers submit bids.
- Sellers submit asks.
- Trades execute when a bid meets or exceeds an ask.
- Participants adjust prices over multiple rounds.
Each agent received a private valuation (buyers) or cost (sellers). The goal was to see if the market cleared at the theoretical equilibrium price, which maximizes total surplus.
The paper tested multiple model families (GPT-4, Claude 3.5, Llama 3.1) in buyer and seller roles across repeated auction rounds. Agents used Chain-of-Thought (CoT) prompting to expose reasoning traces.
Key Findings
Convergence failure. Human markets typically converge to equilibrium within a few rounds. LLM agent markets either converged much slower or not at all. Efficiency (total surplus captured) was lower.
Heterogeneity across models. Different model families exhibited different bidding strategies. Some models were more aggressive, others more conservative. This heterogeneity did not average out to equilibrium behavior.
Urgency over strategy. Lexical analysis of CoT traces showed that when agents decided to execute a trade rather than continue adjusting prices, the reasoning shifted from strategic considerations (e.g., "I should wait for a better price") to urgency cues (e.g., "I need to close this deal now"). This suggests agents may lack the patience or temporal reasoning to let markets clear naturally.
No evidence of explicit collusion. The paper did not find agents coordinating to fix prices, but the lack of convergence itself is a failure mode. Markets that do not clear leave value on the table.
Architecture for Market Mechanism Testing
If you want to replicate this or test your own agents in auction environments, here is the plumbing you need.
State Representation
Each agent must track:
- Private valuation or cost. The agent's reservation price.
- Price history. Bids and asks from prior rounds.
- Trade history. Which transactions executed and at what price.
- Budget or inventory constraints. If applicable.
The state object passed to the agent might look like this:
state = {
"role": "buyer",
"valuation": 50,
"round": 3,
"price_history": [
{"round": 1, "bids": [45, 48], "asks": [52, 55], "trades": []},
{"round": 2, "bids": [47, 49], "asks": [51, 53], "trades": [{"price": 51}]}
],
"budget_remaining": 200
}
Instrumentation
You need to log:
- Bid/ask decisions. What price did the agent submit?
- CoT traces. The reasoning behind each decision.
- Trade execution. Which bids and asks matched.
- Convergence metrics. Distance from theoretical equilibrium over time.
Store these in a time-series database (InfluxDB, TimescaleDB) or structured logs (JSON lines) for post-hoc analysis.
Preventing Inference-Time Gaming
If your agents can access the internet or have been trained on auction theory papers, they might game the evaluation. Mitigations:
- Isolated inference. Run agents in a sandboxed environment with no network access.
- Prompt injection detection. Monitor for attempts to reference the paper or known equilibrium strategies.
- Randomized valuations. Use private valuations drawn from distributions the agent has not seen in training data.
Collusion Detection
Even without explicit coordination, agents might exhibit emergent collusion (e.g., all sellers refusing to lower asks). Detect this by:
- Price clustering. If all asks remain above equilibrium for many rounds, flag it.
- CoT keyword analysis. Search for phrases like "wait for others to lower" or "hold the line."
- Counterfactual testing. Replace one agent with a known-rational baseline and see if the market clears.
Trade-Off Table: Human vs. LLM Agent Markets
| Dimension | Human Markets | LLM Agent Markets |
|---|---|---|
| Convergence speed | Fast (3-5 rounds typical) | Slow or absent (10+ rounds, often no equilibrium) |
| Efficiency | High (90%+ surplus captured) | Lower (varies by model, often <80%) |
| Strategy diversity | Moderate (some aggressive, some patient) | High (model-dependent, unpredictable) |
| Temporal reasoning | Strong (humans wait for better prices) | Weak (urgency overrides strategy) |
| Collusion risk | Requires explicit coordination | Emergent patterns possible without intent |
| Observability | Limited (self-reports, surveys) | High (CoT traces, full logs) |
Implementation Sketch
Here is a minimal double auction orchestrator in Python:
import random
from typing import List, Dict
class DoubleAuction:
def __init__(self, buyers: List, sellers: List):
self.buyers = buyers
self.sellers = sellers
self.trade_history = []
self.round = 0
def run_round(self):
self.round += 1
bids = [b.submit_bid(self.get_state(b)) for b in self.buyers]
asks = [s.submit_ask(self.get_state(s)) for s in self.sellers]
# Match highest bid with lowest ask
bids_sorted = sorted(bids, key=lambda x: x['price'], reverse=True)
asks_sorted = sorted(asks, key=lambda x: x['price'])
trades = []
for bid in bids_sorted:
for ask in asks_sorted:
if bid['price'] >= ask['price']:
trades.append({
'buyer': bid['agent_id'],
'seller': ask['agent_id'],
'price': (bid['price'] + ask['price']) / 2,
'round': self.round
})
asks_sorted.remove(ask)
break
self.trade_history.extend(trades)
return trades
def get_state(self, agent):
return {
'role': agent.role,
'valuation': agent.valuation,
'round': self.round,
'price_history': self.trade_history,
'budget_remaining': agent.budget
}
Each agent's submit_bid or submit_ask method calls an LLM with the state object and a prompt like:
You are a buyer in a double auction. Your private valuation is $50.
Current round: 3
Recent trades: [{"price": 51, "round": 2}]
Recent bids: [47, 49]
Recent asks: [51, 53]
Submit your bid for this round. Explain your reasoning step by step.
Parse the LLM response for the bid price and log the CoT trace.
Failure Modes
Agents ignore price history. If the LLM does not condition on past rounds, it will not adjust bids toward equilibrium. Solution: include price history in the prompt and test with ablations.
Urgency bias. Agents may execute trades too early because the prompt implies time pressure. Solution: use neutral language ("you may submit a bid") rather than urgent framing ("you must decide now").
Model-specific quirks. One model family might always bid aggressively, another conservatively. Solution: test multiple models and log per-model metrics.
No convergence signal. If the orchestrator runs a fixed number of rounds, agents have no incentive to converge. Solution: allow indefinite rounds until a stopping condition (e.g., no new trades for N rounds).
Observability Stack
To debug why agents fail to converge, you need:
- Bid/ask logs. Time-series of all submitted prices.
- CoT traces. Full reasoning for each decision.
- Convergence dashboard. Plot distance from equilibrium over time.
- Agent comparison. Side-by-side CoT traces for different models in the same market.
Store logs in structured JSON:
{
"round": 3,
"agent_id": "buyer_gpt4_1",
"role": "buyer",
"valuation": 50,
"bid": 48,
"cot": "The last trade was at $51, which is above my valuation. I should bid lower to avoid overpaying. I'll try $48.",
"timestamp": "2026-09-03T10:15:32Z"
}
Query with SQL or jq to find patterns (e.g., all agents who bid above valuation, all CoT traces mentioning "urgency").
When to Use This
You are deploying agents in procurement or negotiation. If your agents will participate in real markets, test them in simulated auctions first. Measure convergence and efficiency before risking capital.
You are building multi-agent coordination systems. Auction mechanisms are a proxy for any scenario where agents must coordinate without explicit communication. If they fail here, they will fail in task allocation, resource sharing, or collaborative planning.
You need to detect emergent collusion. Even if agents do not intend to collude, their behavior might produce collusive outcomes. Instrumented auctions let you catch this early.
When to Avoid This
Your agents do not interact with markets. If you are building a single-agent assistant or a pipeline with no competitive dynamics, auction testing is overkill.
You cannot afford the inference cost. Running dozens of agents through multiple auction rounds burns tokens. Budget accordingly.
You need real-time decisions. Auctions with CoT reasoning are slow. If you need sub-second bids, you will need a different architecture (e.g., fine-tuned models without CoT).
Technical Verdict
LLM agents do not naturally converge in double auctions the way humans do. If you are building systems that rely on market mechanisms for price discovery or resource allocation, you cannot assume the rules will work as designed. Instrument your auctions to log bids, CoT traces, and convergence metrics. Test multiple model families. Watch for urgency bias and emergent collusion patterns. If your agents fail to converge in simulation, they will fail in production.
The paper's testing framework is open source. Use it to validate your agents before deploying them in real markets. The cost of a failed auction in production is higher than the cost of running a few hundred test rounds.
Top comments (0)