01 / controlled
A2Scripted
One agent, deterministic counterparts, oracle-scored outcomes.
In short
- What it measures. How much of a market's welfare an agent captures while trading against other agents—economic competence, not task completion.
- The finding. Hidden information is the open frontier: even the strongest agent captures about .41 of achievable welfare with values visible, and about .15 once discovery is hidden.
- The shape. The gap holds across providers and price tiers—a capability frontier, not a model ranking.
- How it's scored. Five-agent agent-to-agent markets; an oracle grades realized ÷ achievable welfare (the AER); results are checked on private held-out seeds.
01 / Individual decisions
Can one agent find the right economic action?
A2Scripted isolates a single model against deterministic counterparties and scores the economic outcome with an oracle. It separates basic allocation skill from everything that live interaction adds.
AER (Aggregate Efficiency Ratio) is the share of the available gains from trade the agents actually capture — 1.00 reaches the fully efficient outcome, 0 is no improvement over the starting allocation, and a negative value means trades left the group worse off.
1.00 AER
Frontier performance on visible construction reaches the oracle ceiling.
current difficulty · saturated
4 / 4
Solved: procurement bundling, private-value auction, cyclic clearing, and coupled B2B award + re-split.
gpt-5.5 · n=30
−0.23
Surplus efficiency on hidden-reservation pricing—the scripted frontier is not saturated here.
market-ext · n=20
Key question
Does the task stay hard when the values and constraints are visible?
With the market held still, we can ask whether an agent can identify a welfare-improving action before asking it to infer, persuade, or coordinate with anyone else.
A2Scriptedcontrolled capability cell
Key finding
Visible construction is solved at the frontier. Hidden-reservation pricing is not.
Frontier models saturate the visible, full-information construction cells. The open problem moves to strategic inference and pricing when a counterparty's reservation value is private—so fixed scripted construction becomes a floor, not the headline.
strong optimizerweak strategist
02 / Emergent A2A market
How much does an agent capture against live counterparties?
A2A replaces deterministic counterparts with live agents that negotiate, reveal information, and settle through a verified ledger. Aggregate efficiency then measures how much of the available welfare an agent captures once the market reacts and information is hidden—which is exactly where capability falls short of the visible ceiling.
- 01Under testthe model being scored takes one seat in a five-seat market.
- 02Frozen panelthe other four seats are a fixed model pinned at temperature 0—reactive counterparts that negotiate, not scripts.
- 03Verified ledgerfree-form offers and consent are compiled, verified, and scored—the same arena pipeline.
A2A swaps A2Scripted's deterministic counterparts for negotiating LLM agents. The AER then measures how much of the available welfare the under-test model captures once counterparts can bargain and hide information.
Best agent · welfare captured, visible → hidden
Open the cross-provider leaderboard →
.15Against a live panel, the strongest agent captures .41 of achievable welfare with values in view—and just .15 once discovery is hidden.
held-out · gemini-3.5-flash · no agent clears .42 in any case
held-out · gemini-3.5-flash · no agent clears .42 in any case
Aggregate efficiency ratio
Bars show development AER◇ held-out AERmax .42
03 / Integrated arena
Where does market coordination break?
The arena composes discovery, bargaining, consent, and partial clearing in one economy. It tests whether local agent skill survives the full interaction loop—and whether the result survives new counterparties and unseen seeds. Where the individual cases decompose the economy one capability at a time, the arena composes them back: the same frontier, at market scale.
- 01Negotiateagents exchange free-form offers and consent.
- 02Compilethe compiler turns each accepted agreement into concrete asset transfers.
- 03Verifythe verifier gates feasibility, budget, and authorization—invalid transfers are rejected.
- 04Settlevalid transfers apply to a shared ledger; partial clearing is allowed.
- 05Scorethe oracle grades realized ÷ achievable welfare (the AER).
The compiler and verifier are fixed parts of the environment, not the agent under test, so a score reflects the agent's own behavior. Invalid or unauthorized transfers are rejected before scoring rather than counted as trades.
Arena finding
Discovery is the sharpest failure surface.
When counterparties are hidden, the leading margin compresses. On held-out case03, gemini-2.5-flash remains below the deterministic greedy floor (.095 vs .108). The earlier clearing weakness narrows away, so the supported claim is discovery-specific.
case03 · hidden counterpartiesunsaturated
Robustness
Change the panel. Change every seed.
A four-case panel swap keeps gemini-3.5-flash first under both a frozen-LLM and a scripted-rational panel, and first across 30 private held-out seeds per case. The leader is panel-robust; the deterministic floor is not—it makes no deals under the scripted panel on discovery and consent.
leader panel-robust · four casesheld-out leader · 4 / 4
03 · b / Market mode
All seats live: some populations collapse, some hold.
One model in every seat—a homogeneous population. The one-vs-panel score misses two things: a scale-dependent collapse for some models, and its absence for others.
.235 → .085
A homogeneous population captures less of the optimum as the market grows from 5 to 15 agents—coordination is a second frontier axis.
5 · 10 · 15 agents = .235 · .188 · .085
≤ .36
Capture never exceeds ~0.36 across five friction knobs; visibility helps but tops out near 0.15.
under-trading ceiling
.036 → 0
One adversarial quarter of the population erases an already-thin surplus—little is left to disrupt.
defection 0 → 25%
.29 → .47
Self-play welfare is trainable: a Qwen-4B shared-policy population improves with RLVR.
shared-policy self-play
The gemini and Qwen readings reuse pre-pipeline self-play runs (welfare-ratio, mean over 3 seeds). The deepseek and glm populations are measured in the official AER (n=20 per case), directly comparable to the leaderboard—and they show the collapse is not universal. Market mode stays a distinctive axis, not yet leaderboard-grade.
04 / RL environment
Can the missing market skill be trained—and transfer?
The RL environment turns capability cells into training tasks, then evaluates on an unseen integrated arena. Reward comes from realized utility and valid exchange, not from matching a preferred answer string.
1.6×
held-out reduced-arena welfare
step 0 → step 40
step 0 → step 40
0.29 → 0.47
A scoped Qwen3-4B probe improved clean held-out arena welfare over 40 training steps while keeping completion stable.
What survived the audit
A compiler-free re-score preserved and slightly strengthened the trend, supporting a real economic gain rather than better output formatting.
Early evidence · broader arena and curriculum transfer remains open05 / Questions
Common questions.
What AERead measures, how the market works, and what the pilot does and does not claim.
- What is AERead?
- AERead (AgentEcon Readiness) is a welfare-grounded benchmark that measures whether an LLM can act effectively in an economy—how much of the available gains from trade it captures in agent-to-agent markets—rather than whether it can complete isolated tasks.
- What is the Aggregate Efficiency Ratio (AER)?
- AER is the share of achievable market welfare an agent captures. 1.00 reaches the fully efficient outcome, 0 is no improvement over the starting allocation, and a negative value means trades left the group worse off.
- What is the frontier problem AERead identifies?
- With counterparty values in view, models make real progress. Once information is hidden—discovery, private pricing, consent—welfare capture collapses toward a trivial deterministic baseline. Characterizing that hidden-information frontier, not ranking models, is the point.
- How does the evaluation market work?
- Each case is a five-agent market in which the model under test takes one seat against a four-agent frozen panel. Free-form offers and consent are compiled, verified, and settled on a shared ledger, and an oracle grades realized divided by achievable welfare.
- Is AERead about ranking models?
- No. The load-bearing result is the structure of the gap: welfare capture stays far from the achievable frontier and widens as information hides, across providers and price tiers—a capability frontier, not a model ranking.
- How is AERead different from task-completion benchmarks?
- Task-completion benchmarks score whether a model finished a task. AERead scores how much economic welfare an agent captures while trading against other agents under realistic information conditions, so it measures economic competence rather than task completion.
- What does agent-to-agent (A2A) mean in AERead?
- A2A means the model under test negotiates against other live agents—a frozen panel pinned at temperature 0—rather than a scripted environment. Offers and consent are free-form and settled on a shared ledger, so the score reflects behavior in a reactive market.
- Is any model close to solving the hidden-information frontier?
- No. In pilot V0, once information is hidden every model tested falls toward a trivial deterministic baseline. The strongest agent captures about 0.41 of achievable welfare with values visible and about 0.15 once discovery is hidden.
- Who created AERead?
- AERead is developed by Chenyu Li, Zeyu Sun, and Guangting Yu.