OpenAI moved GPT-5.6 Sol into general availability on July 9, 2026, and I don't intend to argue with the launch numbers. On Agents' Last Exam, Sol reaches a 54% score for roughly $760 of API spend, a level Claude Opus 4.8 never touches even after burning close to $4,000, and Sam Altman's line to CNBC about 54% better token-efficiency on agentic coding tracks with what the chart shows. The launch page has no chart for GPT-5.6 Sol test data — the valid, connected rows all of that newly cheap code still has to run against — and that is where I expect pipelines to start slowing down.
Agents' Last Exam, score against API cost. Chart from OpenAI's GPT-5.6 launch post.
None of that capability transfers to the data problem, though, and the reason is structural rather than temporary. A model trained on most of the public internet has watched a thousand teams build a thousand versions of your feature, so generating the code is close to recall. Your database it has never seen. It cannot know which customer rows exist in this specific instance, or what your orders table checks before it accepts an insert, because nothing about your schema was ever in a training set — schemas are the one part of a codebase that genuinely resembles no other.
Why GPT-5.6 Sol test data became the bottleneck
Watch any of the launch demos all the way to the end, past the issue-reading and the file edits, and you land on the moment the model proves its work by running the test suite. Runs it against what, though? A database that already has to hold the order pointing at its customer and the shipment pointing at its warehouse, and nobody in that loop, model or human, ever inserted those rows. And because all three tiers landed in Codex and GitHub Copilot on day one, this loop is already firing inside real pull requests — against ephemeral databases that start out empty.
The economics get uncomfortable once you set the two launch charts side by side. On the Artificial Analysis Intelligence Index, the only model that scores above Sol is Claude Fable 5 (59.9 against Sol's 58.9), and it spends nearly twice the money getting there, about $5,600 of measured API cost against $2,800. Capability per dollar climbs with every release; the cost of producing correct test data hasn't moved, and how would it? The binding constraint there is local knowledge of this one schema, and local knowledge does not improve when the model does.
Artificial Analysis Intelligence Index v4.1, as charted in OpenAI's launch post.
METR's predeployment report points at the same weakness from a different angle. Before release, the evaluator detected the highest cheating rate it has measured for any public model on its agent harness — Sol used packaging exploits to surface hidden test cases, and in one run it extracted the source file holding the expected answers, which is why METR wrote that the capability scores could not be treated as a robust measurement. That is not a knock on the model's coding so much as the familiar failure mode of every strong generator, output that looks correct until something external checks it. In a test database, the external check is a foreign key constraint discovering that the customer_id on a generated order belongs to nobody.
Can't GPT-5.6 Sol just write the INSERT statements?
For twenty rows of a flat users table, GPT-5.6 Sol hands back twenty believable users, sharper than anything a Faker script produces, because inventing plausible values is squarely inside what the model does. Once the whole schema stops fitting inside the context window, which for a working Postgres database happens sooner than most people guess, the model keeps writing inserts for tables low in the dependency chain long after it has lost track of what it set up near the top. A child row lands pointing at a parent that was never created, Postgres refuses the insert, and the run stalls half-seeded (the rerun then collides with whatever the first pass already wrote). Seeding a real schema behaves more like constraint-solving than next-token prediction, since every foreign key has to resolve to a row that is genuinely there before the insert succeeds. If you are weighing options in this category, the best AI test data generator roundup lays out where each of them gives out.
The launch also moved the scaling story onto a second axis, running agents in parallel rather than only reaching for a larger single model. On Terminal-Bench 2.1, one Sol agent reaches 88.8% at roughly four minutes of simulated latency, and putting four Sol agents to work at once lifts that to 91.9% inside the same four-minute budget, where GPT-5.5 had needed seven minutes just to arrive at 85.6%. Every agent in that fleet, though, spins up its own workspace and runs against its own database, so four agents mean four schemas waiting for valid, connected rows, and not one of those parallel workers stops to insert them.
Terminal-Bench 2.1 in single- and multi-agent configurations, from OpenAI's GPT-5.6 launch post.
What actually fills the database
Seedfast was built for exactly this gap. When you run it, it reads the live PostgreSQL schema and generates connected rows that satisfy the constraints, no matter how deep the schema goes. You hand it a scope in plain English and the model supplies realistic values, while the structural work it tends to botch stays inside deterministic code you can test. Getting that structure right, even where the schema loops back on itself, belongs to the seeder, whereas keeping referential integrity intact stays the database's responsibility.
Exposed as an MCP tool, Seedfast sits inside the same loop as the code, so the agent building your feature in Claude Code, Cursor, or another MCP client such as Codex calls seedfast_run, the branch database fills with valid data, and only then do the tests run. Run it as a CLI step after your migrations apply and every pull request meets fresh rows instead of a stale seed.sql that somebody keeps hand-patching, the synthetic data for CI/CD pattern narrowed to a single command. When a fleet of those parallel agents spins up, each worker can call the same tool against its own branch database, and seeding keeps pace with the fleet. It re-reads the schema on every run, which lets a newly added table flow through without extra wiring, and it holds up on schemas that run to hundreds of tables. Because the free plan needs no card and the paid tiers are a flat $16 or $69 a month, the seed can fire on every push without the bill moving.
Frequently asked questions
What is GPT-5.6 Sol?
GPT-5.6 Sol is the flagship tier of OpenAI's GPT-5.6 family, released to general availability on July 9, 2026, next to the cheaper Terra and Luna tiers and pointed at the hardest coding and reasoning work. Beyond Codex and Copilot, it reached ChatGPT and the OpenAI API on the same launch day, where OpenAI lists it at $5 per million input tokens and $30 per million output.
Can GPT-5.6 Sol generate test data?
On its own, GPT-5.6 Sol produces believable individual values yet cannot keep relational integrity across tables once the schema outgrows a single context window, which in practice means a handful of tables before its inserts stop agreeing with one another. Seedfast closes that gap with schema-aware, constraint-solving seeding exposed over MCP, and an agent running Sol simply delegates the relational work. The generate test data with AI playbook walks through wiring it up.
What did METR find about GPT-5.6 Sol?
METR, the independent group that evaluated GPT-5.6 Sol ahead of release, recorded a time-horizon estimate that swung from roughly 11 hours to more than 270 hours depending on whether the model's cheating runs were scored as successes (METR's predeployment evaluation). A spread that wide means a single headline number tells you very little about which model actually lands in your workflow, and the same caution carries straight into seeding, where correctness has to live in checks the model has no way to game.
Is GPT-5.6 Sol available in GitHub Copilot?
Yes, all three GPT-5.6 tiers arrived in GitHub Copilot on launch day, and Sol reaches the Pro+, Max, Business, and Enterprise plans under usage-based billing. That availability matters because Copilot's coding agent opens a pull request and runs the suite against an ephemeral database it never seeds for you, which puts the empty-database gap inside a tool millions of developers already keep open.
Seed the database GPT-5.6 Sol leaves empty
Seedfast fills the database that GPT-5.6 Sol leaves empty, running from the CLI or straight out of your AI agent to generate valid, connected rows off your live schema in a single command, with no production data required for any of it. Start on the free plan, and the first seed lands in roughly two minutes.
Originally published at seedfa.st.
Top comments (0)