DEV Community

Cover image for BioBasicsBench: GPT-6 Astra vs GPT-6.1 Sol, Astra Wins
Shamuddin
Shamuddin

Posted on Originally published at aitecharchive.com

BioBasicsBench: GPT-6 Astra vs GPT-6.1 Sol, Astra Wins

GPT-6 Astra leads BioBasicsBench with a 66.7% mean reward at the highest reasoning setting, ahead of GPT-6.1 Sol at 62.0% and well ahead of every older OpenAI model tested, according to OpenAI's November 6, 2026 write-up. Astra is the pick if you want the strongest result, but the lead over Sol is narrow and the confidence intervals overlap. The same score also means about a third of the available credit on routine wet-lab tasks was missed, so it is not a basis for unsupervised lab judgment, which is OpenAI's own caution.

TL;DR

  • BioBasicsBench has 100 wet-lab tasks built with the biotech company Ligo, released by OpenAI on November 6, 2026 (OpenAI).
  • GPT-6 Astra scored 66.7% (95% CI 60.3 to 73.1%) at xhigh effort; GPT-6.1 Sol scored 62.0% (55.6 to 68.4%) (OpenAI).
  • The score is partial credit against expert rubrics, not the share of tasks fully solved (Nerova).
  • Only OpenAI models were tested, and no public task set or evaluation code is available for independent reproduction (Nerova).
  • Treat agent readings of gels, plates and microscopy as decision support, not sign-off.
  • Last verified: November 9, 2026.

What is BioBasicsBench and what does it measure?

BioBasicsBench is a benchmark of 100 tasks that test whether an AI agent can interpret raw experimental data, spot errors, and recognise when the evidence cannot support a conclusion (OpenAI). It was developed with Ligo, a company focused on enzyme design. The tasks fall into five areas: protein methods, imaging and cytometry, quantification and assays, nucleic acid methods, and molecular cloning. They range from reading a DNA sample's molecular weight off a gel to detailed assay analysis (OpenAI).

The inputs are deliberately messy. They include gel and blot images, plate photographs, microscopy stacks and time-lapse, sequencing reads, plate-reader exports and recordings of bench work, with reference sheets such as ladder charts and plasmid maps to help interpretation. Gels and blots are the largest category at 31 tasks, followed by protocols and notes at 27, according to the data mix table in the OpenAI post. Categories overlap, so those counts do not sum to 100.

How were the BioBasicsBench tasks written and checked?

Scientists wrote each task from their own experiments and paired it with an expert-written grading rubric (OpenAI). Every task then went through three reviews: an internal review by a Ligo scientist who did not write it, an external review by a PhD-level scientist working in a wet lab, and a final internal review. Independent coverage of the release, such as AI Affairs, restates the same process.

The grading matters for reading the headline number. Mean reward is the average task score as a percentage of the available rubric credit, so partial answers earn partial marks, and the 95% intervals come from bootstrap resampling of whole tasks (OpenAI; Subagentic). OpenAI ran seven of its models across 37 model and reasoning-effort settings, five attempts per task per setting, for 18,500 results in total.

How does GPT-6 Astra compare with the other models on BioBasicsBench?

Astra comes first on the xhigh leaderboard, with Sol second and a large drop to the older generations. The scores, intervals and output tokens per task below come from OpenAI's post; the final column is my own subtraction from those figures.

Model Mean reward 95% CI Output tokens per task Gap to Astra
GPT-6 Astra 66.7% 60.3 to 73.1% 4,704 n/a
GPT-6.1 Sol 62.0% 55.6 to 68.4% 5,238 4.7 points
GPT-5.6 Sol 43.8% 37.6 to 50.0% 9,099 22.9 points
GPT-6 Luna 38.3% 32.3 to 44.5% 13,120 28.4 points
GPT-5.5 35.4% 29.7 to 41.0% 11,135 31.3 points
GPT-5.4 33.2% 28.3 to 38.4% 13,717 33.5 points
GPT-5.2 31.2% 26.2 to 36.3% 27,620 35.5 points

The verdict between the top two is Astra, with a caveat. A 4.7-point gap sits inside overlapping intervals, so the data do not separate Astra from Sol with confidence. The real break is between the two GPT-6 models at the top and every older model in the table. For a wider comparison of the two newest model families outside biology, see our GPT-6 Astra vs Claude Fable 5.1 verdict.

Where does GPT-6 Astra still fail in the lab?

OpenAI published failure cases alongside the scores, and they are instructive (OpenAI):

  • Wrong colonies from a colony-PCR gel. Astra assumed a larger PCR product meant successful cloning. The reference answer selects lanes 5, 9 and 10; Astra chose 4, 6, 7 and 8.
  • Ignoring contradicting evidence. It discounted observations that clashed with its preferred reading, in a case where vacuole contrast reverses with focal depth.
  • Overcounting colonies. On one plate it overcounted, failing to resolve close neighbours, missing visible colonies and marking empty agar.

None of these is exotic. They are the mistakes a rushed junior scientist makes, which is exactly why a confident wrong answer is the risk to manage. OpenAI states the point directly: "Higher benchmark scores do not yet mean models can make reliable judgments in the lab."

What did GPT-6 Astra get right that earlier models missed?

Three examples stand out, all from OpenAI's post. Astra recognised a z-stack, meaning cells imaged at different focal depths, instead of inventing a biological explanation for the differences. It noticed a plate labelled with the wrong antibiotic (LB Kan50) and rejected it as a control. And it recognised that a dead-cell dye detector could pick up light from a second dye, so brightness alone could not justify calling cells dead.

These are control-checking behaviours rather than pattern reading. The agent is asking whether the evidence is valid before interpreting it. OpenAI's hypothesis, which it presents as an observation and not a proven finding, is that stronger native multimodal capabilities may be key to improving wet-lab analysis.

What should research leads take from the BioBasicsBench results?

  1. Use it as decision support. Have an agent draft a reading of a gel, plate or microscopy image, then have a scientist confirm it. A score of 66.7% in partial credit says nothing about which third will be wrong.
  2. Expect lower cost per task as scores rise. GPT-5.2 used 27,620 output tokens per task for 31.2%, while Astra reached 66.7% on 4,704 (OpenAI). Newer models are both better and cheaper per task on this benchmark.
  3. Watch the control-checking behaviour. The mislabelled antibiotic plate and the ambiguous dye are the new capability being measured, and they are the cases most worth testing on your own data.

Remember the limits. These are the developer's own results on its own models, with no Anthropic, Google or third-party scores (Neuralbrew). OpenAI also frames tracking these capabilities as relevant to biosafety and biosecurity.

Why does reading bench evidence matter now?

Physical automation is moving faster than evidence interpretation. Nature reported on November 4, 2026 that Shanghai Jiao Tong University's AutoDNA self-driving lab operates more than 20 types of instrument across thousands of operations (Nature). The same week, Google DeepMind added drug-discovery tools to Antigravity (MobiHealthNews).

Robots that run experiments need something that reads the results reliably. BioBasicsBench measures that missing layer, routine interpretation of gels, plates and traces, and the answer today is useful but not dependable. For related work on scientific agents, see our comparison of Antigravity cheminformatics against El Agente Farmaco, the GDPX benchmark results for Opus 5.5 and Gemini 4 Carbon, our look at bacterial versus human particle structure with Claude, and the broader agentic AI in health care comparison.

FAQ

Q: What is BioBasicsBench?
A: It is a 100-task benchmark from OpenAI and Ligo, released on November 6, 2026, that tests whether AI agents can interpret raw wet-lab data, find errors and recognise when evidence is insufficient (OpenAI).

Q: Which model scored highest on BioBasicsBench?
A: GPT-6 Astra scored highest at 66.7% mean reward at xhigh effort, with GPT-6.1 Sol second at 62.0% (OpenAI).

Q: Does 66.7% mean Astra solves two thirds of the tasks?
A: No. The metric is mean reward, the average share of available rubric credit, so partial answers count (Nerova).

Q: Were Claude or Gemini models tested?
A: No. Only seven OpenAI models were evaluated, so there are no Anthropic, Google or third-party scores to compare (Neuralbrew).

Q: Can I reproduce the results myself?
A: Not at present. No public task set or evaluation code has been released (Nerova).

Q: Are AI agents ready to run lab decisions unsupervised?
A: OpenAI says higher benchmark scores do not yet mean models can make reliable judgments in the lab, so human review remains necessary (OpenAI).

Corrections log

No corrections to date. Figures were checked against OpenAI's primary post on November 9, 2026.

This article was produced with AI assistance and reviewed against the primary sources linked above. It contains no medical or patient advice.

Top comments (1)

Collapse
 
dralexperez profile image
Alex Perez •

Great breakdown of the benchmark details, especially pointing out that 66.7% is partial rubric credit rather than fully solved tasks. Having spent years reading messy gels and troubleshooting molecular cloning, the most encouraging part here isn't the raw score, it's the control-checking behavior like catching an incorrect antibiotic plate or dye bleed-through. If AI agents can reliably act as a second pair of eyes for experimental controls before we scale up an assay, that alone saves huge amounts of bench time.