TL;DR
Get business pricing on tech for your team
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
OpenAI and biotech company Ligo introduced BioBasicsBench, a benchmark of 100 tasks designed to test whether AI agents can interpret raw wet-lab evidence, identify errors and recognize when data do not support a conclusion. OpenAI reports that GPT-6 Astra scored 66.7% of available credit at the “xhigh” reasoning setting, compared with 43.8% for GPT-5.6 Sol; the company says image interpretation, control checks and experimental context remain areas for improvement.
OpenAI and biotech company Ligo have introduced BioBasicsBench, a 100-task benchmark designed to measure whether AI agents can interpret the raw evidence generated in everyday wet-lab experiments, spot errors and recognize when results are too uncertain to support a conclusion. OpenAI reported that GPT-6 Astra scored an average 66.7% of available credit at its “xhigh” reasoning setting, compared with 43.8% for GPT-5.6 Sol, while saying the results leave room for improvement.
The benchmark uses experimental materials such as gel and blot images, microscopy data, sequencing files, instrument readouts and protocol videos. Its 100 tasks span five areas: protein methods; imaging and cytometry; quantification and assays; nucleic acid methods; and molecular cloning. Questions range from reading a DNA sample’s molecular weight from a gel to analyzing more involved assays.
OpenAI says scientists built the tasks from data generated in their own experiments and paired each question with expert-written grading criteria. A Ligo scientist who did not write a task reviewed it, followed by an external review from a PhD-level wet-lab scientist and a final internal review. Agents provide a conclusion, measurement or recommendation, with supporting reasoning when requested. Scoring awards credit for individual rubric criteria, so a partly correct response can earn partial credit.
The evaluation covered seven models and 37 model-and-reasoning settings. OpenAI says each setting was tested five times on each of the 100 tasks, producing 18,500 results. The reported Astra and Sol scores apply to the “xhigh” setting, which allows more reasoning per task. OpenAI presents the results as progress on routine analysis, not as evidence that agents can reliably perform all wet-lab work.
Why Routine Lab Interpretation Matters
Wet-lab experiments do not always produce clean, publication-ready figures. Researchers may have to interpret ambiguous images, failed runs and conflicting observations, while checking whether controls worked as intended. An agent that misreads a result or misses a control problem could encourage a scientist to draw an unsupported conclusion or spend time on a follow-up experiment that does not answer the research question.
BioBasicsBench’s emphasis on raw evidence and error recognition addresses a practical test of AI’s potential use in research: whether a system can help with ordinary analysis rather than only respond to polished examples. The reported scores also show why benchmark performance needs careful interpretation. Partial-credit results measure several skills together, and a single average does not establish that an agent is dependable for a particular experiment or safe to use without expert review.
For laboratories considering AI assistance, the benchmark may help identify where models perform well and where human checking remains necessary. Its value will depend on how representative its tasks are of work outside the benchmark and whether later testing confirms similar performance in real laboratory settings.
Products Worth Considering
Jackery Explorer 1000 v2 Portable Power Station, 1070Wh, 1500W
Jackery Explorer 300 Portable Power Station with SolarSaga 100W Solar Panel
Apple AirPods Pro (2nd Generation) Wireless Earbuds with USB‑C Charging, Up to 2X More Active Noise Cancelling, Transparency Mode, Adaptive
As an affiliate, we earn on qualifying purchases.
How BioBasicsBench Was Built
OpenAI says existing benchmarks often rely on charts and images selected for scientific papers. Those materials may not capture the uncertainty or troubleshooting common in routine experiments. BioBasicsBench instead draws on experimental data from scientists’ own work, including photographs, raw files and records that may need to be interpreted together.
The benchmark includes different evidence formats, and a single task can use more than one. Supporting references, such as plasmid maps and ladder charts, may accompany the experimental material. The source report lists overlapping data categories, so counts for those categories should not be added as if each represented separate tasks.
OpenAI reports evaluating GPT-5.2, GPT-5.4, GPT-5.5, GPT-5.6 Sol, GPT-6.1 Sol, GPT-6 Luna and GPT-6 Astra across multiple reasoning-effort settings. The comparison highlighted in the report is 66.7% versus 43.8% of available credit for Astra and Sol at “xhigh.” These figures are benchmark scores, not measurements of how much faster a laboratory completed research or how often an AI-generated finding was independently validated.
Limits of the Reported Scores
The supplied report material does not establish how closely the 100 tasks represent the full range of laboratory conditions, research fields or experimental failures. It also does not provide enough information here to assess the spread of scores across individual tasks, the uncertainty around the averages or how model performance varies among the benchmark’s five subject areas.
The reported comparison does not show whether a higher benchmark score leads to better decisions in active research or reduces the time and cost of experiments. OpenAI identifies image interpretation, control checking and use of experimental context as areas needing more improvement. The source material also does not describe independent testing of the benchmark or report performance from AI agents working alongside researchers in live laboratories.
Accordingly, the results should be read as a measurement on a defined set of tasks, not as proof that an agent can replace a scientist, run experiments independently or safely make consequential research decisions. The benchmark’s scores alone cannot settle those questions.
Further Testing in Laboratory Settings
The report describes the benchmark and its evaluation but does not announce a date for a new test or a next release. A useful next step would be to examine performance by task type and skill, including whether agents reliably flag insufficient evidence and failed controls rather than guessing at an answer.
Further evaluations could also test whether the benchmark’s findings hold across new experiments and laboratories, and compare model suggestions with expert decisions in practical workflows. Until such evidence is available, researchers should treat the reported scores as benchmark results and review any AI analysis against the original data and experimental controls.
Key Questions
What is BioBasicsBench?
BioBasicsBench is a 100-task benchmark from OpenAI and biotech company Ligo. It tests AI agents on routine wet-lab analysis, including interpreting data, identifying errors and recognizing when evidence does not justify a conclusion.
What did GPT-6 Astra score?
OpenAI reports that GPT-6 Astra earned an average 66.7% of available credit at the “xhigh” reasoning setting. GPT-5.6 Sol earned 43.8% at that setting. These are benchmark scores, not measures of laboratory productivity.
What kinds of evidence are included?
The tasks use materials such as gels and blots, microscopy images, sequencing files, instrument exports, lab notes and protocol videos. Some tasks include multiple evidence types and supporting reference sheets.
Does the benchmark show that AI can replace wet-lab scientists?
No. The report evaluates models on a defined set of analysis tasks. It does not establish that AI agents can replace scientists, run experiments independently or make reliable research decisions without human review.
Source: openai.com
