Open Discovery Challenge: Benchmarking LLMs on Drug Candidate Discovery

TL;DR: VIDRAFT launched the Open Discovery Challenge, a public benchmark that pits leading frontier LLMs — OpenAI, Claude, Gemini, and DeepSeek — against each other on the task of finding drug candidates. The challenge evaluates AI reasoning and knowledge retrieval in a real-world biomedical discovery context, giving ML engineers a concrete signal for how general-purpose models perform on specialized scientific tasks.

What it is

The Open Discovery Challenge is a publicly announced benchmark initiative by VIDRAFT, a Korean Pre-AGI AI startup, covered by AP News in August 2026. The challenge directly compares the drug-candidate discovery capabilities of today's most prominent large language models:

• OpenAI (GPT-series)
• Anthropic Claude
• Google Gemini
• DeepSeek

The core question the challenge asks: Which of these frontier AI systems is actually better at identifying viable drug candidates? Rather than measuring generic NLP performance, the benchmark is domain-specific — it probes each model's ability to reason about biomedical targets, molecular candidates, and the logic underlying early-stage drug discovery.

This is a meaningful distinction for engineers working on life-sciences AI, because general leaderboard performance (e.g., MMLU, HumanEval) does not reliably predict how a model behaves when tasked with specialized scientific reasoning in domains like pharmacology or medicinal chemistry.

How it works

At a conceptual level, the Open Discovery Challenge appears to frame drug candidate discovery as a structured reasoning and retrieval task. Each participating model is presented with scenarios — likely involving biological targets, disease contexts, or molecular constraints — and evaluated on the quality and relevance of its outputs as potential drug candidates.

The high-level methodology follows a pattern common in scientific AI benchmarking:

1. Task definition — A set of drug discovery problems is curated, each with a defined biological context (e.g., a target protein, a disease indication).
2. Model prompting — Each frontier LLM receives the same inputs under consistent conditions, ensuring fair comparison.
3. Evaluation — Outputs are assessed against domain-relevant criteria, such as whether proposed candidates are chemically plausible, mechanistically justified, or consistent with known pharmacology.
4. Cross-model comparison — Results across OpenAI, Claude, Gemini, and DeepSeek are aggregated to surface relative strengths and failure modes.

The "open" framing of the challenge suggests VIDRAFT is positioning this as a community-accessible evaluation — not a closed internal test — which is what makes it relevant to the broader developer and research audience.

Benchmarks & results

The AP News press release establishes the challenge and its competitive framing but does not publish granular numerical results in the source material available at time of writing. Specific accuracy scores, ranking tables, or per-model metrics have not been disclosed publicly in this announcement.

What the source does convey qualitatively:

• The challenge is designed to differentiate the four models on a non-trivial scientific task, implying that meaningful performance gaps are expected and observable.
• The framing as an open challenge — asking "which AI is better?" — suggests the results are intended for public release, not internal use only.

Engineers should watch VIDRAFT's official channels for when full benchmark results are published. As with any domain-specific benchmark, interpreting the numbers will require understanding the evaluation criteria and dataset composition, details that should accompany any formal results release.

How to try it

Based on the information available in the source article, VIDRAFT has not published a public repository, Hugging Face dataset, or self-hosted API endpoint for the Open Discovery Challenge at this time. The announcement is a press release establishing the challenge's existence and framing.

To stay current on access:

• Follow VIDRAFT's official announcements for Hugging Face model/dataset releases, GitHub repositories, or API access details as they become public.
• If VIDRAFT releases an OpenAI-compatible API in the future, standard tooling will apply — but no endpoint, model name, or credentials should be assumed from this announcement alone.

If you are working on life-sciences AI and want to run your own informal comparison in the meantime, all four named models (OpenAI, Claude, Gemini, DeepSeek) have publicly accessible APIs you can query with your own drug discovery prompts today.

FAQ

Q: Is this benchmark open-source, or is it a proprietary internal evaluation?
A: The "Open Discovery Challenge" name and the public AP News press release suggest it is intended as a publicly visible benchmark, not a closed internal study. However, dataset and evaluation code have not been published in the source material reviewed here. Treat it as open-in-intent until VIDRAFT releases artifacts.

Q: Why does domain-specific benchmarking matter if these models all score similarly on standard leaderboards?
A: General benchmarks (MMLU, BIG-Bench, etc.) measure broad knowledge and reasoning but are poor proxies for specialized tasks. Drug candidate discovery requires integrating chemistry, biology, and mechanistic reasoning in ways that expose different failure modes than general QA. A model that tops the MMLU leaderboard may still hallucinate chemically implausible structures or miss key pharmacological constraints — which is precisely what domain-specific challenges like this one are designed to surface.

Q: Which model won the challenge?
A: Specific results have not been published in the source article. VIDRAFT's announcement frames the question competitively but has not yet released ranked outcomes publicly.

---

Originally reported by AP News (미국) (2026-08-19) — source article.