Open Discovery Challenge: How to Build a Verifiable Judge for AI-Designed Malaria Drugs

Generative models can propose thousands of plausible molecules in a day. The harder question is no longer whether an AI can draw a molecule. It is whether anyone can tell if that molecule is potent, selective, safe enough to investigate, and possible to synthesize.

That is the premise of the Open Discovery Challenge, a public leaderboard opened by VIDRAFT and FINAL-Bench for AI-designed malaria drug candidates.

The target is PfDHODH, the malaria parasite's dihydroorotate dehydrogenase. A useful candidate must inhibit the parasite enzyme, avoid the homologous human enzyme, survive long enough to matter, and cross both the red-blood-cell membrane and the parasite membrane.

The original announcement and entrant guide are available on Hugging Face:

Who Tells You Whether the Molecule Your AI Just Designed Is Any Good?

Why verification is now the bottleneck

Molecule generation has become accessible. Verification has not.

A language or molecular model can emit valid-looking SMILES strings indefinitely, but structural plausibility alone does not answer the questions that determine whether a candidate deserves further work:

• Does it kill the parasite in a whole cell?
• Does it bind PfDHODH strongly enough?
• Does it avoid human DHODH?
• Is its ADMET profile acceptable?
• Is it genuinely novel rather than a decorated known scaffold?
• Can it realistically be synthesized?

The Open Discovery Challenge turns those questions into a published six-axis scoring system.

Axis	Points	Question
Whole-cell activity	30	Does the parasite actually die?
Target binding	20	Does the molecule bind PfDHODH efficiently for its size?
Selectivity	20	Does it prefer the parasite enzyme over human DHODH?
ADMET	15	Is the predicted absorption, distribution, metabolism, and toxicity acceptable?
Novelty	10	Is it structurally distinct from known antimalarial space?
Synthesis	5	Can the structure plausibly be made?

Publishing a rubric is easy. Making the rubric behave like a fair scientific judge is the difficult part.

Fourteen defects were found before opening

The team reports finding fourteen defects while validating the scorer. Several are useful engineering lessons for anyone building scientific-AI evaluation systems.

1. A toxicity gate rejected approved drugs — and coffee

Initial toxicity thresholds looked conventional, but all three approved antimalarials failed. Caffeine failed too.

The underlying predictors were biased against large, lipophilic molecules, and their outputs had been converted directly into hard cutoffs. This led to a practical validation rule:

Every threshold must let approved drugs through before it is allowed to reject anyone.

That rule subsequently exposed a molecular-weight cap that excluded a 531.9 Da reference drug and a reactivity detector that repeatedly rejected an approved compound.

This is a broadly applicable lesson: a gate should be validated against positive controls before it is trusted on novel candidates.

2. Correcting size bias almost promoted caffeine

Raw binding scores tend to reward heavier molecules. Normalizing by heavy-atom count is a standard correction, but it created the opposite bias.

Compound	Binding efficiency per heavy atom
DSM265, a clinical candidate	0.369
Caffeine	0.354

Caffeine nearly tied the clinical candidate. The correction removed the mass advantage but over-rewarded small, weak binders. Adding a potency floor alongside the efficiency term fixed the failure mode.

Metric normalization does not remove incentives; it changes them. Every transformed metric needs adversarial controls.

3. A whole-cell model had never seen failure

The first activity model predicted caffeine as a 1 μM active compound. The issue was not the model architecture but the training data.

The dataset contained compounds with measured activity, while records such as "no effect at 100 μM" had no numerical value and were filtered out. The model had effectively been trained only on compounds that worked.

After adding 5,190 failure records and retraining, the separation between the clinical candidate and caffeine widened from 1.00 to 1.74 log units.

Negative results are not empty data. For scientific prediction, they define the decision boundary.

4. A broken call path looked like a model failure

An approved drug with a literature IC50 of 13 nM initially came back at 248 μM, a four-order-of-magnitude error. The first conclusion was that the docking tool could not support absolute values.

That conclusion was withdrawn after the same configuration, called directly, returned 13 nM. The model was not the problem; the integration path was.

This is why end-to-end validation should be paired with component-level probes. Otherwise, orchestration bugs get misdiagnosed as scientific limitations.

5. Fingerprint reconstruction failed silently

The novelty scorer called the wrong function when reconstructing chemical fingerprints. No exception was raised, and the resulting numbers looked plausible. Only the novelty axis would have been quietly wrong.

A round-trip identity check caught it:

1. Store a fingerprint.
2. Read it back.
3. Compare it with itself.

Silent semantic corruption is more dangerous than a crash because ordinary monitoring sees a healthy system.

6. A nominal 90% lower bound covered only 83%

Candidates are graded using uncertainty-aware lower bounds rather than point predictions. The initial bound was nominally 90%, but measured coverage was 83%.

The statistical correction assumed exchangeable training and test data, while evaluation deliberately held out complete chemical scaffolds. That scaffold split was scientifically appropriate but violated the correction's premise.

The margin was widened until measured coverage reached 90.01%. The published number is the measured coverage, not the nominal label.

Controls make the scoreboard inspectable

The leaderboard includes known compounds and inert controls alongside submitted candidates.

Reference	Score	Role
DSM265	50.9	Clinical-stage antimalarial
Brequinar	4.0	Human-enzyme inhibitor; wrong target
Teriflunomide	2.8	Human-DHODH drug; wrong target
Ibuprofen	1.9	Inert control
Caffeine	1.8	Inert control

The scorer therefore exposes an immediately testable property: a relevant clinical candidate should rank above wrong-target drugs and inert compounds.

Selectivity also receives a positive control. Brequinar is detected binding human DHODH at 6 nM, while DSM265 is predicted 46 times weaker against the human enzyme. The human-side model is not simply calling everything weak.

Novelty is coupled to potency

Novelty is useful only when paired with function.

The challenge multiplies novelty by potency, creating two deliberate zero-value cases:

• A novel but inactive molecule earns no novelty reward.
• A potent analogue of a known drug also earns almost no novelty reward.

The points exist only where a candidate is both new and predicted to work.

Other axes follow the same principle. ADMET, synthesis, and novelty scores scale with how much of a viable candidate the molecule is. A perfectly safe compound that does nothing is not a drug candidate.

How entries are submitted

Participants do not need a proprietary molecular model. Any model capable of producing valid SMILES or InChI can be used. The official guide provides starter prompts for:

• parasite-versus-human selectivity;
• novel scaffold exploration;
• whole-cell permeability;
• iterative improvement from leaderboard feedback.

Submissions are screened before scoring. Duplicate structures, molecular formulas without explicit structures, PAINS motifs, prohibited covalent warheads, and structures above the mass limit are rejected. Predicted mutagenicity or extreme insolubility is scored and shown but ranked below clean candidates.

The guide repeatedly asks entrants to vary their constraints. Unmodified starter prompts tend to produce the same molecules, and duplicates are credited to the first submission.

What this challenge does — and does not — establish

This is a computational prioritization system, not clinical evidence and not a replacement for experimental assays.

Its value is narrower and practical: it gives AI-generated molecules a transparent, control-validated first-pass judge and exposes the numbers behind each axis. It can help determine which candidates deserve scarce wet-lab attention.

Malaria caused approximately 597,000 deaths in 2023, according to the WHO figures cited in the original article. Roughly three-quarters were children under five, and 95% of deaths occurred in Africa. In diseases where commercial incentives are weak, open verification infrastructure may matter as much as open generation.

The central lesson is simple:

Scientific AI needs a judge that can itself be tested.

Explore the full rubric, prompts, controls, and entrant instructions in the original Open Discovery Challenge article.

---

This article is an independent platform adaptation of the public FINAL-Bench/VIDRAFT announcement. All challenge-specific results and figures are attributed to the original project article. Computational predictions are not experimental or clinical validation.