My drug valuation desk said a Phase 2 cancer drug was worth $1.29 billion. After two small fixes, the same drug with the same sales forecast was worth $283.5 million.
Neither bug crashed anything. The screen looked great the whole time. That is the kind of bug this post is about.
The setup
The desk is called Somota. My company Codebat finished the first version in March 2025, under the name PharmFlow. It values a drug with risk-adjusted NPV (rNPV): forecast the cash flows, then weight them by the chance the drug actually reaches the market.
This week my AI coding assistant was tracing where every input on the screen came from. Most inputs had a source. One did not.
Bug 1: a success rate with no source
The probability of success for the cancer asset was 30%. I had typed it in by hand, a long time ago, and nothing on the screen said so.
The best public benchmark I know of is the BIO, Informa Pharma Intelligence and QLS report, Clinical Development Success Rates 2011-2020 (February 2021). It gives the odds of passing each phase transition, by therapy area. For oncology:
| Transition | Success rate |
|---|---|
| Phase 1 to Phase 2 | 48.8% |
| Phase 2 to Phase 3 | 24.6% |
| Phase 3 to filing | 47.7% |
| Filing to approval | 92.0% |
A drug in Phase 2 still has three of those doors in front of it. The chance it gets through all of them is the product:
BENCH = {
"oncology": {"p12": 0.488, "p23": 0.246, "p3n": 0.477, "pna": 0.920},
}
STAGE_STEPS = {
"Phase 1": ("p12", "p23", "p3n", "pna"),
"Phase 2": ("p23", "p3n", "pna"),
"Phase 3": ("p3n", "pna"),
}
def bench_pos(asset):
"""Likelihood of approval from the asset's current phase."""
b, v = BENCH[asset["area"]], 1.0
for k in STAGE_STEPS[asset["stage"]]:
v *= b[k]
return round(v, 4)
bench_pos({"area": "oncology", "stage": "Phase 2"}) # 0.108
So 10.8%, about one in nine. The model had 30%.
Bug 2: costs multiplied by the same probability
The second bug was in how development costs were risk-adjusted. The old code did roughly this:
risked_dev_cost = development_cost * launch_probability
That treats every dollar of trial spending as if it only happens when the drug succeeds. It is like a college saying you will probably drop out, so it will only charge you 30% of the tuition.
Real trial bills do not work that way. You pay for the trial you are running now in full. You only pay for the next trial if this one works. So each year of spending gets its own weight: the chance of still being in development that year.
def cost_weights(asset):
"""Development cost is spent over 3 years. Year 1 is the current phase,
paid for sure. Each later year is paid only if the trials before it passed."""
b, s = BENCH[asset["area"]], asset["stage"]
if s == "Phase 1":
w = [1.0, b["p12"], b["p12"] * b["p23"]]
elif s == "Phase 2":
w = [1.0, b["p23"], b["p23"] * b["p3n"]]
else: # Phase 3 runs two of the three years, then filing
w = [1.0, 1.0, b["p3n"]]
return [round(x, 4) for x in w]
cost_weights({"area": "oncology", "stage": "Phase 2"}) # [1.0, 0.246, 0.1173]
The result
Same asset: $1.5B peak sales, 12.5% discount rate, $600M development cost to go, three years to launch.
| Old | Fixed | |
|---|---|---|
| Probability of success | 30%, hand-typed | 10.8%, sourced |
| Development cost weights | x 0.30 every year | 1.0, 0.246, 0.117 |
| Risk-adjusted NPV | $1,292.2M | $283.5M |
Then I ran a tornado chart: push each input up and down and see how far the value moves.
- Success rate, plus or minus 10 points: $755.8M swing
- Discount rate: $216.1M swing
- Peak sales: $206.6M swing
The one input nobody had sourced was the one that moved the price the most.
A check you can copy
This works on any model, not only drug valuations.
- Put a source label next to every input on the screen: public data, computed from public data, or typed by hand.
- Run a sensitivity check and sort the inputs by how far they move the output.
- Look at the top of that list first. If the biggest driver is typed by hand, go find a source before anyone trusts the number.
- Keep the old value one key press away, so you can show exactly what changed and by how much.
The desk now shows the study and the three phase transitions right under the success rate, and the old math stays available behind an OLD/NEW toggle.
The walkthrough
I walk through both bugs on the live desk, driven from a Stream Deck, in a 7-minute video: https://youtu.be/T_ay0f1kI74
The free one-page checklist these checks come from is here: https://docs.google.com/forms/d/e/1FAIpQLSfceCw9W10yY0VeNEvnt_cCAVX9SoxLT8QlHgkSgWUMus1Bng/viewform?usp=pp_url&entry.1357287784=pf003dev
Illustrative asset. Public data only. Not investment advice.
Top comments (0)