DEV Community

Raymond Chang
Raymond Chang

Posted on

One Hand-Typed Number Made My Drug Valuation 4.5x Too High

My drug valuation desk said a Phase 2 cancer drug was worth $1.29 billion. After two small fixes, the same drug with the same sales forecast was worth $283.5 million.

Neither bug crashed anything. The screen looked great the whole time. That is the kind of bug this post is about.

The setup

The desk is called Somota. My company Codebat finished the first version in March 2025, under the name PharmFlow. It values a drug with risk-adjusted NPV (rNPV): forecast the cash flows, then weight them by the chance the drug actually reaches the market.

This week my AI coding assistant was tracing where every input on the screen came from. Most inputs had a source. One did not.

Bug 1: a success rate with no source

The probability of success for the cancer asset was 30%. I had typed it in by hand, a long time ago, and nothing on the screen said so.

The best public benchmark I know of is the BIO, Informa Pharma Intelligence and QLS report, Clinical Development Success Rates 2011-2020 (February 2021). It gives the odds of passing each phase transition, by therapy area. For oncology:

Transition Success rate
Phase 1 to Phase 2 48.8%
Phase 2 to Phase 3 24.6%
Phase 3 to filing 47.7%
Filing to approval 92.0%

A drug in Phase 2 still has three of those doors in front of it. The chance it gets through all of them is the product:

BENCH = {
    "oncology": {"p12": 0.488, "p23": 0.246, "p3n": 0.477, "pna": 0.920},
}
STAGE_STEPS = {
    "Phase 1": ("p12", "p23", "p3n", "pna"),
    "Phase 2": ("p23", "p3n", "pna"),
    "Phase 3": ("p3n", "pna"),
}

def bench_pos(asset):
    """Likelihood of approval from the asset's current phase."""
    b, v = BENCH[asset["area"]], 1.0
    for k in STAGE_STEPS[asset["stage"]]:
        v *= b[k]
    return round(v, 4)

bench_pos({"area": "oncology", "stage": "Phase 2"})   # 0.108
Enter fullscreen mode Exit fullscreen mode

So 10.8%, about one in nine. The model had 30%.

Bug 2: costs multiplied by the same probability

The second bug was in how development costs were risk-adjusted. The old code did roughly this:

risked_dev_cost = development_cost * launch_probability
Enter fullscreen mode Exit fullscreen mode

That treats every dollar of trial spending as if it only happens when the drug succeeds. It is like a college saying you will probably drop out, so it will only charge you 30% of the tuition.

Real trial bills do not work that way. You pay for the trial you are running now in full. You only pay for the next trial if this one works. So each year of spending gets its own weight: the chance of still being in development that year.

def cost_weights(asset):
    """Development cost is spent over 3 years. Year 1 is the current phase,
    paid for sure. Each later year is paid only if the trials before it passed."""
    b, s = BENCH[asset["area"]], asset["stage"]
    if s == "Phase 1":
        w = [1.0, b["p12"], b["p12"] * b["p23"]]
    elif s == "Phase 2":
        w = [1.0, b["p23"], b["p23"] * b["p3n"]]
    else:  # Phase 3 runs two of the three years, then filing
        w = [1.0, 1.0, b["p3n"]]
    return [round(x, 4) for x in w]

cost_weights({"area": "oncology", "stage": "Phase 2"})   # [1.0, 0.246, 0.1173]
Enter fullscreen mode Exit fullscreen mode

The result

Same asset: $1.5B peak sales, 12.5% discount rate, $600M development cost to go, three years to launch.

Old Fixed
Probability of success 30%, hand-typed 10.8%, sourced
Development cost weights x 0.30 every year 1.0, 0.246, 0.117
Risk-adjusted NPV $1,292.2M $283.5M

Then I ran a tornado chart: push each input up and down and see how far the value moves.

  • Success rate, plus or minus 10 points: $755.8M swing
  • Discount rate: $216.1M swing
  • Peak sales: $206.6M swing

The one input nobody had sourced was the one that moved the price the most.

A check you can copy

This works on any model, not only drug valuations.

  1. Put a source label next to every input on the screen: public data, computed from public data, or typed by hand.
  2. Run a sensitivity check and sort the inputs by how far they move the output.
  3. Look at the top of that list first. If the biggest driver is typed by hand, go find a source before anyone trusts the number.
  4. Keep the old value one key press away, so you can show exactly what changed and by how much.

The desk now shows the study and the three phase transitions right under the success rate, and the old math stays available behind an OLD/NEW toggle.

The walkthrough

I walk through both bugs on the live desk, driven from a Stream Deck, in a 7-minute video: https://youtu.be/T_ay0f1kI74

The free one-page checklist these checks come from is here: https://docs.google.com/forms/d/e/1FAIpQLSfceCw9W10yY0VeNEvnt_cCAVX9SoxLT8QlHgkSgWUMus1Bng/viewform?usp=pp_url&entry.1357287784=pf003dev

Illustrative asset. Public data only. Not investment advice.

Top comments (0)