DEV Community: NeuPortal

OpenAI's finance AI scores 69.9%. Build a 50-question eval harness for your own documents.

NeuPortal — Fri, 11 Sep 2026 08:53:23 +0000

ChatGPT for Financial Services arrived on 10 September with a benchmark score attached: a shade under seventy per cent correct across a large pile of treasury filings. Strong number, honestly reported by OpenAI.

It also tells you almost nothing about your documents.

That is not a complaint about the vendor. It is what benchmarks are. A score is an average over one corpus, and your corpus is a different corpus. If you are the engineer who has to answer "can we ship this", you need a number computed on your own material, and you need it before someone else picks one for you.

Here is a harness that produces one. It is deliberately small. No eval framework, no vector database, no orchestration layer. A spreadsheet, a loop, and a scoring function.

The shape of the thing
from dataclasses import dataclass

Five fields carry the whole design, and two of them are the ones teams skip.

kind exists because "document question answering" is four unrelated tasks wearing one label. Pulling a stated figure off a page is not the same skill as computing a ratio from three of them, which is not the same as deciding whether this filing's definition of adjusted earnings matches the one used last quarter. A system can be excellent at the first and useless at the third, and a pooled score will read as respectable either way.

difficulty exists because errors are not distributed evenly. They cluster on the items that were ambiguous or contested, which are exactly the items a junior colleague would have escalated instead of answering. If your test set is all routine questions, you have measured the easy half and learned nothing about your exposure.

Building the set

Fifty cases. Not five hundred. Fifty that a domain expert wrote, where each expected answer is one that person will defend in a meeting.

Pull them from your own documents, and deliberately include:

figures that were later restated, with the original still sitting in the file
a parent and a subsidiary with similar names and different numbers
at least one internally inconsistent document, because you have them
two questions whose correct answer is "the document does not say"

That last category is the most valuable and the most often omitted. A system that never declines to answer is not more capable, it is less honest, and you want that to appear in your numbers rather than in production.

Running it twice
def run_suite(cases, ask, show_sources: bool):
rows = []
for c in cases:
out = ask(c.question, with_citations=show_sources)
rows.append({
"qid": c.qid,
"kind": c.kind,
"difficulty": c.difficulty,
"answer": out.text,
"cited_doc": out.citation.doc if out.citation else None,
"cited_page": out.citation.page if out.citation else None,
"correct": None, # graded by a person, below
"cite_valid": (out.citation is not None
and out.citation.doc == c.source_doc),
})
return rows

Two passes matter, and they measure different systems.

Pass one, citations hidden. This measures the model.

Pass two, citations visible, graded by a reviewer who is allowed to open the source. This measures your review process. The delta between the two passes is the number nobody has: how many wrong answers your humans actually catch when the evidence is right in front of them.

Most organisations have never measured the second one. They assume the review step works because it exists.

Note cite_valid is tracked separately from correct. This is the distinction the whole exercise turns on. A citation establishes provenance. It does not establish that the retrieved figure answers the question asked. A perfectly valid citation pointing at a superseded figure scores cite_valid=True, correct=False, and that combination is the interesting one. Count it explicitly:

def report(rows):
by = {}
for r in rows:
for axis in (("kind", r["kind"]), ("difficulty", r["difficulty"])):
b = by.setdefault(axis, {"n": 0, "ok": 0})
b["n"] += 1
b["ok"] += bool(r["correct"])
for (axis, val), b in sorted(by.items()):
print(f"{axis:10} {val:12} {b['ok']}/{b['n']} {b['ok']/b['n']:.0%}")

trap = [r for r in rows if r["cite_valid"] and not r["correct"]]
print(f"\nsourced but wrong: {len(trap)}  <- the ones review will wave through")

Reading the output

Never read the total first. The total is the least informative line in the report, for the same reason a company's average salary tells you nothing about any individual.

Read the segment rows. If one kind is far below the others, you have found the workflow that cannot ship yet, and you can route those questions to a person while shipping the rest.

Then read the direction of the errors. Where the answer is numeric, record signed error rather than a pass/fail flag. If mistakes scatter both ways, that is noise and you can buffer against it. If they all lean the same way, that is bias, it will not average out with volume, and it will produce the identical mistake every time at scale.

This is not a theoretical distinction. In a forecasting system we run in the open, the pooled accuracy sat above where it needed to be for weeks while one slice inside it had stopped working entirely - it captured a twenty-fifth of outcomes where the interval was constructed for half, and the ones it missed were all beyond the same edge. The overall figure was computed correctly and hid the whole thing. We found it by cutting along an axis we had never reported.

The sourced but wrong counter deserves its own attention. Those cases pass every automated check you are likely to build. The link resolves, the page exists, the number appears on it. Only a person who understands the domain catches them, which means that count is effectively a measure of how much human review your workflow genuinely requires.

Keeping it alive

Re-run monthly, pinned to a model version, and store the results. Models get updated without changing their name, your documents change, and a score from March is not evidence about September. A harness that runs once is a slide. A harness that runs monthly is a control.

Total cost: an afternoon for the code, a day of a domain expert's time for the fifty cases. In exchange you get to replace "the vendor reports roughly seventy per cent" with a number about your own work, broken out by task type, with the direction of the errors attached.

That is the difference between citing a benchmark and having evidence.

We run this discipline on our own forecasting models and publish the results, failures included, at neuportal.ai/experiment

Build the harness before somebody else picks the number for you.

Educational content - not financial advice.

ai #machinelearning #python #testing

GPT-6 Astra for Developers: What Changed in Codex, and a 90-Minute Test Plan for Your Repo

NeuPortal — Wed, 09 Sep 2026 11:01:11 +0000

OpenAI shipped GPT-6 Astra on 3-4 September 2026. It is on Plus, Pro, Business and Enterprise, in the API, and on AWS, with an Astra Pro tier for the top plans. There is no new pricing structure; usage draws from existing allowances, with credits for overage. The API page did not state a per-token price at launch.

Set the benchmark headlines aside for a minute. For anyone who writes code for a living, one change in this release is worth more than the rest of the announcement combined, and it can be tested this afternoon.

The change that matters: Codex remembers across context windows

Every agentic coding session eventually hits the same wall. The context window fills. The tool compresses what came before into a summary. The summary drops something - a constraint you gave at the start, a test that failed on step three, the reason you rejected an approach. Two hours later the agent reintroduces the thing you rejected, and you cannot tell whether it is being stupid or just forgetful.

It is forgetful. And the Astra release attacks that directly.

Codex now holds onto its notes when a context window rolls over. The accumulated details persist instead of being re-summarised on every rollover, and what came earlier remains searchable, so the agent can pull a requirement or a test result back out of the past rather than reconstructing it from a lossy digest. It shipped as experimental, and OpenAI says it becomes the default over the coming weeks.

That is a precise fix for a precise failure. Which means you can test it precisely.

A 90-minute test plan

Pick a real ticket, not a toy. The right candidate has a constraint that is easy to state and easy to violate - "never call the payments service from a background job," "this endpoint must stay backward-compatible with v2 clients," "do not touch the migration files." Something the agent will be tempted to break twenty steps later when it is deep in unrelated code.

Minute 0-10. State the constraint once, at the start. Do not repeat it. Write down the exact wording somewhere the agent cannot see.

Minute 10-60. Give it the ticket. Let it run long enough to blow through at least one context window - you want the rollover to happen. Do not intervene when it wanders; wandering is the test.

Minute 60-80. Read the diff for the constraint. Not for correctness in general - specifically for whether the rule from minute zero survived. Then ask the agent, in a fresh message, why it made a particular choice that depends on that rule. See whether it retrieves the original reasoning or invents a new one.

Minute 80-90. Run the same ticket with the notes feature off, if your setup lets you toggle it. Compare. If you cannot toggle, run it against the previous model.

If the constraint survives the rollover, the feature does what it says. If it does not, you have learned that faster than any benchmark would have told you.

Computer use: nearly twice as fast, and why that matters more than it sounds

OpenAI reports that computer-use tasks in ChatGPT run at close to 2x the previous speed, and that the same optimisation lifted the prior model, GPT-5.6 Sol, by around 60%. The pitch is multi-step workflows that finish in documents, spreadsheets and presentations rather than drafts.

If you have never delegated a UI-driven workflow to an agent, the speed number looks cosmetic. If you have, you know that a fifteen-step task at the old pace was slow enough that you stopped delegating it. Halve the time and a whole class of tedious work moves back across the line. Measure one of yours.

The security rating, and what to do about it on your side

Astra is the first OpenAI model designated critical for cybersecurity under the company's preparedness framework. In OpenAI's description, that means it can find and use vulnerabilities nobody knew about in fortified systems with no operator at the controls. It scored 100% on ExploitBench. The most capable form is limited to vetted testers, and there is a program called Daybreak Blue to extend defensive access. The chief scientist said that guarding against harm nobody intended might turn into the limiting factor on progress - a striking thing to say about your own release.

For your codebase, two consequences.

Finding holes in your stack just got cheaper. Not for you specifically - for all comers, invited or not. Assume the review you have been putting off is now cheap enough for someone else to run.

And the identical ability works in the defender's hands. The general model, even without the gated tier, is strong enough to be useful pointed at your own code. Put it in the security review rotation. Daybreak Blue is the formal channel if you need the full version.

What is a claim, not a measurement

Three things from the launch are worth labelling in your notes as unverified: "the smartest and most aligned model anywhere," "the strongest model it has built for software work," and Greg Brockman's suggestion that Astra may represent AGI, which he called speculative. The three benchmark scores - ExploitBench 100%, ARC-AGI-3 99.9%, FrontierMath Tier 4 98% - are measurements, but OpenAI's word for them is "saturates," which means the benchmarks have nothing left to measure; the model is not the thing that ran out.

One more technical note that matters for how you evaluate it. Astra uses an approach OpenAI names recurrent depth (the literature says looped transformers), which spends extra computation on tough problems with no increase in model size. The trade is that it keeps back part or all of its reasoning where previous models exposed it. Practically: you will inspect fewer traces and assert on more outputs. Build your tests accordingly.

The bottom line for engineers

Test the Codex memory on a real constraint. Time one delegated workflow. Put the model on your security review. Write down which launch claims you are treating as unmeasured. Four items, one afternoon, and you will know more about whether Astra helps your team than any announcement can tell you.

We publish our own forecasts under a similar discipline - committed before the outcome, scored afterwards in public with the misses kept - at neuportal.ai/experiment

Run the constraint test before you trust the summary.

Educational content - not financial advice.

Your Time-Series Validation Score Is Inflated, and Your Test Suite Will Never Tell You

NeuPortal — Wed, 02 Sep 2026 11:15:18 +0000

Every machine learning engineer learns early that leakage inflates validation scores. We check for target leakage. We check for train-test contamination. We use time-based splits instead of random ones on temporal data.

Then we build a feature over a rolling window, evaluate the model, and quietly reintroduce a related problem that no leakage check is designed to catch.

It is not leakage. The trouble is a sample size you never actually had.

Where it comes from

Temporal modelling almost always involves windows. You want a label describing what happens over the next thirty days, so you compute it at every timestep. You want a feature summarising the trailing thirty days, so you compute that too. Standard practice, and correct as far as it goes.

Take a daily price series with about 3,300 rows and generate a thirty-day forward label at each step. Your dataset now has roughly 3,275 labelled examples. Your training loop sees 3,275. Your metrics are computed over 3,275. Every confidence estimate you produce inherits that figure.

Now consider two consecutive rows. Row one carries a label describing days 1 through 30. Row two carries a label describing days 2 through 31. Twenty-nine of the thirty days feeding those labels are shared.

These are not two independent examples. They are one example with a small perturbation, and your dataset contains twenty-nine more just like it before you reach a genuinely new observation.

Why this is not the same problem as leakage

Leakage means information from the future has contaminated the past. Time-based splits fix it, and most teams handle this correctly now.

This is different. There is no contamination across the split boundary. Every row is causally valid. The problem is that your effective sample size is roughly your row count divided by the window length, and every uncertainty estimate in your pipeline assumes otherwise.

Divide instead of sliding: 3,300 rows with a thirty-day window yields about 110 genuinely independent examples. Not 3,275. The ratio equals the window length exactly. A seven-day window inflates sevenfold. A ninety-day window inflates ninetyfold.

What breaks downstream

Uncertainty estimates scale with the root of how many truly separate examples you hold, so overstating by thirty compresses everything by a factor near 5.5.

Confidence intervals on your metrics land at roughly a fifth of their warranted width. Bootstrap distributions over your validation scores are far tighter than reality. Statistical tests comparing model A against model B will declare significance on differences that are noise. Hyperparameter selection will confidently pick a configuration that simply got a favourable draw from your hundred-odd real examples.

Worst of all, cross-validation does not save you. K-fold on overlapping windows spreads near-duplicate rows across folds, so your held-out fold contains examples that share twenty-nine days with something the model trained on. The fold boundary looks clean. The information boundary is not.

Nothing errors. Every assertion passes. Your data is valid, your split is temporally correct, your code is right. The defect lives in an assumption underneath the metric, and assumptions raise no exceptions.

How bad it gets at long windows

Push the window to 365 days on the same series and you can generate roughly 2,940 labelled examples from nine years of history.

Nine. That is the number of independent observations available for a model predicting annual behaviour. A confidence estimate spanning 2,940 rows that describe nine underlying events is neither cautious nor bold. It carries no information at all.

And those nine are not nine draws from a stationary process either - across nine years of any real-world series, the data generating process itself has usually changed more than once.

Five things to do about it

Report effective sample size alongside row count in every experiment log. Row count over window length is a crude estimator and vastly better than nothing. Put it in the same table as your metrics so nobody reads the metrics without it.

Compute uncertainty on non-overlapping subsets. Train on everything if you like - more correlated examples still help the fit. But derive intervals, error bars and significance tests from the independent subset only.

Use blocked cross-validation with purging and embargo. Blocked splits keep contiguous segments together. Purging removes examples whose windows straddle the boundary. An embargo gap prevents the fold edge from sharing information at all. This is standard in financial ML and underused everywhere else that windows appear.

Prefer block bootstrap over the standard variety. Resampling individual rows destroys the autocorrelation that created the dependence, which quietly restores the original error while looking rigorous.

Treat window length as a modelling constraint, not a free parameter. A window that leaves you a handful of independent examples is not a modelling choice awaiting better regularisation. It is a problem your dataset cannot support, and the correct output is a scoped-down claim rather than a wider error bar.

The general lesson

The pattern generalises well past finance. Any domain with sliding windows over correlated sequences carries it: sensor streams, clinical monitoring, demand forecasting, telemetry, anything with a rolling aggregate. Wherever consecutive examples share most of their underlying observations, your row count is a measure of computation rather than a measure of evidence.

More rows from the same underlying history do not add information. They add duplicates with slightly different noise, and every statistical procedure downstream will thank you for them by becoming more confident about less.

Count separately. Then decide what you are entitled to claim.

We publish our own forecasts under this constraint - committed before the outcome, scored afterwards in the open with failures retained - at neuportal.ai/experiment

Divide your row count by your window length before your next standup.

Educational content - not financial advice.

The Bug Class Where Your Code Is Correct, Your Tests Pass, and It Never Runs

NeuPortal — Sun, 02 Aug 2026 12:06:58 +0000

Across one of our agents, the large majority of genuine defects - eight out of nine, when I went back and classified them - belonged to a single category. Not off-by-one, not a race, not a bad regex.

The category is: the code was written correctly, it was tested correctly, and it never executed on the path that mattered.

Every one of them looked healthy on a dashboard. Every one had passing unit tests. The tests passed because they called the function directly, and the function was fine. The wiring was not.

Case 1: the feature that shipped switched off

We publish forecast intervals, and at some point we added conditioning: instead of reading historical quantiles across all market conditions, filter the sample to periods resembling the current one. It matters a lot - unconditional intervals carry a permanent allowance for turbulence that a calm market has not earned.

The code was written. It was correct. It had a toggle:

useConditioning = input.bool(false, "Condition on volatility regime")

That false is the entire bug.

The mechanism existed, its tests passed, and it never ran. Every chart we published for weeks drew the unconditional band while our documentation described the conditioned one. Nothing errored. Nothing looked wrong. Coverage came out at 84% against a stated 50%, and because over-coverage produces no failures - every outcome lands inside a too-wide band - there was no symptom to investigate.

A default value is a code path. Treat it as one.

Case 2: the function that only advances when its branch is taken

This one is language-specific but the shape generalises, and it is nastier because there is no toggle to find.

Some functions carry internal state across invocations. Moving averages, correlations, anything that maintains a rolling window. In Pine Script these are the ta.* family, but the pattern exists anywhere you have a stateful helper that assumes it is called once per tick.

Write this and it looks fine:

ma(src, len, kind) =>
    kind == "EMA" ? ta.ema(src, len) : ta.sma(src, len)

It compiles. It returns plausible numbers. It is wrong.

Only one branch executes per bar, so only one of the two averages advances its internal state. The other one is being fed a history with holes in it. The value it returns is not the moving average of the series - it is the moving average of the subset of bars where that branch happened to be taken.

The fix is to compute both unconditionally and select afterwards:

ma(src, len, kind) =>
    e = ta.ema(src, len)
    s = ta.sma(src, len)
    kind == "EMA" ? e : s

Slightly more arithmetic, correct answers. I found this in two separate places in the same codebase on the same day, which tells you how natural the wrong version feels to write.

Case 3: the check that checked the wrong thing

A smaller one, from the same week, because it completes the pattern.

We had a text sanitiser that converts typographic characters to ASCII, because certain publishing platforms decode UTF-8 as a legacy codepage and an em dash arrives as mojibake. Fine. It also contained this:

while "  -  " in text:
    text = text.replace("  -  ", " - ")

The intent was to clean up the double space left behind when a spaced em dash becomes a spaced hyphen. The effect was to collapse aligned indentation in every file it touched. Run in check mode it reported false positives on source files. Run in fix mode it would have silently mangled them.

The rule was correct for the case it was written for and wrong for every other input. A cleanup step that runs globally is not a cleanup step, it is a transformation.

Why unit tests do not catch any of this

Because unit tests call the function.

A test for the conditioning logic imports the conditioning function, feeds it data, and asserts the output. It passes. It says nothing about whether production reaches that function. A test for ma() calls ma() with kind="EMA" and gets a correct EMA, because in that test every invocation takes the EMA branch and the state advances properly.

The defect lives in the relationship between components, and unit tests are specifically designed not to look there. That is usually a virtue. Here it is the blind spot.

What actually finds them

Three things, in order of how much they have paid off for us.

Review the call graph, not the components. The productive question is not "is this correct" but "under what conditions does this execute, and did I verify it under those conditions". For every function you care about, trace backwards to the entry point and check that the path is reachable with production configuration.

Treat every default as a decision. Any flag, any optional parameter, any if enabled branch. Write down what runs when nobody touches anything, because that is what runs.

Assert on observable output, not on internals. Our conditioning bug would have been caught in a day by a check that compared the published band width against the conditioned band width and complained when they matched. That check is trivial and we did not have it, because we knew the code was correct.

The cheap version

If you take one thing: after adding a feature behind a flag, grep for the flag and read every line that references it. Not to check the logic - to check that production sets it.

That is a two-minute habit and it would have saved us several weeks of publishing numbers that quietly described a different calculation than the one we were documenting.

The bug was never in the code. It was in the assumption that written means running.

AI and Volatility: Forecasting How Much a Market Moves, Not Which Way

NeuPortal — Fri, 24 Jul 2026 13:16:35 +0000

Ask almost any machine pointed at a market the same question and it will answer confidently: where is the price going next. It is the question with the screenshots and the viral threads, and it is the one machine learning is worst at, because a liquid market has already absorbed whatever the model just noticed. There is a different question you can ask the same machine, quieter and far more useful: not which way, but how far. How much is this asset likely to move over the next day? That is a volatility forecast, and unlike a price target it is something a model can genuinely deliver — and, just as importantly, something you can hold it to afterwards.

This is the honest home of AI in markets, and it is a very different thing from prediction. It is worth walking through carefully, because the difference between a volatility forecast that means something and one that is decoration is measurable, and most of the genre fails the measurement.

Why direction is the wrong question for a liquid market

A deep market is not a puzzle sitting still. It is an adversary that has already priced whatever your model just discovered. By the time a directional pattern is visible in the data, it is visible to everyone with the same data, and the price reflects it. Forecasting the direction of the next move, in that setting, is close to calling a coin the market has already flipped.

This is not a limitation that a bigger model removes. It is the structure of the problem. The information that would tell you which way the price is about to go is exactly the information a liquid market competes away fastest. So a system built to answer that question is built to lose, slowly, in a way that only shows up over enough calls to be inconvenient to count.

Volatility is predictable in a way returns are not

Volatility is different, and the difference is a real statistical property: it persists. Calm days cluster with calm days, violent days with violent days, and a shock today raises the odds of a large move tomorrow. Returns are close to unpredictable; the size of the moves is not. That autocorrelation of magnitude — volatility clustering — is stable enough to learn from.

Give a model realised volatility over several lookbacks, options-implied surfaces where they exist, funding rates and open interest, and it can return a forward range that carries information even when the centre of that range is genuinely unknowable. Notice how much humbler that output is than an arrow on a chart. It does not say what will happen. It says how wide to expect the outcomes to be. And that single estimate is what everything downstream depends on: how large a position holds risk constant, where a stop is noise and where it is real, when to brace before a violent session instead of flinching after it.

The band most people draw is wrong in both directions

Here is where measurement separates from vibes. The standard way to turn a volatility number into a band is to multiply by the square root of the horizon — sigma times root-t. It is one line of code, it is everywhere, and for fat-tailed assets it misprices the distribution in a way that is worth stating precisely.

We measured it against the entire Binance history rather than a flattering recent window — 3,261 daily bars for Bitcoin back to 2017. The quantity of interest is the ratio of an empirically-measured 80% band to the sigma-root-t band at each horizon. For Bitcoin it runs about 0.80 at one day, roughly 0.88 at seven days, and about 1.00 by thirty days. Read that carefully: at short horizons the parametric band is too wide, and by a month it is about right. The error changes sign as the horizon extends, so there is no single correction factor that fixes it.

The reason the short-horizon band is too wide despite genuinely fat tails is that the excess kurtosis — around sixteen on daily returns, against three for a normal distribution — lives in the extreme tails, not in the tenth-to-ninetieth-percentile shoulders. So the 80% interval is actually narrower than a Gaussian would imply, while the 99% interval is much wider. Fat tails and a narrow 80% band coexist. A parametric shortcut hides exactly that, and hiding it is how a band ends up quietly lying about what it knows.

Read the band off the data, and count your samples honestly

The fix is to stop parameterising and read the interval straight off the empirical distribution of realised moves over the matching horizon, tilting the midpoint only with a momentum lean that engages when a trend gate clears — never with a hand-drawn line. Every number then has a stated source: it is a quantile of real history, not an assumption.

There is one trap in doing this, and it is a subtle one. The multi-day moves overlap — consecutive thirty-day windows share twenty-nine days of data — so the samples are heavily autocorrelated. If you report the raw count of overlapping windows as your sample size, you overstate your evidence by roughly the horizon. Bitcoin's thirty-day band, drawn from about 3,231 overlapping windows, rests on only around 107 independent months. That is a materially different epistemic object, and collapsing the two is how a backtest manufactures confidence it has not earned. We print the independent count on every chart for exactly this reason: a band should show how much history actually stands behind it, not how much it can appear to.

Coverage: the honesty metric that cuts both ways

The metric for an interval forecast is coverage, and its most important feature is that it fails in both directions. If you claim a 50% range, the outcome should land inside it about half the time across many days — not most of the time. A band that contains the price ninety percent of the time is not precise, it is padded, and padding is cowardice dressed as confidence: it can never be caught being wrong, which is exactly why it is worthless. A band too narrow gets caught immediately. Both are failures, and the only way to tell which one you are looking at is to score the same forecaster over many out-of-sample days against outcomes fixed in advance.

This is the measure a volatility model lives or dies by, and it is the one almost no public market analysis reports, because reporting it means publishing the times the band was wrong. Over-coverage has to count as a miss or the whole exercise is theatre. Say so in those words, or the number means nothing.

Why a volatility forecast has to be committed before the fact

A forecast is only evidence if it existed before the event. This is the plainest thing in the field and the most routinely ignored, because the entire "AI called this move" genre survives on screenshots taken afterward, on ranges that were never written down until they looked good.

The fix is not a better model. It is a timestamp. We write each forecast down first, serialise it, hash it with SHA-256, and anchor that hash to the Bitcoin blockchain through OpenTimestamps before any of it is public. Then we score it openly, the misses on the same page as the hits, with no filter that hides them. The Bitcoin block does not prove the forecast was good — the coverage score does that. It proves the number existed before the outcome did, which is the one claim no amount of after-the-fact narration can fake. One practical note from building this, because it is the kind of detail that quietly discredits an honest record: hash the exact bytes you publish. Write the file, hash the file, timestamp the file — if a reader runs the hash themselves and gets a different digest because you re-serialised in between, it reads as fraud even when nothing was wrong.

What a volatility model is not

The deflation belongs here, because leaving it out is how the genre gets away with itself. None of this is an edge. Reading volatility well lowers the cost of being wrong; it does not tell you the future, and it will not beat the market. No method reliably beats a liquid market, and anyone promising that is selling something — usually a subscription, sometimes a token, always a screenshot.

What an honest volatility model buys you is not prophecy. It is a band whose width means what it says, scored in the open where it is allowed to look bad, committed before the candle closed so the record cannot be curated later. A forecast is a risk object before it is anything else, and the machine earns its keep not in the arrow on the chart but in the honest width of the band around it — and in being able to prove, afterwards, that the width was honest.

Educational content — not financial advice, and not a betting tip.

Grading a probabilistic forecast: Brier score & log loss in Python · tags: python, machinelearning, datascience

NeuPortal — Fri, 17 Jul 2026 20:13:40 +0000

If your model outputs probabilities ("70% chance of X"), accuracy is the wrong way to grade it. A proper scoring rule rewards calibration and punishes confident wrongness — its best score comes only from reporting your true belief. The two workhorses: Brier score and log loss.
import numpy as np

def brier(p, y): # p: predicted prob, y: 0/1 outcome
return np.mean((p - y) ** 2) # lower is better

def log_loss(p, y, eps=1e-15):
p = np.clip(p, eps, 1 - eps)
return -np.mean(y*np.log(p) + (1-y)*np.log(1-p))

p = np.array([0.9, 0.6, 0.3, 0.8]); y = np.array([1, 0, 0, 1])
print(brier(p, y), log_loss(p, y))
Hedge everything to 0.5 to "look safe" and a proper rule penalizes you; inflate to 0.99 and, when you're wrong, it penalizes you far more. Accuracy rewards bluffing; proper scoring rules make bluffing expensive.

Pair it with a reliability diagram (bucket predictions by probability, plot predicted vs realized frequency — the diagonal is honesty) and you can actually tell whether a probabilistic model is any good, instead of cherry-picking the calls it got right.

We score every forecast this way in public — wins and losses both — at neuportal.ai/experiment.

Educational content — not financial advice.

Survivorship Bias: Why the Data You Can See Is Already Filtered

NeuPortal — Wed, 15 Jul 2026 08:13:14 +0000

Imagine judging how safe a sport is by interviewing only the people at the finish line. Everyone you talk to is fine, so you conclude the sport is harmless — never noticing that the people who got hurt are not in the room to be counted. That is survivorship bias: drawing conclusions from a sample that has already been filtered down to the winners.

The Planes That Came Back

In WWII, analysts studied bullet holes on returning bombers and wanted to add armor where the holes clustered. The statistician Abraham Wald pointed out the flaw: they were only looking at the planes that came back. The areas with the fewest holes on survivors — the engines — were exactly where a hit was fatal. The armor belonged where the survivors had no holes.

How It Poisons Financial Data

Funds that perform badly get closed and drop out of databases. Delisted companies fall out of indices. Blown-up strategies never get written about. What's left is a highlight reel, and any "average return" computed from it is flattering by construction.

Why Backtests Inherit It

Test a strategy on "the companies currently in the index" and you've quietly excluded every company that went to zero. The backtest looks robust because it was never shown the wreckage. Combined with overfitting, the equity curve is confident precisely because the hard cases are absent.

How We Guard Against It

The only real defense is keeping the failures in the data. In our public forecasting experiment every call is locked and Bitcoin-timestamped before the event, so we can't delete the ones that turn out wrong — the losers stay permanently on the record next to the winners. Right now the market is ahead of our model, and that number stays visible on purpose. A track record only means something when nothing has been edited out of it: neuportal.ai/experiment

Educational content — not financial advice, and not a betting tip.

Monte Carlo Simulation for Tournament Forecasting: From a Match Model to Bracket Probabilities

NeuPortal — Sat, 11 Jul 2026 12:54:59 +0000

Suppose you have a decent model for a single game вАФ say a Poisson model that, given two teams, spits out the probability of a home win, a draw, and an away win. Now someone asks the bigger question: "What's the probability this team lifts the trophy?" It's tempting to reach for a calculator and start multiplying. Resist that instinct. For anything past the simplest bracket, hand-multiplication quietly falls apart, and Monte Carlo simulation is the tool that actually works.

This article explains how to go from a match-level model to tournament-level probabilities by simulating the whole event thousands of times вАФ how the loop works, how many runs you need, how to attach confidence intervals, and where the approach can mislead you.

Why multiplying probabilities by hand breaks down

Picture a knockout bracket. For a team to win, it has to survive the round of 16, the quarter-final, the semi, and the final. If those four matches had fixed opponents and fixed win probabilities, you could just multiply: 0.7 √Ч 0.6 √Ч 0.55 √Ч 0.5 and be done.

The problem is that the opponents are not fixed. Who your team meets in the quarter-final depends on who won the other round-of-16 tie вАФ which is itself uncertain. To do this by hand you'd have to enumerate every possible path through the bracket, weight each path by the probability that this exact set of results occurred, compute your team's chance along that specific path, and sum over all of them. The number of paths explodes combinatorially. Add group stages with tie-breakers, seeding, re-seeding, byes, or extra-time-then-penalties, and the bookkeeping becomes hopeless.

Worse, the naive approach silently assumes independence and a single path, throwing away exactly the branching structure that makes a tournament a tournament. You need something that respects the bracket without writing down every branch. That something is simulation.

From a match model to a tournament: the core idea

Monte Carlo simulation flips the problem from calculating to playing. Instead of computing the probability of a path, you just play the tournament out once, at random, according to your match model. Then you do it again. And again вАФ tens of thousands of times.

Each simulated tournament is one plausible history of the event. In one run, the favourite crashes out early; in another, it cruises to the title; in a third, a mid-table side goes on an improbable run. No single run means anything. But run the whole thing 50,000 times and count how often each team ends up as champion, and those counts вАФ divided by the number of runs вАФ converge on the probabilities you actually wanted. You never enumerate a single path by hand. You let the branches sort themselves out.

The engine underneath is your match model. A common choice for football is a Poisson model: estimate each side's expected goals from attack/defence strength, then treat goals as Poisson-distributed. But the tournament layer doesn't care what the match model is. Poisson, an Elo-style win probability, a machine-learning classifier вАФ any model that turns "Team A vs Team B" into an outcome you can sample from will slot straight in.

The simulation loop, step by step

One simulated tournament is just a loop: sample every match in the current round, advance the winners, repeat until one team remains. Then wrap that in an outer loop and tally the results.

import numpy as np
from collections import Counter

def play_match(a, b, ratings, knockout=True):
    lam_a, lam_b = expected_goals(a, b, ratings)   # your match model
    goals_a = np.random.poisson(lam_a)
    goals_b = np.random.poisson(lam_b)
    if goals_a == goals_b and knockout:
        return resolve_tie(a, b, ratings)          # extra time / penalties
    return a if goals_a > goals_b else b

def simulate_once(bracket, ratings):
    alive = bracket.first_round()                  # list of (teamA, teamB)
    while len(alive) >= 1:
        winners = [play_match(a, b, ratings) for (a, b) in alive]
        if len(winners) == 1:
            return winners[0]                      # champion
        alive = pair_up(winners)                   # next round's matchups

N = 50_000
champs = Counter(simulate_once(bracket, ratings) for _ in range(N))

for team, wins in champs.most_common():
    p = wins / N
    se = (p * (1 - p) / N) ** 0.5
    print(f"{team}: {p:.3%}  ¬±{1.96*se:.3%}")

Notice the resolve_tie step. In a knockout, a level score can't stand, so you need a rule for extra time and penalties вАФ often modeled as something close to a coin flip, sometimes tilted by strength. That little function is a real modeling decision, not a detail, and we'll come back to why.

How many runs? Convergence and confidence intervals

Because the estimate is a proportion вАФ champions counted over runs вАФ the law of large numbers guarantees it settles toward the true model-implied probability as the number of runs grows. The useful fact is how fast. The Monte Carlo standard error of an estimated probability p over N runs is:

SE = sqrt( p * (1 - p) / N )

That square root is the whole story. Error shrinks with the square root of N, so to halve your uncertainty you need four times the runs. A worked example: for a coin-flip-ish p вЙИ 0.5 at N = 10,000, the standard error is about 0.005 вАФ half a percentage point. A 95% interval is roughly ¬±1.96 √Ч SE, so ¬±1 point. Push to N = 100,000 and that tightens to about ¬±0.3 points.

Two practical rules follow. First, report the interval. If one team comes out at 12.3% and another at 12.1%, and your Monte Carlo error is ¬±0.5 points, those two numbers are indistinguishable вАФ pretending otherwise is false precision. Second, rare events need more runs. Estimating a longshot's 0.5% title chance to a sensible relative accuracy takes far more runs than nailing the favourite's 30%, because when p is tiny you see very few successes. For headline numbers, 10,000 runs are usually plenty; for stable tails, 100,000 or more.

Reading the output: probabilities, not predictions

The output is a distribution, not a call. "Team A: 28% ¬± 0.4%" does not say Team A will win; it says that across many simulated tournaments built on this model, Team A came out on top about 28% of the time. Same discipline applies to every stage вАФ you get reach-the-final and reach-the-quarters probabilities from the same runs for free, just by tallying at each round.

This is also why simulation beats a single point forecast: it hands you the full shape of what's plausible, including the ugly-but-real chance the whole thing goes sideways.

Garbage in, garbage out: the limits

Here's the discipline. Monte Carlo simulation is an amplifier, not an oracle. It faithfully propagates whatever your match model believes вАФ including everything your match model gets wrong. Run a biased model 100,000 times and you get a beautifully tight, confidently wrong answer.

Keep three failure modes in view. Model error dwarfs Monte Carlo error: the ¬±0.4% from your runs is the easy uncertainty. The real uncertainty lives in your goal estimates, your ratings, and that resolve_tie coin-flip assumption вАФ and it's much larger. Independence is an assumption, not a fact: real tournaments carry injuries, fatigue, and momentum across matches, while a naive loop treats each game as a clean slate. And garbage inputs stay garbage вАФ stale ratings or a mis-specified home advantage don't get laundered by volume. More runs only make a wrong answer more precise, never more correct.

None of this makes the method less valuable. It makes it honest: a way to turn a match model into tournament probabilities that you can then test against reality, rather than a machine for manufacturing certainty.

The bottom line

Monte Carlo simulation is the right tool for tournament forecasting because it respects the branching structure a bracket creates, which hand-multiplication cannot. Build a match model you trust, sample outcomes, advance winners, repeat thousands of times, and count. Report confidence intervals so you don't over-read the noise, run more iterations for rare events, and never forget that the simulation is only as good as the model feeding it.

The honest test of any such model isn't how clean the code looks вАФ it's whether the probabilities hold up once the games are played. That's the part worth committing to in public, before kickoff.

Educational content вАФ not financial advice, and not a betting tip.

At NeuPortal we lock each forecast as explicit probabilities before the event, timestamp it onto the Bitcoin blockchain so it can't be backdated, and then score it in the open. The running board вАФ every locked forecast and its result, wins and losses alike вАФ is public at neuportal.ai/experiment. Check the claims yourself; that's the entire point.

How to Prove a Prediction Was Made Before the Event (with OpenTimestamps)

NeuPortal — Sat, 11 Jul 2026 12:47:26 +0000

Everyone who has ever been right about something loud enough to remember it will tell you they called it. The screenshot arrives after the match, after the candle, after the election. And there is no way to know whether it was written on Monday or edited on Friday. This is the quiet rot at the center of most "track records": a prediction you cannot date is not a prediction at all. It is a memory with good lighting.

The technical name for the problem is look-ahead. If a forecast can be created, tweaked, or cherry-picked after the outcome is known, then it carries zero information about skill. The only fix is to make the timing of a prediction independently checkable вАФ to prove a document existed in a specific form before a specific moment, without asking anyone to trust you, your server clock, or your database. That is precisely what OpenTimestamps does, using the Bitcoin blockchain as a shared, tamper-evident clock.

Why timing is the whole game

A forecast is a bet against the future. Its value comes entirely from the fact that the future was unknown when the forecast was fixed. The instant you allow post-hoc editing, every desirable property collapses: calibration becomes meaningless, Brier scores become fiction, and "I predicted this" becomes unfalsifiable.

So an honest forecasting system needs one hard guarantee before anything else: this exact text existed at this exact time, and has not changed since. Note what that guarantee does not require. It does not require publishing the forecast publicly in advance (you might want it sealed). It does not require a notary, a lawyer, or a trusted timestamping company that could be subpoenaed, hacked, or simply go out of business. It requires a clock that nobody controls and nobody can wind backward.

What "proof of existence" actually means

The building block is a cryptographic hash вАФ typically SHA-256. Feed any file into it and you get a 64-character fingerprint. Change a single comma and the fingerprint changes completely. Crucially, the process only runs one way: from the fingerprint you cannot reconstruct the document, and you cannot craft a different document that produces the same fingerprint.

That property gives us a clean trick. Instead of proving "this document existed at time T," we only need to prove "this hash existed at time T." If the hash is provably that old, and the hash uniquely identifies the document, then the document is provably that old too. You never have to reveal the content to prove its age вАФ you only reveal the hash. This is the difference between timestamping and publishing, and it is why sealed-then-scored forecasting is possible at all.

How OpenTimestamps works, step by step

OpenTimestamps (created by Bitcoin developer Peter Todd) turns that hash into a permanent, verifiable anchor on Bitcoin. The flow looks like this:

Hash the document. Your forecast file is reduced to a single SHA-256 fingerprint on your own machine. The content never leaves your control.
Submit the hash to a calendar server. Free, public aggregator servers collect hashes from users all over the world during a short window.
Aggregate into a Merkle tree. The server combines thousands of these hashes into a single Merkle tree вАФ a structure where every submitted hash is a leaf, and pairs of hashes are hashed together, level by level, up to one final value called the Merkle root. Your hash is now cryptographically bound to that root.
Anchor the root in a Bitcoin transaction. The server publishes just the Merkle root into a Bitcoin transaction. One transaction economically timestamps every hash in the tree at once, which is why the service can be free.
Wait for a block. When Bitcoin miners include that transaction in a block, the block header carries a timestamp and, more importantly, the immense proof-of-work behind it. From this moment on, your hash is provably older than that block.

The output you keep is a small .ots proof file. It contains the exact Merkle path from your document's hash up to the Bitcoin block вАФ every sibling hash needed to walk from your leaf to the anchored root. That path is the whole proof, and it is self-contained.

Why Bitcoin is the "honest clock"

You could, in principle, timestamp against any authority. The reason Bitcoin is the right one is that its clock is expensive to lie to. Each block is secured by proof-of-work вАФ a global, ongoing expenditure of real energy. To move a confirmed transaction to an earlier or later block, an attacker would have to re-mine that block and every block after it, out-racing the entire honest network. Nobody can quietly backdate a hash into last week's block, because last week's block is buried under a mountain of accumulated work.

Just as important, no single party owns the record. There is no company whose database you must trust, no admin who can edit a row, no jurisdiction that can compel a change. The timestamp is agreed on by a decentralized network and replicated across thousands of nodes. That combination вАФ costly to forge, owned by no one, checkable by everyone вАФ is what earns Bitcoin the description of an honest clock. For forecasting, it means the proof of "when" survives even if the forecaster disappears.

How anyone can verify a `.ots` proof

The point of all this is independence: you should not have to trust the person showing you a proof. Given the original document and its .ots file, anyone can verify it вАФ for example with the open-source client:

ots verify forecast.txt.ots

The verifier recomputes your document's hash, walks the Merkle path in the .ots file to reconstruct the anchored Bitcoin transaction, looks that transaction up in the blockchain, and reads the block's timestamp. If the document was altered by even one byte, the hash won't match and verification fails outright. If everything lines up, you get a concrete statement like "existed as of block N, mined on this date." No account, no API key, no faith in the publisher вАФ just math and a copy of the blockchain (or a public block explorer).

Applying it to accountability forecasting

This is the backbone of how NeuPortal (neuportal.ai) runs its accountability experiment. Every forecast is locked вАФ written, hashed, and timestamped through OpenTimestamps вАФ before the event it describes. Once reality delivers the outcome, the prediction is scored with a Brier score and compared against prediction-market baselines. Because the lock happened first and is anchored to Bitcoin, there is no room to quietly delete a bad call or sharpen a good one after the fact.

The proofs are public. Our timestamped forecast records live in an open repository (github.com/neuportalai-ui/experiment-proofs), and the underlying dataset is archived with a citable DOI on Zenodo (10.5281/zenodo.21294229). Anyone can pull a forecast, run ots verify, and confirm for themselves that it predates the event. That is the entire difference between a highlight reel and a track record: one asks for your trust, the other hands you the tools to withhold it.

The disclaimer that matters

Timestamping proves when a forecast was made and that it hasn't changed. It says nothing about whether the forecast was any good вАФ a confidently wrong prediction timestamps just as cleanly as a brilliant one. Proof of timing is a floor, not a ceiling: it makes honest scoring possible, but the scoring still has to happen in public, across many predictions, wins and losses alike. Read any timestamped record with that in mind. Educational content вАФ not financial advice, and not a betting tip.

Want to check the receipts? Browse our public, Bitcoin-timestamped forecast proofs at github.com/neuportalai-ui/experiment-proofs, cite the archived dataset via Zenodo DOI 10.5281/zenodo.21294229, and follow the live scoring at neuportal.ai. Don't trust the claim вАФ verify the timestamp.

NeuPortal Research

How AI Forecasting Works: A Plain-English Guide to Machines That Predict the Future

NeuPortal — Fri, 10 Jul 2026 10:57:56 +0000

AI forecasting explained in plain English: how machines predict the future with probabilities, the main methods, and how to judge if a forecast is trustworthy.

Ask most people whether a computer can predict the future and you’ll get one of two answers: a confident “yes, algorithms know everything now” or a dismissive “no, it’s all guesswork.” The truth sits in a more interesting middle. AI forecasting doesn’t tell you what will happen. It estimates how likely different outcomes are, based on patterns in data. Understanding that distinction is the key to reading any AI prediction without being fooled by it.

This guide explains, in plain English, how AI forecasting actually works — the main methods, how forecasts are scored, where they’re used, and how to tell a trustworthy forecast from one that merely sounds confident.

What “Forecasting” Really Means

A forecast is not a promise. When a weather model says there’s a 70% chance of rain tomorrow, it isn’t wrong if the day stays dry. It’s making a probabilistic statement: out of many similar days, roughly seven in ten would see rain.

This is the single most important idea in AI forecasting. Good forecasts deal in probabilities, not certainties. “The team will win” is a claim that’s either right or wrong. “The team has a 62% chance of winning” is a forecast — and you can only judge it fairly across many predictions, not from a single result.

So the honest answer to “can AI predict the future?” is this: it can estimate probabilities for future events, sometimes very well and sometimes poorly, but it cannot see the future. Anyone who tells you otherwise is selling something.

The Main Families of AI Forecasting Methods

“AI forecasting” is an umbrella term covering several very different techniques. Here are the big families, explained simply.

Statistical time-series models. These are the classic workhorses of predictive analytics. Methods like ARIMA and exponential smoothing look at a sequence of past values — daily sales, hourly temperatures, monthly traffic — and project the trend and seasonal patterns forward. Time series forecasting is what powers a lot of the “expected demand next week” numbers you never see.

Probabilistic and Bayesian models. Instead of a single answer, these produce a full range of possible outcomes with probabilities attached. Bayesian methods are especially good at updating beliefs as new evidence arrives: start with a prior estimate, see fresh data, revise. They naturally express uncertainty, which is exactly what a forecast should do.

Regression and Poisson models. Regression links an outcome to explanatory factors — how price, weather, and day of the week combine to shape sales. Poisson models specialize in counting events that happen at some average rate, like the number of goals in a match or support tickets in an hour. Simple, transparent, and often surprisingly hard to beat.

Machine learning forecasting. Here algorithms like gradient-boosted trees and neural networks learn patterns from large, messy datasets without being told the exact rules. Machine learning forecasting shines when relationships are complex and nonlinear, and when there’s plenty of data. The trade-off: models can become black boxes, and they can “learn” noise that doesn’t repeat.

LLM-based forecasting. The newest entrant. Large language models can read news, reports, and context, then reason in words toward a probability estimate. This is promising for messy, real-world questions that pure number-crunching can’t capture. But language models can also sound authoritative while being wrong, so their forecasts need the same scoring discipline as any other method.

No single family is “best.” The right tool depends on the question, the data available, and how much uncertainty you’re willing to live with.

How AI Forecasts Are Evaluated

Here’s where most forecasting hype falls apart. A prediction being right once tells you almost nothing. Flip a coin, call heads, get heads — you didn’t predict anything. Real evaluation looks at many forecasts over time.

Calibration. A well-calibrated forecaster is right about as often as they claim. When they say “70% chance,” the event should happen roughly 70% of the time across all their 70% predictions. Probability calibration is the gold standard, because it measures honesty rather than luck.

Brier score and log loss. These are numerical scores that reward forecasts for being both confident and correct, while punishing confident wrong calls harder than hesitant ones. A lower Brier score means better forecasts. Scores like these let you compare two forecasters fairly instead of trading anecdotes.

The lesson: judge a forecasting system by its track record across many predictions, ideally scored with calibration and a metric like the Brier score — never by a single lucky hit.

Where AI Forecasting Is Used

Predictive analytics quietly runs a large part of the modern world:

Weather. The original probabilistic forecasting domain, and still one of the most rigorous.
Demand and supply chain. Retailers and manufacturers forecast demand to decide what to stock and where.
Sports. Models estimate win probabilities and expected scores from team and player data.
Finance and economics. Institutions forecast risk, volatility, and macro indicators — though markets are famously hard to beat.
Elections. Poll-based models express results as probabilities, which is why a “likely” winner can still lose.
Health and medicine. Models forecast disease spread, patient risk, and resource needs.

In every one of these, the output is a probability or a range — not a guarantee.

The Limits and Failure Modes

Being a smart reader of AI predictions means knowing how they go wrong.

Overconfidence. Many systems (and people) state probabilities that are too extreme. A model that says “95% certain” far more often than it turns out to be right is poorly calibrated, even if it sounds impressive.

Unfalsifiable claims. “Something big will happen soon” can never be scored, so it isn’t a real forecast. Trustworthy forecasts are specific, time-bound, and checkable.

Cherry-picked track records. Anyone can highlight their best calls and bury the misses. A track record only means something if it includes every forecast, scored consistently.

Garbage in, garbage out. Models trained on biased, stale, or thin data will project those flaws forward. And the future can simply break from the past — a model cannot foresee a genuinely new kind of event it has never seen before.

None of this makes forecasting useless. It makes unscored, cherry-picked forecasting useless.

How to Judge Whether an AI Forecast Is Trustworthy

You don’t need to be a data scientist to be a critical reader. Ask:

Is it a probability, or a promise? Real forecasts give odds and admit uncertainty. Stated certainty is a red flag.
Is there a scored track record? Look for calibration and Brier scores across many public predictions, not testimonials.
Was the forecast locked in before the event? A prediction recorded after the fact, or quietly edited, proves nothing.
Are the misses shown too? Honest forecasters publish their whole record, wins and losses alike.
Does it explain its reasoning and data? Transparency beats a confident tone every time.

If a forecast fails these tests, treat it as entertainment, not evidence.

One example of the “lock it in and score it” approach is NeuPortal (neuportal.ai), an experiment that records AI forecasts before events happen and then scores them in public — the kind of transparency that separates a real track record from a highlight reel.

The Bottom Line

AI forecasting is genuinely useful and increasingly common, but it is a tool for estimating probabilities, not a crystal ball. The methods range from century-old statistics to brand-new language models, yet they are all judged by the same honest standard: calibration and track record over many predictions. Learn to ask for probabilities, scored history, and pre-committed forecasts, and you’ll be a far sharper reader of any machine that claims to predict the future.

This is educational content, not betting or financial advice.

DEV Community: NeuPortal

OpenAI's finance AI scores 69.9%. Build a 50-question eval harness for your own documents.

ai #machinelearning #python #testing

GPT-6 Astra for Developers: What Changed in Codex, and a 90-Minute Test Plan for Your Repo

Your Time-Series Validation Score Is Inflated, and Your Test Suite Will Never Tell You

Where it comes from

Why this is not the same problem as leakage

What breaks downstream

How bad it gets at long windows

Five things to do about it

The general lesson

The Bug Class Where Your Code Is Correct, Your Tests Pass, and It Never Runs

Case 1: the feature that shipped switched off

Case 2: the function that only advances when its branch is taken

Case 3: the check that checked the wrong thing

Why unit tests do not catch any of this

What actually finds them

The cheap version

AI and Volatility: Forecasting How Much a Market Moves, Not Which Way

Why direction is the wrong question for a liquid market

Volatility is predictable in a way returns are not

The band most people draw is wrong in both directions

Read the band off the data, and count your samples honestly

Coverage: the honesty metric that cuts both ways

Why a volatility forecast has to be committed before the fact

What a volatility model is not

Grading a probabilistic forecast: Brier score & log loss in Python · tags: python, machinelearning, datascience

Survivorship Bias: Why the Data You Can See Is Already Filtered

The Planes That Came Back

How It Poisons Financial Data

Why Backtests Inherit It

How We Guard Against It

Monte Carlo Simulation for Tournament Forecasting: From a Match Model to Bracket Probabilities

Why multiplying probabilities by hand breaks down

From a match model to a tournament: the core idea

The simulation loop, step by step

How many runs? Convergence and confidence intervals

Reading the output: probabilities, not predictions

Garbage in, garbage out: the limits

The bottom line

How to Prove a Prediction Was Made Before the Event (with OpenTimestamps)

Why timing is the whole game

What "proof of existence" actually means

How OpenTimestamps works, step by step

Why Bitcoin is the "honest clock"

How anyone can verify a .ots proof

Applying it to accountability forecasting

The disclaimer that matters

How AI Forecasting Works: A Plain-English Guide to Machines That Predict the Future

How anyone can verify a `.ots` proof