IntelligenceMax updates a learner estimate after every answer and displays it on the familiar IQ scale. The arithmetic resembles item response theory. That resemblance is useful, but it is easy to ask the formula to carry more meaning than it can bear.
A formula can be internally consistent while its inputs remain uncertain. Here, the distinction begins with the item parameters.
The update rule
For ability θ, item difficulty b, and discrimination a, the model assumes:
P(correct) = sigmoid(a × (θ - b) / 15)
The first estimate has a normal prior centred at 100 with a standard deviation of 15. After an answer u, where correct is 1 and incorrect is 0, the implementation takes one capped local update using the score and Fisher information:
information = (a / 15)² × p × (1 - p)
posterior precision = prior precision + information
delta = (u - p) × (a / 15) / posterior precision
The per-answer change is capped at 1.5 points. Displayed uncertainty also has a floor, including a 15 / √n evidence floor based on the number of answers.
Suppose the current estimate is 100 with a standard error of 5. A question has b = 115 and a = 1.2. The assumed probability of a correct answer is about 0.23. A correct response moves the estimate up by about 1.5 points after the cap; an incorrect response moves it down by about 0.45.
The calculation is the easy part to show. The inputs are where the uncertainty lives.
Model-assigned is not empirically calibrated
The AI that generates each question also assigns its difficulty and discrimination. Those values are checked for shape and range, but they are not fitted from a norming sample. Calling the update "IRT-style" describes the logistic form. It does not establish that 115 means the same thing as a score of 115 on a validated instrument.
The current system can calculate the answer probability implied by the learner estimate and the assumed item parameters. It does not yet establish that questions labelled 115 are answered at the predicted rate across an appropriate population. That would require empirical item calibration, stable items, enough responses per item, and checks for differential item functioning.
The product also reports a separate confidence-calibration score. That score is a Brier-style measure comparing a user's stated confidence with correctness. It does not validate item difficulty or the ability estimate.
What this number can support
Within those limits, the estimate can support an adaptive practice loop. It gives the system a consistent way to react to surprising answers and prefer questions near the current estimate.
It cannot establish a clinical Full Scale IQ, a permanent change in general intelligence, transfer to school or work, or treatment of a cognitive condition. Those conclusions require validated instruments and, for change claims, a study that separates training from expectation, retesting, selection, and ordinary fluctuation.
The transfer literature matters here. Reviews of commercial brain training and working-memory training find the strongest evidence on trained tasks and close relatives. Broad gains are less convincing, especially against active control groups. A rising practice estimate may represent learning within the task family. It is not evidence by itself that a distant ability changed.
What I would test next
An empirical calibration report should group responses by predicted probability and compare prediction with observed accuracy. A simple first diagnostic would publish reliability bins, sample counts, and a Brier score or log loss for the model's P(correct). Stable reusable items would permit item-level residuals. Generated one-off items make that harder and may require hierarchical evaluation by prompt version, domain, and assigned difficulty band.
Until that work exists, the precise statement is:
This is a logistic, IRT-shaped practice estimate that uses model-assigned item parameters. It is not a normed IQ score.
The current measurement notes are public at https://intelligencemax.ai/science.
Sources:
- Simons et al. (2016): https://doi.org/10.1177/1529100616661983
- Melby-Lervåg, Redick, and Hulme (2016): https://doi.org/10.1177/1745691616635612
Disclosure: I built IntelligenceMax. The technical question I most want readers to challenge is whether useful empirical calibration is possible for generated, mostly one-off items without quietly turning the system into a fixed bank.
Top comments (0)