DEV Community: Naoyuki Umetani

How Accurate Is Statistical Computation in JavaScript? — A Comparative Analysis: rStats.js vs. R

Naoyuki Umetani — Mon, 09 Nov 2026 06:35:25 +0000

Introduction

When performing statistical analysis in JavaScript, one of the primary concerns is: "Can the calculation results truly be trusted?" In scenarios such as the tails of probability distributions, cases involving extremely large parameters, or instances with infinitesimal probabilities, the choice of numerical computation method can lead to significant discrepancies in the results.

To address this challenge directly, I am developing rStats.js, a statistical computation library for JavaScript. In this project, I use R as a key benchmark to continuously verify the accuracy of these statistical calculations.

In this article, I will explain the methodology used to compare rStats.js against R, as well as the level of precision I aim to achieve.

1. Automated Verification Against R 4.6.2

In rStats.js, the general policy is to compare every public-facing function against R version 4.6.2. This goes beyond simply testing a few representative input values; the test suite incorporates a diverse range of input conditions, including:

Typical input values
Extremely small probabilities (e.g., 1e-300)
The extreme tails of distributions
Boundary values (e.g., 0, 1, or cases where degrees of freedom approach infinity)
NaN and Infinity
Negative values
Extremely large parameters

The expected values—calculated using R—are generated via dedicated R scripts and stored as JSON fixtures. Consequently, there is no need to have R installed when running the tests; the results from R are pre-saved, allowing the JavaScript test suite to verify against them.

2. Precision Standards Vary by Function Type

When evaluating the accuracy of numerical computations, applying a uniform margin of error across all functions is not appropriate. In rStats.js, the allowable margin of error is tailored to the specific nature of each calculation.

Calculation Type	Relative Tolerance
Special functions, probability densities, and distribution functions	1e-14
Descriptive statistics	1e-14
Hypothesis testing	1e-12
Linear models	1e-12
Linear algebra	1e-14 – 1e-13
Numerical methods	1e-14 – 1e-15
Mixed-effects models	Primarily 1e-12 – 1e-9

These values are not merely design targets; they are tolerances established based on the results of running a large number of test cases.

Note that not all results can be evaluated solely by simple relative error. In cases where values are close to zero or where numerical cancellation occurs, evaluation is performed using absolute error or an appropriate scale.

3. R’s Results Are Not Always Correct

This is an aspect to which we pay particular attention during the development of rStats.js.

While R is an excellent environment for statistical computing, its numerical calculations are not necessarily accurate under all conditions. In scenarios involving extremely small probabilities or exceptionally large parameter values, even R can be subject to rounding errors or loss of precision.

Therefore, if our results do not match those of R, we do not automatically assume that the error lies with rStats.js. Instead, we calculate high-precision reference values when necessary to verify which result is mathematically correct.

To calculate these reference values, we primarily use the following methods:

Arbitrary-precision arithmetic (256–1300 bits) via Rmpfr
Exact rational number arithmetic via gmp
High-precision computation via mpmath
Independent verification using numerical integration or mathematically derived series

For instance, if the value of a probability distribution function does not match R’s output, but it does align with our high-precision reference value, we consider that result to be "more accurate than R." ## 4. Cases where more accurate results are returned than in R

Here are a few specific examples.

4-1. Extremely small probabilities

Consider the following calculation:

pgamma(1e-100, 3, lower.tail = FALSE, log.p = TRUE)

R result: -1.6666666666665175e-301
High-precision reference value: -1.6666666666666667e-301
rStats.js: -1.666666666666667e-301

In this case, the result from rStats.js is much closer to the high-precision reference value.

4-2. Non-central t-distribution calculations

For the non-central t-distribution, there are cases where R fails to correctly calculate extremely small probabilities. For example:

pt(1, 5, ncp = 40)

R: 1.009e-303
High-precision reference value: 1.0887e-287
rStats.js: Matches the high-precision reference value

As shown here, there are instances where simply replicating R's output is insufficient.

Note that the values presented here are taken from the verification examples provided in docs/differences-from-r.md on GitHub.

5. Evaluating quantile functions using a different approach

Quantile functions for probability distributions require a different evaluation method than standard functions. In regions where the cumulative distribution function is very flat, even a tiny difference in probability can lead to a significant discrepancy in the quantile value. Therefore, simply comparing the output against R's results using relative error is not an appropriate approach. rStats.js employs a "backward error" approach, where the result of a quantization function is fed back into the corresponding distribution function for evaluation. In other words, it assesses how closely the value returned by rStats.js—when substituted back into the distribution function—approximates the target probability.

Even if a result differs from that of R, it is considered acceptable provided the backward error is equal to or less than that of R. For discrete distributions, it is also verified that the function returns the smallest integer that satisfies the definition.

6. Verification of Random Number Generation

In addition to statistical functions, rStats.js implements random number generation. It includes a setSeed() function (analogous to R’s set.seed()) and supports various random number generation algorithms, such as Mersenne-Twister and L'Ecuyer-CMRG.

For random number generation, the following aspects are verified:

Discrete random numbers and sampling results match those of R.
The state of the L'Ecuyer-CMRG generator matches that of R.
Continuous random number values fall within an acceptable margin of error.
The total count of random numbers consumed matches that of R.

For discrete results, an exact match is required, while for continuous results, the standard is generally a deviation of no more than 1e-12.

Note that for continuous random numbers, results may not match down to the least significant bit; this is due to differences between JavaScript’s Math functions and the mathematical libraries used by the R runtime environment. Such discrepancies are intentional and are considered to be within an acceptable range.

7. Documenting Cases Where Results Differ from R

The goal of rStats.js is not necessarily to achieve an exact match with R in every instance. Because the priority is to ensure mathematically and statistically correct results, the library may occasionally produce outputs that differ from those of R.

Rather than silently excluding such cases from the test suite, these instances are documented as knownDifferences—with the underlying reasons explained—and are made publicly available in the docs/differences-from-r.md file on GitHub. In some cases—such as when R returns NaN or Infinity for undefined results—rStats.js is designed to throw an error instead. By explicitly documenting these discrepancies, we ensure that users are aware of how the library’s behavior differs from that of R.

8. Implementation Supporting Numerical Precision

To achieve this level of precision, rStats.js employs not only standard double-precision floating-point arithmetic but also "Double-Double" arithmetic where necessary. Double-Double arithmetic combines two double-precision floating-point numbers to provide a greater number of significant digits than is otherwise possible.

For instance, this technique is used in calculations such as:

Computing the exponent part of exponential functions.
Evaluating continued fractions for incomplete beta functions.
Evaluating continued fractions for incomplete gamma functions.
Asymptotic expansions for large parameters.
Operations prone to catastrophic cancellation (loss of precision).

Rather than applying high-precision arithmetic to every calculation, we limit its use to areas where precision is a critical concern, thereby minimizing the impact on the performance of standard calculations.

9. Transparency Regarding Validation Results and Limitations

rStats.js provides full transparency regarding its validation methods and its differences from R via GitHub:

Within the scope of our validation, no known precision issues have been identified. However, this does not constitute a guarantee of mathematical perfection for all possible input values. Certain precision limitations remain outside the validated range—for example, when dealing with Bessel functions of extremely high order.

We believe that what matters most is not merely claiming high precision, but being transparent about the validation methods used and the inherent limitations of the implementation. ## Conclusion

I believe that JavaScript is not merely a language for developing web applications; it can also serve as a platform for performing reliable statistical computations.

In rStats.js, we verify the accuracy of numerical calculations by using R as a key benchmark, alongside high-precision reference values. Moving forward, I intend to not only expand the range of available statistical methods but also further enhance the reliability and verifiability of these numerical computations.

rStats.js is an open-source project licensed under the MIT License.
GitHub: https://github.com/numero-uno-jpn/rStats.js/

I welcome feedback from developers interested in statistical computing, numerical analysis, and data analysis using JavaScript.

Achieving Precision Equal to or Greater Than R in JavaScript: Fundamentals of Double-Double Arithmetic and Its Usage in rStats.js

Naoyuki Umetani — Mon, 09 Nov 2026 06:35:05 +0000

rStats.js is a statistics library for JavaScript and the browser. Its function names, arguments, default values, and return values are modeled after R, and every function has been tested by comparing its output against that of R. It covers areas such as probability distributions, random number generation, various statistical tests, analysis of variance (ANOVA), linear models, mixed-effects models, and survival analysis. It has no external dependencies and is released under the MIT License.

GitHub: https://github.com/numero-uno-jpn/rStats.js
Demo: https://numero-uno-jpn.github.io/rStats.js/demo/ (You can compare the results of all functions against the values produced by R.)
npm: npm install @numero-uno-jpn/rstats

The background behind its creation and an overall introduction can be found in my previous article.

Why use "double-double" arithmetic?

In JavaScript, numbers are represented as 64-bit doubles, which provide a precision of approximately 16 decimal digits. When performing calculations using standard doubles, the discrepancy from the true value can sometimes exceed 1e-14—for instance, in the tails of probability distributions. rStats.js addresses this by employing "double-double" arithmetic (combining two doubles to achieve approximately 32 digits of precision) for these specific cases, ensuring accuracy equal to or better than that of R.

In this article, I will explain the fundamentals of double-double arithmetic using JavaScript code, describe where and how it is applied within rStats.js, and discuss how I optimized the performance of these otherwise computationally intensive operations.

When I asked an AI how to achieve precision equal to or better than that of R, it suggested "double-double" arithmetic. While I was aware of the technique, I had never used it before. Although the concept made sense immediately, I was concerned that it might be too computationally expensive. In fact, this became an issue (discussed later).

Calculations with `double` (double-precision floats) result in more errors than one might expect

Let’s start with a well-known example:

0.1 + 0.2; // 0.30000000000000004

When adding values that differ significantly in magnitude, the smaller value is lost due to rounding.

let s = 0;
for (const x of [1e16, 1, -1e16]) s += x;
s;  // 0 (the correct result is 1)

As soon as 1 is added to 1e16, the 1 is lost due to rounding.

The same thing happens in R. While R’s sum function may utilize long double—which offers higher precision than double—if the environment supports it, the actual level of precision varies by environment. In the version of R (4.5.2) I used on Windows to write this article, .Machine$longdouble.digits was 53, meaning it had the same precision as a standard double. Consequently, the following sum is incorrect in this R environment:

sum(c(1e20, 1e3, 1e-10, -1e20, -1e3))  # -1000 (the correct result is 1e-10)

In contrast, the sum function in rStats.js returns 1e-10 for this calculation.

While developing rStats.js, I was surprised to discover quite a few instances where R’s own results were imprecise. I have compiled these findings in a document titled Differences from R. Furthermore, the fact that the precision of intermediate calculations in R can vary depending on the environment was, in itself, a surprise. WebR is an implementation of R compiled into WebAssembly, allowing it to run in browsers or within Node.js. Since WebAssembly instructions only support 32-bit and 64-bit floating-point numbers, WebR performs all calculations using 64-bit doubles. In the version of R for Windows that I examined, the precision of long double was also identical to that of double. Learning about these types of numerical errors convinced me of the utility of rStats.js.

Fundamentals of Double-Double Arithmetic

Double-double arithmetic represents a single number as the sum of two double values: hi + lo. The hi component holds the bulk of the value, while the lo component captures the remainder that cannot be represented by hi alone. This approach yields a precision of approximately 106 bits (roughly 32 decimal digits).

The foundation for this lies in the "error-free transformations" described by Dekker (1971) and Knuth (in The Art of Computer Programming, section 4.2.2). These methods allow for the precise decomposition of an arithmetic operation (such as addition or multiplication) into two parts: the rounded result and the error term discarded during the rounding process. ### Addition: twoSum

// Returns [s, e] such that a + b = s + e holds exactly
function twoSum(a, b) {
const s = a + b;
const bb = s - a;
const e = (a - (s - bb)) + (b - bb);
return [s, e];
}

twoSum(0.1, 0.2);  // [0.30000000000000004, -2.7755575615628914e-17]

Here, s is the result of a standard addition, and e represents the portion lost due to rounding. In the first example, the sum can be corrected by adding back the lost portion.

let hi = 0, lo = 0;
for (const x of [1e16, 1, -1e16]) {
const [s, e] = twoSum(hi, x);
hi = s;
lo += e;
}
hi + lo;  // 1

Multiplication: twoProd

To extract the error from a multiplication, the simplest approach in many other languages is to use FMA (an instruction that performs a fused multiply-add, with only a single rounding step). However, JavaScript lacks an FMA operation. Instead, we use Veltkamp’s method to split each number into two parts—a high part and a low part—and then calculate their products without rounding. Since the product of two 26-bit values fits within 52 bits, it can be represented exactly using a double-precision float (which has a 53-bit significand).

const SPLITTER = 134217729;  // 2^27 + 1

// Splits 'a' into 'hi' and 'lo' components, each with a 26-bit significand (a = hi + lo)
function split(a) {
const t = SPLITTER * a;
const hi = t - (t - a);
return [hi, a - hi];
}

// Returns [p, e] such that a * b = p + e holds exactly
function twoProd(a, b) {
const p = a * b;
const [ah, al] = split(a);
const [bh, bl] = split(b);
const e = ((ah * bh - p) + ah * bl + al * bh) + al * bl;
return [p, e];
}

twoProd(0.1, 0.1); // [0.010000000000000002, -8.326672684688674e-19]

javascript

Addition, multiplication, and division for "double-double" numbers are constructed by combining these two operations. As an example, addition is implemented as follows—this follows the same procedure as ddAdd in rStats.js.

// A version of twoSum with fewer steps; valid when |a| >= |b|
function quickTwoSum(a, b) {
const s = a + b;
return [s, b - (s - a)];
}

// Addition of two double-double numbers: (ah + al) + (bh + bl)
function ddAdd([ah, al], [bh, bl]) {
let [s, e] = twoSum(ah, bh);
const [t, f] = twoSum(al, bl);
e += t;
[s, e] = quickTwoSum(s, e);
e += f;
return quickTwoSum(s, e);
}

// Sum of 1e16 + 1 and -1e16 + 0.5
ddAdd([1e16, 1], [-1e16, 0.5]);  // [1.5, 0]

This approach uses twoSum to add the high-order components (ah and bh) and the low-order components (al and bl) separately. It then incorporates the resulting error terms step-by-step, finally formatting the result back into a high-order (hi) and low-order (lo) pair.

The code provided here is a minimal implementation intended to illustrate the underlying mechanism. The implementation in rStats.js (src/special/dd.ts) includes additional handling for edge cases: if an overflow occurs, the error term is set to 0; if a value is so large that SPLITTER * a would overflow, the value is scaled down (by a power of 2) to bring it closer to 1 before the split is performed.

Usage in rStats.js

rStats.js does not use double-double arithmetic for every calculation. It is employed only in scenarios where standard double-precision floating-point numbers are insufficient to achieve the required precision of $1 \times 1 0^{- 14}$ . For example, the following cases are relevant:

Exponentials of small probabilities: When calculating probabilities in the form $exp (- E)$ , the absolute error in $E$ translates directly into a relative error in the probability. As the probability approaches $1 0^{- 300}$ , $E$ reaches several hundred, and the error using double precision reaches $1 0^{- 13}$ . At $E = 700$ , the mere act of rounding $E$ introduces an error of $6 \times 1 0^{- 14}$ .
Continued fractions for the incomplete beta function: When parameters are large and $x$ is near the mean, the result becomes highly sensitive to the rounding of $x$ . For $a = 1 0^{5}$ and $x = 0.999$ , using only double precision results in a deviation of $6 \times 1 0^{- 13}$ .
Summations for statistical tests: Calculations for means, variances, and correlations are performed using double-double precision.

Conversely, in cases where the upper bound of the error indicates that standard double precision is sufficient, the double-double operations are omitted.

The impact of this can be verified by disabling double-double arithmetic internally. When disabled, only about half of the test results maintain a precision of $1 0^{- 14}$ . In some cases, the degradation is significant: the beta function (beta) shows a deviation of up to $3 \times 1 0^{- 5}$ , and the pt function (Student's t-distribution) deviates by up to $6 \times 1 0^{- 7}$ .

The absence of FMA (Fused Multiply-Add) in JavaScript also influences the choice of R—the reference implementation used for comparison. The arm64 version of R utilizes FMA, which causes the final digit of the second value generated by set.seed(42); rnorm(3) to differ from that of the x86_64 version. rStats.js...

Why I Built rStats.js: Bringing Reliable Statistical Computing to JavaScript

Naoyuki Umetani — Mon, 09 Nov 2026 06:13:46 +0000

Introduction

I am developing rStats.js, an open-source library for performing statistical analysis in JavaScript.

My goal is to make it easy to access numerically accurate and statistically sound statistical computations from within JavaScript applications.

JavaScript is used not only in web browsers but also in a wide range of other environments, such as Node.js. Meanwhile, R and Python are the dominant languages in the field of statistical analysis; I felt that the options for performing robust statistical computing in JavaScript still had significant room for improvement.

Consequently, I decided to develop a JavaScript library that focuses not just on implementing statistical methods, but also on the precision of numerical calculations and the validity of the underlying algorithms.

Why Develop My Own Instead of Using Existing Statistical Libraries?

There are already several statistical libraries available for JavaScript. I have examined many of these existing libraries while working on my own project.

In doing so, I placed a high priority on the accuracy and verifiability of statistical computations.

Simply having a statistical function implemented is not the same as ensuring that its output is correct under various conditions. In fields like medical research or the analysis of clinical data, even a minor discrepancy in a calculation result could affect how research findings are interpreted.

With rStats.js, I emphasize the following:

Adopting statistically sound algorithms.
Verifying the precision and stability of numerical computations.
Using R’s statistical functions as a key benchmark.
Ensuring correct behavior even with boundary values or extreme inputs.
Providing an API that is easy to integrate into real-world applications.

Using R as a Key Reference

In the world of statistical analysis, R is a cornerstone.

With rStats.js, I aim to enable similar types of analysis in JavaScript by using R’s functions and statistical methods as a reference.

However, the goal is not to blindly replicate R’s behavior in every instance. We conduct comparative verification against R, while also utilizing high-precision reference values whenever necessary. In cases where our results differ from those produced by R, we prioritize the outcome that is mathematically and statistically correct.

Our focus is not merely on replicating the results of existing software, but on ensuring that the results themselves are accurate.

Bringing Statistical Analysis to JavaScript Applications

A primary goal of rStats.js is to enable the integration of statistical analysis capabilities not only into dedicated statistical software but also into general-purpose web applications and business systems.

Potential use cases include:

BI and data analysis tools
Survey and research platforms
Medical and clinical research systems
Educational and learning applications
Browser-based statistical analysis tools

Performing statistical calculations directly within a browser offers significant advantages regarding privacy and data management, as it eliminates the need to transmit data to an external server.

Future Outlook

We are currently continuing to expand our range of statistical methods and verify our numerical computations.

Moving forward, we aim to cover a broader spectrum of statistical techniques. We are also committed to enhancing API stability, improving documentation, expanding our test suite, and providing practical code samples—all with the goal of creating a library that developers can rely on with confidence.

Ultimately, we aspire to see this library adopted not only for individual research and study but also integrated into other open-source projects and commercial applications.

The Project

rStats.js is an open-source project released under the MIT License.

GitHub: https://github.com/numero-uno-jpn/rStats.js/

We welcome feedback and contributions from developers looking to perform statistical calculations in JavaScript, as well as from anyone with an interest in statistical algorithms and numerical computation.

Please take a look at the project!

DEV Community: Naoyuki Umetani

How Accurate Is Statistical Computation in JavaScript? — A Comparative Analysis: rStats.js vs. R

Introduction

1. Automated Verification Against R 4.6.2

2. Precision Standards Vary by Function Type

3. R’s Results Are Not Always Correct

4-1. Extremely small probabilities

4-2. Non-central t-distribution calculations

5. Evaluating quantile functions using a different approach

6. Verification of Random Number Generation

7. Documenting Cases Where Results Differ from R

8. Implementation Supporting Numerical Precision

9. Transparency Regarding Validation Results and Limitations

Achieving Precision Equal to or Greater Than R in JavaScript: Fundamentals of Double-Double Arithmetic and Its Usage in rStats.js

Why use "double-double" arithmetic?

Calculations with double (double-precision floats) result in more errors than one might expect

Fundamentals of Double-Double Arithmetic

Multiplication: twoProd

Usage in rStats.js

Why I Built rStats.js: Bringing Reliable Statistical Computing to JavaScript

Introduction

Why Develop My Own Instead of Using Existing Statistical Libraries?

Using R as a Key Reference

Bringing Statistical Analysis to JavaScript Applications

Future Outlook

The Project

Calculations with `double` (double-precision floats) result in more errors than one might expect