I Replaced My sklearn Pipeline With Pure Rust. The Docker Image Shrank 400x

Murat Genç — Sat, 01 Aug 2026 14:37:07 +0000

How datarust gives you StandardScaler, OneHotEncoder, LogisticRegression, and JSON model persistence — without BLAS, Python, or the guilt of a 900MB Docker image.

There's a moment in every ML project where the notebook stops being fun and the deployment starts being painful.

You've got your Pipeline. You've got your metrics. You hand it to the platform team, and they ask: "Why does this need 900 megabytes?"

Because scikit-learn pulls in NumPy, SciPy, joblib, threadpoolctl, and a Fortran BLAS library. That's why.

I've been in that meeting. I've been the person who built the beautiful sklearn pipeline and then watched it drown in Docker layers. So I started wondering: what if the entire preprocessing + classical ML workflow existed as a pure Rust library — zero BLAS, zero Python runtime — that compiled to a single static binary?

That library is datarust. It just hit v0.6, and I want to show you the same workflow you'd write at work — mixed data types, preprocessing, training, evaluation, serialization — but without the Python tax.

The setup: two commands, zero configuration

cargo new churn_classifier && cd churn_classifier
cargo add datarust --features datasets,serde

That's it. No pip install. No virtual environment. No fighting with BLAS backends. The default build has zero external dependencies.

The data: mixed types, real problems

Real data is messy. You've got numbers (tenure, monthly_charge), categories (contract_type), missing values, and different scales. In datarust, data lives in two containers: Matrix for numbers and StrMatrix for strings.

They're deliberately separate types — the compiler won't let you accidentally feed a string column to a scaler.

use datarust::Matrix;
use datarust::matrix::StrMatrix;

// Numeric: tenure (months), monthly_charge ($), age
let numeric = Matrix::new(vec![
    vec![3.0,  85.0, 24.0],
    vec![12.0, 70.0, 31.0],
    vec![f64::NAN, 95.0, 45.0],   // missing tenure
    vec![48.0, 55.0, 52.0],
])?;

// Categorical: contract_type
let categorical = StrMatrix::from_strings(vec![
    vec!["MonthToMonth"],
    vec!["OneYear"],
    vec!["MonthToMonth"],
    vec!["TwoYear"],
])?;

// Target: 1 = churned, 0 = stayed
let y = vec![1.0, 0.0, 1.0, 0.0];

Already, something nice has happened: Matrix::new validates that all rows have the same length. No silent broadcasting bugs. If your data is ragged, you find out at construction, not three functions deep.

Preprocessing: the part that usually hurts

This is where most Rust ML libraries tap out. They give you a model — maybe a linear regression — but ask you to handle scaling, encoding, and imputation yourself. datarust takes the sklearn approach: these are first-class citizens.

I want to do four things:

Impute the missing tenure value (replace with the mean)
Scale the numeric columns to zero mean / unit variance
One-hot encode the contract type
Do all of the above in a single, composable step

Here's the ColumnTransformer:

use datarust::compose::{ColumnTransformer, Remainder, Table};
use datarust::encoder::{HandleUnknown, OneHotEncoder};
use datarust::imputer::{ImputeStrategy, SimpleImputer};
use datarust::scaler::StandardScaler;
use datarust::transformer_kind::TransformerKind;
use datarust::CategoricalTransformerKind;

let table = Table::new(numeric, categorical)?;

let mut ct = ColumnTransformer::new()
    .remainder(Remainder::Drop)
    .add_numeric(
        "tenure_imputed",
        vec![0],           // column 0 of numeric matrix
        TransformerKind::SimpleImputer(SimpleImputer::new(ImputeStrategy::Mean)),
    )
    .add_numeric(
        "features_scaled",
        vec![1, 2],        // columns 1, 2 of numeric matrix
        TransformerKind::StandardScaler(StandardScaler::new()),
    )
    .add_categorical(
        "contract_encoded",
        vec![0],           // column 0 of categorical matrix
        CategoricalTransformerKind::OneHotEncoder(
            OneHotEncoder::new().handle_unknown(HandleUnknown::Ignore),
        ),
    );

let x = ct.fit_transform(&table)?;

That's it. The output x is a single numeric Matrix — imputed, scaled, encoded — ready to feed into any model.

There's one design choice here that I genuinely love: the type system enforces correctness. add_numeric takes a TransformerKind, add_categorical takes a CategoricalTransformerKind. You can't accidentally put a OneHotEncoder (which expects strings) on a numeric column. The compiler catches it.

After years of sklearn's everything-is-a-string-column-name dynamism, this feels like putting on glasses for the first time.

Training the model

Now for the classifier. datarust v0.6 ships LogisticRegression with both binary and multiclass support:

use datarust::linear_model::{LogisticRegression, LogisticSolver};
use datarust::traits::Predictor;

let mut model = LogisticRegression::new()
    .with_solver(LogisticSolver::Svd)
    .with_max_iter(100);

model.fit(&x, &y)?;
let predictions = model.predict(&x)?;
let probabilities = model.predict_proba(&x)?;  // (n, 2) matrix

The model uses Newton-Raphson (IRLS for binary, full multinomial for multiclass) under the hood — the same algorithm sklearn uses, implemented in pure Rust with a Cholesky or SVD linear solver. No BLAS. No LAPACK.

And here's something I didn't expect to care about: it's fast. On benchmarks comparing against scikit-learn 1.6 on an M-series Mac, datarust's ColumnTransformer was 179–620× faster than sklearn's on the same data. Part of that is the flat memory layout (a single contiguous Vec<f64> instead of Python objects), and part is simply not crossing the Python/C boundary on every operation.

I'm not claiming datarust beats sklearn everywhere — sklearn's PCA (which calls LAPACK) is dramatically faster on large matrices. But for preprocessing and linear models, pure Rust holds its own.

Evaluation: don't stop at accuracy

A common mistake is to stop at accuracy. For churn — where maybe 20% of customers leave — accuracy is misleading. A model that predicts "nobody churns" is 80% accurate and completely useless.

use datarust::metrics::classification::*;

let acc = accuracy_score(&y, &predictions)?;
let prec = precision_score(&y, &predictions)?;
let rec = recall_score(&y, &predictions)?;
let f1 = f1_score(&y, &predictions)?;
let auc = roc_auc_score(&y, &probabilities.column(1))?;
let ll = log_loss(&y, &probabilities.column(1), 1e-15)?;

For this churn problem, I care most about recall — how many of the customers who will churn did I catch? A false positive (offering a discount to someone who wasn't leaving) costs a little; a false negative (losing a customer I could have saved) costs a lot.

And if this were a multiclass problem, the same metrics auto-detect the label count and switch to macro-averaging. No average='macro' parameter needed.

Cross-validate properly

Evaluating on training data is the oldest sin in machine learning. Let's do it properly:

use datarust::model_selection::StratifiedKFold;

let skf = StratifiedKFold::new()
    .with_n_splits(5)
    .with_shuffle(true)
    .with_random_state(42);

let mut fold_accuracies = Vec::new();
for (train_idx, test_idx) in skf.split(&y)? {
    let x_train = x.select_rows(&train_idx)?;
    let x_test = x.select_rows(&test_idx)?;
    let y_train: Vec<f64> = train_idx.iter().map(|&i| y[i]).collect();
    let y_test: Vec<f64> = test_idx.iter().map(|&i| y[i]).collect();

    let mut fold_model = LogisticRegression::new().with_solver(LogisticSolver::Svd);
    fold_model.fit(&x_train, &y_train)?;
    let fold_pred = fold_model.predict(&x_test)?;
    fold_accuracies.push(accuracy_score(&y_test, &fold_pred)?);
}

Stratified means each fold preserves the churn/stay ratio — critical for imbalanced data.

Save the model: JSON, not pickle

You trained the model. Now you need to deploy it. In Python, you'd joblib.dump a pickle file — a binary blob that only Python can read, tied to the exact library versions that created it.

datarust uses JSON:

use datarust::pipeline::Pipeline;

let mut pipe = Pipeline::new()
    .push("scaler", TransformerKind::StandardScaler(StandardScaler::new()))
    .with_estimator(LogisticRegression::new());

pipe.fit(&x, &y)?;

// Save to disk
datarust::serialize::save_json(&pipe, "churn_model.json")?;

// ... later, in the serving binary ...
let model: SupervisedPipeline<LogisticRegression> =
    datarust::serialize::load_json("churn_model.json")?;

// Already fitted — predict immediately
let predictions = model.predict(&new_data)?;

The saved model is human-readable JSON. You can cat it. You can diff it. You can load it in a Rust service, a CLI tool, or a WASM module — anything that can parse JSON and link the datarust crate.

Try that with a pickle file.

The deployment payoff

The entire workflow above — preprocessing, training, evaluation, cross-validation, serialization — runs in a single Rust binary with zero external dependencies by default.

$ cargo build --release
$ ls -lh target/release/churn_service
-rwxr-xr-x  2.3MB  churn_service

That 2.3-megabyte binary contains the entire ML pipeline.

You can:

Cross-compile it to any target Rust supports (Linux, macOS, Windows, WASM, ARM)
Run it in the browser via WASM — load the JSON model, call predict, return the result
Embed it in a microservice written in Go, Node, or Python via FFI
Ship it as a CLI tool that data scientists can run locally without installing Python

Compare that to a Python sklearn Docker image:

	datarust	Python + sklearn
Binary size	2.3 MB	~30 MB (venv)
Docker image	~8 MB	~900 MB
WASM support	native	pyodide (~30 MB)
External deps	0	NumPy, SciPy, joblib

What's not there yet

I'm not going to pretend this library does everything sklearn does. Here's what's missing:

No trees or ensembles. No RandomForest, no GradientBoosting. This is the biggest gap, planned for v0.7.
No SVM. No SVC with RBF kernels.
No deep learning. That's deliberate — candle and burn own that space in Rust.
PCA is slower than sklearn's on large matrices. sklearn calls LAPACK; datarust uses a pure-Rust Jacobi eigensolver.
No CSV reader built-in. You construct the Matrix yourself. (A csv feature is planned.)

If your workflow absolutely needs RandomForest or a transformer, datarust isn't there yet. But if you're doing logistic regression, linear models, clustering, or — especially — preprocessing pipelines that need to run outside Python, it's ready today.

The full feature set

Here's what v0.6 gives you, all with zero dependencies:

Preprocessing:
StandardScaler, MinMaxScaler, RobustScaler, MaxAbsScaler, Normalizer, Binarizer, KBinsDiscretizer, QuantileTransformer, PowerTransformer

Encoding:
OneHotEncoder, OrdinalEncoder, LabelEncoder, TargetEncoder, FrequencyEncoder

Imputation:
SimpleImputer (mean/median/most_frequent/constant), KnnImputer

Feature engineering:
PolynomialFeatures, VarianceThreshold, SelectKBest, PCA, TruncatedSVD

Models:
LinearRegression, Ridge, Lasso, LogisticRegression, KMeans

Metrics:
MSE, MAE, RMSE, R², accuracy, precision, recall, F1, ROC-AUC, PR-AUC, log loss, confusion matrix, Cohen's kappa, Matthews correlation, silhouette score

Utilities:
Pipeline, ColumnTransformer, KFold, StratifiedKFold, cross_val_score, train_test_split, JSON serialization

Get started

cargo add datarust --features datasets,serde

If you've ever wanted to run an sklearn-style pipeline without the Python tax, give it a try.

datarust is MIT-licensed and available on crates.io, with full documentation at genc-murat.github.io/datarust. The roadmap lives at github.com/genc-murat/datarust — if you want to contribute a DecisionTree or a CountVectorizer, now's a good time.

DEV Community: Murat Genç