DataLens: The Data Tool That Refused to pip install Anything

Saurav Gopinath ek — Tue, 08 Sep 2026 03:22:48 +0000

What DataLens actually does

Before the stdlib war stories, here's the tool itself. DataLens is a data quality engine — you point it at a messy CSV/JSONL file and it:

Streams the file instead of loading it all into memory, so it handles files bigger than your RAM
Infers types per column — detects 15 types including EMAIL, UUID, IP_ADDRESS, DATE, not just "string vs number"
Flags problems: missing values, type mismatches, outliers (IQR + Z-score), duplicates, format violations
Runs anomaly detection through the autoencoder we hand-built (explained below) to catch multi-column issues a simple rule can't
Auto-cleans the data with a confidence score attached to each fix, so nothing gets silently changed
Lets you run SQL directly on the dataset via an in-memory sqlite3 engine — no separate database setup
Compiles to one file — datalens_single.py — that runs anywhere Python 3.14 runs, no pip install needed

Example usage:

python datalens.py analyze data.csv
python datalens.py clean data.csv --apply
python datalens.py query data.csv "SELECT * FROM data WHERE age > 30"

That's the "what." Now the "how it almost broke us": the anomaly detector needed a neural net, and our rulebook said no third-party packages. No NumPy. No pandas. No scikit-learn. Just Python 3.14's standard library.

Our first reaction was denial. You cannot build an ANN without a matrix library — everyone knows that. numpy.dot() is basically load-bearing infrastructure for machine learning in Python. We spent an embarrassing amount of time trying to convince ourselves some obscure math submodule secretly did vectorized linear algebra. It doesn't. There is no shortcut. If you want matrix multiplication in pure stdlib Python, you write nested for loops and you like it.

What we normally would have installed

In any other project, this is a two-second decision: pip install numpy, import it, move on with your life. Matrix ops, broadcasting, vectorized activation functions — all free. Neither of us had ever really had to think about how A @ B works under the hood, because neither of us had ever had to write it ourselves.

What it actually took to replace it

Quick context if you're not deep in ML: an autoencoder is a neural net that learns to compress data down and rebuild it back — if it can't rebuild something well, that "something" is probably an anomaly. Backprop is just how the network learns from its mistakes, by working backward from the error and adjusting itself.

An autoencoder needs: matrix multiplication, transpose, element-wise activation functions (sigmoid, ReLU), and gradient computation for backprop. Without NumPy, every one of those is a hand-rolled function operating on nested Python lists. Matrix multiply becomes three nested loops instead of one line. A forward pass that would be a single .dot() call turns into a small file of helper functions: matmul(), transpose(), add_bias(), sigmoid(), sigmoid_derivative(). We split it — one of us built the forward pass and activation functions, the other took backprop and the training loop — and then spent a good while debugging the seam where the two met.

The genuinely hard part wasn't the math — it was performance. Pure Python loops over lists of lists are slow, and profiling a dataset with a few thousand rows through even a small autoencoder made that obvious fast. We ended up leaning hard on Python's array module instead of plain lists for the weight matrices. In plain terms: a normal Python list stores each number as a separate object scattered in memory, while array packs numbers back-to-back like a real numeric array in C — less overhead, faster access. It's the closest thing the standard library has to a "no dependency" NumPy array, and neither of us knew it existed until this project forced us to find it.

The stdlib corner nobody told us about

The other surprise was sqlite3. We'd always thought of it as "the toy database module," something you use for a quick local cache, not real analytics. Turns out it's a fully capable SQL engine sitting in the standard library, in-memory mode and all — sqlite3.connect(":memory:") gives you a real query engine with joins, aggregates, and indexes, with zero setup. We ended up building DataLens's entire SQL analytics layer on top of it instead of hand-rolling a query parser, which honestly felt like cheating in the best way.

The thing that turned out harder than the docs made it look

The docs for array make it sound like a drop-in replacement for lists with a type constraint. What they don't emphasize is that array only holds primitive numeric types — no nested structures — so representing a 2D matrix means either flattening to 1D and doing manual index math (row * width + col) everywhere, or nesting arrays inside a list and losing some of the contiguous-memory benefit you wanted in the first place. We went with flattening, and every single matrix operation had to be rewritten around that indexing scheme. It works, and it's fast, but it made debugging genuinely painful — a transpose bug three layers into backprop just looks like "the model isn't learning," not "your index math is wrong," and it took the two of us comparing notes line by line to actually find it.

Why bother

The constraint felt arbitrary at first — obviously NumPy would make all of this trivial and safe. But writing the matrix ops by hand meant we actually understood, for the first time, what a forward pass and backward pass are doing numerically, instead of trusting a black box. And the deployment story is real: DataLens compiles down to one portable .py file that runs on any machine with Python 3.14, no pip install, no dependency resolution, no supply-chain risk. For a data quality tool meant to run in locked-down or air-gapped environments, that's not a nice-to-have — it's the whole point.

Turns out "zero dependency" isn't a limitation you work around. It's a forcing function that makes you actually learn the thing you'd normally outsource — and a decent excuse to argue with your teammate about whose indexing bug it was.

Repo: github.com/Akshith1413/DataLens

If you've ever hand-rolled something you normally pip install, drop it in the comments — curious what stdlib corners other people have found.

DEV Community: Saurav Gopinath ek

DataLens: The Data Tool That Refused to pip install Anything