DEV Community: Nishant Banginwar

K-Means Doesn't Find Your Clusters. It Finds Its Own.

Nishant Banginwar — Fri, 14 Aug 2026 03:30:14 +0000

Classic Machine Learning Through the Eyes of an SRE — Part 5

Supervised learning at least tells you when it's wrong. Unsupervised learning will happily hand you a confident answer to a question your data never contained.

K-Means is where that lesson starts, because K-Means always gives you the K clusters you asked for. Ask it for 5 clusters of your client accounts and you will receive exactly 5 clusters, with centroids and an objective value. Whether your accounts actually form 5 groups — or any groups at all — is not something the algorithm can tell you.

It answers the question "what's the best way to split this into 5 compact groups?", not "are there 5 groups here?"

Those are very different questions. Only one of them was yours.

The bet it makes

Two bets, actually.

Round blobs. Assigning each point to its nearest centroid creates a Voronoi partition — convex cells with straight boundaries. Pair that with an objective that minimizes squared distance to the center and you get a strong preference for compact, roughly isotropic groups.

If your real segments are elongated, chained, or density-based — say, a "slow-burn escalation" pattern that snakes across metrics — K-Means will slice that snake into three neat blobs and report success.

That tendency toward compact, similarly scaled groups deserves its own warning. If one segment is enormous and another is a handful of accounts, the distance-minimization objective can produce unintuitive partitions. Nothing in the output tells you that the resulting groups correspond to meaningful business segments.

You know K. The number of clusters is an input, not an output. The elbow method and silhouette scores help you argue about it, but they're heuristics, not oracles. K is your hypothesis about the world, wearing the algorithm's clothes.

SEARCH, not solve

Mechanically, K-Means iterates: place centroids, assign points, move centroids to the mean, repeat until convergence. That's Lloyd's algorithm, and it's a heuristic — the global K-Means optimization problem is NP-hard in general, so what you get is a local optimum that can depend on initialization.

Run it with different initializations, and you can get different clusters. k-means++ initialization and multiple restarts manage this, but "manage" is the honest verb.

It's worth naming what's actually being minimized: the within-cluster sum of squares, usually called inertia. And there's a catch in it that explains why the elbow method has to exist at all.

The optimal inertia never increases as K rises — at K = n, every point is its own cluster and inertia is zero. So you cannot choose K by simply minimizing the objective. The elbow exists precisely because the metric can't answer the business question, which leaves you looking for the point where additional complexity stops being worth the improvement.

That word SEARCH matters, and it's my own shorthand rather than standard terminology, for how each unsupervised algorithm arrives at an answer.

K-Means SEARCHES: iterate and hope.
DBSCAN DEFINES: declare a density rule and traverse what follows.
PCA SOLVES: an eigendecomposition/SVD gives a direct solution rather than an iterative local search.
Hierarchical BUILDS: greedy merges, all the way up.

Four strategies for finding answers without labels, and K-Means is the anchor I compare the rest against — most of what they do is best understood as a reaction to something K-Means gets wrong.

The failure that hides

With no labels, there's no accuracy metric to catch you. The failure signature isn't an error — it's a plausible-looking segmentation that quietly mismatches reality.

Concrete version from my world: cluster IT-services accounts for a QBR deck. K-Means produces "5 client segments." The deck ships. Strategy gets built on those segments.

Nobody ever asks whether K=5 actually represents meaningful structure or whether the clusters cut across every important business boundary — because the output looks like insight, and there's no ground truth to embarrass it.

In supervised land, a bad model gets caught by the test set.

In unsupervised land, the test set is a stakeholder meeting three months later.

What I'd tell my ops team

Treat K-Means output as a HYPOTHESIS generator, not a report.

Before anything downstream consumes the clusters:

Check stability — do the clusters survive resampling and re-initialization?
Check the geometry — look at silhouette scores per cluster, not just the average.
Name each cluster in business language — if a cluster can't be described in one sentence a delivery head recognizes, it may be an artifact rather than a useful segment.

And scale your features first. Like SVM, K-Means runs on distance, so unscaled features can silently decide the clustering for you.

The supervised block taught me that the threshold is a business decision. The unsupervised block starts with a harder version: sometimes the QUESTION is a business decision.

K-Means will answer whatever K you hand it. Choosing K responsibly is your job, not its.

Production takeaway

Treat clusters as a hypothesis, never a report.

Before anything downstream consumes them: test stability across resamples and re-initialization, read silhouette scores per cluster rather than only the average, and name each cluster in one sentence a delivery head would recognize.

If it can't be named, it's an artifact.

Common interview mistake

Presenting K as something the algorithm discovered. K is your input and your hypothesis; the elbow and silhouette are arguments, not oracles. Worth knowing why: the optimal inertia never increases as K rises, so the objective itself can never choose K for you.

Second, describing Lloyd's algorithm as if it finds the globally optimal clustering. The K-Means optimization problem is NP-hard in general, so Lloyd's converges to a local optimum depending on initialization. Multiple initializations and k-means++ reduce that sensitivity without changing the underlying problem.

Third, and easiest to miss: skipping feature scaling on a distance-based algorithm.

Where I'd use this in a real production system

Account or customer segmentation for coverage models · workload profiles for capacity planning · a first-pass structure check before designing a supervised label.

Series: what ML algorithms bet about your world, production/ops lens. Previous: SVM. Next: DBSCAN — the first algorithm with no loss function at all.

The Kernel Trick Is the Oldest Move in Engineering

Nishant Banginwar — Tue, 11 Aug 2026 03:22:24 +0000

Classic Machine Learning Through the Eyes of an SRE — Part 4

When a computation is too hard, don't compute harder. Change coordinates until it becomes easy.

Every engineer has made this move. Pick the right data structure and the impossible query goes O(1). Re-index the table and the report that took an hour takes a second. Move the problem into a space where it's trivial, solve it there, come back.

That's the kernel trick. SVM's famous move isn't building a curvy model — it's finding a FLAT cut in a transformed space, which corresponds to a curved boundary back in your original features. The separator stays linear in the transformed space. The space did the work.

And here's the part that makes it a trick rather than just a projection: the data never actually goes up there. The optimization only ever needs inner products between pairs of points, and a kernel function computes what that inner product would be in the high-dimensional space, directly from the original coordinates. You get the geometry of a space you never built. Some kernels correspond to infinitely many dimensions, which would otherwise be an awkward amount of memory to allocate.

The bet it makes

SVM bets that the most ROBUST boundary is the one with the widest margin — maximum distance from the nearest points on each side. And here's the part that rewired me: only those nearest points matter. They're the support vectors. The non-support-vector points don't directly determine the final boundary at all.

Compare that to the forest, which averages over EVERYTHING. SVM is the opposite extreme: the borderline cases that become support vectors define the decision boundary. In delivery-risk terms — the projects that teach you where the line is aren't the disasters or the easy wins. They're the borderline ones that barely breached and barely survived. SVM formalizes that.

Everything old returns

After trees and forests threw away gradient descent, SVM brings some of the regression toolkit back: an explicit loss, convex optimization, and iterative optimization. The families really do have different mechanics, and crossing back is noticeable.

Two things I got wrong on the first pass, both the same trap. I used "convex" as if it were a tunable knob. It isn't. It's a property of the problem, and the knobs are C and gamma. I also named gradient descent as the "loss," when it's the optimizer. The three architecture slots (hypothesis, loss, optimization) are separate, and letting them blur means you don't actually know which part you're tuning when things go wrong.

Also: regularization isn't an afterthought in SVM. The maximum-margin objective is directly tied to controlling the weight norm — the same role L2 regularization plays elsewhere. The safety and the simplicity come from the same place.

The correctness trap

Feature scaling for SVM is not hygiene. It's correctness.

Margins are measured in distance. If ticket-count ranges from 0–10,000 and CSAT from 1–5, ticket-count can dominate the distance calculation entirely, and the margin stops meaning what you think it means. Unscaled features don't degrade an SVM — they quietly change the question it's answering.

Ops translation: this is a units bug. Like averaging milliseconds with seconds in a latency dashboard — the chart still renders, the number is still a number, and it's wrong. The system fails silently and the output looks fine. Those are the failures that survive review.

Where the bet fails

The margin bet assumes the borderline cases are TRUSTWORTHY. If your labels are noisy exactly at the boundary — and in ops data they usually are, because borderline breaches are exactly where "was that a breach?" gets argued — then the points defining your model are your least reliable ones. C controls how much you let noisy borderline points bend the line; gamma controls how local the influence of individual training points becomes in an RBF kernel. Tune them together or not at all.

And the kernel choice is a hypothesis about your domain's shape, not a preprocessing detail. Picking RBF because "it usually works" is betting your world is locally clumpy without asking.

Two things it won't give you

A probability. An SVM returns a decision score related to distance from the boundary, not a calibrated probability. If you want 0.83, you need calibration — traditionally Platt scaling, which fits a logistic model to the SVM scores using additional cross-validation. Which means the threshold conversation from the first article in this series doesn't go away here, it just gets an extra step in front of it. Anyone treating the raw decision score as a probability is reading a number that doesn't mean what they think.

Scale. For classical kernel SVMs, training can become roughly quadratic or worse with the number of samples, depending on the solver, because the optimization works over pairs of points. Fine for thousands of rows. Impractical well before you reach a million. That's the honest reason forests and gradient boosting took over large tabular problems, and it has nothing to do with which one is smarter.

What the supervised set finally gave me

SVM completed the supervised set for me, and with it the real prize: a map of four different ways to handle non-linearity.

Polynomials bend the line. Trees carve local boxes. Forests average those boxes. SVM changes the space so a flat boundary becomes useful.

I don't want to leave this series knowing four algorithms. I want to leave knowing what question to ask when I meet a fifth:

What bet about the shape of my data am I willing to make?

Production takeaway

Feature scaling is a correctness requirement here, not hygiene — unscaled features silently change the question the model is answering, and the output still looks fine. Treat kernel choice as a documented hypothesis about your domain's shape, tune C and gamma together, and if anything downstream needs a probability, calibrate explicitly rather than passing the decision score along as though it were one.

Common interview mistake

Saying the kernel trick projects your data into a higher dimension. It doesn't — that's the entire point of the word "trick." It computes what the inner products would be in that space without ever building it.

Two more worth knowing: calling gradient descent "the loss" (it's the optimizer; the loss is hinge plus a weight-norm penalty), and treating convexity as a tunable knob. And don't describe regularization as bolted on — the maximum-margin objective is already controlling the weight norm.

Where I'd use this in a real production system

Smaller, high-dimensional problems: text and log classification, one-class novelty detection on infra metrics, cases where the borderline examples are the whole story and you have clean labels.

Series: what ML algorithms bet about your world, production/ops lens. Previous: random forest. Next: K-Means — and the uncomfortable discovery that it finds ITS clusters, not yours.

Random Forest Is Horizontal Scaling for Predictions

Nishant Banginwar — Fri, 07 Aug 2026 03:42:00 +0000

Classic Machine Learning Through the Eyes of an SRE — Part 3

The random forest is the first ML algorithm that made me feel at home. Not because of the math — because it's an SRE idea wearing a stats costume.

Many independent workers. No single point of failure. Majority vote. If one worker goes weird, the fleet absorbs it. We've been building systems this way for decades; the forest just applies it to prediction.

The problem it exists to fix

Last article: a single decision tree is readable but unstable — small data change, whole tree flips, explanation rewrites itself. That instability is variance, and it's exactly what scared me about trusting one tree in production.

The forest's move: grow hundreds of trees, each on a random resample of the data, and — this is the part that matters — force each split to choose from only a random subset of features.

That second randomization is the whole difference between a random forest and plain bagging. Bagging alone gives you many trees on resampled data, but if one feature is strongly predictive, every tree grabs it first and they all end up looking alike. Starving each split of features is what makes the trees genuinely different from each other. The randomness isn't sloppiness. It's manufactured disagreement.

The instability doesn't get fixed. It gets CANCELLED. Each tree is still jumpy, but they're jumpy in different directions, and the average is calm.

What surprised me

No new loss function. Each tree still minimizes impurity exactly like a lone tree. The forest adds zero new objectives. The entire gain is a bias-variance bargain: variance drops hard, bias barely moves. You give up readability and get back trustworthiness.

Embarrassingly parallel. Trees are independent, so training scales horizontally — throw cores at it. Boosting, its sequential cousin, is the opposite: each model depends on the last. Map-reduce versus a pipeline.

The smoothness illusion. A forest's decision boundary looks smooth, almost like regression's curve. I initially logged that as "the forest resembles regression." It only looks like regression. Regression starts smooth. A forest averages thousands of tiny box-shaped decisions until the edges blur into something that appears smooth. The cleanest transfers into the forest come from the single tree, not from regression. I kept reaching for the wrong parent.

This is also why random forests became the default baseline for tabular business data. Before reaching for deep learning, many teams still ask one question: can a forest already solve this?

The bet it makes

Same as the tree — the world is chunky boxes — plus one more: your errors are DIVERSE. Averaging only cancels mistakes that point in different directions. If every tree shares the same blind spot, the vote is unanimous and unanimously wrong.

That's the production failure worth internalizing: a forest fails quietly and confidently when its diversity is fake. Bootstrap resamples from a biased dataset are all biased the same way. A hundred voters reading the same newspaper is one voter.

Ops translation: redundancy without diversity is not redundancy. Three replicas in the same rack. Five monitors on the same network path. We've all been burned by correlated failure — the forest can be too.

What I'd tell my ops team

For delivery-risk scoring in IT services, the forest is usually the grown-up default: tabular data, mixed feature types, non-linear interactions, and you care more about being right than explaining every path.

There's also a free instrument most people ignore. Because every tree trains on a bootstrap sample — rows drawn with replacement — roughly one-third of the data never gets selected for any given tree. Those untouched rows become its out-of-bag set. Score each row using only the trees that never saw it and you get a validation estimate without holding anything back. Free monitoring, built into the training process. I don't know another algorithm that hands you that.

The mental shift is to operate a forest like a fleet. Individual trees will be wrong in different ways, and that's fine — your job is to watch the behaviour of the fleet, not any single member. It also means you lose the thing a single tree gave you for free. When someone asks "why did it flag this project," the honest first answer is "347 of 500 trees voted yes," and that's rarely what they wanted to hear.

The lesson I keep coming back to: strategy transfers everywhere, mechanics transfer WITHIN a family (tree → forest) and break ACROSS families (regression → tree). Every algorithm makes a different bet about the world. Learn the bet first, and the equations start making sense.

Production takeaway

After every retrain, compare which features the forest is leaning on. A feature that suddenly dominates is your instability alarm — with no single tree to read, it's the only one you get.

Use out-of-bag error as free validation. Every row can be scored by the trees that never saw it, so you get a health signal without holding data back.

Plan for the explainability gap before someone asks. The forest doesn't expose a single decision path, so answering "why" needs another tool — SHAP or LIME — or a single tree running alongside purely for narration. Decide which before it's a meeting question.

Common interview mistake

First, assuming more trees always means better generalization. Averaging only cancels errors that point in DIFFERENT directions, and with a leaked feature or a biased sample all trees are wrong the same way. Redundancy without diversity isn't redundancy.

Second, and this one gets asked constantly: describing a random forest as just bagged decision trees. Bagging randomizes the ROWS. A random forest also randomizes the COLUMNS available at each split, and that second randomization is what stops every tree from latching onto the same dominant feature.

Where I'd use this in a real production system

Delivery-risk scoring on tabular data · churn prediction · fraud risk · anywhere accuracy matters more than a readable decision path.

Series: what ML algorithms bet about your world, production/ops lens. Previous: decision trees. Next: SVM — and the oldest engineering trick in the book.

Decision Trees Aren't Trained. They're Grown.

Nishant Banginwar — Tue, 04 Aug 2026 03:16:57 +0000

Classic Machine Learning Through the Eyes of an SRE — Part 2

The second algorithm I studied broke everything I'd just learned from the first.

Logistic regression taught me that training means gradient descent: guess, measure error, adjust the weights, repeat until convergence. So when I opened decision trees, I went looking for the optimizer.

There wasn't one.

A decision tree isn't optimized the way I expected. It's grown.

At each step it finds the locally best split, commits to it, and recursively repeats the process. No backtracking. No second chances. There is optimization happening — each split minimizes impurity — but only locally, one step at a time. Finding the globally optimal tree is NP-hard, so the algorithm doesn't even try.

That felt surprisingly familiar.

In incident response or capacity planning, we rarely know the perfect answer. We make the best decision with the information we have, knowing a different first choice might have led somewhere else. Decision trees simply turn that idea into an algorithm.

The bet a tree makes

Every machine learning algorithm makes a different bet about the world.

Logistic regression assumes relationships are smooth. Risk gradually increases as signals change.

Decision trees make the opposite assumption. They assume the world is made of boxes.

A project isn't slightly riskier because velocity drops. It's risky when several conditions happen together: a fixed-price contract, a new account manager, and a month-end delivery. Inside that box, projects fail. Outside it, they're usually fine.

This is exactly how many operational systems work. Severity matrices, routing rules, escalation policies, approval workflows — they're all collections of decision boxes.

That's why trees immediately felt intuitive to me.

The hidden cost of flexibility

Trees make very few assumptions about the data. That sounds like an advantage.

The price is instability.

Change a small part of the training data and the first split can change. Since every later split depends on that first decision, the entire tree can be completely different after retraining.

Same data. Different explanation.

I actually made this mistake while learning. My first notes said that because trees make fewer assumptions, they must be more stable.

Exactly backwards.

Fewer assumptions mean more freedom to fit whatever the sample contains. More freedom means higher variance. I had confused flexibility with reliability.

What this looks like in production

Imagine retraining a churn model every month.

Last month the first split said ticket volume is the biggest predictor. This month it says response time is.

The model might perform equally well. But if people treat the tree as an explanation rather than just a prediction, you've just changed the organization's understanding of reality.

That's why I would version decision trees the same way we version configuration.

Don't just monitor accuracy. Diff the structure. If the explanation changes dramatically between retrains, someone should know why.

Strategy transfers. Mechanics don't.

This algorithm taught me something more useful than decision trees.

Some things transferred directly from logistic regression: frame the business problem first, understand the cost of false positives and false negatives, watch for data leakage, and treat turning predictions into actions as a business decision. Those are strategies.

Other things didn't transfer at all — gradient descent, differentiable loss, model coefficients. Those are mechanics.

Since then, every time I learn a new algorithm, my first question is: what assumptions is this algorithm making about the world?

The answer usually predicts how it learns — and how it eventually fails.

What I'd tell my SRE team

Use a single decision tree when the model is the runbook.

If someone needs to explain every decision to a customer, auditor, or compliance reviewer, it's hard to beat a tree.

Just remember that the explanation itself is unstable. Version it. Diff it. Treat changes like configuration changes.

Production takeaway

A decision tree is the model that is the runbook. Use it when humans need to read, audit, and defend every decision path. Monitor the structure — not just the accuracy.

Common interview mistake

Two answers I now know are wrong: "decision trees are trained using gradient descent," and "decision trees are more stable because they make fewer assumptions." Neither is true. Trees are grown greedily, and fewer assumptions usually mean higher variance, not greater stability.

Where I'd use it

Ticket triage · escalation routing · customer churn explanations · compliance workflows · any system where the business needs to understand why the model made a decision.

Part 2 of "Classic Machine Learning Through the Eyes of an SRE" — a decade in production, now moving into AI platform engineering, documenting it in public. Next: random forests, the first algorithm that felt like distributed systems disguised as machine learning.

Logistic Regression Doesn't Make Decisions—Your Business Does

Nishant Banginwar — Thu, 30 Jul 2026 03:25:31 +0000

Classic Machine Learning Through the Eyes of an SRE — Part 1

For years, every Monday morning started the same way.

Delivery managers would sit down, review a portfolio of projects, and instinctively rank which ones needed attention first.

Nobody called it machine learning.

They looked at dozens of signals: velocity slowing down, customer escalations increasing, risks piling up, and that usually proactive project manager suddenly going quiet.

Somehow all of those signals became a single mental estimate: "This project feels risky."

Looking back, they were doing something surprisingly similar to logistic regression. Not mathematically. Conceptually. They were combining signals, estimating the probability of something going wrong, and then deciding where to act.

The important part is that they made the decision—not the probability.

The bet the algorithm makes

Obviously, logistic regression is far more rigorous than human intuition. But the mental model is surprisingly similar: combine signals, estimate a probability, then decide what to do next.

Underneath, it's the same weighted-sum core as linear regression, squashed through a sigmoid so the output lands between 0 and 1. Its bet about your world is that risk rises smoothly with the signals — no cliffs, no chunky exceptions. For delivery risk, that's mostly right: a project with slightly worse velocity is slightly riskier.

One mechanical detail worth knowing: unlike linear regression, there's no closed-form solution here. You iterate with gradient descent until it converges. This is the first algorithm where "training" literally means guess, measure, adjust, repeat.

The threshold was never in the model

A logistic regression model might tell you there's a 73% chance of an SLA breach.

It never tells you to escalate.

That decision belongs to the business. The threshold isn't part of the algorithm. It's a product decision. An operational decision. Sometimes even a financial decision.

This is one of the most common misunderstandings I see. Teams accept the default 0.5 cutoff as if it were a property of mathematics. It isn't. If a missed breach costs ten times what a false alarm costs, 0.5 is the wrong line — and no amount of model tuning will tell you that.

Log loss and the cost of confidence

Log loss also changed how I think about confidence.

It doesn't just penalize mistakes. It penalizes being confident and wrong.

That reminded me of production systems. A noisy alert is frustrating. A dashboard showing green while customers are down? That's the kind of mistake nobody forgets.

Whether it's monitoring or machine learning, confidence attached to the wrong answer is usually more dangerous than uncertainty.

Two mistakes I made while learning this

One mistake I caught myself making was explaining logistic regression using mean squared error. That's linear regression. Logistic regression uses log loss.

I also used to say a train/test split prevents overfitting. It doesn't. It helps detect overfitting. Regularization is what helps reduce it.

Tiny wording differences. Completely different mental models.

When the bet is simply wrong

If the model is consistently wrong for one segment of your data, retraining alone often won't help — that's the smooth-line assumption failing against a boxy reality. If the relationship is inherently nonlinear, you may need different features, or a different class of model altogether.

That's a design decision, not a tuning problem.

The SRE lens

After spending a decade in SRE and DevOps, I've started looking at machine learning models the same way I look at monitoring systems.

The prediction isn't the product. The operational decision is.

A probability without a response plan is just another number on a dashboard.

The model computes. People—and increasingly AI systems—still have to decide what happens next.

That's where the real engineering begins.

Production takeaway

Probabilities don't create business value. Operational decisions do. Define what happens at high scores, at low scores, and in the uncertain middle before you ship the model.

Common interview mistake

Confusing probability with classification, or assuming a 0.5 threshold is always appropriate.

Where I'd use this in a real production system

SLA breach prediction · incident escalation risk · ticket priority prediction · customer churn probability · fraud risk scoring.

Part 1 of "Classic Machine Learning Through the Eyes of an SRE" — a decade in production, now moving into AI platform engineering, documenting it in public. Next: decision trees, the first algorithm that isn't trained at all.

The Algorithm Is the Last Decision You Make in Machine Learning

Nishant Banginwar — Mon, 27 Jul 2026 07:19:09 +0000

Every delivery manager I've worked with runs a prediction model on Monday morning. It's called dread.

Open the portfolio, scan 30 projects, and your gut sorts them: these five worry me, those twenty are fine, and that one — that one is going to blow up this month. No math. But it's a real model: inputs (velocity, escalations, that PM who's gone quiet), weights (learned from years of pain), and an output (who gets your attention today).

I spent 10 years in SRE and DevOps building that kind of gut. Now I'm transitioning into AI engineering, and the first thing I forced myself to do was NOT train a model. I wrote thinking documents instead. Problem framing before code. It felt like a waste of build time.

It turned out to be the point.

The hard part isn't the math.

When I started, I assumed the difficulty in ML was the mathematics. Gradients, loss functions, the Greek letters. It isn't. Most of that is decided for you once the problem is framed properly.

The hard part is everything upstream of the algorithm: what question are you actually asking, what does a wrong answer cost, and what happens when the model speaks? Get those wrong and the most elegant model in the world predicts the wrong thing, accurately.

Here's what that looks like on a real problem from my domain.

Walk-through: Predicting SLA breaches in IT services

The task sounds simple: Predict which delivery projects will breach SLA this month. Grab data, train a classifier, done. Except four questions decide everything before a single model runs:

What exactly is the event? A breach reported this month? Detected this month? Root-caused to this month? Each definition changes your training data, your labels, and what the system actually predicts. Pick carelessly and you'll ship a model that answers a question nobody asked.
Prediction to what action? If a risky project just triggers an email to a PM, false alarms are cheap. If it pulls two engineers off another client's project, every false alarm has a victim. Same model, same score — completely different system.
What does confident-wrong cost? A 0.9 breach-risk on a healthy project wastes money and trust. A 0.1 on a project that then breaches burns a client relationship. Those costs are not symmetric, and no algorithm knows that. Only you know what each failure direction costs in money and reputation.
Where is the dangerous zone? For us it's 0.5 — the model shrugging. A 0.5 means "I have no idea," and at that point the Monday-morning dread model in a good manager's head beats the classifier. What probability triggers what action is a business decision. It was never the model's to make.

Notice what's missing from all four: Any mention of logistic regression, random forests, or neural networks. The algorithm question ("Is this classification? regression? unsupervised?") ANSWERS ITSELF once these four are settled. That's why it's the last decision, not the first.

The framework I stole from ops

None of this was new to me — I just didn't know it applied to ML. In SRE we never deploy a monitoring alert without asking: What exactly fires it, who gets paged, what do they do, and what does a false page cost at 3 AM? An alert without those answers is noise with a pager attached.

A prediction without those answers is the same thing. A number with nowhere to go.

So before I touch any model now, I write four things down:

The decision this prediction feeds, in one sentence, including who acts on it.
The price of both failure directions. If I can't price them, I'm not ready to set a threshold.
The exact event definition, with its time window.
Only then: which family of algorithm fits the shape of this question.

Item 4 takes ten minutes. Items 1–3 take days. And they're the reason item 4 becomes easy.

The algorithm is the last decision you make.

This is the first post in a series — I'm working through the classic ML algorithms one at a time (logistic regression next), each through this same lens: what the algorithm bets about your world, and what that means in production. I'm a decade-long SRE/DevOps engineer (currently on Google's Piper infrastructure via Movate) moving into AI platform engineering, documenting it in public. If you've watched an ML project fail for non-ML reasons, tell me the story in the comments.