Randomness isn't the opposite of order. It's just order written in a language we haven't learned yet.
As software engineers, we spend our lives trying to make systems deterministic.
We write unit tests.
We build idempotent APIs.
We retry failed requests.
We eliminate race conditions.
Then production happens.
Suddenly requests arrive in bursts, servers fail at inconvenient times, users behave unpredictably, and latency charts look like modern art.
That's where probability stops being a university subject and starts becoming an engineering tool.
Probability isn't about predicting exactly what will happen.
It's about understanding what usually happens, and designing systems that survive everything else.
A Probability Generating Function Is Basically a ZIP File
One of the coolest ideas in probability is something called a Probability Generating Function (PGF).
The name sounds intimidating, but the idea is surprisingly elegant.
Imagine taking an entire probability distribution and compressing it into a single function.
That's exactly what a PGF does.
Instead of storing files, it stores probabilities.
Mathematically, it looks like this:
G(s) = E[sˣ]
The mysterious variable s isn't an input you'll ever substitute meaningful values into during modeling.
Think of it like a placeholder that keeps every possible outcome organized.
If you've ever worked with JSON, you can think of a PGF as a serialized representation of a random variable.
The really clever part comes later.
Instead of unpacking the whole distribution, you can query it.
Need the expected value?
Take one derivative.
Need the variance?
Take another.
It's remarkably similar to querying metadata instead of scanning an entire database table.
Randomness Usually Comes in Two Flavors
Although real-world systems feel chaotic, most engineering problems fall into two categories.
1. Fixed Experiments (Binomial)
Suppose your backend sends 100 push notifications.
Each notification either succeeds or fails.
Every request is independent.
Nothing fancy.
This is exactly what the Binomial Distribution models.
Some examples include:
- API requests succeeding
- Login attempts
- Feature flag rollouts
- CI tests passing
- Email delivery success
Whenever you're counting how many successes occur in a fixed number of attempts, you're probably dealing with a Binomial process.
2. Events Over Time (Poisson)
Now imagine you're monitoring production traffic.
You don't know exactly when requests will arrive.
You only know the average rate.
That's the world of the Poisson Distribution.
Examples include:
- HTTP requests per second
- Kafka messages
- Database writes
- Customer arrivals
- Server interrupts
One fascinating property of the Poisson model is that its average and variance are identical.
In practical terms:
As traffic grows, uncertainty grows with it.
A system handling 10 requests per second behaves very differently from one processing 10,000—even if both are perfectly healthy.
Why Every Performance Dashboard Eventually Looks Like a Bell Curve
If you've ever opened Grafana or Datadog, you've probably seen something that resembles a bell curve.
That's no coincidence.
Many independent random effects naturally combine into what's called the Normal Distribution.
Latency.
CPU temperature.
Manufacturing tolerances.
Sensor measurements.
Human height.
They all tend to cluster around an average.
Engineers love this because it makes systems predictable.
A useful rule of thumb is the 68–95–99.7 Rule.
| Range | Percentage of Values |
|---|---|
| Within 1 standard deviation | 68% |
| Within 2 standard deviations | 95% |
| Within 3 standard deviations | 99.7% |
Instead of obsessing over individual measurements, we can understand the entire system just by knowing its average and spread.
Small Datasets Lie More Than Big Ones
Imagine benchmarking a new caching algorithm.
You only have eight measurements.
Can you confidently say it's faster?
Probably not.
Small datasets exaggerate randomness.
That's why statisticians use the Student's t-distribution instead of the normal distribution.
I like thinking of it as the skeptical bell curve.
It assumes your sample might be misleading.
Its wider tails are basically mathematics saying:
"You probably don't have enough evidence yet."
Ironically, one of the hardest engineering skills isn't collecting data.
It's knowing when you don't have enough.
Uniform Randomness Isn't as Simple as It Looks
Sometimes every outcome is equally likely.
That's the Uniform Distribution.
A classic example is a load balancer spreading requests evenly across workers.
Suppose every build in your CI pipeline takes somewhere between 30 seconds and 4 minutes.
Initially, every completion time is equally plausible.
Now imagine the build has already been running for three minutes.
The original probability model no longer applies.
You've learned something new.
Your probability space has changed.
Good probabilistic models evolve as new information arrives.
Good software systems should too.
Treat Your Statistical Models Like Production Code
Here's something I wish more engineers talked about.
Just because a mathematical model looks reasonable doesn't mean reality agrees.
That's why we have Goodness of Fit tests.
Think of them as integration tests for probability distributions.
Different tests answer different questions.
Chi-Squared Test
Compares expected observations with actual observations.
Great when your data naturally falls into buckets.
Kolmogorov–Smirnov Test
Instead of comparing buckets, it compares entire cumulative distributions.
Useful when working with continuous measurements.
Anderson–Darling Test
Sometimes the average isn't where systems fail.
Sometimes it's the outliers.
Anderson–Darling pays much more attention to the tails of the distribution.
If you're building fraud detection, distributed systems, recommendation engines, or financial software, those rare edge cases are often the ones that matter most.
Engineering Is Really About Managing Uncertainty
We often describe software engineering as building reliable systems.
That's only half true.
We're really building reliable systems on top of unreliable environments.
Networks fail.
Users behave unpredictably.
Hardware ages.
Traffic spikes.
Probability gives us the vocabulary to reason about all of it.
Probability Generating Functions compress uncertainty into something we can analyze.
Probability distributions reveal recurring patterns hiding beneath noisy data.
Goodness of Fit tests keep us honest when reality refuses to match our assumptions.
Just like we wouldn't merge untested code into main, we shouldn't trust statistical models that haven't survived contact with real-world data.
Engineering isn't about eliminating randomness.
It's about understanding it well enough to build systems that thrive in spite of it.
If you enjoyed this article, follow me for more posts where mathematics meets software engineering, AI, distributed systems, and mobile development.
Top comments (0)