Why Data Structures Still Matter When You're Doing Machine Learning

It's easy to assume DSA is "just for interviews" once you're writing model.fit() and letting a library handle the rest. But the moment your ML system has to run on real, large-scale, latency-sensitive data — not a clean Kaggle CSV — data structures and complexity analysis stop being theoretical and start being the difference between a system that scales and one that falls over.
A few places where this showed up directly in projects I've built.

1. Preprocessing at scale is a data structures problem Feature engineering on large datasets is full of classic DSA patterns hiding in plain sight:

Deduplication of scraped or user-submitted records → hash sets, O(n) instead of O(n²) pairwise comparison
Sliding-window features (rolling averages, time-based aggregations) → deques for O(1) amortized window updates instead of recomputing from scratch
Frequency-based features (word counts, category counts for encoding) → hash maps, but watch out for high-cardinality categorical features blowing up memory

pythonfrom collections import deque

def rolling_average(stream, window_size):
window = deque(maxlen=window_size)
for value in stream:
window.append(value)
yield sum(window) / len(window)
A naive rolling average recomputed from a slice on every step is O(n·k). The deque version is O(n) total. On a dataset with millions of rows, that's the difference between a preprocessing job that finishes in minutes versus one that doesn't finish at all.

1. Graphs show up more than you'd expect
   Any time your data has relationships — user-item interactions, dependency chains, entity co-occurrence — you're working with a graph whether you call it that or not. Recommendation systems, fraud-detection pipelines, and even NLP dependency parsing all lean on graph traversal and shortest-path concepts:
   pythondef bfs_connected_component(graph, start):
   visited = {start}
   queue = deque([start])
   component = []

   while queue:
   node = queue.popleft()
   component.append(node)
   for neighbor in graph[node]:
   if neighbor not in visited:
   visited.add(neighbor)
   queue.append(neighbor)

   return component
   Understanding why BFS is O(V + E) — not just being able to write it — matters when you're deciding whether a graph-based feature is even feasible to compute on a graph with millions of edges.

2. Complexity analysis decides your architecture, not just your code
   This is the part that's easy to underrate. Knowing Big-O isn't just about passing a coding interview — it directly informs decisions like:

Whether to precompute and cache a similarity matrix (O(n²) space) or compute similarities on demand
Whether a k-nearest-neighbors lookup needs an approximate structure (KD-tree, ball tree, or an ANN library) once your dataset crosses a certain size, because brute-force O(n) search per query stops being viable
Whether a batch job's O(n log n) sort-and-merge step will actually finish inside your maintenance window as data grows

None of these are "algorithm interview" questions. They're architecture decisions that determine whether your ML pipeline runs in 10 minutes or 10 hours next quarter when the dataset doubles.

1. A concrete example: efficient top-k without sorting everything A common ML pipeline task — get the top-k highest-scoring items from a large candidate set — gets solved wrong surprisingly often: pythonimport heapq

def top_k(candidates, k):
# candidates: list of (score, item) tuples
return heapq.nlargest(k, candidates, key=lambda x: x[0])
Sorting the entire candidate list is O(n log n). A heap-based nlargest is O(n log k) — a meaningful difference when n is in the millions and k is small, which is exactly the shape of most recommendation and ranking problems.
The takeaway
Libraries like scikit-learn and TensorFlow abstract away a lot, but they don't abstract away the cost of how you get data into and out of them. The preprocessing, feature engineering, and serving layers around a model are still ordinary software engineering — and that's where DSA fundamentals quietly decide whether your ML system is production-ready or just a notebook that happens to work on a laptop.

I write about backend systems, applied ML, and the engineering fundamentals that hold it all together.