I read Rich Sutton’s The Bitter Lesson the other night, and one idea kept bothering me in a good way.
It made me think about a LeetCode problem I’ve probably solved fifty times: two numbers in an array that add up to a target.
The obvious solution is a loop inside another loop, checking every possible pair.
It’s slow. It’s not particularly clever. But it works.
Then you learn the usual better approach: use a hashmap, reduce the complexity, and move on.
That comparison kept coming back to me while thinking about Sutton’s argument.
A lot of AI history can be viewed through a similar lens. We spent years trying to build the “smart solution” by encoding what we knew about the problem directly into the system.
For chess, that meant increasingly sophisticated ways of representing positions, evaluating moves, and incorporating human chess knowledge. For vision, it meant manually designing features and trying to capture the structure we believed mattered.
These approaches weren't stupid. Quite the opposite—they were often extremely clever.
But there was another direction that kept becoming more powerful: give the system a general mechanism for search or learning, and then scale the resources behind it.
Deep Blue is a good example. Its strength didn't come from reproducing the way a grandmaster thinks. A huge part of its advantage came from being able to search an enormous number of positions.
AlphaZero took a different route. Rather than being built around human chess knowledge or trained on a database of human games, it learned through self-play and search.
That distinction is what made the nested-loop analogy interesting to me.
In DSA, brute force is usually something you escape from. If you have the same problem and roughly the same input size, the nested loop doesn't become a fundamentally different algorithm just because computers get faster.
AI operates under a different constraint.
The scale keeps moving.
Compute increases. Data increases. Models get larger. Training becomes more practical. Systems can run experiments and searches at scales that simply weren't available before.
So an approach that looks absurdly inefficient at one point in time can become surprisingly effective when the available scale changes.
The important lesson isn't simply that “brute force beats intelligence.”
It's that human cleverness and machine scale behave very differently.
A human can spend years designing better rules, features, heuristics, and representations. But all of that is constrained by what we already know about the problem.
General methods have a different property: they can keep benefiting from more computation and more experience without requiring us to explicitly specify every useful piece of knowledge.
That's the part of The Bitter Lesson that I find most interesting.
We tend to associate progress with making the system more sophisticated from our perspective more knowledge, more rules, more carefully engineered components.
But some of the biggest shifts in AI have come from doing less of that and putting more emphasis on methods that scale.
The nested loop is still the nested loop.
What changes is the machine running it, the amount of computation behind it, and the scale at which you're willing to run it.
And once I started looking at AI through that lens, I began noticing the same pattern in a lot of places.
Top comments (0)