Down the memory lane
Some developers don’t understand memory.
They understand new, delete, or garbage collection. And again — that’s just the surface.
Memory is not a flat, uniform space.
It’s a hierarchy:
• registers
• multiple levels of cache
• RAM
• virtual memory managed by the OS
Each layer has different latency and behavior.
But we usually learn memory management like this:
“allocate object → use it → free it”
So we assume: as long as memory is “managed”, everything is fine.
It’s not.
Performance is often dominated by:
• cache locality
• allocation patterns
• fragmentation
• allocator contention
That’s why two identical algorithms can have wildly different performance
depending on how data is laid out in memory.
Using smart pointers doesn’t mean you understand memory. It means you’ve delegated part of the problem. Which is fine — until it isn’t.
If you care about performance, you need to understand:
- how data moves,
- how it’s cached, and how it’s allocated.
Because in the end:
It’s not about what your code does.
It’s about how it lives in memory.
Building blocks
Data types are one of the most misunderstood fundamentals.
Because we don’t really learn what they are.
We learn their names: int, float, bool - feel like simple building blocks.
But underneath, they are:
• bit patterns
• rules of interpretation
• operations defined over those bits
Without that model, decisions become guesswork.
Like using signed integers for sizes.
A size is a count. It cannot be negative.
But if your type allows negative values, you’ve already introduced invalid states into your system.
This leads to subtle bugs:
• overflow issues
• signed/unsigned comparison traps
• edge cases that “should never happen” — but do
The deeper issue is this:
A type is not a label. It’s a constraint on reality.
If you don’t understand:
• binary representation
• overflow behavior
• how arithmetic actually works at the hardware level
then you’re not choosing types. You’re guessing.
And sometimes guessing is enough.
Until it isn’t.
We teach developers what to write.
We rarely teach them what it means.
Concurrency
Some developers don’t learn concurrency. They learn async/await or whatever concurrency utilities their favorite language or framework provides. And those are not the same thing.
Threads are not a language feature. They exist at the OS level, run on physical CPU cores, and compete for shared memory and cache.
But many of us first encounter concurrency through: Python, C++, Java, or some framework API.
So our mental model becomes: “this is how concurrency works.”
It’s not.
It’s just how one runtime exposes it.
That’s why things feel unpredictable:
• “Why do I still get race conditions?”
• “Why is my async code slower than sync?”
• “Why doesn’t this scale across cores?”
Because underneath:
• the OS scheduler decides who runs
• context switching is not free
• cores share memory with non-uniform latency
• caches can invalidate each other
If you don’t understand that layer, you’re not really reasoning about concurrency.
You’re just using it.
To be clear — that’s fine at the beginning.
But the moment you care about performance or correctness under load,
you have to go down the stack.
Hardware → OS → runtime → language.
Not the other way around.
Otherwise you’re not solving concurrency problems.
You’re operating an API.
Abstractions are just tools
In previous chapters I talked about:
• concurrency
• memory
• data types
At first glance, these look like different topics.
They’re not.
They all share the same root problem:
We learn abstractions… without understanding what they abstract.
We learn:
• async/await instead of how threads are scheduled
• new/delete instead of how memory behaves
• int/float instead of how data is represented
So our mental model becomes: “this is how the system works”
When in reality: “this is how one layer exposes it”
And most of the time, that’s enough.
Until it isn’t.
The cracks appear when:
• performance matters
• systems scale
• bugs become non-deterministic
• behavior stops matching intuition
That’s when abstraction stops helping and starts hiding the problem.
This doesn’t mean everyone must become a hardware engineer.
But it does mean:
If you want to reason about systems, you need to understand the layers below them.
A simple rule that helped me:
When something feels “weird” or unpredictable — you’re probably missing a layer in your mental model.
Go one level deeper:
hardware → OS → runtime → language.
That’s usually where the answer is.
Abstractions are tools.
But understanding is leverage.
Mental model
A fair question may come up:
“How do I actually learn all these lower layers without going down a 5-year rabbit hole?”
Short answer:
You don’t need to learn everything. You need to learn just enough to build a mental model.
Here’s what actually works.
Learn one layer below your current work
If you’re writing application code → learn runtime behavior
If you’re using runtime features → learn OS basics
If you’re already there → peek into hardware
Don’t jump straight into CPU manuals. Go one step down.Learn through problems, not theory
Don’t start with textbooks.
Start with questions like:
• “Why is this slower than expected?”
• “Why does this break under load?”
• “Why does this behave differently on another machine?”
Then dig until you hit the real cause. That’s where learning sticks.Build small, focused experiments
You don’t need big projects.
Try things like:
• write a tiny thread pool
• measure cache effects with different data layouts
• intentionally create a race condition and observe it
• compare contiguous vs scattered allocations
You’ll learn more from one experiment than from hours of passive reading.Use tools that show reality
Start simple:
• profilers (CPU, memory)
• timing measurements
• system monitors
Look at what the system is actually doing — not what you think it’s doing.Accept partial understanding
You don’t need to master everything.
If you walk away knowing:
• caches exist and affect performance
• threads are scheduled, not “run” by your code
• memory is not uniform
you’re already ahead of most developers.Go deeper only when it pays off
Not every problem needs low-level knowledge.
But when something feels:
• slow
• unpredictable
• “magical”
that’s your signal to go down a layer.
You don’t need to become a systems expert.
But you do need to know when the abstraction stops being enough.
That’s the real skill.
Top comments (0)