Memory management in C# - Part 1: why we should care

One could argue that as software programmers our main goal is to efficiently manage hardware resources.

There are many of those, like network cards, hard drive disks, SSDs, I/O devices, telemetry devices, and arguably by far most commonly CPUs and RAM modules (notice the different abstraction layers in those examples).

So this time let’s talk about RAM modules. And C#.

Please notice this article greatly simplifies the contents. If you want to expand on the knowledge, please use the references at the end. There are too many great books on the subject by people much more knowledgeable than me :-)

“Wait, but isn’t it all abstracted away from us, as application developers?” Yes, and here’s why we should still care.

Where’s my memory? (a recap)

There are lots of interesting online explanations on Heap vs. Stack memory, especially theoretical and introductory. But how and when should we actually choose to use them?

Naturally, we are constrained to the Stack. When executing a program, the OS uses a stack implementation. Nothing too fancy, it’s literally (with some simplification) a stack of function calls (i.e. the Call Stack), internally implemented as a contiguous array of so-called stack frames.

Every language has a calling convention which specifies what goes into this stack (i.e. what is a stack frame). Generally, data is included regarding

• Return addresses, for actually JUMPing between function calls
• Parameter data
• Some space for return values
• Scoped variables

For our purposes, let’s leave other implementation details to the C# language specification, and the lower level stuff for the OS and the likes of Intel and AMD and their Instruction Set Architectures

From another perspective, the Call Stack tracks where the program is in terms of execution. It has to be a stack basically due to the efficiency of pushing and popping functions in it, which is exactly how a program executes in the first place!

And there it is, your parameter and variable memory is in “the Stack”. When the function returns, it finishes execution and gets popped from the stack, meaning the memory is automatically free’d up.

Since C# is a general purpose programming language, we could arguably write any solution with Stack memory only 👀 It’s fast, easy to write, and there’s no need for explicit memory management (the GC will end up managing it as well, but we’ll discuss this in the next sections). Here are some good use cases:

• Recursive functions
• Procedural programming
• Functional programming

Of course, it is limited. The C# (.NET) stack size for 32bit architectures defaults to 1MB per Thread, which is a lot for most applications - we can go tens of thousands of levels deep for simple recursive functions. If you exceed it, be prepared for an infamous StackOverflow exception (that said please don't be afraid of recursive code).

The other problem is scope, meaning we’d have to pass down local variables as parameters all the way through the code execution, which would make complex code less readable and a nightmare to maintain.

Should it be all “new”?

But don’t panic! (and make sure to bring your towel). We still have the Heap. As with the Stack, it also resides within RAM. But it's a different data structure, for a different kind of access.

A heap can be implemented just as well from a simple array. The rules are different though, since it is a tree-like structure that allows for some optimized read access at the cost of more complex writes. So yes, it is slower than the Stack (it can even end up with some fragmentation). So why use it? It's virtually limitless! We can easily throw data at it, and we get the reference to where that data is.

As opposed to a value which holds the value itself (int x = 2), a reference is a pointer which holds an address to where the value is dynamically allocated (var x = new string[10]). The address size can vary, but on an 32-bit machine, they are usually 4 bytes long (pointing to anything in the Heap, including functions!). The difference is commonly abstracted away from the programmer in C#, so it is a good idea to have the C# Value type docs and the C# reference types docs close in case of doubts.

So how do we use the heap in C#? The short, over-simplified and slightly off answer is with the new keyword (or instantiations). The complete answer is:

• A reference type is allocated on the heap
• A value type is created where the scope is: if it's a field in a class, along in the heap. If it's a local variable in a method, most likely on the stack (notice C# has tons of features and there are more complex scenarios, for lambda expressions there have shared scopes for example)

The above list is short and ambiguous on purpose. The language specification makes no guarantees on what goes where, and the compiler will ultimately and optimally decide.

To another extent, the C# heap is a global Managed Heap. That also means there is a runtime data structure running along with our code: the Garbage Collector.

So my code is produces Garbage?

Jokes apart, the Garbage Collector (GC) doesn't mean to offend us. It is responsible for effectively managing the C# memory (Heap and Stack), and can be our best friend when used correctly.

To reiterate, the GC runs with our code and goes reclaiming dead objects (pieces of memory we're not using). Hence we don’t need to explicitly free memory, which is a great and dangerous power. Most content out there will tell you to leave it all to the GC, since it knows better.

If you’ve gone this far though, you’re like me and is not particularly happy with having some data structure we can’t control taking care of so much of our program. So like Morpheus, I’m hereby admittedly inviting you to take the red pill with me.

Collecting the garbage

Remember the reference type from the previous sections? Let’s assume the OOP nature of C# and start calling it “object” (it has to be an object anyways given how an OOP compiler makes Semantic Analysis).

In a “managed” language like C#, objects have lifetimes: a short-lived object has been allocated recently and is likely closed related to others also just allocated, while a long-lived is older and maybe less related.

For completeness sake, the GC also differentiates between Large Objects (85,000 bytes and larger) and regular ones, with a separate heap for each type

The criteria to define an object lifespan is based on so-called GC generations. There are three of them:

• Gen 0: consists of youngest objects, like a recently allocated temporary variable. It’s the cheapest to manage, being the first option to free space for new allocations and also where GC collections are more frequent. All new objects implicitly go to Gen 0, except Large Objects, which go straight to Gen 3. We like Gen 0
• Gen 1: literally the mid way of object lifetimes, in between short and long-lived objects. After the Gen 0 collection, the memory is compacted and objects - like collections, for example - are promoted to Gen 1. Gen 1 is not checked every time for collection, only when the Gen 0 collection couldn’t feee up enough space for newer allocations. We have mixed feelings for Gen 1.
• Gen 2: long-lives objects, usually alive for the duration of a process (like Singletons, for example). Surviving a Gen 2 collection (i.e. a full collection) means sticking around until the memory is determined unreachable. We’re not fans of Gen 2, but it’s serves its purpose
• Gen 3: is just a nickname space for Large Objects only

Objects can survive collections, being promoted from Gen 0 to Gen 1 and Gen 1 to Gen 2. In simples terms, the GC will try to find the sweet spot between not getting too intrusive in collections and not letting the memory become too big.

GC - Ghost Component

What we’ll discuss now may seem like a lot of developer work for a GC that automatically sorts out the memory usage. It isn’t. The thing is the GC makes thread pauses (for ephemeral collections - Gen 0 and Gen 1 - lasting a couple of milliseconds) to actually cleanup the memory. In general, what we want is to make these pauses less frequent and faster. Most of the real challenge goes to understanding how have we implemented the solution - how we have used the memory.

So you may have heard about “problems with the GC” or “memory leaks in C#”. These are usually due to Gen 2 objects that are hanged around or maybe a concentration of Large Objects.

Some objects interact with “out-of-process” resources (i.e. unmanaged resources), like disk I/O, and network connections (and any other general OS resources). When that happens, explicit cleanup becomes necessary (after all we wouldn’t want the GC to step in and shut down a network port, and it wouldn’t know if the port is being used anyways).

With regards to such unmanaged resources, a case-by-case analysis is necessary. For the GC to be able to collect them, we should implement the IDisposable.Dispose method. We should also consider using destructors, or in C# terms, the Object.Finalize method in case there’s a chance of Dispose not being called by a developer.

Finally, if you make too many allocations, you may get a nasty OutOfMemory exception. That ultimately means the OS was not able to provide addressable space for allocations. In such a case, remember: the .NET CLR, .exe and other .dll modules are sharing the memory with your application.

Analyzing and profiling

In the next parts, we’ll actually analyze and profile a C# program in practice. Then, we’ll discuss tools, trade-offs and myths in terms of memory management. See you there!

PS.: Please leave your comments and feedback. This very condensed article should be used as a general discussion and not a static reference. Let me know if I should expand on any of the content :)