This book offers insights into C++, including the algorithms and practices used in game development, and explores the language's strengths and weaknesses, its established workflows, and hands-on solutions, because C++ still dominates the game industry today thanks to its combination of high performance, flexibility, and deep low-level control.
Hotspots and bottlenecks
The key task in optimization is to find hotspots, which are code fragments with a high computational load, and bottlenecks, which are components that limit overall throughput, and these problems usually fall into three categories, namely abstractions, smoothness, and hardware limitations.
Abstractions, or more precisely premature abstractions, add unnecessary layers to the engine architecture and make the whole system more complex than it needs to be, and the usual suspects here are multiple inheritance, one-size-fits-all solutions to game development problems, and heavy use of design patterns in places where simple procedural code would do the job just fine.
The same category includes writing beautiful code instead of fast code, when performance-critical sections get buried under unnecessary layers of ceremony and checks, which is exactly what the KISS principle warns us about.
Most developers agree that smoothness matters more than peak performance, and the things that work against it are well known, because sporadic memory allocations on the main thread make the frame rate uneven, algorithms with unpredictable complexity make frame times inconsistent, and blocking resource loads cause stuttering. A stable 30 FPS beats a frame rate that jumps between 60 and 20, because players feel those jumps immediately, which is why a consistent frame rate is such an easy-to-understand and measurable indicator of quality.
Hardware limitations are best treated as creative constraints, because brute-forcing a platform's physical limits instead of applying smart optimizations only adds complexity without any real payoff. Raising the polygon count without a proper level-of-detail system will leave the player with low FPS, and the same goes for complex shaders that ignore fill-rate limits, memory-hungry data structures that don't account for the cache hierarchy, and so on.
A modern CPU-GPU architecture is a complex data-processing pipeline with many parallel execution units, and in this "digital hell" data moves through a hierarchy of caches, travels between the CPU and GPU over the bus, and gets synchronized across threads with memory barriers and atomic operations, so the system almost never does exactly what you wrote in the code.
You can get solid and stable performance by balancing the workload across all of these components, and even then you may not be able to push the CPU beyond 70 to 80%, but you can make much better use of the GPU, memory bandwidth, and caches. The real challenge is finding a delicate balance right at the edge of what's possible for every component at once, and if you get it wrong, resource contention follows, one component becomes the critical link, and the rest of the system suffers from pipeline stalls, context-switching overhead, and reduced throughput.
Hotspots
In software systems, hotspots are localized sections of code with unusually high resource consumption, whether that's CPU time, memory, or disk I/O, and they're characterized by a disproportionately high execution frequency, heavy CPU utilization, high memory pressure, or significant system call overhead.
In the context of code optimization, the Pareto principle, also known as the 80/20 rule, means that 20% of the source code accounts for 80% of a program's total execution time, which makes hunting down hotspots a highly effective strategy. Searching iteratively helps you find the code blocks that consume the most resources so you can target them and cool them down, but once one issue is fixed, the workload shifts and previously minor sections turn into new hotspots that need attention. This process goes on until the results are acceptable or the time allocated for the task runs out, and it's worth remembering that an engineer's time is a finite resource too, because the more time a developer spends polishing one part of the code, the less time they have for everything else.
Modern profilers provide detailed analysis down to the level of individual CPU instructions, which makes it possible to identify temporal hotspots, code with a high number of cache misses, heavy I/O operations, and resource-lock contention in multithreaded applications.
Bottlenecks
Imagine a dish being prepared in a café. The chef can't make the whole thing at once, so they hand the side dish off to an assistant while working on the main part themselves, and then the two parts come together and a waiter takes the finished plate to the customer. How long will the whole process take? Suppose the chef finishes first, but the assistant simply can't work any faster, and now the assistant is the bottleneck that slows down the entire system.
Games work exactly the same way, because there's always a "slow assistant" somewhere, whether it's the CPU, the GPU, memory, or something in the code that holds everything else back, and the developer's job is to find that weak link and bring its performance in line with the rest. Optimization almost always involves trade-offs, and in 99% of cases there's no silver bullet that will fix the problem right away.
The first and most common trade-off is between performance and memory, since sometimes it's more efficient to store extra data than to recalculate it from scratch. This is called memoization or caching, and the idea is that you calculate something once, store the result, and then simply pull it from memory whenever you need it, which costs more memory but gets you the result faster.
The second trade-off is between accuracy and speed. Sometimes you can cut corners and speed up a system by simplifying a data type or using a less precise algorithm, and shaders are a good example, because they provide both log and logp, where the former is more accurate but slower and the latter is faster but less precise. In the final image, players are unlikely to notice that a neighboring polygon has shifted by a quarter of a pixel, but the game may run noticeably faster, and the same principle applies to almost any data model, pathfinding system, or game AI. The key question is how much precision you're willing to give up, or, in more formal terms, how much you can shift resources from lower-priority computations to higher-priority ones.
The third classic trade-off is simplicity versus speed, and this one has more to do with people and the team than with hardware. A classic example is a young, talented developer who has just discovered design patterns and responds by building five levels of nested caching in an attempt to optimize the texture-loading system.
Sure, they'll get that system built, and it may even run faster thanks to intrusive pointers, custom allocator pools, and a lock-free hash table, but the result is predictable, because nobody, not even the author a month or two later, can understand, debug, or modify that code without risking breaking something. Other developers on the team start avoiding those files, they groan every time they have to touch the system, and code reviews keep getting comments like "What the hell is this? Can we just rewrite it in plain C++?"
An experienced developer knows that real performance gains don't come from showing off their knowledge of algorithms, but from understanding the trade-off between how maintainable the code is and how much the optimization actually pays off.
Yes, you can squeeze out another 5% of performance with custom memory pools, but that will take two weeks to implement and another month to debug all the weird edge cases, so it might be smarter to use a simple linear allocator and trade some memory for time.
Mature optimization means fixing the truly critical bottlenecks with clean, well-documented solutions that you can easily switch off with conditional compilation if something goes wrong. In the real world, shipping the game matters more than cool algorithms, and having code the team can maintain matters more than having code that looks impressive in a conference talk.
Myths and misconceptions
Software optimization is inherently endless, because there will always be something left to improve, but it's important to understand that past a certain point you start hitting diminishing returns, and every additional effort delivers a smaller and less noticeable performance gain.
On top of that, developers often spend time chasing dead ends that provide no real benefit (but make for great slides and conference talks) and only make the system harder to maintain. The key skill here is knowing when to stop and ask whether further improvements will actually add value or turn into a witch hunt.
One of the most common mistakes is making decisions based on assumptions. You can't just "guess" what's slowing the system down, because those guesses are usually wrong, and the only reliable approach is to measure and analyze, then analyze and measure again, until guesses and gut feelings turn into well-founded engineering decisions.
Optimization doesn't start after the code is written, and in fact 80% of performance problems can be eliminated at the architecture design stage, because proper decomposition, a clear object hierarchy, and well-thought-out dependency management help you avoid bottlenecks in the first place. Despite the overhead that comes with languages like C# and Java, a well-structured system often outperforms code written in pure C++ that ignores architectural constraints.
When it comes to optimization, it's important to focus on worst-case scenarios rather than average ones. Most developers test their games and applications on powerful workstations, which often gives them a distorted picture, and on weaker hardware the bottlenecks can be completely different, with memory bandwidth or CPU power becoming the limiting factor instead of the GPU.
To analyze performance properly, developers need to deliberately test on configurations with minimal or weak hardware specs, because those machines are much closer to what a good portion of your players actually experience. Players can tell the difference between 55 and 60 FPS, but they rarely complain about it, while a drop to 20 FPS feels like serious lag even if it only happens once in a while, and that's why consistent frame times often matter more than a high average frame rate.
Debug and release builds give fundamentally different results (Pic. 3.1). In debug mode, compiler optimizations are turned off and extra checks are turned on, which can slow the code down several times over, and not evenly. For valid performance profiling, work only with release builds and enable debug symbols or disable optimization for specific sections when you need to, because this lets you strike a balance between traceability and accurate measurements while still giving you a realistic picture of performance.
Picture 3. 1
That said, debug builds are useful in the early stages of testing, when they can reveal excessive CPU usage or inefficient algorithms before those problems become critical.
Finally, the key requirement for any benchmark is control over external variables, since poorly designed tests can create the illusion of optimization. For example, the first measurement might be taken in fullscreen mode while the second runs in windowed mode, at a different refresh rate, or at a different resolution, and the frame rate goes up because the environment changed rather than the code, which makes the whole comparison meaningless. To avoid mistakes like this, keep every environment parameter fixed and change only one thing between runs, namely the piece of code you're optimizing. The best practice is to automate your benchmarks, which removes human error and gives you reproducible results.
Many people think optimization time should be added to sprints once the game is nearly finished, or, even worse, only when performance problems show up. But ignoring optimization in the early stages of development means risking increasingly tangled processes and unpredictable performance problems at the worst possible moment.
Another myth is that reactive optimization works well enough on its own. Yes, hundreds of games ship after being optimized at the last minute, but this approach usually ends with cutting features or simplifying mechanics to make up for low FPS. Reactive optimization creates a false sense of control, because fixing problems late in development takes far more time and effort than taking a systematic approach throughout the whole project.
The next myth is that optimization needs large, dedicated blocks of time, when in reality it's much more effective to work it in gradually. With practices like performance budgets, you can set limits for FPS, CPU, and GPU usage up front and organize the team's work around those numbers, and continuous monitoring with incremental code adjustments produces better results than occasional optimization pushes in the final weeks of development.
There's also a persistent belief that every optimization is justified. In reality, there are plenty of areas where programmers can sink time without seeing any obvious benefit, so only optimize what actually affects performance and weigh the potential payoff against the time the changes will take. This approach keeps you from wasting effort and keeps the focus on the bottlenecks that are really slowing the game down.
Finally, there's a misconception that performance is a single metric, when a game that "feels slow" can mean any combination of low FPS, longer response times, high CPU and GPU load, and frame rate drops. To optimize even one of these metrics effectively, developers first need to understand which ones are critical for their particular project, how they relate to each other, and what's actually holding the game back.
A systematic approach like this is the only way to avoid the common misconceptions about optimization. Will anyone actually approve such changes, though? Most likely not, because it's extremely unlikely that the project architect, tech lead, or CTO will sign off on real architectural changes, and that's not really the programmer's call to make anyway, so pushing for it tends to step on the architect's toes. Besides, changing an existing, working, well-tested system can break the systems around it, and there's no way to predict when the error will surface, only that it inevitably will. That's exactly why a developer who just joined the team can't rewrite half the engine "the right way," and what's left for them is code-level and algorithmic tweaks, which can still squeeze out a performance improvement of at least 10%.
This brings us to the reality that game development forces us to balance our "ideal" technical solutions against the practical constraints of the project. Even a programmer who sees through every optimization myth often has to work within an existing architecture they can't change and settle for local improvements at the algorithm and code level, but this approach can still deliver tangible results, such as the 10 to 15% performance gains that can make all the difference for a successful release.
Knowing where your influence ends matters, and so does choosing your battles wisely. Optimization was never just a technical skill, because it's also a team sport where every change needs to be justified, measured, and agreed upon, and in the end the best optimization is the one players actually notice, not the one that gets you a few nods in a technical chat.
Tools
Any work aimed at improving performance is wasted if you can't see what's actually happening in your code. It's like trying to repair a watch blindfolded or tune a musical instrument with your ears plugged, because you can turn the screws and pluck the strings all you want, but the result will be purely random.
In everyday life, we get immediate feedback, so if you trip, you fall, if you take a wrong turn, you may end up at a dead end, and if you add too much salt, you ruin the dish. These signals help us correct course right away and remember not to make the same mistake twice. In programming, however, things work differently, because code can execute millions of times per second, process gigabytes of data, and interact with dozens of system components, while all we see is the end result, say, an FPS counter that refuses to go above 50.
Modern software, and games in particular, can run to hundreds of thousands of lines of code stacked on top of complex operating systems, graphics drivers, and various libraries that add even more code. Without specialized analysis tools, all you can really do is guess whether the CPU can't keep up, whether the bottleneck is loading textures from disk, or whether memory is leaking somewhere. Profilers, performance monitors, and debuggers can mislead you too, but you can't do without them, because they show which functions take the longest to execute, how much memory each part of the program uses, how often the application hits the disk, and where the CPU sits idle waiting for data. Without this information you're flying blind, and with it the solution often becomes obvious, although measuring and analyzing performance correctly is its own kind of science that sometimes borders on alchemy.
Any tool that measures a program's performance will inevitably affect that performance. Some tools are light enough on their feet that they barely slow the program down, and those are called non-intrusive, while others can seriously drag down an application's performance, and those are called intrusive.
How much interference you get depends on how much data you need and how often you need it. A simple FPS counter has virtually no impact on the game, while a detailed analyzer that logs every function call and its execution time can slow things down considerably. Some tools, such as Valgrind, go even further and run the entire program inside a virtual machine to collect highly detailed information about every operation, and you can guess what that does to performance, since the program can easily run ten times slower than usual or worse.
There are several ways to measure performance, and sampling profilers are the gentlest and simplest of them. They periodically take a snapshot of the program to see which code is running at that moment, and by collecting a series of these snapshots, developers can build a statistical picture of what the program spends most of its time doing.
A more intrusive approach relies on interception tools that sit between the program and external resources, so PIX, for example, intercepts every call the program makes to the GPU through the DirectX API, while network analyzers peek at all network traffic.
Getting even more detail means adding special commands and profiling markers to the code to collect data, and this process is called instrumentation.
Choosing the right tool depends on your specific goals and on how much performance you're willing to sacrifice for a more detailed analysis. The one rule that really matters is that the tool shouldn't distort the measurements more than the problem you're hunting for, since there's little point in optimizing a function that accounts for only 1% of execution time if the profiler itself slows the program down by 50%. That's why all three types of tools are often used, each in its own circumstances.
Theory and terminology
Profilers is an umbrella term for the performance analysis tools used across most of game and software development, and a profiler's main job is to capture how the workload breaks down during execution in as much detail as the developer allows.
The classic use case is measuring CPU time, but modern profilers can track much more, including cache misses, memory allocations, disk access, thread usage, and synchronization. Depending on the level of instrumentation, they can give you either an aggregated view of the workload that shows which modules or systems put the most load on the CPU and where, or a per-frame or per-interval snapshot with the call stack and the exact distribution of work across individual instructions.
Profilers fall into two categories, general-purpose and specialized. General-purpose profilers are usually the ones built into an IDE or a platform SDK, such as the Visual Studio Profiler, and they give you a broad sense of where performance is leaking. Specialized profilers are low-level tools, such as Sony Razor, Intel VTune, PIX, or Tracy, that collect data from CPU performance counters and OS events and let you analyze code behavior with much greater precision, and only these low-level tools let developers evaluate how specific algorithmic steps or memory access patterns affect the overall performance of a piece of code.
Total time shows the cumulative execution time of a function, including all the nested calls it makes to other functions. In theory, it helps identify hotspots in an application and gives a complete picture of how much time the program spends in a particular section of code along with all its dependencies, but in practice it can give a distorted view of performance.
As part of its optimizations, the compiler may inline functions directly into the code of the calling function, which blurs the execution time of the parent function even though the inlined code is what actually does the work. This metric also doesn't account for indirect costs, such as cache pollution or data eviction, which can seriously undermine how reliable the measurement is.
For example, when you analyze functions whose performance depends heavily on their input data, such as sorting an almost-sorted array versus an unsorted one, the profiler will report an average value, which can hide real performance problems that only show up in certain use cases.
Self time measures only the time spent executing instructions inside the function itself, excluding the time spent in all the functions it calls. This helps identify the code blocks that actually do the expensive work rather than just handing it off to other functions, and functions with heavy algorithms, loops, or number crunching typically show high self time.
The ObjectUpdate() function may have a high total time because it calls all the other subsystems responsible for updating the game state, but a low self time because it doesn't do much work on its own, while a collision-physics function may show significant self time, which tells you that this particular section needs optimization. The problems here are much the same as with the previous metric, because self time can't reflect a function's actual impact on the overall performance of the system. A function may have a low self time while triggering many expensive operations through other functions, which creates a false impression of efficiency. On top of that, this metric ignores function call overhead, so if a function makes thousands of small calls, the cost of creating stack frames, passing parameters, and returning values can be significant, or even close to the actual execution time, and still stay invisible in the self-time numbers.
Another issue is "aggressive optimization," which is especially common with Clang-based compilers that inline small functions and partially embed their code at the call site. As a result, self-time measurements get distorted, because code that belongs to a different function is counted as self time for the caller (Pic. 3.2).
Picture 3. 2
The Calls metric shows how often a particular piece of code runs during a profiling session, which helps reveal how different functions are actually used in the application. Even if their individual execution times are short, functions with a high call count are good candidates for optimization, because the cumulative overhead of all those calls can be significant.
Analyzing call counts is one of the first steps in finding inefficient algorithms or unnecessary computations (with one caveat, which is covered in the note below). A function that gets called thousands of times per frame may need a whole range of optimization techniques, and a high call count can also point to recursion, resource leaks, or other architectural problems in the application (Pic. 3.3).
Picture 3.3
Overall, this is a fairly useful metric, but the call count alone doesn't show how those calls are distributed over time. A function may be called 100,000 times in a session, but if all those calls happen while a level is loading, they have no impact on gameplay performance, and in that case the profiler will show a high call count without telling you that the function sits idle while frames are being rendered. The call count can also be misleading when it comes to spotting real architectural problems, because a high number of calls may simply be the result of a well-designed architecture with lots of small, specialized functions rather than a sign of inefficiency.
NOTE
Modern compilers have gotten very good at inlining, so focusing only on call counts can lead to premature or incorrect conclusions\.
Total waste time tracks how long a function spends with its thread blocked while waiting for resources to become available, I/O operations to complete, or system events to occur (Fig. 3.4). This time doesn't reflect active execution of the function, but rather delays caused by external factors, and a high waste time can point to indirect performance problems that affect the function's overall execution time. This metric is usually analyzed when refactoring code or reducing dependencies between functions.
Picture 3.4
Author: Sergei Kushnirenko
Sergei has over 20-year experience in coding and game development. He graduated from ITMO National Research University and began his career developing software for naval simulators, navigation systems, and network solutions. For the past fifteen years, Sergei has specialized in game development: at Electronic Arts, he worked on optimizing The Sims and SimCity BuildIt, and at Gaijin Entertainment, Sergei headed up the porting of games to the Nintendo Switch and Apple TV platforms. Sergei actively participates in open-source projects, including the ImSpinner library and the Pharaoh (1999) game restoration project.
Top comments (0)