One Line of PyTorch Code That Fixes a Silent Memory Leak

If you have ever seen your GPU memory climb steadily during training even though your batch size never changes, the cause is almost always the same thing: accumulating loss or metric tensors directly in a Python list instead of detaching them first. Every tensor you append still carries its computation graph with it, so PyTorch keeps every intermediate activation alive in memory for the entire run. The fix is a single change, replace losses.append(loss) with losses.append(loss.detach().item()), and the graph gets freed immediately after each backward pass. It is a small habit, but it is one of the most common reasons training jobs that should fit comfortably in memory end up crashing hours in. If you are debugging a similar leak, torch.cuda.memory_summary() is worth checking before assuming you need a bigger GPU. More on scaling PyTorch workloads properly: Hire PyTorch Developers