DEV Community

Deep Fix
Deep Fix

Posted on

How to Fix Python Memory Leaks in Production – Proven Techniques & Tools

#python #memory #debugging #devops

Introduction

Memory leaks in Python can silently degrade performance, especially in long‑running production services. This guide walks you through diagnosing, fixing, and preventing leaks with real‑world examples.

Why Python Still Leaks

  • Reference cycles involving objects with __del__.
  • Unreleased C extensions.
  • Global caches and mutable default arguments.

Quick Checklist

  1. Enable tracemalloc.
  2. Run objgraph to spot unexpected object graphs.
  3. Profile with memory_profiler.
  4. Review third‑party libraries.

Step‑by‑Step Troubleshooting

Reproduce the Leak Locally 

# leak_demo.py
cache = []

def add():
    # each call adds a 1 MiB string
    cache.append('x' * 1024 * 1024)

if __name__ == '__main__':
    while True:
        add()
Enter fullscreen mode Exit fullscreen mode

Run the script and watch RSS grow.

Capture a Snapshot with tracemalloc 

import tracemalloc, time

tracemalloc.start()

# ... run your workload ...

time.sleep(5)                     # let the workload settle
snapshot = tracemalloc.take_snapshot()
top_stats = snapshot.statistics('lineno')

print("[ Top 10 memory consumers ]")
for stat in top_stats[:10]:
    print(stat)
Enter fullscreen mode Exit fullscreen mode

The output points to the file and line responsible.

Visualise Object Graphs 

pip install objgraph
Enter fullscreen mode Exit fullscreen mode 
import objgraph, gc

gc.collect()
objgraph.show_most_common_types(limit=5)
objgraph.show_backrefs(objgraph.by_type('list')[0], max_depth=3)
Enter fullscreen mode Exit fullscreen mode

If you see a growing list of a custom class, you likely have a cache that never clears.

Fix Common Patterns

Mutable default arguments 

# Bad
def append_item(item, container=[]):
    container.append(item)
    return container

# Fixed
def append_item(item, container=None):
    if container is None:
        container = []
    container.append(item)
    return container
Enter fullscreen mode Exit fullscreen mode

Unclosed file handles 

# Bad
def read_data(path):
    f = open(path)
    return f.read()   # file never closed

# Fixed
def read_data(path):
    with open(path) as f:
        return f.read()
Enter fullscreen mode Exit fullscreen mode

C extension leaks

Upgrade the extension or wrap the call in a subprocess to isolate the leak.

Production‑Ready Monitoring

  • Prometheus exporter for process_resident_memory_bytes.
  • Grafana alerts when RSS grows > 20 % over a 10‑minute window.
  • Automatic heap dumps via gdb when thresholds are breached.

Automated Patch Tool

We’ve packaged a reusable script that scans for the three patterns above and rewrites the offending files. Download the pre‑configured script here. For a full CI integration, you can also Get the complete patch tool and embed it in your pipeline. Need the source? Access the full repository fix.

Wrap‑Up

Fixing memory leaks is iterative: detect, isolate, patch, and monitor. By combining tracemalloc, objgraph, and disciplined code reviews, you can keep your Python services healthy in production.

Top comments (0)