DEV Community

qing
qing

Posted on

Python Generators: Memory-Efficient Iteration

Python Generators: Memory-Efficient Iteration

Generators allow you to iterate over large datasets without loading everything into memory at once.

Complete Code Example

def fibonacci():
    a, b = 0, 1
    while True:
        yield a
        a, b = b, a + b

# Take first 10 Fibonacci numbers
fib = fibonacci()
first_10 = [next(fib) for _ in range(10)]
print(first_10)  # [0, 1, 1, 2, 3, 5, 8, 13, 21, 34]

# Generator expression (memory efficient)
total = sum(x**2 for x in range(1_000_000))  # No list in memory!

# Pipeline with generators
def read_data(n):
    for i in range(n):
        yield {"id": i, "value": i * 2}

def filter_data(data, threshold):
    for item in data:
        if item["value"] > threshold:
            yield item

def transform(data):
    for item in data:
        yield {**item, "label": f"item_{item['id']}"}

pipeline = transform(filter_data(read_data(100), 50))
results = list(pipeline)
print(f"Got {len(results)} results")
Enter fullscreen mode Exit fullscreen mode

Why This Matters

Understanding these Python features will help you write more:

  • ✅ Readable and maintainable code
  • ✅ Memory-efficient applications
  • ✅ Pythonic, idiomatic solutions

Summary

Mastering Python's built-in features is key to becoming a better developer. Practice these examples in your own projects, and you'll quickly see the benefits.


Found this helpful? Follow me for more Python tutorials and tips! 🐍

Follow for more Python content!


💡 Related: **Content Creator Ultimate Bundle (Save 33%)* — $29.99*

Top comments (0)