Optimizing Python Loops for Speed and Memory Efficiency with Generators

#python #codeoptimization #pythontips #performance

Loops are core to Python programming—but if written carelessly, they can slow your code down and waste memory.

Let’s explore how to go from naive loops → list comprehensions → generators for faster, cleaner, and memory-efficient Python code.

Naive Loop Example

A simple loop works fine for small datasets but doesn’t scale well:

def square_even(nums):
    result = []
    for n in nums:
        if n % 2 == 0:
            result.append(n * n)
    return result

✅ Easy to read
❌ Builds a full list in memory (bad for millions of items)

Using List Comprehensions

Python’s list comprehensions are faster and more concise:

def square_even_lc(nums):
    return [n * n for n in nums if n % 2 == 0]

✅ Runs faster
❌ Still consumes memory proportional to list size

Enter Generators (Lazy Evaluation)

Generators produce items on demand, making them ideal for large or streaming data.

def square_even_gen(nums):
    for n in nums:
        if n % 2 == 0:
            yield n * n

Use it like this:

for val in square_even_gen(range(10_000_000)):
    print(val)

✅ Minimal memory usage
✅ Ideal for reading files or streaming data
❌ One-time use (exhausted after iteration)

Quick Performance Comparison

import time

nums = range(10_000_00)

start = time.time()
square_even(nums)
print("Naive:", round(time.time() - start, 3), "s")

start = time.time()
square_even_lc(nums)
print("List comprehension:", round(time.time() - start, 3), "s")

start = time.time()
sum(square_even_gen(nums))
print("Generator:", round(time.time() - start, 3), "s")

Typical Result:

List comprehension: fastest for small/medium data
Generator: best for huge datasets or limited memory

Conclusion

Optimizing loops isn’t just about speed—it’s about writing scalable and memory-smart Python.
Use list comprehensions for everyday code, and generators when dealing with big data or streaming tasks.