I Made Python 2,000x Faster by Changing One Line of Code

No caching. No multiprocessing. No C extensions. Just choosing the right data structure.

Python has a reputation for being slow. But sometimes the problem isn't Python itself: it's the data structure we choose.

I compared two ways to answer one question: is this ID already in the collection? In one benchmark run, changing the collection from a list to a set made repeated unsuccessful lookups about 2,378 times faster.

This isn't a claim about general API performance. It's a specific, reproducible experiment. Let's see why it happens.

The problem

Imagine a service that holds 10,000 integer IDs and needs to check thousands of incoming IDs.

user_ids = list(range(10_000))

def user_exists(user_id):
    return user_id in user_ids

This is correct code. But how does it behave when most lookups miss?

The benchmark

Here's the complete benchmark (also available as a separate .py file):

import platform
import statistics
import timeit

N = 10_000
Q = 5_000

ids_list = list(range(N))
ids_set = set(ids_list)
queries = range(N, N + Q)  # All queries are misses.


def using_list():
    return sum(q in ids_list for q in queries)


def using_set():
    return sum(q in ids_set for q in queries)


assert using_list() == using_set() == 0


def benchmark(func):
    timings = timeit.repeat(func, number=1, repeat=7)
    return statistics.median(timings)


list_time = benchmark(using_list)
set_time = benchmark(using_set)

print(f"Python: {platform.python_version()}")
print(f"List: {list_time * 1000:.3f} ms")
print(f"Set:  {set_time * 1000:.3f} ms")
print(f"Speedup: {list_time / set_time:,.0f}x")

The collection contains 10,000 integer IDs. Every one of the 5,000 queries is absent, so this tests a particularly unfavorable case for a list. The structures are built before timing begins. Each function is run seven times, and the median is reported.

The results

On the test environment (Python 3.13.5), I measured:

Approach	Median time
list	350.413 ms
set	0.147 ms
Ratio	~2,378×

These are measurements from one execution environment, not universal performance constants. Rerunning the test may produce different values.

Why does list take longer?

For a list, x in values scans its elements until it finds x or reaches the end. Since all our test queries miss, each check requires a full scan: O(n) per lookup.

By contrast, a set uses hashing. For ordinary integer keys and typical hashing behavior, its membership check is O(1) on average. That does not mean sets can never slow down; collisions and unusual inputs matter.

The one-line change

# Before
user_ids = list(range(10_000))

# After
user_ids = set(range(10_000))

The membership expression stays identical:

if user_id in user_ids:
    ...

The algorithmic cost, however, is different.

Don't replace every list with a set

Sets have trade-offs:

• They don't keep duplicate elements.
• They don't support positional indexing.
• Their elements must be hashable.
• They typically use more memory.
• Constructing a set takes time, so the conversion may not pay off for a single lookup.

Also, the test deliberately uses only missing IDs. A list can find an element at the beginning quickly, and on tiny collections a list may be perfectly adequate. Ordering, duplicate values, and memory use may matter more than lookup speed.

The real takeaway

Before adding threads, caches, or new infrastructure, ask whether your hot path uses the right data structure.

This benchmark does not show that a FastAPI endpoint will become 2,000× faster. It shows that repeated in-memory membership checks can differ dramatically depending on the collection type and workload.

Measure your actual bottleneck, then optimize it.

What is the biggest speedup you've achieved by changing a surprisingly small amount of code?

---

GitHub · Portfolio