Linear Search Bottlenecks: How in_array() Inside a Loop Quietly Wrecks Your App's Speed

Your app works fine with two hundred users. Then it hits two hundred thousand, and it crawls.

Nobody touched the database. Nobody added a slow third party API call. The support tickets start coming in about pages that take eight, ten, sometimes fifteen seconds to load. The team checks the server. The server is fine. They check the queries. The queries are fine. Everyone assumes it must be something big and dramatic.

It almost never is. Most of the time, the real bug is one small, ordinary looking line sitting quietly inside a loop, doing exactly what it was written to do, just far too many times.

That line usually looks like this in PHP.

if (in_array($userId, $bannedUsers)) {
    // block access
}

Or like this in JavaScript.

if (allowedIds.includes(user.id)) {
    // grant access
}

Or like this in Python.

if user_id in banned_users:
    # deny access

None of these lines look dangerous. They read like plain English. That is exactly the problem. They are so easy to write and so easy to trust that almost nobody stops to ask what is happening underneath them, especially when they end up inside a loop that runs for every user, every product, or every request.

What in_array() Is Actually Doing

in_array(), .includes(), and Python's in keyword all do the same basic job when checking membership in a list or array. They perform a linear search. That means the function starts at the first element and checks it, then the second, then the third, and keeps going one by one until it either finds a match or reaches the end of the list.

In computer science terms, that is O(n) complexity. The time it takes grows in a straight line with the size of the array. Check ten items and it is instant. Check ten thousand items and you will actually feel it. Check ten thousand items thousands of times inside a loop, and you have just built an O(n times m) bottleneck, which is the technical way of saying "this gets exponentially worse as your data grows."

Here is the pattern that causes it almost every time.

$bannedUsers = getBannedUserList(); // 15,000 entries

foreach ($allRequests as $request) {
    if (in_array($request['user_id'], $bannedUsers)) {
        // reject
    }
}

If $allRequests has 10,000 entries and $bannedUsers has 15,000 entries, that inner in_array() call can run up to 15,000 comparisons, and it does that for every single one of the 10,000 requests. That is potentially 150 million comparisons to answer a question that should take microseconds.

This is not a rare or exotic mistake. It shows up in permission checks, blacklists and whitelists, duplicate detection, tag matching, category filtering, and anywhere a developer needs to ask "is this thing in that list" while looping over a second collection.

The Numbers Do Not Lie

This is not a theoretical concern. Independent benchmarks on real PHP arrays have shown in_array() taking several seconds to search a large dataset repeatedly, while the hash based alternative finishes the same job in a few hundredths of a second on identical hardware. That is not a small gap. That is the difference between a page that feels instant and one that makes a user close the tab.

The frustrating part is that this bug is invisible during development. Your local test array has twelve items. Your staging environment has two hundred. Everything performs beautifully. Then the product succeeds, real users show up, the dataset grows into the thousands or millions, and the exact same code that sailed through every code review becomes the reason your server is on fire during a traffic spike.

A job platform matching candidates against a growing skills list. An e-commerce store checking product IDs against a large catalog on every page view. An authentication layer validating a user against a permissions array on every single request. These are not edge cases. This is the everyday shape of a real, growing application, and it is exactly where this crime hides.

The Professional Fix

The fix does not require a new library, a caching layer, or a rewrite. It requires changing the shape of the data you are searching, from a plain list into a structure built for instant lookups.

PHP: flip it into keys and use isset()

// Before: O(n) per check
$bannedUsers = getBannedUserList();
foreach ($allRequests as $request) {
    if (in_array($request['user_id'], $bannedUsers)) {
        // reject
    }
}

// After: O(1) per check
$bannedUsers = array_flip(getBannedUserList());
foreach ($allRequests as $request) {
    if (isset($bannedUsers[$request['user_id']])) {
        // reject
    }
}

array_flip() turns the array values into keys. PHP arrays are hash maps under the hood, so isset() on a key resolves in close to constant time, no matter how large the array gets. The lookup that took seconds now finishes before you notice it happened.

JavaScript: reach for a Set

// Before: O(n) per check
const allowedIds = getAllowedIds(); // large array
requests.forEach(req => {
    if (allowedIds.includes(req.userId)) {
        // grant access
    }
});

// After: O(1) per check
const allowedIds = new Set(getAllowedIds());
requests.forEach(req => {
    if (allowedIds.has(req.userId)) {
        // grant access
    }
});

A Set is built specifically for membership checks. .has() does not walk the array looking for a match. It hashes the value and jumps straight to the answer.

Python: swap the list for a set

# Before: O(n) per check
banned_users = get_banned_user_list()  # a list
for request in all_requests:
    if request['user_id'] in banned_users:
        # reject
        pass

# After: O(1) per check
banned_users = set(get_banned_user_list())
for request in all_requests:
    if request['user_id'] in banned_users:
        # reject
        pass

The code barely changes. The only difference is the data type, a list swapped for a set. Python's in operator behaves completely differently depending on which one you hand it. Against a list, it is a linear scan. Against a set, it is a hash lookup. Same keyword, same syntax, a completely different performance story.

In every one of these three languages, the fix is not clever. It is not advanced. It takes one line to build the right data structure once, before the loop, instead of forcing the search to happen over and over inside it. That single change is often the difference between a feature that scales quietly and one that pages the on call engineer at 2am.

Why This Crime Keeps Happening

The honest answer is that in_array(), .includes(), and in are not wrong functions. They exist for good reasons and they are perfectly fine for small, fixed size lists. The mistake is not the function itself. It is using it without asking a simple question first: how big can this list get, and how many times am I about to search it.

That single question, asked at the moment you write the loop instead of the moment your server starts timing out, is the difference between code that merely works and code that actually holds up once real traffic arrives. It is a small habit, but it is the kind of habit that separates developers who ship features from developers whose features survive contact with production.

This exact pattern, a familiar, harmless looking line quietly turning into a performance or security liability once real scale hits it, is the entire premise behind Code Crimes: Security & Performance Mistakes in Modern Code. The book walks through more than two hundred real vulnerability and performance case studies across Python, PHP, and JavaScript, the same three languages covered here, each one broken down into what the mistake actually is, what it costs in a real production system, and the professional fix a senior engineer would reach for.

Linear search bottlenecks like this one are just a single chapter's worth of crimes. The book covers dozens more, from insecure defaults to N plus one queries to authentication shortcuts that quietly leave a door open.

You can read more about the book here: Code Crimes: The Book Every Developer Needs Before Their Next Code Review

Or grab it directly on Amazon: https://www.amazon.com/dp/B0H678BFCK

If a single line like in_array() inside a loop can cost your app seconds per request at scale, it is worth asking what else is quietly hiding in your codebase, waiting for the traffic to arrive that finally exposes it.