Using Python set() to make Django queries efficient

Recently, I worked on cleaning up some Django code that was running slower than it should. The task was simple: check if a folder was part of a user’s home folders. But the way it was written caused unnecessary delays. Here’s the process I followed to fix it.

---

The Problem: Repeating Database Queries

In the original code, a function checked if a folder was in the user’s home folders by calling a database query every time it ran. It looked like this:

def is_a_home_folder(folder):
    if folder in get_user_home_folders():
        return True
    return False

This function was part of a larger operation:

def can_delete_element(request, element, from_tag=False):
    if is_a_home_folder(element):
        return False
    # Other checks...

For a system with a lot of elements, running a query every time made the program slow. It became clear that this could be improved.

---

The Fix: Cache the Data

To speed things up, I changed the code to fetch all the home folders once and reuse that data throughout the function. Here’s the updated version:

def is_a_home_folder(folder, home_folders):
    return folder in home_folders

def can_delete_element(request, element, from_tag=False):
    # Fetch the home folders once
    home_folders = set(get_user_home_folders())

    if is_a_home_folder(element, home_folders):
        return False
    # Other checks...

---

What This Changes

1. Fewer Queries: Instead of querying the database each time, the home folders are fetched once at the start.
2. Faster Checks: Using a set instead of a list makes lookups much faster.
3. Easier to Read: Separating data fetching from the main logic makes the function more straightforward.

---

Why This Works

The logic here is simple: avoid doing the same work over and over. If you can fetch data once and reuse it, you save time and make the code easier to maintain. This doesn’t just apply to database queries—it’s useful for any repetitive task.

---

Conclusion

By making small adjustments like this, you can improve both performance and clarity in your code. It’s a good reminder that even simple changes can have a big impact. Have you run into similar issues while coding? Let me know how you handled it.