DEV Community: Egor

Claude vs Gemini vs GPT: What a 2026 Study of 26 AI Models Reveals About Long Context

Egor — Wed, 14 Oct 2026 14:28:14 +0000

26 AI models. 6,438 evaluation instances. Contexts up to 1 million tokens. The results challenge how we compare modern language models.

Introduction

Imagine giving an AI assistant an entire software repository.

Thousands of files. Hundreds of modules. API documentation, database schemas, and years of development history.

The model accepts everything without complaining.

You ask it to identify a subtle architectural problem.

It confidently provides an answer.

But is that answer actually reliable?

In 2026, the question is no longer simply whether a language model can accept a million tokens.

The more interesting question is:

How much of that context can the model actually use correctly?

A recent research paper, ATLAS: All-round Testing of Long-context Abilities across Scales, provides some surprising answers.

The researchers evaluated 26 models on 6,438 instances across eight capability dimensions, considering input lengths from 8K to 1M tokens.

One finding stands out:

Gemini 3.1 Pro Preview led at 128K, but Claude Opus 4.6 led at 1M.

The best model depends on how you measure it.

Let's examine what the researchers tested, what their results actually mean, and why these findings matter for developers building real AI systems.

Research note: This article analyzes published 2026 research. It is not an independent benchmark, and it does not claim that the author personally tested all 26 models.

1. The Problem With AI Benchmarks

When comparing language models, we often see a familiar table:

Model	Context window
Model A	128K
Model B	256K
Model C	1M

The implication seems obvious.

Model C can process more information, so it must be better at large-document tasks.

But this reasoning contains a hidden assumption:

That accepting more information means using that information effectively.

Consider two different tasks.

Task A: Simple retrieval

A document contains the sentence:

"The deployment identifier is QZ-7194."

The model is asked to retrieve that identifier.

Task B: Complex reasoning

A repository contains dozens of service definitions, configuration files, and dependency relationships.

The model must identify which changes could introduce a security regression.

The second task requires much more than finding one string.

It may involve combining evidence from many locations, resolving contradictions, and understanding dependencies.

A model that performs well on Task A might struggle on Task B.

That's exactly why evaluating long-context AI requires more than a single benchmark number.

2. ATLAS: A Different Way to Measure Long-Context Intelligence

Research: ATLAS: All-round Testing of Long-context Abilities across Scales

Authors: Deli Huang and colleagues

Published: May 27, 2026

Paper: https://arxiv.org/abs/2605.28079

The ATLAS researchers investigated two major weaknesses in traditional long-context evaluation.

First, model performance can deteriorate as input length increases.

Second, success on foundational tasks such as retrieval does not necessarily transfer to more complex application workloads.

Their framework introduces several methodological improvements.

Evaluating across context lengths

Instead of measuring performance at only one input size, ATLAS considers scores across a predefined context-length grid, from 8K to 1M tokens.

This makes it possible to analyze how performance changes as more information enters the prompt.

A model that performs exceptionally well at 128K may deteriorate more sharply than another model at longer lengths.

Separating foundational abilities from applications

ATLAS distinguishes foundational operations from downstream workloads.

This matters because retrieving information and applying it correctly are different capabilities.

For instance, a model may identify relevant functions in a repository without understanding how they interact.

Measuring balanced performance

The researchers introduce ATLAScore, a harmonic-mean-based aggregate across capability categories.

This is intended to penalize highly imbalanced capability profiles.

A model that dominates one task but struggles on others should not automatically receive the strongest overall evaluation.

The framework also includes uncertainty propagation when computing its final aggregate scores.

Together, these changes provide a more nuanced picture than a single retrieval accuracy number.

3. The Result: The Leader Changes at 1 Million Tokens

One of the most interesting findings concerns model rankings.

According to the ATLAS paper:

Evaluation scope	Leading model
Up to 128K tokens	Gemini 3.1 Pro Preview
Up to 1M tokens	Claude Opus 4.6

This does not mean Claude is universally better than Gemini.

Nor does it mean Gemini is inherently unreliable.

It means the relative ranking changes under the specific evaluation conditions used by ATLAS.

The researchers also reported that seven models changed position by at least two ranking places when the evaluation expanded from the 8K–128K range to the 8K–1M range.

That's important.

Imagine selecting a model for your application after reviewing results obtained only at 128K.

Your production workload, however, frequently processes 600K-token documents.

The benchmark you used may not reflect your actual operating conditions.

And if models deteriorate at different rates, their relative performance at 600K may differ from their relative performance at 128K.

The key insight: model quality is not independent of context length.

A useful benchmark should measure the range your application actually uses.

4. Why Context Length Changes the Difficulty

Let's examine a simplified example.

Imagine this information appears inside a long repository:

service_a.py:
Service A sends requests to Service B.

service_b.py:
Service B uses Database C.

config.yaml:
Database C has public network access enabled.

security_policy.md:
Production databases must not expose public access.

Now ask:

"Does Service A indirectly depend on a database with a potentially unsafe network configuration?"

Answering correctly requires several steps.

First, identify the dependency between Service A and Service B.

Second, identify Service B's database.

Third, find Database C's configuration.

Finally, compare that configuration with the security policy.

No single sentence contains the entire answer.

As the context grows, the model must identify relevant information among more irrelevant material while preserving the connections between different pieces of evidence.

This creates distinct possible failure modes:

Missing one of the necessary pieces of evidence
Retrieving a similar but incorrect detail
Correctly retrieving the evidence but connecting it incorrectly
Producing a plausible conclusion unsupported by the source material

The example is illustrative, not an ATLAS test case.

But it demonstrates why longer context can increase difficulty even when every relevant document fits into the model's advertised context window.

5. Another 2026 Study Reveals a Deeper Problem

ATLAS evaluates long-context capabilities across models and scales.

A separate 2026 study, CorpusQA: A 10 Million Token Benchmark for Corpus-Level Analysis and Reasoning, investigates a different question:

Can AI systems analyze entire collections of documents when the evidence is distributed throughout the collection?

Authors: Zhiyuan Lu and colleagues

Originally published: January 21, 2026

Revised: April 25, 2026

Paper: https://arxiv.org/abs/2601.14952

CorpusQA includes tasks involving financial, educational, and real-estate documents.

Some questions require aggregating numerical information from many documents.

For example, imagine analyzing hundreds of financial reports to identify every company satisfying several accounting conditions.

A traditional retrieval system might find a few relevant documents.

But if answering the question requires analyzing every company, retrieving only five documents is insufficient.

This is a fundamentally different problem from finding one answer-bearing passage.

The published results

CorpusQA evaluated several models at different context lengths.

For Gemini 2.5 Pro, the authors reported:

Context length	Overall accuracy
128K	80.19%
1M	50.73%

These results show a decline of 29.46 percentage points under the study's conditions.

Importantly, CorpusQA evaluated Gemini 2.5 Pro, not Gemini 3.1 Pro.

Its results cannot be directly substituted for the newer model's performance.

But they illustrate how difficult corpus-level analysis can remain, even when a model supports very long inputs.

The study also found that conventional retrieval-augmented generation performed poorly on tasks requiring globally distributed evidence.

The lesson isn't that RAG is useless.

It's that retrieving a handful of relevant chunks is not the same as analyzing an entire document collection.

6. Why Traditional RAG Can Fail on Global Questions

A typical RAG pipeline looks like this:

User question
      |
      v
Vector search
      |
      v
Top-K relevant chunks
      |
      v
LLM
      |
      v
Generated answer

This works well when a question can be answered using a relatively small number of relevant passages.

For example:

"What is the API rate limit?"

A good retrieval system can locate the appropriate documentation paragraph and pass it to the model.

Now consider another question:

"Across all customer reports from the last year, which failure category increased the most?"

The answer requires a global aggregation.

Retrieving only the most semantically similar documents may exclude a large part of the dataset.

No matter how capable the LLM is, it cannot correctly aggregate information it never receives.

This is an information-coverage problem, not merely a language-model problem.

In the CorpusQA evaluation at 1M tokens, standard RAG achieved 17.38% accuracy, compared with 44.95% for the Memory Agent approach using Gemini 2.5 Pro.

This illustrates how different approaches to managing large document collections can significantly affect performance.

A better architecture for some analytical tasks

For questions requiring exact aggregation, a structured processing pipeline may be more appropriate:

Document collection
        |
        v
Information extraction
        |
        v
Structured records
        |
        v
SQL / analytical computation
        |
        v
Evidence verification
        |
        v
LLM-generated explanation

This design separates factual computation from language generation.

The LLM can interpret the user's question and explain the result, while deterministic code handles calculations.

Of course, the approach has its own limitations.

Extraction errors, incomplete schemas, and stale data can still produce incorrect answers.

But it creates opportunities to validate individual steps instead of relying entirely on one enormous prompt.

7. What About Vendor Benchmarks?

Independent research is only one source of evidence.

Model developers also publish technical evaluations.

For example, Anthropic's Claude Opus 4.6 System Card discusses long-context benchmarks such as MRCR and GraphWalks.

Source: https://www.anthropic.com/system-cards

MRCR evaluates the ability to locate and distinguish between similar pieces of information across long inputs.

GraphWalks examines structured reasoning over graph-like information represented within the context.

These evaluations are useful because they test capabilities beyond ordinary question answering.

However, developer-published benchmark results need careful interpretation.

Providers may use different inference settings, reasoning budgets, input limits, filtering rules, or evaluation configurations.

A benchmark score obtained under one provider's setup should not automatically be treated as directly comparable with an independent benchmark.

In particular, the ability to answer a question using a long prompt depends on more than the model checkpoint.

It can also depend on:

Reasoning budget
Tool usage
Prompt construction
Context truncation
Sampling configuration
Dataset filtering
Evaluation metrics

These differences can materially affect observed results.

That's one reason ATLAS is interesting: its evaluation explicitly attempts to compare long-context capabilities across multiple tasks and lengths.

8. What This Means for AI Coding Assistants

Consider an AI coding assistant with access to a large repository.

There are two possible designs.

Approach A: Put everything in the prompt

Entire repository
       |
       v
Large context window
       |
       v
LLM response

This is conceptually simple.

But it can be expensive and may introduce large amounts of irrelevant information.

It also doesn't guarantee that the model will correctly connect all important details.

Approach B: Search, inspect, and verify

User question
      |
      v
Repository search
      |
      v
Relevant files
      |
      v
Dependency analysis
      |
      v
LLM reasoning
      |
      v
Tool-based verification
      |
      v
Final answer

This approach can use search tools, static analysis, tests, and targeted context retrieval.

For some software engineering tasks, those tools may provide more reliable evidence than simply expanding the prompt.

For instance, when checking whether a function is actually called, repository search can identify references.

When checking whether a proposed patch works, an automated test can provide execution evidence.

An LLM can assist with interpretation, but it doesn't have to perform every operation internally.

The broader principle is:

Use the context window as one resource in an engineering system, not as a replacement for the system itself.

9. How I Would Benchmark Modern AI Models

The published results suggest a practical evaluation protocol for comparing models on real development workflows.

I would test each model across four dimensions.

Test A: Context length

Evaluate the same task family at:

8K
32K
128K
256K
512K
1M

Only include lengths genuinely supported by each tested model and API configuration.

Test B: Evidence position

Place answer-relevant information near the beginning, middle, and end of the input.

Measure whether accuracy changes.

Test C: Reasoning complexity

Use several difficulty levels:

Level 1: Retrieve one explicitly stated fact.

Level 2: Connect information from two files.

Level 3: Combine evidence from several documents.

Level 4: Perform global analysis across an entire corpus.

Test D: Tool-assisted reasoning

Compare direct long-context prompting against workflows using retrieval, code execution, or other verification tools.

For every experiment, record:

Metric	Why it matters
Answer accuracy	Measures correctness
Context length	Reveals scale sensitivity
Input/output tokens	Measures resource usage
Latency	Measures responsiveness
Cost	Measures practical efficiency
Evidence completeness	Checks whether necessary information was found

Model versions and inference settings should be fixed and documented.

Tasks should be repeated using multiple independent examples, with uncertainty estimates reported alongside aggregate scores.

This is a proposed experimental design, not a claim that these tests have already been run for this article.

10. Limitations and Research Caveats

There are several reasons to interpret these findings carefully.

First, ATLAS is a May 2026 preprint. Its results are useful research evidence, but they should not be mistaken for a universally accepted model ranking.

Second, the comparison covers a specific collection of 26 models and evaluation settings. It does not establish the performance of every model released later in 2026.

Third, CorpusQA evaluates a different set of models and tasks. Its numerical results cannot be directly compared with ATLAS scores.

Fourth, performance at 1M tokens may involve implementation-specific constraints, including inference settings and context handling.

Finally, benchmark results do not automatically predict real-world performance.

An AI system answering legal questions, writing code, analyzing scientific papers, or processing financial records may require very different capabilities.

The correct model and architecture should be selected through task-specific evaluation.

Conclusion: We Need Better Questions Than "How Many Tokens?"

The rapid growth of context windows is an important technological achievement.

But the 2026 research tells a more complicated story.

The ATLAS study found that model rankings change when evaluation extends toward one million tokens.

CorpusQA demonstrated the difficulty of analyzing large document collections when relevant evidence is dispersed across many sources.

Together, these studies suggest that increasing context capacity alone is not enough to guarantee reliable information processing.

For developers, the practical implications are straightforward:

Don't select an LLM based only on advertised context size.

Evaluate it at the lengths your application actually uses.

Test retrieval and reasoning separately.

Measure the quality of evidence, not only the fluency of the generated answer.

And consider whether structured tools, retrieval systems, or deterministic computation can make your application more reliable.

The future of long-context AI may not be about putting everything into one prompt.

It may be about building systems that know what to retrieve, what to remember, what to compute, and what to verify.

A million-token context window is a capability.

Reliable reasoning over a million tokens is a different achievement.

References

[1] ATLAS — All-round Testing of Long-context Abilities across Scales (2026)

Deli Huang et al.

https://arxiv.org/abs/2605.28079

[2] CorpusQA — A 10 Million Token Benchmark for Corpus-Level Analysis and Reasoning (2026)

Zhiyuan Lu et al.

https://arxiv.org/abs/2601.14952

[3] Claude Opus 4.6 System Card (2026)

Anthropic.

https://www.anthropic.com/system-cards

Discussion

Do you think future AI systems will rely primarily on larger context windows, or will retrieval, memory, and tool-assisted reasoning become more important?

And have you encountered cases where an LLM missed important information even though it was present in the prompt?

I'd be interested in hearing about real-world experiences.

GitHub: https://github.com/coffee1337

Portfolio: https://coffee1337.github.io/

I Made Python 2,000x Faster by Changing One Line of Code

Egor — Wed, 14 Oct 2026 08:33:28 +0000

No caching. No multiprocessing. No C extensions. Just choosing the right data structure.

Python has a reputation for being slow. But sometimes the problem isn't Python itself: it's the data structure we choose.

I compared two ways to answer one question: is this ID already in the collection? In one benchmark run, changing the collection from a list to a set made repeated unsuccessful lookups about 2,378 times faster.

This isn't a claim about general API performance. It's a specific, reproducible experiment. Let's see why it happens.

The problem

Imagine a service that holds 10,000 integer IDs and needs to check thousands of incoming IDs.

user_ids = list(range(10_000))

def user_exists(user_id):
    return user_id in user_ids

This is correct code. But how does it behave when most lookups miss?

The benchmark

Here's the complete benchmark (also available as a separate .py file):

import platform
import statistics
import timeit

N = 10_000
Q = 5_000

ids_list = list(range(N))
ids_set = set(ids_list)
queries = range(N, N + Q)  # All queries are misses.


def using_list():
    return sum(q in ids_list for q in queries)


def using_set():
    return sum(q in ids_set for q in queries)


assert using_list() == using_set() == 0


def benchmark(func):
    timings = timeit.repeat(func, number=1, repeat=7)
    return statistics.median(timings)


list_time = benchmark(using_list)
set_time = benchmark(using_set)

print(f"Python: {platform.python_version()}")
print(f"List: {list_time * 1000:.3f} ms")
print(f"Set:  {set_time * 1000:.3f} ms")
print(f"Speedup: {list_time / set_time:,.0f}x")

The collection contains 10,000 integer IDs. Every one of the 5,000 queries is absent, so this tests a particularly unfavorable case for a list. The structures are built before timing begins. Each function is run seven times, and the median is reported.

The results

On the test environment (Python 3.13.5), I measured:

Approach	Median time
`list`	350.413 ms
`set`	0.147 ms
Ratio	~2,378×

These are measurements from one execution environment, not universal performance constants. Rerunning the test may produce different values.

Why does `list` take longer?

For a list, x in values scans its elements until it finds x or reaches the end. Since all our test queries miss, each check requires a full scan: O(n) per lookup.

By contrast, a set uses hashing. For ordinary integer keys and typical hashing behavior, its membership check is O(1) on average. That does not mean sets can never slow down; collisions and unusual inputs matter.

The one-line change

# Before
user_ids = list(range(10_000))

# After
user_ids = set(range(10_000))

The membership expression stays identical:

if user_id in user_ids:
    ...

The algorithmic cost, however, is different.

Don't replace every list with a set

Sets have trade-offs:

They don't keep duplicate elements.
They don't support positional indexing.
Their elements must be hashable.
They typically use more memory.
Constructing a set takes time, so the conversion may not pay off for a single lookup.

Also, the test deliberately uses only missing IDs. A list can find an element at the beginning quickly, and on tiny collections a list may be perfectly adequate. Ordering, duplicate values, and memory use may matter more than lookup speed.

The real takeaway

Before adding threads, caches, or new infrastructure, ask whether your hot path uses the right data structure.

This benchmark does not show that a FastAPI endpoint will become 2,000× faster. It shows that repeated in-memory membership checks can differ dramatically depending on the collection type and workload.

Measure your actual bottleneck, then optimize it.

What is the biggest speedup you've achieved by changing a surprisingly small amount of code?

GitHub · Portfolio

This 3-Line Python Mistake Can Disable Your Entire API Authentication

Egor — Tue, 13 Oct 2026 18:24:18 +0000

Ever written a security check that looked perfectly reasonable... until you realized it could let everyone in?

Here's a tiny Python mistake that can turn an API key check into an authentication bypass.

No complicated exploit. No special tools.

Just a missing environment variable.

The innocent-looking code

Imagine a FastAPI application protecting an internal endpoint with an API key.

import os

from fastapi import FastAPI, Header, HTTPException

app = FastAPI()

API_KEY = os.getenv("API_KEY", "")


@app.get("/admin")
async def admin(x_api_key: str = Header(default="")):
    if API_KEY and x_api_key != API_KEY:
        raise HTTPException(status_code=401)

    return {"message": "Welcome, admin!"}

Looks reasonable, right?

We read the secret from an environment variable, compare it against the incoming header, and reject unauthorized requests.

Except there's one pretty serious problem.

What happens if API_KEY isn't configured?

The bug hiding in plain sight

Look closely at this condition:

if API_KEY and x_api_key != API_KEY:

Python's and operator short-circuits.

When API_KEY is an empty string, the first condition is false.

So Python never evaluates the comparison.

And the application never raises the authentication error.

That means this request:

curl http://localhost:8000/admin

Can return:

{
  "message": "Welcome, admin!"
}

No API key required.

The authentication mechanism has silently failed open.

And the scariest part?

The application can start successfully, pass basic health checks, and expose the protected endpoint without an obvious error.

Fail-open vs. fail-closed

This is the difference between two important security concepts.

Fail-open: When something goes wrong, access is allowed.

Fail-closed: When something goes wrong, access is denied.

Authentication should generally fail closed.

If your application cannot load its authentication secret, it should not simply allow requests through.

The absence of a security configuration should be treated as an error, not permission.

Fix #1: Reject missing configuration

Instead of silently providing an empty default:

API_KEY = os.getenv("API_KEY", "")

Require the environment variable:

API_KEY = os.environ["API_KEY"]

if not API_KEY.strip():
    raise RuntimeError("API_KEY must not be empty")

Now the application fails during startup if the API key is missing or blank.

This is a much safer failure mode.

An unavailable service is inconvenient.

An accidentally unprotected service can be disastrous.

Fix #2: Compare secrets safely

For secret comparisons, Python provides hmac.compare_digest().

Here's a better implementation:

import os
import hmac

from fastapi import FastAPI, Header, HTTPException

app = FastAPI()

API_KEY = os.environ["API_KEY"]

if not API_KEY.strip():
    raise RuntimeError("API_KEY must not be empty")


@app.get("/admin")
async def admin(x_api_key: str | None = Header(default=None)):
    if x_api_key is None or not hmac.compare_digest(
        x_api_key, API_KEY
    ):
        raise HTTPException(
            status_code=401,
            detail="Unauthorized",
        )

    return {"message": "Welcome, admin!"}

This version improves several things:

Missing configuration prevents successful startup.
Missing credentials are rejected.
Incorrect credentials are rejected.
Secret comparison uses a function designed to avoid content-dependent short-circuit comparisons.

Constant-time comparison is useful defense in depth, but it doesn't replace TLS, proper secret management, access controls, or rate limiting.

Fix #3: Test the failure, not just the success

Developers often test authentication with a valid API key.

Maybe they also test an incorrect key.

But how often do they test a missing configuration?

Here's a minimal example of a regression test for the original logic:

def vulnerable_check(expected: str, provided: str) -> bool:
    if expected and provided != expected:
        return False

    return True


def test_missing_secret_must_not_grant_access():
    assert vulnerable_check("", "") is False

This test fails against the vulnerable implementation.

And that's exactly what we want it to reveal.

The corrected standalone version:

import hmac


def secure_check(expected: str, provided: str) -> bool:
    if not expected.strip():
        return False

    return hmac.compare_digest(expected, provided)


def test_missing_secret():
    assert secure_check("", "") is False


def test_wrong_secret():
    assert secure_check("secret123", "wrong") is False


def test_correct_secret():
    assert secure_check("secret123", "secret123") is True

These tests cover the comparison logic. An actual FastAPI application should also have integration tests proving that missing configuration prevents startup and unauthenticated requests return HTTP 401.

The bigger lesson

This isn't really about FastAPI.

The same mistake can appear in:

Django applications
Flask APIs
Microservices
Webhook authentication
Internal admin dashboards
Background worker APIs

And it's not limited to API keys.

Any security feature that depends on optional configuration deserves a careful review.

For example:

if SECRET and validate_token(token):
    ...

Or:

if ENABLE_AUTH:
    check_permissions()

What happens when those values are missing, empty, or incorrectly configured?

The important question isn't just:

"Does authentication work when everything is configured correctly?"

It's also:

"What happens when the security configuration breaks?"

Final thoughts

Not every serious security bug requires complex cryptography or sophisticated attacks.

Sometimes all it takes is an empty string and one if statement.

Three things worth remembering:

Never silently disable authentication because a secret is missing.
Validate critical security configuration during application startup.
Write tests for failure conditions, not only successful requests.

Secure software isn't only about handling the expected path correctly.

It's also about making sure the unexpected path doesn't become an open door.

Have you ever discovered a security bug caused by a seemingly harmless default value?

Share it in the comments. I'd love to hear about it.

If you enjoy Python, backend architecture, and application security, feel free to connect:

GitHub: https://github.com/coffee1337

Portfolio: https://coffee1337.github.io/

Your FastAPI App Is Async. So Why Is It Still Slow?

Egor — Tue, 13 Oct 2026 10:03:12 +0000

Your FastAPI App Is Async. So Why Is It Still Slow? 🐍

Understanding the difference between writing async code and building an actually concurrent Python backend.

You built your API with FastAPI.

You used async def everywhere.

You even added await to your database calls.

So why does one slow request still seem to affect everything else?

Here's something that surprises a lot of Python developers:

Using async doesn't automatically make your application non-blocking.

And in some cases, a perfectly valid asynchronous endpoint can make your application perform worse than a regular synchronous one.

Let's explore why.

1. The `async def` Illusion

Consider this FastAPI endpoint:

from fastapi import FastAPI
import time

app = FastAPI()

@app.get("/slow")
async def slow_endpoint():
    time.sleep(5)
    return {"message": "Done"}

At first glance, everything looks fine.

The endpoint uses async def, just like many FastAPI examples.

But there's a problem:

time.sleep(5)

This call blocks the thread running the event loop.

While it executes, the event loop cannot make progress on other coroutines running on that same thread.

If multiple requests arrive, they can end up waiting for each other.

The endpoint is declared asynchronous, but its implementation blocks asynchronous execution.

The fix

Replace the blocking call with an asynchronous one:

import asyncio
from fastapi import FastAPI

app = FastAPI()

@app.get("/slow")
async def slow_endpoint():
    await asyncio.sleep(5)
    return {"message": "Done"}

Now something important changes.

When Python reaches:

await asyncio.sleep(5)

The coroutine suspends and gives control back to the event loop.

Other requests can make progress while the first one waits.

This is the foundation of asynchronous programming.

2. Concurrency Is Not Parallelism

This distinction matters.

Concurrency means multiple tasks can make progress during overlapping periods.

Parallelism means multiple tasks actually execute at the same time, typically on different CPU cores.

AsyncIO is primarily about cooperative concurrency.

Imagine three API requests, each waiting two seconds for an external service.

With blocking operations, a single-threaded execution model might look like this:

Request A: [----2s----]
Request B:             [----2s----]
Request C:                         [----2s----]

Total: ~6 seconds

With asynchronous waiting:

Request A: [----2s----]
Request B: [----2s----]
Request C: [----2s----]

Total: ~2 seconds

These are simplified timelines, not benchmark results. Real application performance also depends on worker configuration, connection pools, infrastructure, and external services.

The key idea is that asynchronous tasks can overlap their waiting time.

But what happens when the work is CPU-intensive?

That's where things get interesting.

3. AsyncIO Won't Magically Speed Up CPU-Heavy Code

Imagine your API needs to perform an expensive calculation.

def expensive_calculation():
    total = 0

    for i in range(30_000_000):
        total += i * i

    return total

Now put it inside an async endpoint:

@app.get("/calculate")
async def calculate():
    result = expensive_calculation()
    return {"result": result}

This is still blocking.

Why?

Because expensive_calculation() is regular Python code that continuously uses CPU time.

There is no asynchronous waiting point where the coroutine can yield control.

Adding async to the endpoint doesn't transform CPU-bound work into non-blocking work.

A better approach

For CPU-intensive operations, you may need:

A separate process
A process pool
A background job system
A dedicated worker service

For example, Python's ProcessPoolExecutor can execute work in separate processes.

import asyncio
from concurrent.futures import ProcessPoolExecutor

from fastapi import FastAPI

app = FastAPI()

pool = ProcessPoolExecutor(max_workers=2)


def expensive_calculation():
    return sum(i * i for i in range(30_000_000))


@app.get("/calculate")
async def calculate():
    loop = asyncio.get_running_loop()

    result = await loop.run_in_executor(
        pool,
        expensive_calculation,
    )

    return {"result": result}

The CPU-heavy function runs outside the event-loop process, allowing the event loop to continue serving other requests.

This is a simplified example. In a real deployment, the process pool should be managed as part of the application's lifecycle, and worker counts should be chosen based on available resources.

For expensive or long-running jobs, returning a task ID and processing work asynchronously in a dedicated worker is often a better API design.

4. The Hidden Problem: Synchronous HTTP Clients

Here's another common mistake.

import requests

@app.get("/users")
async def get_users():
    response = requests.get(
        "https://jsonplaceholder.typicode.com/users",
        timeout=10,
    )

    return response.json()

The requests library performs synchronous HTTP operations.

So even though the endpoint is asynchronous, the outgoing request blocks the event-loop thread until it finishes.

A better choice for async endpoints is an asynchronous HTTP client.

For example, HTTPX:

import httpx

@app.get("/users")
async def get_users():
    async with httpx.AsyncClient(
        timeout=10.0
    ) as client:
        response = await client.get(
            "https://jsonplaceholder.typicode.com/users"
        )

        response.raise_for_status()

        return response.json()

Now the endpoint can yield control while waiting for the external API.

For production applications, it's usually better to reuse an AsyncClient rather than create a new one for every request.

Reusing the client allows connection pooling and reduces unnecessary connection overhead.

5. What About Database Queries?

The same principle applies to databases.

An asynchronous endpoint doesn't automatically make database access asynchronous.

For example:

@app.get("/users/{user_id}")
async def get_user(user_id: int):
    user = sync_db_query(user_id)
    return user

If sync_db_query() blocks, your event loop is blocked.

With SQLAlchemy's asynchronous API, database operations can look like this:

from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession


async def get_user(
    session: AsyncSession,
    user_id: int,
):
    result = await session.execute(
        select(User).where(User.id == user_id)
    )

    return result.scalar_one_or_none()

This example assumes that User is a defined SQLAlchemy model and the session uses an asynchronous database driver.

However, there's another important detail:

Async database access doesn't make slow SQL queries fast.

If a query takes three seconds because of missing indexes, inefficient joins, or a poor execution plan, adding await won't fix that.

Async improves how your application handles waiting.

Database optimization improves how quickly that waiting ends.

Both matter.

6. More Concurrency Can Actually Make Things Worse

Suppose your API suddenly receives 1,000 requests.

Each request needs a database connection.

Your PostgreSQL connection pool supports 20 simultaneous connections.

What happens?

The application cannot magically execute 1,000 database queries concurrently using only 20 available connections.

Some requests must wait for a connection to become available.

And increasing the pool size isn't always the right answer.

Too many connections can overload the database and reduce overall throughput.

This is where concepts like backpressure, rate limiting, bounded concurrency, and connection pooling become important.

For example, you can limit concurrent operations using an asyncio.Semaphore.

import asyncio

semaphore = asyncio.Semaphore(10)


async def fetch_external_data(client, url):
    async with semaphore:
        response = await client.get(url)
        response.raise_for_status()
        return response.json()

This example limits calls using that semaphore to ten concurrent operations within the process.

It doesn't create a global limit across multiple application workers or servers.

That's an important difference in distributed deployments.

7. How to Know If Your API Is Actually Fast

One of the biggest mistakes in backend development is optimizing code without measuring anything.

An endpoint taking 100 milliseconds doesn't tell you much by itself.

You should also consider:

Latency

How long does an individual request take?

Throughput

How many requests can your application handle per second?

P95 and P99 latency

How long do the slowest 5% and 1% of requests take?

Error rate

Does your application start returning errors under load?

Resource utilization

Are you constrained by CPU, memory, database connections, or an external API?

Tools such as Locust, k6, and Prometheus can help you investigate these questions.

The important part is testing under realistic workloads.

A backend that responds quickly to one request may behave completely differently when hundreds of users arrive simultaneously.

8. A Practical Rule for FastAPI Developers

Here's a mental model worth remembering:

Workload	Common approach
Async HTTP requests	`httpx.AsyncClient`
Async database operations	Async driver + SQLAlchemy AsyncSession
Waiting for timers	`asyncio.sleep()`
Blocking I/O library	Sync endpoint or thread pool
CPU-intensive calculation	Process pool or dedicated worker
Long-running background job	Job queue + worker
High-volume external API calls	Connection pooling + bounded concurrency

These are guidelines, not universal rules.

For example, FastAPI can execute regular def path operations in a thread pool, which is useful when integrating synchronous libraries.

But thread pools have limits, too.

The goal isn't to use async everywhere.

The goal is to choose the right execution model for your workload.

Final Thoughts

AsyncIO is powerful, but it's easy to misunderstand.

A Python application doesn't become scalable simply because its functions contain async and await.

The real performance improvements come from understanding:

What blocks the event loop
Where your application spends time waiting
Which operations are CPU-bound
How database and HTTP connection pools behave
When concurrency needs to be limited
How your application performs under realistic load

The best backend architecture isn't the one with the most asynchronous code.

It's the one that understands where the bottlenecks are.

And sometimes, replacing a single blocking function can make a bigger difference than rewriting an entire service.

💬 Let's Discuss

What's the most unexpected performance bottleneck you've encountered in a Python backend?

Was it a blocking library, a database query, an external API, or something completely different?

I'd love to hear your experience in the comments.

Interested in Python backend development, FastAPI, and building scalable applications?

GitHub: https://github.com/coffee1337
Portfolio: https://coffee1337.github.io/

Thanks for reading! 🚀

AI Python Mentor

Egor — Mon, 12 Oct 2026 15:01:18 +0000

Building an AI-Powered Python Learning Platform with FastAPI, Next.js, and RAG

Building an AI-powered application is relatively easy when all you need is a chat interface connected to an LLM API.

Building a complete learning platform around that AI is a different challenge.

I wanted to explore how to combine a structured curriculum, practical programming exercises, personalized learning, and an AI tutor into a single full-stack system.

That's how AI Python Mentor started.

In this article, I'll walk through the architecture, technology choices, AI integration, and some of the engineering challenges behind the project.

GitHub: https://github.com/coffee1337/ai-python-mentor

The Idea

AI Python Mentor is a full-stack platform designed to help users learn Python and backend development.

Instead of simply providing AI-generated explanations, the application combines several learning mechanisms:

A structured curriculum covering Python and backend concepts
Programming exercises and knowledge checks
An AI tutor with access to lesson context
Personalized learning roadmaps
Progress tracking and spaced repetition
Career-oriented skill gap analysis
Practical portfolio projects

The goal is to build an environment where AI supports structured learning rather than replacing it.

Technology Stack

The application uses:

Backend

Python 3.12
FastAPI
PostgreSQL
SQLAlchemy and Alembic
Pydantic
AsyncIO

Frontend

Next.js 15
React 19
TypeScript

LLM Gateway
Retrieval-Augmented Generation (RAG)
Lesson-based knowledge indexing
Token usage and cost accounting

Infrastructure

Docker and Docker Compose
GitHub Actions
Background workers
Database migrations

I chose FastAPI because it provides a solid foundation for asynchronous API development, explicit validation, and modular backend design.

PostgreSQL handles relational data, while SQLAlchemy and Alembic manage database access and schema evolution.

System Architecture

The application follows a modular monolith approach.

The core API is responsible for authentication, learning progress, lessons, exercises, and integrations.

Specialized tasks can be handled by separate processes.

                    Browser
                       |
                       v
                 Next.js Frontend
                       |
                       v
                  FastAPI API
                       |
        +--------------+--------------+
        |              |              |
        v              v              v
    PostgreSQL     AI Gateway     Background
                       + RAG        Workers
        |
        +---- Learning Progress
        +---- User Sessions
        +---- Exercise Attempts
        +---- AI Usage Records

Separate execution architecture:
    Execution Dispatcher
              |
              v
       Isolated Python Runner

One of my architectural goals was to keep core learning features independent of the availability of external AI services.

For example, if the AI Gateway is unavailable, users should still be able to access lessons and other non-AI functionality.

This separation makes the application easier to maintain and evolve.

Building an AI Tutor with RAG

A general-purpose LLM can explain Python concepts, but it doesn't automatically know which lesson a student is studying or which version of the material is currently published.

That's where Retrieval-Augmented Generation becomes useful.

AI Python Mentor uses a local knowledge index built from immutable snapshots of published lesson content.

The basic flow is:

Student asks a question
          |
          v
Identify current lesson
          |
          v
Retrieve relevant lesson context
          |
          v
Build LLM request
          |
          v
Generate contextual explanation
          |
          v
Track usage and cost

By retrieving context from the learning materials, the AI tutor can ground its explanations in the content the student is actually studying.

This is particularly important for educational applications, where a technically correct answer may still be inappropriate for the learner's current level.

The platform also tracks AI usage and costs.

This is an important consideration when building products that rely on paid LLM APIs.

Handling Python Code Execution

Another challenge is supporting practical programming exercises.

Users submit Python solutions following a simple contract:

def solve(payload):
    # User implementation
    ...

Exercises can define public examples and hidden tests, while the platform stores submissions and attempt history.

However, executing untrusted Python code directly inside the main API would introduce significant security risks.

For that reason, the system separates execution dispatching from the primary application and defines a dedicated Python Runner architecture.

The production execution model is designed around isolation and resource restrictions, including container or sandbox boundaries.

Important: Untrusted code execution is disabled by default. The production Runner still requires the appropriate isolated deployment and security configuration.

The key design decision is simple: the main web API should never become the execution environment for arbitrary user code.

Authentication and Security

The backend includes several account security features:

Cookie-based sessions
Argon2 password hashing
CSRF protection
Email confirmation and password recovery
Session revocation
Account export and deletion
Administrative audit logging

These features are important because real applications require much more than a working login endpoint.

For production deployments, secure cookies and HTTPS are also required.

Background Processing

Not every operation should run directly inside an HTTP request.

The architecture includes background processing for workloads such as notifications and execution dispatching.

Separating those workloads helps keep request handling responsive and makes longer-running operations easier to manage.

Testing and CI

The repository includes automated checks for backend functionality, database migrations, frontend builds, worker logic, and browser flows.

GitHub Actions helps validate changes across these areas.

I also added support for native Windows development without requiring Docker for every local workflow.

That was useful because it made the development environment more accessible while keeping Docker-based deployment options available.

Current Status

AI Python Mentor is still under active development.

The core learning platform, backend, AI integration, database layer, and CI infrastructure have been implemented.

Current work focuses on product polish, learning content quality, AI evaluation, and production deployment.

I prefer being transparent about this: having a production-oriented architecture does not mean every planned production component has already been deployed.

What I Learned

This project reinforced several engineering lessons:

1. AI should be a component, not the entire architecture.

A useful AI product still needs reliable authentication, data models, application logic, and operational safeguards.

2. Structured context matters.

RAG is particularly valuable when answers should reflect specific, versioned learning materials.

3. Security boundaries should be designed early.

This is especially important when an application accepts executable code from users.

4. Operational concerns are product features.

Usage quotas, cost tracking, background jobs, and testing all contribute to the long-term viability of an AI application.

What's Next?

My next priorities include improving AI evaluation, polishing the user experience, refining educational content, and preparing the infrastructure for production deployment.

You can explore the code and architecture here:

GitHub: https://github.com/coffee1337/ai-python-mentor

Portfolio: https://coffee1337.github.io

I'm also open to freelance and contract projects involving Python backends, FastAPI, AI/LLM integrations, RAG systems, and SaaS MVP development.

If you're building something similar, feel free to reach out!

Tags: #python #ai #fastapi #webdev

DEV Community: Egor

Claude vs Gemini vs GPT: What a 2026 Study of 26 AI Models Reveals About Long Context

Introduction

1. The Problem With AI Benchmarks

2. ATLAS: A Different Way to Measure Long-Context Intelligence

Evaluating across context lengths

Separating foundational abilities from applications

Measuring balanced performance

3. The Result: The Leader Changes at 1 Million Tokens

4. Why Context Length Changes the Difficulty

5. Another 2026 Study Reveals a Deeper Problem

The published results

6. Why Traditional RAG Can Fail on Global Questions

A better architecture for some analytical tasks

7. What About Vendor Benchmarks?

8. What This Means for AI Coding Assistants

Approach A: Put everything in the prompt

Approach B: Search, inspect, and verify

9. How I Would Benchmark Modern AI Models

Test A: Context length

Test B: Evidence position

Test C: Reasoning complexity

Test D: Tool-assisted reasoning

10. Limitations and Research Caveats

Conclusion: We Need Better Questions Than "How Many Tokens?"

References

Discussion

I Made Python 2,000x Faster by Changing One Line of Code

The problem

The benchmark

The results

Why does list take longer?

The one-line change

Don't replace every list with a set

The real takeaway

This 3-Line Python Mistake Can Disable Your Entire API Authentication

The innocent-looking code

The bug hiding in plain sight

Fail-open vs. fail-closed

Fix #1: Reject missing configuration

Fix #2: Compare secrets safely

Fix #3: Test the failure, not just the success

The bigger lesson

Final thoughts

Your FastAPI App Is Async. So Why Is It Still Slow?

Your FastAPI App Is Async. So Why Is It Still Slow? 🐍

1. The async def Illusion

The fix

2. Concurrency Is Not Parallelism

3. AsyncIO Won't Magically Speed Up CPU-Heavy Code

A better approach

4. The Hidden Problem: Synchronous HTTP Clients

5. What About Database Queries?

6. More Concurrency Can Actually Make Things Worse

7. How to Know If Your API Is Actually Fast

8. A Practical Rule for FastAPI Developers

Final Thoughts

💬 Let's Discuss

AI Python Mentor

Building an AI-Powered Python Learning Platform with FastAPI, Next.js, and RAG

The Idea

Technology Stack

System Architecture

Building an AI Tutor with RAG

Handling Python Code Execution

Authentication and Security

Background Processing

Testing and CI

Current Status

What I Learned

What's Next?

Why does `list` take longer?

1. The `async def` Illusion