DEV Community

Cover image for Python 3.15 Is Not a Speed Release for Data Teams. It Is an Operations Release.
Firat Celik
Firat Celik

Posted on

Python 3.15 Is Not a Speed Release for Data Teams. It Is an Operations Release.

Python 3.15.0 shipped on October 9, 2026 (release post). The headline number most people will repeat is in the JIT section of What's New: a "7-8% geometric mean performance improvement" over the standard interpreter on x86-64 Linux, and 11-12% on AArch64 macOS.

For a data team, the tempting reading is "upgrade the job images, pipelines get faster." I installed 3.15.0 on a VM, ran a small pandas/pyarrow workload with the JIT off and on, and then spent most of my time on three features that do not touch throughput at all. Those three are the ones that change how you run Python next week.

The thesis

For data teams, Python 3.15 is an operations release, not a speed release. The features that matter are about startup (PEP 810 lazy imports), observability (the Tachyon sampling profiler and frame pointers on by default) and supply chain (PEP 829 .start files replacing import lines in .pth files).

Each of them also has a trap that the release notes do not spell out. I hit all of them on the VM, and they are documented below with the commands to reproduce them.

The antithesis, at full strength

The best counter-case is not silly. It goes like this:

  • The JIT is real now. What's New lists a new tracing frontend, basic register allocation, more constant propagation and reference count elimination. The pyperformance range is wide, "from roughly 15% slowdown to over 100% speedup", but the geometric mean is positive on both platforms.
  • Data engineers write more Python-level loops than they admit. Row-wise apply, per-record parsing, UDFs, validation code. That is exactly the code a JIT can see.
  • Free-threading got a stable ABI. PEP 803 adds abi3t: extensions that target the Stable ABI can now be compiled so they are compatible with free-threaded builds. That is the long road to real multi-core Python for CPU-bound pipeline code.

So the antithesis is: the startup and tooling features are nice, but the throughput work is what will show up on the cloud bill.

Here is what that looked like on the VM.

Setup

Everything below ran on one shared VM: 8 vCPU Intel Xeon, 15 GiB RAM, Debian 13, kernel 6.12. Python 3.15.0 is the uv-managed build (Clang 22, tail-calling interpreter, JIT compiled in but off by default, turned on with PYTHON_JIT=1). Libraries: numpy 2.5.4, pandas 3.0.6, pyarrow 26.0.0, all installed from cp315 wheels three days after the release. The VM is shared with other workloads, so I report medians of repeated fresh processes and repeat whole rounds.

One thing I did not expect: the VM had uv 0.12.24, and uv python install 3.15 installed 3.15.0rc3, not the final release. uv 0.13.0 installed 3.15.0. If your images resolve "3.15" through an older tool, check python -VV before you trust anything. That matters again in the profiler section.

JIT off vs on

Three workloads, timed in-process, 7 fresh processes per setting, two rounds:

Workload JIT off (median) JIT on (median) Change
Pure-Python parsing loop, 1M rows 337.6 / 332.4 ms 309.7 / 304.6 ms about 8% faster
pandas row-wise apply, 100k rows 1176.9 / 1183.6 ms 1126.8 / 1144.3 ms about 3 to 4% faster
pandas vectorized groupby().sum(), 2M rows 63.5 / 76.3 ms 63.3 / 71.4 ms within noise (single runs ranged 60 to 103 ms)

jit_bench.py (the three workloads)
"""Time three data-team-shaped workloads in-process. Run with PYTHON_JIT=0 or 1."""
import sys
import time

import numpy as np
import pandas as pd


def pure_python_parse(n=1_000_000):
    total = 0
    for i in range(n):
        line = f"{i},{i % 97},{i * 3}"
        a, b, c = line.split(",")
        if int(b) > 50:
            total += int(c)
    return total


def vectorized_groupby(n=2_000_000):
    rng = np.random.default_rng(42)
    df = pd.DataFrame({"k": rng.integers(0, 50_000, n), "v": rng.random(n)})
    return df.groupby("k")["v"].sum().iloc[0]


def rowwise_apply(n=100_000):
    df = pd.DataFrame({"email": [f" User{i}+spam@Example.com " for i in range(n)]})
    return df.apply(lambda r: r["email"].strip().lower().replace("+spam", ""), axis=1).iloc[0]


WORKLOADS = {"pure_python_parse": pure_python_parse,
             "vectorized_groupby": vectorized_groupby,
             "rowwise_apply": rowwise_apply}

fn = WORKLOADS[sys.argv[1]]
fn(10_000)  # warm-up
t0 = time.perf_counter()
fn()
print(f"{(time.perf_counter() - t0) * 1000:.1f}")
Enter fullscreen mode Exit fullscreen mode

Run it as PYTHON_JIT=0 python jit_bench.py pure_python_parse and PYTHON_JIT=1 ..., several times each.

What the numbers support (interpretation): the JIT does what the release notes say for Python-level code. The pure-Python loop landed right in the 7-8% band. But a pipeline whose hot path is already in numpy, pandas or Arrow does not get faster, because the JIT never sees that code. And the row-wise apply is the honest middle: 3 to 4% is real, but the fix for a 1.2 second apply is to vectorize it, not to wait for a JIT.

The free-threading ABI is a packaging milestone. What's New says moving an extension to abi3t "usually requires some non-trivial changes to the source code", and it pays off only when the extensions you depend on ship those wheels and you run a free-threaded build (cp315t). That is worth tracking, not something you collect by bumping a base image.

So the antithesis is partly right: there is a free win for Python-level code. It is small, it is opt-in, and it is experimental. The three features below change your runbooks this week.

1. Lazy imports fix startup, not throughput

The wrong assumption: "lazy imports make Python faster."

The mechanics: lazy import pandas as pd binds pd to a lightweight proxy. The module is loaded the first time the name is used, a step PEP 810 calls reification. If loading fails, the error is raised at first use, and the traceback shows both places. Lazy imports are only allowed at module scope: inside a function, a class body or a try/except they are a SyntaxError. I confirmed the try case on 3.15.0, which rules out the classic try: import orjson as json fallback.

I wrote a toy pipeline CLI, pipectl, with the imports most data tools have at the top: numpy, pandas, pyarrow, requests. It has a version command, a check command that validates a JSON config, and a run command that aggregates 2M rows and writes Parquet. The lazy variant changes only the import lines:

lazy import json
import argparse
import sys

lazy import numpy as np
lazy import pandas as pd
lazy import pyarrow as pa
lazy import pyarrow.parquet as pq
lazy import requests
Enter fullscreen mode Exit fullscreen mode

Each command was run 30 times in a fresh process after warm-up, in three clean rounds:

Command Eager (median per round) Lazy (median per round)
python -c pass (floor) 9.3 ms
pipectl version 351.4 / 381.2 / 344.4 ms 20.0 / 19.6 / 19.9 ms
pipectl check cfg.json 352.1 / 342.7 / 352.1 ms 21.2 / 21.1 / 20.8 ms
pipectl run (2M rows) 453.2 / 441.0 / 450.8 ms 400.5 / 392.8 / 400.5 ms

-X importtime shows where the eager 350 ms goes: pandas about 180 ms cumulative, numpy about 40 ms, requests about 35 ms (medians of three runs, and importtime adds its own overhead).

The version and check rows are the point of the feature: about 94% less wall time, because those commands never touch pandas. That is more than the "50-70% in practice" that PEP 810 cites for command-line tools, and it should be: my CLI is the extreme case where the cheap commands use none of the heavy imports.

The run row is the one to read twice. Lazy saved about 50 ms, so did it make the work faster? No. I ran a third variant, eager but with the import requests line deleted:

pipectl run, 20 runs each Median
eager 454.9 ms
eager without import requests 410.4 ms
lazy 413.3 ms

The whole gain is the import that the run path never uses. Lazy imports remove the imports you don't use. They don't make the ones you do use any cheaper. If your job imports pandas and uses pandas, it pays the full 180 ms either way.

Where this pays off for data teams (interpretation): CLIs, --help in CI, pre-commit hooks, tests that import a package to run one function, notebook kernels, and anything that gets parsed often. The Airflow docs already warn that "import statements also count as top-level code" in Dag files and that expensive imports there hurt the scheduler (Airflow best practices). The lazy keyword looks built for that pattern. I did not test it inside Airflow, and you need an Airflow version that supports 3.15 first.

Trap 1: the global switch broke pandas

3.15 also has an interpreter-wide mode: -X lazy_imports=all or PYTHON_LAZY_IMPORTS=all makes every module-level import in every module potentially lazy, third-party code included. It is tempting to flip it in a job image and collect the startup win without touching code. On 3.15.0 with pandas 3.0.6 and numpy 2.5.4, that failed 5 out of 5 times:

$ PYTHON_LAZY_IMPORTS=all python -c "import pandas as pd; pd.DataFrame({'a': [1]})"
  File ".../pandas/_typing.py", line 401, in <module>
    TakeIndexer: TypeAlias = Sequence[int] | Sequence[np.integer] | npt.NDArray[np.integer]
                                                                    ~~~~~~~~~~~^^^^^^^^^^^^
TypeError: 'lazy_import' object is not subscriptable
Enter fullscreen mode Exit fullscreen mode

The minimal trigger is npt.NDArray[np.integer] with numpy.typing imported under the global mode. numpy alone and requests alone worked. A tiny pure-Python package with the same re-export chain worked too, so this looks specific to how numpy.typing (which defines a module-level __getattr__) interacts with global laziness. I did not chase it further. The lesson does not depend on the root cause: the global mode changes the import semantics of code you did not write, and the lazy keyword in your own entry point does not.

Trap 2: side-effect imports silently stop happening

Plenty of data code imports a module only for what it does at import time: registering a pandas accessor, a SQLAlchemy dialect, a plugin. Linters flag these as unused, and they are the first thing someone will mark lazy:

lazy import pandas as pd
lazy import dq_accessors  # noqa: F401  (registers df.dq at import time)

df = pd.DataFrame({"a": [1, None, 3], "b": [None, None, 1]})
print(df.dq.null_ratio())
Enter fullscreen mode Exit fullscreen mode
AttributeError: 'DataFrame' object has no attribute 'dq'. Did you mean '.eq' instead of '.dq'?
Enter fullscreen mode Exit fullscreen mode

The name dq_accessors is never used, so it is never reified, and the registration never runs. The eager version prints {'a': 0.333, 'b': 0.667}. PEP 810 says this directly: side effects are "deferred until first use", and it recommends explicit initialization functions instead of import-time registration.

Trap 3: the PEP's kill switch does not exist in 3.15.0

PEP 810 describes a third mode, "none", to force every import eager. The shipped 3.15.0 accepts only normal and all, which matches the sys docs. Copying the PEP is not harmless:

$ PYTHON_LAZY_IMPORTS=none python -c pass
Fatal Python error: config_init_lazy_imports: PYTHON_LAZY_IMPORTS: invalid value; expected 'all' or 'normal'
Enter fullscreen mode Exit fullscreen mode

Exit code 1, before any of your code runs. sys.set_lazy_imports("none") raises ValueError. The PEP page itself now says it is "a historical document". If you want an emergency "everything eager" switch, a filter that returns False does it (verified: json loads at the lazy import line):

import sys

sys.set_lazy_imports_filter(lambda importer, name, fromlist: False)
Enter fullscreen mode Exit fullscreen mode

Runbook: lazy imports

  1. Measure first: python -X importtime -m yourtool --help 2> imports.txt, then sort by the cumulative column.
  2. Mark only the heavy imports in your own entry point lazy. Leave third-party internals alone.
  3. Keep side-effect imports eager, or replace them with an explicit register() call.
  4. Do not set PYTHON_LAZY_IMPORTS=all in a job image unless the full test suite passes under it.
  5. If you must support 3.14 too, use __lazy_modules__ = ["pandas", ...] with normal import statements. 3.15 treats those imports as lazy, and older versions just see a module variable and import eagerly (checked on 3.15.0 and 3.13.5 with json).

2. Tachyon: attach to the stuck job by PID, and read the error rate

The wrong assumption: "a sampling profiler is free and always right."

The mechanics: profiling.sampling (PEP 799, named Tachyon) reads the target process's memory from outside. It can run a script, attach to a PID for a time window, or dump a single traceback-style snapshot of a live process. Output formats include pstats tables, collapsed stacks, flame graphs, Firefox Profiler and a live TUI. The docs list the constraints that matter in production:

  • Same Python minor version for profiler and target. If either side is a pre-release, both must be the exact same version.
  • Permissions. On Linux, root, CAP_SYS_PTRACE, or a Yama ptrace_scope of 0. The docs note the default ptrace_scope of 1 restricts ptrace to parent processes.
  • Non-blocking mode (the default) never stops the target, but "can occasionally produce incomplete or inconsistent stack traces". --blocking pauses the target for each sample.

I wrote the classic slow nightly job: a pandas row-wise apply with a per-row Python function, looping forever, and pretended I did not know why it was slow.

slow_job.py
"""A 'stuck' nightly job: a row-wise pandas apply that nobody profiled."""
import time

import numpy as np
import pandas as pd


def normalize_email(row):
    return row["email"].strip().lower().replace("+spam", "")


def enrich(df):
    df["email_norm"] = df.apply(normalize_email, axis=1)
    return df


def main():
    rng = np.random.default_rng(0)
    n = 200_000
    df = pd.DataFrame({
        "id": np.arange(n),
        "email": [f" User{i}+spam@Example.com " for i in rng.integers(0, 10**6, n)],
    })
    while True:
        enrich(df.copy())
        time.sleep(0.01)


if __name__ == "__main__":
    main()
Enter fullscreen mode Exit fullscreen mode

The first thing you want during an incident is "what is it doing right now":

$ python -m profiling.sampling dump --blocking 440208
Stack dump for PID 440208, thread 440208 (main thread, has GIL; most recent call last):
  File "slow_job.py", line 31, in <module>
    main()
  File "slow_job.py", line 26, in main
    enrich(df.copy())
  File "slow_job.py", line 14, in enrich
    df["email_norm"] = df.apply(normalize_email, axis=1)
  ...
  File "slow_job.py", line 10, in normalize_email
    return row["email"].strip().lower().replace("+spam", "")
  File ".venv/lib/python3.15/site-packages/pandas/core/series.py", line 959, in Series.__getitem__
    return self._get_value(key)
Enter fullscreen mode Exit fullscreen mode

No restart, no code change, and the culprit is on screen. On this kernel there is no Yama module (/proc/sys/kernel/yama/ptrace_scope does not exist), so attaching as the same unprivileged user just worked. On a host with the default ptrace_scope of 1, plan for root or CAP_SYS_PTRACE before the incident, not during it.

Then the part I did not expect. Without --blocking, dump failed 8 times out of 20 with RuntimeError: Failed to parse initial frame in chain. With --blocking it succeeded 20 out of 20. For attach, the profiler prints an Error rate line, the percentage of samples it could not read. I measured it on a fixed workload (median of six apply rounds per run, 1 kHz default rate, three repetitions):

Mode Target round time Error rate
No profiler 1170 / 1150 / 1186 ms
attach (non-blocking, default) 1179 / 1180 / 1150 ms 67.8% / 62.0% / 63.7%
attach --blocking 1606 / 1760 / 1657 ms 1.5% / 1.6% / 1.3%

Both halves of the docs are true at once. Non-blocking mode had no overhead I could measure on the target. And on this VM, 62 to 68% of its samples were unreadable in this test (46 to 50% in two earlier 5-second runs against slow_job.py, 38% against a pure-Python recursive function). Blocking mode was clean, and it made the job 37 to 53% slower while attached.

Interpretation: treat a non-blocking profile like a survey with a response rate. If 60% of samples failed, the remaining 40% may still point at the right function (here the row-wise apply dominated either way), but check the error rate before you trust percentages. Use dump --blocking for a one-shot snapshot, where a single pause costs nothing, and keep --blocking profiles short on anything user-facing.

And the version rule bit me right away. A job started on the 3.15.0rc3 that the older uv had installed could not be inspected from 3.15.0:

Can't attach to a pre-release Python interpreter from a process running a different Python version
Enter fullscreen mode Exit fullscreen mode

From rc3 to rc3 it worked. Pin the profiler to the job image's exact interpreter, which in practice means running it from inside that image.

The quieter observability change is PEP 831: CPython is now built with frame pointers by default. What's New says this makes native stack unwinding "faster and more reliable" for system profilers, debuggers, crash analysis tools and eBPF-based tools. The uv build I used has -fno-omit-frame-pointer in its CFLAGS. The catch, in the release notes' words: "A single native component built without frame pointers can break stack unwinding for the whole Python process." Your own Rust or C extensions need the same flags if you want mixed native and Python profiles.

Runbook: profiling a stuck job

  1. python -m profiling.sampling dump --blocking PID first. One snapshot, one short pause.
  2. Then python -m profiling.sampling attach -d 30 --flamegraph -o job.html PID, and read Error rate before the flame graph.
  3. Run the profiler with the job's exact interpreter. A pre-release on either side means exact versions only.
  4. Decide now who is allowed CAP_SYS_PTRACE on worker hosts, and where.

3. .pth import lines become .start files: auditable, not safe

The wrong assumption: "a package can only run code when I import it."

The mechanics: at startup the site module reads every .pth file in site-packages. Lines are paths to add to sys.path, except lines that start with import, which are executed with exec(). Every Python process in that environment runs them: python -c pass, pip, pytest, the notebook kernel, the scheduler. PEP 829 calls this "a broad attack surface".

This is not theoretical. The advisory for the compromised litellm 1.82.7 and 1.82.8 releases (PYSEC-2026-2, published March 24, 2026) describes credential-harvesting malware, and its references point to a litellm_init.pth file inside the 1.82.8 package.

And it is not only malware. setuptools 84.0.0, installed fresh on the VM, still ships distutils-precedence.pth, whose single line starts with import os; ....

What PEP 829 and the site docs change in 3.15:

  • A new name.start file lists entry points as pkg.mod:callable. They run after all .pth path lines are applied.
  • If name.start exists next to name.pth, the import lines in that .pth are ignored.
  • Without a matching .start, import lines still run in 3.15, and the deprecation message appears only with -v. The docs say they will be "silently ignored in Python 3.18 and 3.19", with a warning in 3.20.
  • Errors on one line no longer abort the rest of the file. A failing entry point prints a traceback and startup continues.

On the VM, with a demo zz_demo.pth containing a path line and an import line:

$ python -c "import sys; print('/tmp/extra_path_dir' in sys.path)"
[zz_demo.pth] import line executed at startup
True
Enter fullscreen mode Exit fullscreen mode

python -v shows the new deprecation notice for both the demo file and setuptools' file:

import lines in .../site-packages/distutils-precedence.pth are deprecated, use entry points in a .../site-packages/distutils-precedence.start file instead.
Enter fullscreen mode Exit fullscreen mode

After adding zz_demo.start with one line, zz_demo_boot:init, the import line is suppressed and the entry point runs:

import lines in .../zz_demo.pth are suppressed due to matching .../zz_demo.start file.
Executing entry point: zz_demo_boot:init from .../zz_demo.start
[zz_demo.start] entry point zz_demo_boot:init called
Enter fullscreen mode Exit fullscreen mode

Here is the part to be precise about. A .start file still runs code at every interpreter start. A malicious wheel can ship one just as easily as a .pth. The PEP says so: "The overall pre-start code execution attack surface is not eliminated by this PEP." What 3.15 gives you is a narrower, auditable shape: startup code is a named callable in an importable module, it goes through the import system so audit hooks (PEP 578) can see it, and "you can tell by listing the files in the site-dir, exactly where arbitrary code execution occurs."

That listing is the useful part, and you can do it today on any Python version:

startup_audit.py (runs on Python 3.9+)
"""List everything that can run code at interpreter start in this environment.

Run it with the interpreter you want to audit:
    .venv/bin/python startup_audit.py
"""
import site
import sys
from pathlib import Path

dirs = list(dict.fromkeys(site.getsitepackages() + [site.getusersitepackages()]))
found = 0
for d in map(Path, dirs):
    if not d.is_dir():
        continue
    for f in sorted(d.glob("*.pth")):
        lines = f.read_text(encoding="utf-8-sig", errors="replace").splitlines()
        code = [l for l in lines if l.startswith(("import ", "import\t"))]
        if code:
            found += 1
            shadow = f.with_suffix(".start").exists()
            note = "skipped on 3.15+ (matching .start), runs on older Pythons" if shadow else "RUNS on every start"
            print(f"{f}  [{note}]")
            for l in code:
                print(f"    {l[:110]}")
    for f in sorted(d.glob("*.start")):
        found += 1
        print(f"{f}  [.start entry points]")
        for l in f.read_text(encoding="utf-8-sig").splitlines():
            if l.strip() and not l.lstrip().startswith("#"):
                print(f"    {l.strip()}")
    for name in ("sitecustomize.py", "usercustomize.py"):
        if (d / name).exists():
            found += 1
            print(f"{d / name}  [runs on every start]")
print(f"{sys.version.split()[0]}: {found} startup code hook(s) found")
Enter fullscreen mode Exit fullscreen mode

Output in the demo environment (with setuptools installed):

.../site-packages/distutils-precedence.pth  [RUNS on every start]
    import os; var = 'SETUPTOOLS_USE_DISTUTILS'; enabled = os.environ.get(var, 'local') == 'local'; enabled and __
.../site-packages/zz_demo.pth  [skipped on 3.15+ (matching .start), runs on older Pythons]
    import sys; sys.stderr.write("[zz_demo.pth] import line executed at startup\n")
.../site-packages/zz_demo.start  [.start entry points]
    zz_demo_boot:init
3.15.0: 3 startup code hook(s) found
Enter fullscreen mode Exit fullscreen mode

Most .pth files are harmless. An editable install of a src-layout package with setuptools wrote a path-only .pth on the VM, with no code at all. The ones that execute code are few enough to put on an allowlist.

python -S disables both mechanisms (I checked), but it also skips site-packages entirely, so it is a debugging tool, not a production setting.

Runbook: startup code

  1. Run the audit in your image build and fail the build on any startup hook that is not on an allowlist.
  2. If you maintain a package with a .pth import line, move the code into a zero-argument function and add a matching .start file. Keep the .pth line as import pkg.mod; pkg.mod.callable() while you still support 3.14.
  3. Treat a new .start file in a dependency diff like a new postinstall script in npm: a reason to read the release.

What to do this week

If you run... Do this Skip this
Python CLIs, CI tooling, notebook kernels Measure with -X importtime, add lazy to your entry point's heavy imports PYTHON_LAZY_IMPORTS=all in images
Long batch jobs on VMs or Kubernetes Install the exact job interpreter in the image, rehearse dump --blocking and attach on a staging job, sort out CAP_SYS_PTRACE Trusting a profile without reading Error rate
Anything that installs from PyPI Add the startup audit to the image build, review new .pth and .start files in dependency updates Assuming .start files make startup code safe
Python-heavy loops or row-wise UDFs Try PYTHON_JIT=1 in a benchmark, expect single digits Counting on the JIT to fix a row-wise apply

The JIT will show up in your benchmarks as a few percent. Lazy imports, Tachyon and .start files will show up in your incident timeline: the CLI that starts in 20 ms instead of 350, the stuck job you can inspect without a restart, and the list of files that run code before your main() does. Upgrade for those, and measure the rest.

Top comments (0)