Python 3.15 for data work: lazy imports, frozendict and a new profiler

in Machine Learning & Engineering
Python 3.15 for data work: lazy imports, frozendict and a new profiler

What Python 3.15 changes for data scientists, tested: lazy import cut a pandas CLI’s –help from 132 ms to 10 ms, plus frozendict, unpacking in comprehensions, UTF-8 by default and the new profiling package.

TL;DR — Python 3.15.0 was released on 2026-10-09. It adds lazy import, which delays loading a module until it is first used. In a small CLI, it cut the --help path from 132 ms to 10 ms by not loading pandas. It also adds a built-in frozendict, * unpacking inside comprehensions, UTF-8 as the default text encoding everywhere, and a new profiling package.

Tested on 2026-10-12 with Python 3.15.0 (released 2026-10-09), numpy 2.5.4 and pandas 3.0.6 on macOS (Apple M5).

Key points

  • lazy import pandas as pd (PEP 810) loads pandas only when pd is first used. Scripts that sometimes don’t need a heavy library start much faster.
  • frozendict (PEP 814) is an immutable, hashable dict, so you can use settings as cache keys.
  • [*x for x in lists] (PEP 798) flattens in one comprehension; {**d for d in dicts} merges.
  • UTF-8 is now the default encoding (PEP 686), no matter the system locale. open() without encoding= behaves the same on every OS.
  • The new profiling package (PEP 799) holds the classic tracing profiler and a new sampling profiler. On macOS the sampling profiler needs elevated permissions.

How much faster does lazy import make startup?

A common pattern: a script imports pandas at the top, but some code paths (like --help) never use it.

import sys
lazy import pandas as pd

if "--help" in sys.argv:
    print("usage: report.py FILE")          # this path never touches pandas
    sys.exit(0)
df = pd.DataFrame({"x": range(3)})
print(int(df["x"].sum()))

We timed this script against the same script with a normal import pandas as pd (median of 15 runs each, process start to exit):

import statistics, subprocess, sys, time

def median_ms(args, runs=15):
    times = []
    for _ in range(runs):
        t0 = time.perf_counter(); subprocess.run([sys.executable, *args], capture_output=True, check=True)
        times.append((time.perf_counter() - t0) * 1000)
    return statistics.median(times)

print(sys.version.split()[0])
for mode in ["eager", "lazy"]:
    print(f"{mode:<6} --help: {median_ms([f'cli_{mode}.py', '--help']):6.0f} ms   real run: {median_ms([f'cli_{mode}.py']):6.0f} ms")

3.15.0
eager  --help:    132 ms   real run:    132 ms
lazy   --help:     10 ms   real run:    144 ms

Bar chart of median start-to-exit time on Python 3.15.0: with a normal import, --help and the real run both take 132 ms; with lazy import, --help takes 10 ms and the real run 144 ms

  • The --help path went from 132 ms to 10 ms, because pandas was never loaded. This part was the same in every run.
  • The real run took 132 ms with a normal import and 144 ms with lazy import here. In another run on the same machine it was 141 vs 134 ms, the other way round, so treat that difference as noise: lazy import moves the cost of loading pandas, it doesn’t remove it.

This matters for command-line tools, test collection, and notebooks or scripts that import many libraries but use a few per run.

What other new features help day-to-day data code?

# PEP 798: unpacking in comprehensions
batches = [[1, 2], [3], [4, 5, 6]]
print("flattened:", [*b for b in batches])
configs = [{"lr": 0.1}, {"epochs": 5}, {"lr": 0.01}]
print("merged:", {**c for c in configs})

# PEP 814: frozendict, an immutable and hashable dict
params = frozendict(lr=0.01, epochs=5)
cache = {params: "score 0.93"}               # usable as a dict key
print("lookup:", cache[frozendict(epochs=5, lr=0.01)])
try:
    params["lr"] = 0.1
except TypeError as e:
    print("TypeError:", e)

# PEP 686: UTF-8 is the default text encoding
import locale
with open("tmp.txt", "w") as f:
    print("open() default encoding:", f.encoding, "| locale:", locale.getpreferredencoding(False))

flattened: [1, 2, 3, 4, 5, 6]
merged: {'lr': 0.01, 'epochs': 5}
lookup: score 0.93
TypeError: 'frozendict' object does not support item assignment
open() default encoding: utf-8 | locale: utf-8
  • Flattening batches no longer needs itertools.chain or a double loop.
  • Merging dicts keeps the last value for repeated keys (lr became 0.01).
  • frozendict keys match regardless of argument order, which makes it handy for caching results per hyperparameter set.
  • UTF-8 by default mainly helps Windows users, where open() used to depend on the system locale. On our Mac, both were already UTF-8.

How do you profile with the new profiling package?

Python 3.15 groups profilers under profiling. The tracing profiler (the classic cProfile, now also at profiling.tracing) finds a slow apply:

import pandas as pd
import numpy as np

def load(n=300_000):
    rng = np.random.default_rng(0)
    return pd.DataFrame({"user": rng.integers(0, 5_000, n), "amount": rng.gamma(2.0, 20.0, n)})

def add_tax_slow(df):
    return df.apply(lambda row: row["amount"] * 1.1, axis=1)   # row-by-row: slow

def summarize(df):
    return df.groupby("user")["amount"].agg(["sum", "mean"])

df = load()
df["with_tax"] = add_tax_slow(df)
print(summarize(df).shape)

python -m profiling.tracing -s cumulative slow_pipeline.py

         7848327 function calls (7841461 primitive calls) in 1.157 seconds
   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    1.157    1.157 slow_pipeline.py:1(<module>)
        1    0.000    0.000    1.003    1.003 slow_pipeline.py:8(add_tax_slow)
   300000    0.035    0.000    0.541    0.000 slow_pipeline.py:9(<lambda>)
        1    0.003    0.003    0.004    0.004 slow_pipeline.py:4(load)
        1    0.000    0.000    0.003    0.003 slow_pipeline.py:11(summarize)

(Trimmed to the rows from our script.) add_tax_slow took 1.003 of 1.157 seconds. The fix is a vectorized df["amount"] * 1.1.

The new sampling profiler (python -m profiling.sampling run script.py, nicknamed Tachyon) has lower overhead and can attach to a running process. On macOS, our run on 3.15.0 stopped with “Permission Error: Unable to access process memory on macOS” and a suggestion to rerun with sudo -E, so we did not include its output here.

Should you upgrade now?

If you…Then
Maintain CLIs or scripts with heavy importsTry lazy import for the libraries used on only some paths
Depend on many compiled packagesCheck each one ships wheels for 3.15 (numpy 2.5.4 and pandas 3.0.6 installed as cp315 wheels on our Mac on 2026-10-12)
Read or write text files on WindowsRe-test file I/O: the default encoding changed to UTF-8
Need to support older Pythonslazy import is a syntax error before 3.15; keep it out of shared code for now

Sources

Related: What breaks when you upgrade to pandas 3.0?