DEV Community: Rahul Kumar

Keras #22561 speedups: compiled Keras on PyTorch now generates text 11.6x faster

Rahul Kumar — Fri, 16 Oct 2026 16:29:52 +0000

Between master on Jul 18 and master on Oct 15, 2026, 15 of our PRs landed in Keras. Same GPU, same torch and jax versions, same models; only the Keras source changes. On the issue's test LM, compiled Keras[torch] now generates text 11.6× faster, torch.compile went from slowing Keras down to making it 6.56× faster than eager Keras, and an LLM forward makes 12–23% fewer Python calls on all four backends.

Here is compiled generation at every merge. Pick a config or a workload, and hover a point to see its runs:

Generate-32 throughput, July master vs today

Tokens per second from the 2.1M-parameter test LM, 128 tokens per call (4 sequences × 32 steps); longer bars are faster. In July, compiling a Keras[torch] model made it slower: 1,458 tok/s compiled against 2,139 eager. Today the compiled model runs at 16,975 tok/s: 6.56× faster than eager Keras, within 1.53× of plain compiled PyTorch, down from 17.8×.

Three merges that made torch.compile work with Keras

Three merges each turned one compiled workload from slow to fast in a single step.

The three PRs: #23327 (attention), #23201 (dict ordering in the tree code) and #23647 (identity checks in the tree code). Graph breaks per forward pass, at every merge:

Less Python in every Keras call, on all four backends

Most of the series changed shared code (Layer.call, op tracebacks, dtype handling, input checks), so the saving reaches every backend, not just torch. Python function calls per forward pass, July to today: PyTorch −23% on the LM and −33% on the CNN; JAX −12% on the LM and −17% on the CNN; TensorFlow −13% on the LM and −19% on the CNN; NumPy −16% on the LM and −18% on the CNN. Adding #23336 takes NumPy to −36% on the LM. These counts are exact and repeat call for call on any machine.

Keras[jax]

Under jax.jit the Python runs once at trace time, so Keras[jax] was already close to plain JAX and stays there: 2.30 → 2.45 ms per generate-32 against 2.22 ms for plain JAX. Eager Keras[jax] is where the saving shows in the exact counts: −12% Python calls per forward. The timing agrees: the sweep gives 293 → 267 ms (1.10×) and the back-to-back A/B rounds give 1.08×.

Every config, every workload

Show all 12 configs and workloads (ms per call)

Keras config	Workload	Plain framework (ms)	Jul 18 (ms)	Oct 15 (ms)	Change	+ #23336 (ms)	A/B check
Keras[torch] · eager	CNN forward	0.064	0.668	0.477	1.40×	0.466	1.39×
Keras[torch] · eager	LLM forward	0.189	1.88	1.55	1.22×	1.48	1.20×
Keras[torch] · eager	LLM generate-32	5.85	59.8	49.5	1.21×	47.2	1.20×
Keras[torch] · torch.compile	CNN forward	0.076	1.28	0.160	8.03×	0.161	8.41×
Keras[torch] · torch.compile	LLM forward	0.160	2.69	0.302	8.93×	0.299	9.60×
Keras[torch] · torch.compile	LLM generate-32	4.92	87.8	7.54	11.6×	7.66	12.4×
Keras[jax] · eager	CNN forward	0.508	1.34	1.23	1.08×	1.18	1.15×
Keras[jax] · eager	LLM forward	2.79	8.46	8.38	1.01×	7.90	1.12×
Keras[jax] · eager	LLM generate-32	89.0	293	267	1.10×	269	1.08×
Keras[jax] · jax.jit	CNN forward	0.077	0.079	0.081	0.97×	0.080	1.02×
Keras[jax] · jax.jit	LLM forward	0.120	0.131	0.135	0.97×	0.130	1.05×
Keras[jax] · jax.jit	LLM generate-32	2.22	2.30	2.45	0.94×	2.27	1.05×

A/B check: July ÷ today from 3 separate back-to-back rounds (higher is a bigger speedup).

Each of our PRs, measured where it landed

Each PR was timed on the commit just before and just after it merged, so every row is that one PR's own effect. Δ is just after the merge minus just before it; negative is better.

Show the per-PR table

PR	Merged	What it changed	Δ calls/fwd torch	JAX	TF	NumPy	Δ eager gen-32 (ms)	Δ compiled gen-32 (ms)	Graph breaks
#23183	07-28	Traceback details are built only when an op fails, not on every call	−598	−586	−622	−546	−2.9	−5.2	·
#23327	08-30	Removed the torch.compile graph break in the torch backend's dot_product_attention	+6	0	0	0	+0.40	−12.3	LLM 11 → 0
#23189	09-03	Fixed an always-true device check that sent every tensor through .to()	0	+38	0	0	+0.28	−1.1	·
#23200	09-05	Layer.call stops re-setting built/_called on every call	−1,848	−1,098	−1,188	−1,100	−4.9	−10.5	·
#23182	09-10	O(1) dtype membership and a fast path for canonical dtype strings	−300	−82	−497	−116	−0.89	−1.5	·
#23199	09-10	Dropped a redundant channels_last copy on torch conv inputs	0	−50	0	0	+0.40	−0.29	·
#23201	09-11	Torch tree code made traceable: canonical dict key order	−142	+8	0	0	+0.55	−5.2	CNN 9 → 0
#23296	09-11	Keeps call-context kwargs on the recorded Node (fix)	−109	−78	−14	−46	−0.24	−0.09	·
#23187	09-12	TorchTrainer skips a full flatten walk to find the batch size	0	−26	0	0	+0.14	−0.21	·
#23301	09-12	Layer name scope opened once per call, not twice	−330	−568	−638	−330	−0.89	−0.21	·
#23190	09-15	torch slice() keeps dynamic dims under torch.export	0	−16	0	0	−0.23	−0.16	·
#23647	09-30	Torch tree code stops comparing id()s	0	−8	0	0	−0.58	−49.4	·
#23645	10-07	torch pooling ops run in channels_last	0	−16	0	0	+0.10	−0.05	·
#23184	10-13	Fast path for single-InputSpec input checks	−102	−59	−63	−63	+0.06	+0.05	·
#23920	10-15	Functional models reject a mismatched last axis (fix)	0	−54	0	0	+0.10	+0.02	·
#23336	not merged yet (approved)	Memoized dtype promotion in result_type	−713	−378	−1,471	−3,399	−2.2	+0.12	·

Hot functions in one Keras[torch] LLM forward

Show the hot-functions table

Function	Jul 18	Oct 15	Change	+ #23336
isinstance	2,600	1,740	−33%	1,680
traceback error wrappers	74	48	−35%	48
thread-local config lookups	624	513	−18%	513
standardize_dtype	177	244	+38%	184
tree.flatten	93	67	−28%	67
tree.map_structure	102	62	−39%	62
InputSpec checks	14	14	+0%	14
CallSpec objects built	23	23	+0%	23
inspect.Signature.bind	22	22	+0%	22
convert_to_tensor	86	86	+0%	86
get_device	138	138	+0%	138

How this was measured

Machine: one NVIDIA RTX PRO 6000 Blackwell, kept idle during the runs.
Software: the same for every run: torch 2.13.0, jax 0.11.2, Python 3.14.7. Only the Keras source changes.
Commits: 31 snapshots of master: Jul 18 (a2f8eac8), just before and just after each of the 15 merges, Oct 15 (cc9de2b31), and Oct 15 plus #23336.
Models: the issue's two test models: a small CNN (114k parameters) and a 2-layer transformer LM (2.1M parameters, 32-token greedy generate).
Timing: the issue's benchmark script, one process at a time, warm-up first, a GPU sync after every call. Each number is a median, and the whole sweep ran three times for PyTorch and twice for JAX.
Cross-check: a separate back-to-back run of July against October gives 12.4× compiled and 1.20× eager (the sweep: 11.6× and 1.21×).
Counts: Python calls from cProfile, graph breaks from torch._dynamo.explain. These are exact and repeat on any machine.
Not covered: training, CPU, larger models, and TensorFlow or NumPy timing (call counts only).

The work is tracked in issue #22561. Thank you to the Keras maintainers who reviewed and merged these PRs.

Why two out of three open-source projects don't get a Porchlamp score

Rahul Kumar — Fri, 16 Oct 2026 00:15:14 +0000

TL;DR: Of 11,000 open-source projects, only about 1 in 3 gets a score. The rest aren't hidden: each one is listed with a plain reason, like "no recent activity" or "most outside pull requests go unanswered".

In Porchlamp's October 2026 snapshot, 3,833 of 11,000 GitHub projects received a score; the other 7,167 are listed with the reason they were not ranked.

How are open-source projects ranked fairly?

A ranking is only useful if every project in it is a real choice for a contributor. A project that's archived, has no license, or hasn't had a commit in months isn't one, however good its numbers look.

So before scoring, every project goes through a few simple checks. Fail one, and the project is listed with the reason instead of a rank.

The checks, in plain words

Is it software? Docs-only repos, awesome lists and data dumps are left out. (1,067 projects)
Is it still a real project? Archived repos, mirrors and most forks are left out. (624)
Does it have an open-source license? Without one, you can't safely contribute. (491)
Is anyone still working on it? No human commits in six months means it's stale. This is the biggest group. (2,475)
Does anyone answer outside PRs? If most pull requests from outsiders get no human response within two weeks, it's a black hole. (1,133)

Then there's a last group: projects that pass every check, but where too little could be measured to score them fairly. They're listed as unrated (1,377).

That leaves 3,833 ranked projects.

The black-hole check is careful on purpose

Calling a project a black hole is a strong statement, so the check leans towards giving projects the benefit of the doubt:

It needs at least ten outside PRs. With fewer, there's no verdict.
PRs younger than two weeks don't count yet.
Bots, drafts, maintainers' own PRs and brand-new accounts are ignored.
The rule only fires when the data clearly says "most go unanswered", not on a borderline case.

Undecided isn't failed

Some projects do their reviews somewhere else, on Gerrit or a mailing list, and their GitHub PRs only point there. Porchlamp spots those and leaves them undecided rather than calling them silent. Eight projects are handled that way.

See the reasons yourself

The Not ranked list on Porchlamp shows every project that didn't get a score, with its reason. If you think a check treats a project unfairly, open an issue and say why.

Which of these checks would you loosen, and which would you make stricter?

Where to send your first open-source pull request (the projects that actually answer)

Rahul Kumar — Thu, 15 Oct 2026 23:52:12 +0000

TL;DR: For a first contribution, the projects worth your time are the ones where humans answer outside pull requests and first changes are easy to make. Right now the list starts with llama.cpp, Meshery, open-design, stdlib and Ant Design.

In Porchlamp's October 2026 snapshot of 11,000 GitHub projects, the first-contribution ranking scores projects mostly on two things: how quickly humans respond to outside pull requests (35% of the score) and how easy a first change is (30%).

Which open-source projects are good for a first contribution?

The usual advice for a first open-source contribution is to pick something popular. But popular projects are often the slowest to answer a stranger. Your first PR can sit for weeks with no reply, and that's usually where people give up.

So I built a ranking that asks a different question: if you're new, where will your pull request actually get seen?

What it looks at

Two things carry most of the weight:

Responsiveness (35%): how fast a human answers outside PRs, and how many of those PRs get a decision.
Accessibility (30%): good first issues that are actually open, and whether first-time contributors get merged and come back.

The rest is governance (15%), how much other software depends on the project (10%), and review and CI practices (10%).

The best projects for a first contribution right now

llama.cpp
Meshery
open-design
stdlib
Ant Design
Organic Maps
Apache Maka
Cluster API
freeCodeCamp

A few of these jump up a lot compared with the default ranking. open-design climbs 63 places and stdlib 49. That means their strengths are exactly the things a newcomer needs.

Two labels you'll see

"B (capped)": not a bad grade. It means only part of the project's data could be measured, so the tier can't go higher yet.
The Δ rank column: how far a project moves between the default ranking and this one. It isn't movement over time.

How to use it

Open the Porchlamp first-contribution rankings.
Pick two or three projects in a language you know.
Open each one's pull requests on GitHub and read a few recent ones from outside contributors. Do they get answers?
Then find a good first issue and start small.

A high rank isn't a promise about your particular PR, but it's a much better bet than stars.

If a row looks wrong to you, an issue on the repository with the pull requests that show it is the most useful thing you can send.

Which project gave you your first merged PR, and how long did it take?

Which ecosystems leave your pull request hanging? npm and PyPI more than Rust, Go or Java

Rahul Kumar — Thu, 15 Oct 2026 22:56:51 +0000

TL;DR: If you send a pull request to an npm or PyPI project, you're more likely to hear nothing back than with a Rust, Go or Java one. About 3 in 10 active npm and PyPI projects leave most outside PRs unanswered for two weeks, against about 1 in 5 for crates.io.

In Porchlamp's October 2026 snapshot, 33.8% of active PyPI projects and 30.0% of active npm projects left most outside pull requests unanswered for two weeks, compared with 20.0% of crates.io projects.

Does the ecosystem matter?

To find out, I split my data on outside pull requests by package ecosystem.

A quick definition first. I call a project a black hole when most pull requests from outside contributors get no human reply, review or merge within two weeks. Only active, licensed projects with at least ten outside PRs are judged.

Which ecosystems answer outside pull requests?

Ecosystem	Projects judged	Black holes
PyPI	968	33.8%
npm	1,283	30.0%
Go	609	22.2%
Systems (C, C++ and others)	573	20.1%
crates.io	586	20.0%
Maven	278	19.4%

Two groups stand out:

npm and PyPI: roughly 3 in 10 projects go quiet on outsiders.
Rust, Go, Java and systems projects: closer to 1 in 5.

Rust isn't alone at the good end. Maven is a touch better, and systems projects are level with it.

What I wouldn't read into it

It's not about community character. These are averages over very different projects.
Size isn't controlled for. Bigger projects go silent more often in general, so an ecosystem with more big projects looks worse for that reason alone.
Off-GitHub review isn't counted. Projects that review on Gerrit or a mailing list aren't judged, and they may cluster in some ecosystems.

So if you're picking where to send your first PR, this is a mild nudge, not a guarantee about any single project.

Check the project, not just the ecosystem

The full ranking is on Porchlamp. Here is the Porchlamp ranking of crates.io projects; switch the filter to see any other ecosystem. Every black-hole verdict lists the pull requests behind it. If a project looks misjudged, open an issue with the evidence.

Would you expect the Rust number to hold up once project size is taken into account?

I measured silence: 212,390 outside pull requests across 11,000 GitHub projects

Rahul Kumar — Thu, 15 Oct 2026 22:03:24 +0000

TL;DR: More than 1 in 4 active open-source projects leave most pull requests from outside contributors with no human response for two weeks. The more stars a project has, the more likely it is to be one of them.

In Porchlamp's October 2026 snapshot of 11,000 GitHub projects, 1,133 of the 4,297 active projects (26.4%) left most outside pull requests with no human reply, review or merge within 14 days.

Why I looked

One of my pull requests once sat for four months before anyone merged it. Around the same time, a different project merged five of mine in a single day.

Same person, same kind of code. The only difference was the project. And nothing on a GitHub page tells you which kind you're about to deal with. Stars tell you a project is popular. They don't tell you if anyone will read your PR.

What I measured

I followed six months of pull requests from outside contributors to 11,000 GitHub projects: 212,390 PRs from 110,605 people.

For every PR the question was simple: did a human respond within two weeks? A reply, a review or a merge all count. A bot comment doesn't.

To keep it fair I left out bots, maintainers' own PRs, drafts, PRs the author closed themselves, and brand-new accounts.

Do popular open-source projects answer pull requests?

1. Silence is common. Of the 4,297 active, licensed projects with at least ten outside PRs, 1,133 left most of them unanswered for two weeks. I call these black holes.

2. More stars, more silence.

About 1 in 5 projects with 1k to 10k stars is a black hole. Above 50k stars it's closer to 2 in 5.

3. Popular isn't the same as welcoming. Of the 100 most-starred projects, just 9 reach the top 100 of the default Porchlamp ranking.

What this doesn't mean

It's not a verdict on maintainers. Big projects get far more PRs than anyone can review.
It only sees GitHub. Projects that review on a mailing list or Gerrit aren't judged at all.
It's a snapshot (April to October 2026), so it will drift.

Check a project before you give it a weekend

I turned all of this into a free site, the Porchlamp rankings. It ranks open-source projects by what happens to a stranger's pull request, and you can re-rank it for your goal: a first contribution, career value, deep systems work, or AI-assisted work.

👉 See the Porchlamp rankings of 11,000 open-source projects

Looking for your first project? Here's where to send your first open-source pull request.

Every black-hole verdict links to the pull requests behind it, so you can check the evidence yourself.

What's the longest one of your PRs has ever waited?