TL;DR
- NVIDIA published TensorRT-LLM v1.3.0 as a stable release on 2026-11-20. It follows 30 release candidates (rc0 on 2026-01-22 through rc29 on 2026-09-29), roughly 302 days from first candidate to final.
- The release notes list 13 items marked BREAKING CHANGE. Six remove whole components (the TensorRT backend, the legacy C++ Executor, TensorRT-engine Triton models,
TRTLLMSampler, two-model speculative decoding, and AutoDeploy/Star Attention/WIDEEP), three change defaults silently, and four change the API surface. - The three default changes are the dangerous ones, because nothing crashes at startup: explicit CLI flags now override YAML, KV transfer now times out after 60 seconds, and per-request chat templates are opt-in.
- Usage telemetry is on by default. The documented opt-outs are
TRTLLM_NO_USAGE_STATS=1,DO_NOT_TRACK=1, or--no-telemetry. - The 1.4.0 candidates (rc0 on 2026-10-14, rc4 on 2026-11-18) were already public before this final tag, so 1.3.0 is best read as the stable line, not the newest code.
What actually shipped in TensorRT-LLM 1.3.0?
The GitHub release for NVIDIA/TensorRT-LLM tagged v1.3.0 is dated 2026-11-20 and is not marked pre-release. The notes are long (about 18,000 characters), so it helps to sort them into three buckets before deciding what to do.
Feature headline items (the parts that sell the release):
- Preview support for NVIDIA Rubin (SM107), plus DeepSeek-V4, Kimi K3 and MiniMax-M3 model support on Blackwell and Rubin.
- New serving surface: an Anthropic Messages endpoint (
/v1/messages), a gRPC server, multi-process HTTP frontends, Prometheus metrics, and/start_profileand/stop_profileendpoints. - Speculative decoding additions: DSpark, DFlash, Suffix Automaton and PARD drafters, EAGLE3 dynamic trees, and acceptance-rate-based auto-disable.
- KV cache work: KV Cache Manager V2 enabled by default for most model families, NVFP4 KV cache, cold-page compression, FlexKV and LMCache connectors.
- A new vertical called VisualGen for image and video generation models served through the same
trtllm-serve.
Infrastructure changes that touch your build: base image pytorch:26.09-py3, CUDA Toolkit 13.4.1, PyTorch 2.14.0, C++20, Python 3.10 and 3.12, and TensorRT, ModelOpt and ONNX are no longer dependencies.
Breaking changes, which are the subject of the rest of this article.
Why does a final release arrive after 30 candidates?
The candidate cadence in the chart is derived from the publish timestamps on the GitHub releases API: between two and five candidates every month from January to September. There was no long gap, which suggests the project treats main as continuously releasable and cuts a numbered candidate each time.
Two practical consequences follow:
-
Candidates are snapshots, not a stabilization funnel. Each candidate carries a fresh pile of features, and the notes for individual candidates run to tens of thousands of characters. Pinning to
rc17and "waiting for final" tells you very little about what final contains. -
The final tag came after the next line started. The 1.4.0 candidates began on 2026-10-14, and 1.4.0rc4 was published on 2026-11-18, two days before 1.3.0 final. The API reports the 1.3.0 target as
main. I would not infer more than the dates show, but the practical reading is that "1.3.0" is a stable label for a large feature set, while the active development line is already 1.4.
If you track release trains for a serving stack, the lesson is the same as with other fast-moving engines: a version number tells you less than the changelog diff between your pin and the target. For a related look at how fast a pin ages, see my earlier note on what 26 Dynamo snapshot builds say about pinning.
Which of the 13 breaking changes will break a deployment?
The grouping above is mine; the 13 items are exactly those the release notes mark [BREAKING CHANGE].
Group 1: Removals (6 items). These fail loudly.
| Removed | What replaces it |
|---|---|
TensorRT backend (trtllm-build, TrtLlmArgs, BuildConfig, MedusaDecodingConfig, LookaheadDecodingConfig, trtllm-bench build, trtllm-bench --engine_dir) |
PyTorch backend only; see the project migration guide |
Legacy C++ Executor, legacy runtime and batch-manager APIs, cpp_only build option |
PyTorch backend and LLM API |
| TensorRT-engine Triton backend models | Only the llmapi Triton model remains |
TRTLLMSampler |
TorchSampler, which requires FlashInfer; sampler_type is ignored with a warning |
Two-model speculative decoding (eagle3_one_model, mtp_eagle_one_model fields) |
One-model implementations |
| AutoDeploy, Star Attention, WIDEEP MoE backend | None listed |
These are easy: if you used any of them, your launch fails or a build script breaks, and you find out in staging. The one that deserves a second look is the sampler: disable_flashinfer_sampling is gone, so an environment that deliberately avoided FlashInfer now cannot.
Group 2: Changed defaults (3 items). These fail quietly.
-
CLI flags override YAML. Explicit
trtllm-serve,trtllm-evalandtrtllm-benchflags now take precedence over values in the YAML config. If a wrapper script passes a flag that happened to be ignored before, that flag now takes effect. -
KV cache transfer timeout defaults to 60 seconds. Previously there was no timeout. In disaggregated serving, a slow transfer under load that used to eventually complete now errors. The knob is
cache_transceiver_config.kv_transfer_timeout_ms. -
Per-request chat templates are opt-in. Clients that send their own template per request will see them ignored unless the server starts with
--allow_request_chat_template. Output quality can change without an error.
Group 3: API surface (4 items). These fail for specific clients.
-
logprobsandprompt_logprobsare capped at 100; larger values raise aValueError. - The
/perf_metricsendpoint is removed. Use the request headerX-TRTLLM-return-metrics: 1. - Dynamic speculation fields were renamed to
acceptance_rate_window_sizeandacceptance_rate_threshold, and the threshold is now an acceptance rate between 0 and 1. - Obsolete options such as Eagle
choicesandtrtllm-serve --fail_fast_on_attention_window_too_largeare removed.
How do I audit my configs before upgrading?
Here is a small script that scans a repository of launch scripts and YAML files for the removed or changed names listed above. It is a starting point I wrote for this article, not an official tool; extend the pattern list for your own wrappers.
import pathlib, re, sys
PATTERNS = {
"removed: TensorRT backend": r"trtllm-build|TrtLlmArgs|BuildConfig|--engine_dir|trtllm-bench build",
"removed: decoding configs": r"MedusaDecodingConfig|LookaheadDecodingConfig",
"removed: two-model spec": r"eagle3_one_model|mtp_eagle_one_model",
"removed: sampler": r"TRTLLMSampler|disable_flashinfer_sampling",
"removed: endpoint": r"/perf_metrics",
"removed: flag": r"fail_fast_on_attention_window_too_large",
"changed default: KV timeout": r"cache_transceiver_config",
"changed default: chat template": r"chat_template",
"deprecated: TRITON MoE": r"TritonFusedMoE|backend\W+TRITON",
"deprecated: flags": r"--moe_cluster_parallel_size|--metrics-log-interval",
}
root = pathlib.Path(sys.argv[1])
for p in root.rglob("*"):
if p.suffix not in {".sh", ".yaml", ".yml", ".py", ".md", ".json"} or not p.is_file():
continue
text = p.read_text(encoding="utf-8", errors="ignore")
for label, pat in PATTERNS.items():
for m in re.finditer(pat, text):
line = text.count("\n", 0, m.start()) + 1
print(f"{p}:{line} [{label}] {m.group(0)}")
Run it as python audit.py ./deploy. The "changed default" lines are not errors. They are the places where you should write down what the old behavior was, then confirm the new one in a canary.
What should the canary check?
For the three silent changes, a canary needs specific assertions, not just a health check.
- YAML versus flag conflicts. Dump the effective configuration at startup and diff it against the previous release. Any value that differs is a flag that used to lose and now wins.
-
KV transfer timeout. Replay your longest-context traffic at peak concurrency through the disaggregated path and count timeout errors. If you see them, set
kv_transfer_timeout_msexplicitly rather than leaving the new default in place. -
Chat templates. Send one request with a per-request template and compare output to the old release. Decide whether you want
--allow_request_chat_template, because enabling it is a trust decision about who can send requests. -
Disaggregated auth. The notes deprecate disaggregated serving without
internal_request_auth_key. Set the same key in the disaggregated server config and every worker config now, while it is only a deprecation.
What else changes that is not marked breaking?
A few items deserve a line in your upgrade ticket even though they are not flagged:
-
Telemetry is on by default. Opt out with
TRTLLM_NO_USAGE_STATS=1,DO_NOT_TRACK=1,--no-telemetry, ortelemetry_config=TelemetryConfig(disabled=True). If your cluster blocks egress, check that this does not add noisy retries or startup delay. -
GPT-OSS defaults to the TRTLLM MoE backend on Blackwell. The TRITON MoE backend is deprecated with a three-month migration period; the notes say
CUTLASSreplaces it functionally on Hopper. -
TorchCompileConfig.max_num_streamsnow defaults to 3 instead of 1. That can change memory and scheduling behavior. - Prefix-aware scheduling is enabled by default to avoid duplicate prefill of shared prefixes, which is usually a win but changes batch composition.
-
Known issues to read before you pick hardware: the notes list a disaggregated DSpark issue where the draft KV cache is not transferred to the generation worker (aggregated serving is unaffected), a GLM-5.2 NVFP4 warmup segfault on GB200 when
max_num_tokensexceeds 4096, and compatibility gaps in/v1/responsesfor some input formats, streaming events and tool calls.
Should you move to 1.3.0 or wait for 1.4?
My reading, based only on the published notes and dates:
- Already on a 1.3.0 candidate: move to final. You have already absorbed most breaking changes, and you gain the final fixes.
- On 1.2.x with the PyTorch backend: 1.3.0 is the sensible target. Run the audit script, then the four canary checks.
-
Still on TensorRT engines (
trtllm-build): this is a project, not an upgrade. The backend is gone, so budget time for the migration guide and a re-benchmark. - Wanting the newest model targets: the 1.4.0 candidates are moving, and rc4 lists its own known issues (for example a Kimi K3 NVFP4 KV initialization failure on B300). Candidates are for evaluation. Pin a digest, not a tag.
FAQ
Is TensorRT-LLM 1.3.0 a stable release?
Yes. The GitHub release dated 2026-11-20 is not marked pre-release, unlike the 30 1.3.0rc candidates and the 1.4.0 candidates.
Does 1.3.0 remove the TensorRT backend?
Yes. The notes mark it as a breaking change: PyTorch is the only backend, and trtllm-build and related options are removed.
Will my YAML settings still apply?
Mostly, but explicit CLI flags for trtllm-serve, trtllm-eval and trtllm-bench now take precedence over YAML values, so a flag that used to lose may now win.
How do I turn off usage telemetry?
Set TRTLLM_NO_USAGE_STATS=1 or DO_NOT_TRACK=1, pass --no-telemetry, or set telemetry_config=TelemetryConfig(disabled=True) in the LLM API.
Why did KV transfers start failing in disaggregated serving?
The KV cache transfer timeout now defaults to 60 seconds, where it was previously unlimited. Set cache_transceiver_config.kv_transfer_timeout_ms to a value that matches your longest transfers.
Is the audit script official?
No. It is a simple pattern scan I wrote for this article. Treat matches as places to look, not proof of breakage.
Further reading
- TensorRT-LLM 1.4 Makes trtllm-serve Run 8 HTTP Frontends by Default
- Day-0 Model Support Is a Fork: What 26 NVIDIA Dynamo Snapshot Builds Say About Pinning Your LLM Serving Stack
Top comments (0)