DEV Community

Cover image for TensorRT-LLM 1.3.0 Is Final After 30 Release Candidates: The 13 Breaking Changes to Audit Before You Upgrade
AI Tech News
AI Tech News

Posted on

TensorRT-LLM 1.3.0 Is Final After 30 Release Candidates: The 13 Breaking Changes to Audit Before You Upgrade

TL;DR

  • NVIDIA published TensorRT-LLM v1.3.0 as a stable release on 2026-11-20. It follows 30 release candidates (rc0 on 2026-01-22 through rc29 on 2026-09-29), roughly 302 days from first candidate to final.
  • The release notes list 13 items marked BREAKING CHANGE. Six remove whole components (the TensorRT backend, the legacy C++ Executor, TensorRT-engine Triton models, TRTLLMSampler, two-model speculative decoding, and AutoDeploy/Star Attention/WIDEEP), three change defaults silently, and four change the API surface.
  • The three default changes are the dangerous ones, because nothing crashes at startup: explicit CLI flags now override YAML, KV transfer now times out after 60 seconds, and per-request chat templates are opt-in.
  • Usage telemetry is on by default. The documented opt-outs are TRTLLM_NO_USAGE_STATS=1, DO_NOT_TRACK=1, or --no-telemetry.
  • The 1.4.0 candidates (rc0 on 2026-10-14, rc4 on 2026-11-18) were already public before this final tag, so 1.3.0 is best read as the stable line, not the newest code.

What actually shipped in TensorRT-LLM 1.3.0?

The GitHub release for NVIDIA/TensorRT-LLM tagged v1.3.0 is dated 2026-11-20 and is not marked pre-release. The notes are long (about 18,000 characters), so it helps to sort them into three buckets before deciding what to do.

Feature headline items (the parts that sell the release):

  • Preview support for NVIDIA Rubin (SM107), plus DeepSeek-V4, Kimi K3 and MiniMax-M3 model support on Blackwell and Rubin.
  • New serving surface: an Anthropic Messages endpoint (/v1/messages), a gRPC server, multi-process HTTP frontends, Prometheus metrics, and /start_profile and /stop_profile endpoints.
  • Speculative decoding additions: DSpark, DFlash, Suffix Automaton and PARD drafters, EAGLE3 dynamic trees, and acceptance-rate-based auto-disable.
  • KV cache work: KV Cache Manager V2 enabled by default for most model families, NVFP4 KV cache, cold-page compression, FlexKV and LMCache connectors.
  • A new vertical called VisualGen for image and video generation models served through the same trtllm-serve.

Infrastructure changes that touch your build: base image pytorch:26.09-py3, CUDA Toolkit 13.4.1, PyTorch 2.14.0, C++20, Python 3.10 and 3.12, and TensorRT, ModelOpt and ONNX are no longer dependencies.

Breaking changes, which are the subject of the rest of this article.

Why does a final release arrive after 30 candidates?

Release candidates per month for TensorRT-LLM 1.3.0

The candidate cadence in the chart is derived from the publish timestamps on the GitHub releases API: between two and five candidates every month from January to September. There was no long gap, which suggests the project treats main as continuously releasable and cuts a numbered candidate each time.

Two practical consequences follow:

  1. Candidates are snapshots, not a stabilization funnel. Each candidate carries a fresh pile of features, and the notes for individual candidates run to tens of thousands of characters. Pinning to rc17 and "waiting for final" tells you very little about what final contains.
  2. The final tag came after the next line started. The 1.4.0 candidates began on 2026-10-14, and 1.4.0rc4 was published on 2026-11-18, two days before 1.3.0 final. The API reports the 1.3.0 target as main. I would not infer more than the dates show, but the practical reading is that "1.3.0" is a stable label for a large feature set, while the active development line is already 1.4.

If you track release trains for a serving stack, the lesson is the same as with other fast-moving engines: a version number tells you less than the changelog diff between your pin and the target. For a related look at how fast a pin ages, see my earlier note on what 26 Dynamo snapshot builds say about pinning.

Which of the 13 breaking changes will break a deployment?

Breaking changes grouped by type

The grouping above is mine; the 13 items are exactly those the release notes mark [BREAKING CHANGE].

Group 1: Removals (6 items). These fail loudly.

Removed What replaces it
TensorRT backend (trtllm-build, TrtLlmArgs, BuildConfig, MedusaDecodingConfig, LookaheadDecodingConfig, trtllm-bench build, trtllm-bench --engine_dir) PyTorch backend only; see the project migration guide
Legacy C++ Executor, legacy runtime and batch-manager APIs, cpp_only build option PyTorch backend and LLM API
TensorRT-engine Triton backend models Only the llmapi Triton model remains
TRTLLMSampler TorchSampler, which requires FlashInfer; sampler_type is ignored with a warning
Two-model speculative decoding (eagle3_one_model, mtp_eagle_one_model fields) One-model implementations
AutoDeploy, Star Attention, WIDEEP MoE backend None listed

These are easy: if you used any of them, your launch fails or a build script breaks, and you find out in staging. The one that deserves a second look is the sampler: disable_flashinfer_sampling is gone, so an environment that deliberately avoided FlashInfer now cannot.

Group 2: Changed defaults (3 items). These fail quietly.

  1. CLI flags override YAML. Explicit trtllm-serve, trtllm-eval and trtllm-bench flags now take precedence over values in the YAML config. If a wrapper script passes a flag that happened to be ignored before, that flag now takes effect.
  2. KV cache transfer timeout defaults to 60 seconds. Previously there was no timeout. In disaggregated serving, a slow transfer under load that used to eventually complete now errors. The knob is cache_transceiver_config.kv_transfer_timeout_ms.
  3. Per-request chat templates are opt-in. Clients that send their own template per request will see them ignored unless the server starts with --allow_request_chat_template. Output quality can change without an error.

Group 3: API surface (4 items). These fail for specific clients.

  • logprobs and prompt_logprobs are capped at 100; larger values raise a ValueError.
  • The /perf_metrics endpoint is removed. Use the request header X-TRTLLM-return-metrics: 1.
  • Dynamic speculation fields were renamed to acceptance_rate_window_size and acceptance_rate_threshold, and the threshold is now an acceptance rate between 0 and 1.
  • Obsolete options such as Eagle choices and trtllm-serve --fail_fast_on_attention_window_too_large are removed.

How do I audit my configs before upgrading?

Here is a small script that scans a repository of launch scripts and YAML files for the removed or changed names listed above. It is a starting point I wrote for this article, not an official tool; extend the pattern list for your own wrappers.

import pathlib, re, sys

PATTERNS = {
    "removed: TensorRT backend": r"trtllm-build|TrtLlmArgs|BuildConfig|--engine_dir|trtllm-bench build",
    "removed: decoding configs": r"MedusaDecodingConfig|LookaheadDecodingConfig",
    "removed: two-model spec": r"eagle3_one_model|mtp_eagle_one_model",
    "removed: sampler": r"TRTLLMSampler|disable_flashinfer_sampling",
    "removed: endpoint": r"/perf_metrics",
    "removed: flag": r"fail_fast_on_attention_window_too_large",
    "changed default: KV timeout": r"cache_transceiver_config",
    "changed default: chat template": r"chat_template",
    "deprecated: TRITON MoE": r"TritonFusedMoE|backend\W+TRITON",
    "deprecated: flags": r"--moe_cluster_parallel_size|--metrics-log-interval",
}

root = pathlib.Path(sys.argv[1])
for p in root.rglob("*"):
    if p.suffix not in {".sh", ".yaml", ".yml", ".py", ".md", ".json"} or not p.is_file():
        continue
    text = p.read_text(encoding="utf-8", errors="ignore")
    for label, pat in PATTERNS.items():
        for m in re.finditer(pat, text):
            line = text.count("\n", 0, m.start()) + 1
            print(f"{p}:{line}  [{label}]  {m.group(0)}")
Enter fullscreen mode Exit fullscreen mode

Run it as python audit.py ./deploy. The "changed default" lines are not errors. They are the places where you should write down what the old behavior was, then confirm the new one in a canary.

What should the canary check?

For the three silent changes, a canary needs specific assertions, not just a health check.

  1. YAML versus flag conflicts. Dump the effective configuration at startup and diff it against the previous release. Any value that differs is a flag that used to lose and now wins.
  2. KV transfer timeout. Replay your longest-context traffic at peak concurrency through the disaggregated path and count timeout errors. If you see them, set kv_transfer_timeout_ms explicitly rather than leaving the new default in place.
  3. Chat templates. Send one request with a per-request template and compare output to the old release. Decide whether you want --allow_request_chat_template, because enabling it is a trust decision about who can send requests.
  4. Disaggregated auth. The notes deprecate disaggregated serving without internal_request_auth_key. Set the same key in the disaggregated server config and every worker config now, while it is only a deprecation.

What else changes that is not marked breaking?

A few items deserve a line in your upgrade ticket even though they are not flagged:

  • Telemetry is on by default. Opt out with TRTLLM_NO_USAGE_STATS=1, DO_NOT_TRACK=1, --no-telemetry, or telemetry_config=TelemetryConfig(disabled=True). If your cluster blocks egress, check that this does not add noisy retries or startup delay.
  • GPT-OSS defaults to the TRTLLM MoE backend on Blackwell. The TRITON MoE backend is deprecated with a three-month migration period; the notes say CUTLASS replaces it functionally on Hopper.
  • TorchCompileConfig.max_num_streams now defaults to 3 instead of 1. That can change memory and scheduling behavior.
  • Prefix-aware scheduling is enabled by default to avoid duplicate prefill of shared prefixes, which is usually a win but changes batch composition.
  • Known issues to read before you pick hardware: the notes list a disaggregated DSpark issue where the draft KV cache is not transferred to the generation worker (aggregated serving is unaffected), a GLM-5.2 NVFP4 warmup segfault on GB200 when max_num_tokens exceeds 4096, and compatibility gaps in /v1/responses for some input formats, streaming events and tool calls.

Should you move to 1.3.0 or wait for 1.4?

My reading, based only on the published notes and dates:

  • Already on a 1.3.0 candidate: move to final. You have already absorbed most breaking changes, and you gain the final fixes.
  • On 1.2.x with the PyTorch backend: 1.3.0 is the sensible target. Run the audit script, then the four canary checks.
  • Still on TensorRT engines (trtllm-build): this is a project, not an upgrade. The backend is gone, so budget time for the migration guide and a re-benchmark.
  • Wanting the newest model targets: the 1.4.0 candidates are moving, and rc4 lists its own known issues (for example a Kimi K3 NVFP4 KV initialization failure on B300). Candidates are for evaluation. Pin a digest, not a tag.

FAQ

Is TensorRT-LLM 1.3.0 a stable release?
Yes. The GitHub release dated 2026-11-20 is not marked pre-release, unlike the 30 1.3.0rc candidates and the 1.4.0 candidates.

Does 1.3.0 remove the TensorRT backend?
Yes. The notes mark it as a breaking change: PyTorch is the only backend, and trtllm-build and related options are removed.

Will my YAML settings still apply?
Mostly, but explicit CLI flags for trtllm-serve, trtllm-eval and trtllm-bench now take precedence over YAML values, so a flag that used to lose may now win.

How do I turn off usage telemetry?
Set TRTLLM_NO_USAGE_STATS=1 or DO_NOT_TRACK=1, pass --no-telemetry, or set telemetry_config=TelemetryConfig(disabled=True) in the LLM API.

Why did KV transfers start failing in disaggregated serving?
The KV cache transfer timeout now defaults to 60 seconds, where it was previously unlimited. Set cache_transceiver_config.kv_transfer_timeout_ms to a value that matches your longest transfers.

Is the audit script official?
No. It is a simple pattern scan I wrote for this article. Treat matches as places to look, not proof of breakage.

Further reading

Sources

Top comments (0)