The latest addition to the NVIDIA neural-rendering is NVIDIA DLSS 4. Its main feature, the Multi Frame Generator, is the attempt to generate multiple frames on top of a traditionally rendered frame, which is significantly higher in frame rates (up to a frightening 8x in extreme cases with GeForce RTX 50 Series hardware). Super Resolution, transformer-based models and new Blackwell hardware capabilities are also introduced with DLS 4 and help to accelerate neural rendering.
A brief history: between DLSS - DLSS 4.
DLSS (Deep Learning Super Sampling) was originally an AI upscaler (image reconstruction based on a low-res render). NVIDIA has over the generations of time added frame generation, improved temporal stability, and ray reconstruction. DLSS 4 combines those concepts into one suite and adds Multi Frame Generation as well as hardware/software co-design optimizations made possible by the Blackwell GPU architecture.
What is new in DLSS 4?
- Multi Frame Generation (MFG): AI creates up to three extra frames based on a traditionally rendered frame, and multiplies frame throughput (this is the fundamental DLSS 4 progress). Global MFG On GeForce RTX 50 Series hardware MFG is the high-end performance boost.
- Improved Super Resolution (transformer-based): a more recent type of transformer undoes detail retention and high motion stability.
- Ray Reconstruction and Ray-aware advancements: enhancing AI-based reconstruction of ray-traced effects to maintain render fidelity at reduced render cost.
- Hardware co-design Blackwell-specific features (5th-gen Tensor Cores, AI Management Processor, Optical Flow accelerators) enable the new models to execute MFG applications more quickly and in this way make MFG feasible.
The mechanism of DLSS 4 (high level - no deep math)
- The game makes a traditional frame (or a low-res version)
- The models of NVIDIA calculate optical flow / motion vectors and synthesize the intermediate frames using transformer-based super-resolution and ray reconstruction.
- Multi Frame Generation is able to generate many intermediary frames (up to 3 per rendered frame) which are consistent in time, and latency-sensitive methods such as NVIDIA Reflex are employed to maintain responsiveness on a compatible device. It uses Tensor cores and special hardware units of Blackwell GPUs.
Hardware & compatibility - what gets who.
- GeForce RTX 50 Series (Blackwell): full DLSS 4 support (Multi Frame Generation (hardware-accelerated and fastest performance) and others).
- GeForce RTX 40 Series: is capable of utilizing the Frame Generation and new Super Resolution models, and multi-frame MFG is enabled/optimized with RTX 50 Series hardware in most cases (NVIDIA provides model fallbacks and overrides).
- Other RTX GPUs: will also generally be benefited by new Super Resolution and Ray Reconstruction models when those are enabled by drivers/SDK.
Real-world impact & adoption
NVIDIA has declared wide implementation in hundreds of games and applications; more current proclamations comprise more than 100 games and applications initially with a total of over 175 games and applications being announced as having DLSS 4 / Multi Frame Generation assistance as the ecosystem keeps growing. That involves day one assistance in certain new releases as well as post-release updates of large titles.
Early patches and betas (e.g., patches and some Game Ready drivers by Cyberpunk 2077) have introduced support or previews of early DLSS 4, including indicating what route publishers are taking to provide engines with the capabilities of DLSS 4.
Pros and cons (practical view)
Pros
- frame-rate enhance ray-trace material (supports playable ray-traced 4K in certain instances).
- Improved visual fidelity and naive upscaling with transformer models and ray reconstruction.
Cons
- The biggest benefits would necessitate Blackwell (RTX 50) hardware to realize the full benefits of Multi Frame Generation.
- Frames that have been generated are synthesized - developers/testers may need to ensure that they are made to be temporal stable and have the input latency (games with a very low latency budget can require additional tuning).
- DLSS 4 must be incorporated into third-party engines (that take time and trials to work).
Developers: integration checklist (quick).
- Find the DLSS SDK version to use / docs NVIDIA Developer Read the DLSS developer docs / SDK Read the SDK and pick the correct version. (Plugins and samples of engine offered by NVIDIA.)
- Engine support: ensure your engine (Unreal Engine, Unity, custom) contains DLSS 4 plug-in or an update path. The UE5 builds are also regularly updated with NVIDIA plugins.
- Motion vectors & correct camera motion: make sure that motion vectors and depth/ray-trace inputs are correct - these are the key to the quality of optical-flow and MFG.
- Ray data when performing the reconstruction of the rays: When performing the reconstruction of the rays, feed the necessary ray buffers/parameters in order to get DLSS ray reconstruction to work.
- Test various settings: compare Super Resolution and Frame Generation and Multi Frame Generation and adjust latency and quality tradeoffs.
- Driver and runtime testing: test with the NVIDIA Game Ready drivers and with all of the supported GPU families.
Enabling DLSS 4 as a gamer (short)
- Install the newest GeForce Game Ready driver which supports DLSS 4 on your game. (NVIDIA announces driver notes of game releases with DLSS 4 support.)
- Game options Open graphics - find DLSS, or NVIDIA DLSS and select the options of DLSS 4 ( Super Resolution, Frame Generation, Multi Frame Generation) available to you based on your graphics card.
- In the case with RTX 50 series, Multi Frame Generation should be turned on to obtain the highest uplift; in the case with RTX 40 series, Frame Generation + Super Resolution should be used.
Hardware specifications (what NVIDIA boasts of)
In some demo situations, NVIDIA announcements show very large multipliers in frame rates of some ray-traced workloads - e.g. it claims that frame rate can improve by up to 8x compared to native brute-force rendering on GeForce RTX 50 Series with the full DLSS 4 stack enabled. Experimental results differ depending on game, quality preset, resolution and complexity of the scene.
Quick FAQ
Q: Is DLSS 4 just an upscaler?
A: No - it is a suite: enhanced Super Resolution (upscaling), Frame Generation, Ray Reconstruction and Multi Frame Generation (AI-synthesized intermediate frames).
Q: Do I need an RTX 50 GPU?
A: To use the benefits of full Multi Frame Generation hardware-accelerated, you require RTX 50 (Blackwell); RTX 40 still enjoys the value of updated Frame Generation and Super Resolution models.
Q: Does this enable the playability of path-traced games and 4K?
A: DLSS 4 is specifically designed to lower the performance cost of heavy ray tracing and allow much higher frame rates, hence making high-quality path-traced visuals much more viable on a high-end workstation, however, performance varies based on the game and settings that are selected.
Conclusion
DLSS 4 is a significant advancement in neural rendering: it extends single-frame upscaling to multi-frame AI synthesis and enables that at a realistic level with new features of the GPUs. To the players it offers much higher frame rates and improved ray-traced graphics; to the developers it represents an extra burden of complexity and yet a potent means of creating high end visuals at a low cost. Keep an eye on expanded engine/plugin releases and per-game patches as developers make DLSS 4 available in more games.
Top comments (0)