Neural Texture Compression (NTC) is an algorithm designed to compress all PBR textures used for a single material together. It works best when the texture channels are correlated with each other, for example, detail in the albedo texture corresponds to detail in the normal texture. Up to 16 texture channels can be compressed into one NTC texture set. Typical PBR materials have 9-10 channels: 3x albedo, 3x normal, metalness, roughness, ambient occlusion, opacity.
During compression, the original texture data is transformed into a combination of weights for a small neural network (decoder) and a tensor of latents or features that are sampled and passed through the decoder to reconstruct the texture colors, as is illustrated below. The sampling and decoding processes are fast enough to use them directly in the shaders that normally sample the material textures, such as base pass pixel shaders or ray tracing hit shaders. However, the decoder produces unfiltered data for only one texel, and in order to get filtered textures, we suggest using NTC in combination with Stochastic Texture Filtering (STF). For renderers targeting lower-end hardware, we suggest implementing the "Inference on Load" mode where NTC textures are decompressed when the game or map is loaded, and transcoded to one of the block-compressed formats (BCn) at the same time.
For more background information, please refer to the Random-Access Neural Compression of Material Textures page on the NVIDIA Research website.
NTC can be thought of as an adjustable quality/constant bitrate lossy compression scheme. This means that it will attempt to reconstruct the input images with minimal error while using a fixed amount of data specified as a compression time target. However, unlike block compression schemes (which have a fixed data rate for a given format) the per-texel memory footprint of a NTC-compressed texture bundle will vary based in the specified Latent Shape (which is the composite of the number of high- and low-resolution latent channels, the bit-depth of those high- and low-resolution channels, and the scale factor between them). Each latent shape corresponds to a given per-texel bitrate, and thus for a desired compression level a compatible latent shape can be selected which corresponds to desired bitrate as closely as possible. Furthermore, although NTC compression requires specifying a latent shape (and thus bitrate) it is possible to approximate a constant quality/variable bitrate approach by performing pre-analysis of the bundle to determine what formats are required to achieve a target quality level. See Adaptive Compression for more.
To demonstrate how NTC compares to other methods, consider a material defined by the following bundle of material textures:
- Albedo Color (RGB)
- Normal (XY)
- Roughness
- Metalness
- Ambient Occlusion Factor
Example material from MetalPlates013 from AmbientCG
Assuming 8 bits per channel and optimal channel packing, this corresponds to a bitrate of 64 bits/texel. In a contemporary pipeline this might be block-compressed into one BC7 texture for Albedo (8 bits/texel), one BC5 texture for normals (4 bits/texel), and a third BC7 texture for Roughness, Metalness, and Ambient Occlusion packed as separate channels (for another 8 bits/texel). For NTC we have found that many real-world texture bundles of this format can be compressed with results comparable to BCn (a PSNR of 40 to 50 dB) with a latent shape requiring about 3 bits/texel.
If we assume a 2k-by-2k texture resolution (and ignore the mip chains) we can compute the texture footprint of the bundle at various points in the data pipeline:
| Bundle Compression | Disk Size | PCI-E Traffic | VRAM Size |
|---|---|---|---|
| Raw Image | 32.00 MB | 32.00 MB | 32.00 MB |
| BCn Compressed | 10.00 MB | 10.00 MB | 10.00 MB |
| NTC-on-Load* | 1.52 MB | 1.52 MB | 10.00 MB |
| NTC-on-Sample | 1.52 MB | 1.52 MB | 1.52 MB |
*: Assumes transcoding to equivalent BCn formats at decompression time.
See the Settings and Quality Guide to learn more about various NTC settings and how they affect compression ratios and image quality.
Decompressing texels with NTC requires reading the latent data corresponding to a given texture coordinate and then performing an inference operation by running it through a small Multi-Layer Perceptron (MLP) network whose weights are determined during compression and stored as part of the compressed bundle. While this operation is modest relative to the massive networks employed by many other deep learning applications it still carries a significant computational cost relative to the average pixel shader commonly seen in 3D rendering applications. Fortunately, NTC is able to benefit from new Cooporative Vector extensions for Vulkan and Direct3D 12 which allow pixel shaders to leverage the same hardware acceleration used in large network inference. On Ada- and Blackwell-class GPUs this provides a 2-4x improvement in inference throughput over competing optimal implementations that do not utilize these new extensions.
In order to provide robust backwards compatibility, fallback implementations of the inference code using the dp4a instructions have also been provided. This will allow for the decompression code to be executed reliably on any platform that supports at least Direct3D 12 Shader Model 6; however, there will be substantial performance improvements on newer GPUs. See System Requirements for more details.
NTC texture decompression for graphics applications, both on-load and on-sample, uses an experimental implementation of Cooperative Vectors on DX12. This implementation relies on an NVIDIA customized version of the DirectX Shader Compiler (DXC) that produces code with new DXIL instructions. Shaders using these instructions cannot be signed, and Windows must be in the Developer Mode for them to work.
The experimental implementation will be replaced by the official Microsoft API later in 2025, and NVIDIA driver support for the current version will be removed shortly after that. All non-CoopVec versions of DX12 decompression, as well as all Vulkan versions including CoopVec, are OK to use for shipping.
The DX12 Cooperative Vector support is for testing purposes only. DO NOT SHIP ANY PRODUCTS USING IT.
See the Build Guide for instructions on compiling the SDK.
To experiment with how Neural Texture Compression performs with different latent shapes and texture bundles, follow the NTC Explorer Guide. A few sample material sets have been provided in this package under the ./assets/materials folder.
To see how NTC works on a sample 3D scene follow the instructions in the NTC Renderer Guide for pre-processing a GLTF scene. A sample scene has been provided under the ./assets/models folder.
| Component | Description | Source code |
|---|---|---|
| LibNTC | Library that implements neural texture compression, decompression, and BCn encoding | See Component |
ntc-cli |
Command-line tool for texture compression and decompression | tools/cli |
| NTC Explorer | App for interactive experimentation with neural texture compression and a viewer for NTC files | tools/explorer |
| NTC Renderer | Sample app that demonstrates how to render a GLTF model using NTC materials | samples/renderer |
| BCTest | Test app for evaluating the performance and quality of BCn encoders | support/tests/bctest |
ntc.py |
Python module for developing automation scripts that process materials using ntc-cli |
See Component |
test.py |
Script for basic functional testing of texture compression and decompression | See Component |
| Materials | Example materials for the CLI tool and Explorer | |
| FlightHelmet model | Example model for the Renderer sample |
Operating System:
- Windows 10/11 x64
- Linux x64
Graphics APIs:
- DirectX 12 - with Agility SDK for Cooperative Vector support (optional)
- Vulkan 1.3
GPU for NTC decompression on load and transcoding to BCn:
- Minimum: Anything compatible with Shader Model 6 [*]
- Recommended: NVIDIA Turing (RTX 2000 series) and newer.
GPU for NTC inference on sample:
- Minimum: Anything compatible with Shader Model 6 (will be functional but very slow) [*]
- Recommended: NVIDIA Ada (RTX 4000 series) and newer.
GPU for NTC compression:
- Minimum: NVIDIA Turing and (RTX 2000 series).
- Recommended: NVIDIA Ada (RTX 4000 series) and newer.
[*] The oldest GPUs that the NTC SDK functionality has been validated on are NVIDIA GTX 1000 series, AMD Radeon RX 6000 series, Intel Arc A series.
For Cooperative Vector support on NVIDIA GPUs, please use NVIDIA Graphics Driver version 570 or newer.
For a list of software components needed to build the SDK, please refer to the [Build Guide](##Building NTC SDK).
The following issues are observed with NVIDIA Display Driver 572.16:
- Cooperative Vector inference on DX12 is incompatible with NVIDIA Turing and Ampere GPUs, disabled in LibNTC.
- Image corruption in NTC Explorer when running on Vulkan.
- Non-CoopVec (DP4a) inference performance is lower than expected on NVIDIA Blackwell GPUs.
NTC SDK supports Windows x64 and Linux x64 targets.
Building the NTC SDK on Windows requires the following components:
- Visual Studio 2022 (at least the build tools)
- Windows SDK (tested with 10.0.22621.0 and 10.0.26100.0)
- CMake (tested with v3.28 and v3.31)
- CUDA SDK (tested with 12.4 and 12.8)
Follow the usual way of building CMake projects on Windows:
- Clone the project recursively:
git clone --recursive https://github.com/NVIDIA-RTX/RTXNTC.git
- Building using the "x64 Native Tools Command Prompt for VS 2022":
cd RTXNTC mkdir build cd build cmake .. cmake --build .
- Building using CMake GUI:
- Set "Where is the source code" to the
RTXNTCfolder - Set "Where to build the binaries" to
RTXNTC/build - Configure a solution using
Visual Studio 2022tools for thex64platform. - Generate and open the solution in Visual Studio.
- Build.
- Set "Where is the source code" to the
Visual Studio Code with CMake Tools extension and Ninja build system works fine, too.
Building the NTC SDK on Linux requires the following components:
- C++ compiler (tested with GCC 12.2 and Clang 16.0)
- CMake (tested with v3.25 and 3.31)
- CUDA SDK (tested with v12.4)
- Some development packages, approximately:
sudo apt-get update sudo apt-get install build-essential cmake libx11-dev libxrandr-dev libxinerama-dev libxcursor-dev libxi-dev
Follow the usual way of building CMake projects on Linux:
- Clone the project recursively:
git clone --recursive https://github.com/NVIDIA-RTX/RTXNTC.git
- Create the build folder, configure and build:
mkdir build && cd build cmake .. make -j
The SDK provides several tools for incorporating NTC into a content pipeline and engine runtime.
The LibNTC library provides all functionality necessary to compress, serialize/deserialize, and decompress texture sets using whatever hardware acceleration is available. It also provides implementation of a GPU-based BCn encoder to allow for transcoding decompressed textures into block-compressed format at load time, as well as a shader library demonstrating how to decompress bundle texels directly in pixel shaders at sample time.
Additionally, the ntc-cli tool can be included as part of any script-based pipelines if that is preferable. This tool is based on LibNTC as well and can perform compression and decompression tasks as desired. For more information on syntax please consult the NTC Command-Line Tool Documentation. Texture bundle information can be specified directly from the command line; however, the ntc-cli tool can also be given a Bundle Manifest File which encodes image file path, semantic and formatting information, and desired transcoding formats into one conveient serialized object. Your content pipeline can either track this information using its own database or used the provided manifest format as desired.
Further details about specific usages of LibNTC can be found divided by topic in the following guides, which walk through a proposed pipeline for a typical NTC application:
- Library Initialization
- Compression
- Decompression
The NTC Explorer and NTC Renderer samples demonstrate using LibNTC for compression and decompression and can be used as a further reference for what an integrated content workflow might look like.
Please use GitHub issues or email [email protected] for developer support.
This project includes NVAPI software. All uses of NVAPI software are governed by the license terms specified here: https://github.com/NVIDIA/nvapi/blob/main/License.txt