The focus of this guide is on using Streamline to integrate the NVIDIA Image Scaling (NIS) SDK into an application. For more information about NIS itself, please visit the NVIDIA Image Scaling SDK Github Page
For information on user interface considerations when using the NIS plugin, please see the "RTX UI Developer Guidelines.pdf" document included with this SDK.
The NVIDIA Image Scaling SDK (NIS) provides a single spatial scaling and sharpening algorithm for cross-platform support. The scaling algorithm uses a 6-tap scaling filter combined with 4 directional scaling and adaptive sharpening filters, which creates nice smooth images and sharp edges. In addition, the SDK provides a state-of-the-art adaptive directional sharpening algorithm for use in applications where no scaling is required. By integrating both NVIDIA Image Scaling and NVIDIA DLSS, developers can get the best of both worlds: NVIDIA DLSS for the best image quality, and NVIDIA Image Scaling for cross-platform support.
The directional scaling and sharpening algorithm are combined together in NVScaler while NVSharpen only implements the adaptive-directional-sharpening algorithm. Both algorithms are provided as compute shaders and developers are free to integrate them in their applications. Note that if you integrate NVScaler, you should NOT also integrate NVSharpen, as NVScaler already includes a sharpening pass.
For more information on the NVIDIA Image scaling SDK visit https://github.com/NVIDIAGameWorks/NVIDIAImageScaling
Call slInit as early as possible (before any dxgi/d3d11/d3d12 APIs are invoked)
#include <sl.h>
#include <sl_consts.h>
#include <sl_nis.h>
sl::Preferences pref{};
pref.showConsole = true; // for debugging, set to false in production
pref.logLevel = sl::eLogLevelDefault;
pref.pathsToPlugins = {}; // change this if Streamline plugins are not located next to the executable
pref.numPathsToPlugins = 0; // change this if Streamline plugins are not located next to the executable
pref.pathToLogsAndData = {}; // change this to enable logging to a file
pref.logMessageCallback = myLogMessageCallback; // highly recommended to track warning/error messages in your callback
pref.applicationId = myId; // Provided by NVDA, required if using NGX components (DLSS 2/3)
pref.engineType = myEngine; // If using UE or Unity
pref.engineVersion = myEngineVersion; // Optional version
pref.projectId = myProjectId; // Optional project id
if(SL_FAILED(res, slInit(pref)))
{
// Handle error, check the logs
if(res == sl::Result::eErrorDriverOutOfDate) { /* inform user */}
// and so on ...
}For more details please see preferences
Call slShutdown() before destroying dxgi/d3d11/d3d12/vk instances, devices and other components in your engine.
if(SL_FAILED(res, slShutdown()))
{
// Handle error, check the logs
}Once the main device is created call slSetD3DDevice or slSetVulkanInfo:
if(SL_FAILED(res, slSetD3DDevice(nativeD3DDevice)))
{
// Handle error, check the logs
}As soon as SL is initialized, you can check if NIS is available for the specific adapter you want to use:
Microsoft::WRL::ComPtr<IDXGIFactory> factory;
if (SUCCEEDED(CreateDXGIFactory(__uuidof(IDXGIFactory), (void**)&factory)))
{
Microsoft::WRL::ComPtr<IDXGIAdapter> adapter{};
uint32_t i = 0;
while (factory->EnumAdapters(i, &adapter) != DXGI_ERROR_NOT_FOUND)
{
DXGI_ADAPTER_DESC desc{};
if (SUCCEEDED(adapter->GetDesc(&desc)))
{
sl::AdapterInfo adapterInfo{};
adapterInfo.deviceLUID = (uint8_t*)&desc.AdapterLuid;
adapterInfo.deviceLUIDSizeInBytes = sizeof(LUID);
if (SL_FAILED(result, slIsFeatureSupported(sl::kFeatureNIS, adapterInfo)))
{
// Requested feature is not supported on the system, fallback to the default method
switch (result)
{
case sl::Result::eErrorOSOutOfDate: // inform user to update OS
case sl::Result::eErrorDriverOutOfDate: // inform user to update driver
case sl::Result::eErrorNoSupportedAdapter: // cannot use this adapter (older or non-NVDA GPU etc)
// and so on ...
};
}
else
{
// Feature is supported on this adapter!
}
}
i++;
}
}NIS requires render-res input color after TAA and final-res output color buffers. We can tag resources list this:
// Showing two scenarios, depending if resources are immutable or volatile
// IMPORTANT: Make sure to mark resources which can be deleted or reused for other purposes within a frame as volatile
// FIRST SCENARIO
sl::Resource colorIn = sl::Resource{ sl::ResourceType::eTex2d, myNativeObject, nullptr, nullptr, myInitialState};
sl::Resource colorOut = sl::Resource{ sl::ResourceType::eTex2d, myNativeObject, nullptr, nullptr, myInitialState};
// Marked both resources as volatile since they can change
sl::ResourceTag colorInTag = sl::ResourceTag {&colorIn, sl::kBufferTypeScalingInputColor, sl::ResourceLifecycle::eOnlyValidNow, &myExtent };
sl::ResourceTag colorOutTag = sl::ResourceTag {&colorOut, sl::kBufferTypeScalingOutputColor, sl::ResourceLifecycle::eOnlyValidNow, &myExtent };
// Resources must be valid at this point and valid command list must be provided since resources are volatile
sl::Resource inputs[] = {colorInTag, colorOutTag};
slSetTag(viewport, inputs, _countof(inputs), cmdList);
// SECOND SCENARIO
// Marked both resources as immutable
sl::ResourceTag colorInTag = sl::ResourceTag {&colorIn, sl::kBufferTypeScalingInputColor, sl::ResourceLifecycle::eValidUntilPresent, &myExtent };
sl::ResourceTag colorOutTag = sl::ResourceTag {&colorOut, sl::kBufferTypeScalingOutputColor, sl::ResourceLifecycle::eValidUntilPresent, &myExtent };
// Resources are immutable so they are valid all the time, no need to provide command list since no copies need to be made
std::vector<sl::Resource> inputs = {colorInTag, colorOutTag};
slSetTag(viewport, inputs, _countof(inputs), cmdList);
IMPORTANT When using Vulkan additional information about the resource must be provided (width, height, format, image view etc). See
sl::Resourcefor details.
NIS options must be set so that the NIS plugin can track any changes made by the user:
// Using helpers from sl_nis.h
sl::NISOptions nisOptions{};
nisOptions.mode = NISMode::eNISModeScaler; // use upscaling algorithm or use eNISModeSharpen for sharpening only
nisOptions.hdrMode = NISHDR::eNISHDRNone; // No HDR mode;
// These can be populated based on user selection in the UI
nisOptions.sharpness = myUI->getSharpness();
if(SL_FAILED(result, slNISSetOptions(viewport, nisOptions)))
{
// Handle error here, check the logs
}NOTE: To use NIS sharpening only mode (with no up-scaling) set
sl::NISOptions.modetosl::NISMode::eSharpen
NOTE: To turn off NIS set
sl::NISOptions.modetosl::NISMode::eNISModeOffor simply stop callingslEvaluateFeature, note that this does NOT release any resources, for that please useslFreeResources
On your rendering thread, call slEvaluateFeature at the appropriate location where up-scaling is happening. Please note that myViewport used in slEvaluateFeature must match the one used when setting NIS options and tags (unless options and tags are provided as part of evaluate inputs)
// Make sure NIS is available and user selected this option in the UI
if(useNIS)
{
// NOTE: We can provide all inputs here or separately using slSetTag or slNISSetOptions
// Inform SL that NIS should be injected at this point for the specific viewport
const sl::BaseStructure* inputs[] = {&myViewport};
if(SL_FAILED(result, slEvaluateFeature(sl::kFeatureNIS, *frameToken, inputs, _countof(inputs), myCmdList)))
{
// Handle error
}
else
{
// IMPORTANT: Host is responsible for restoring state on the command list used
restoreState(myCmdList);
}
}
else
{
// Default up-scaling pass like for example TAAU goes here
}IMPORTANT: Plase note that host is responsible for restoring the command buffer(list) state after calling
slEvaluateFeature. For more details on which states are affected please see restore pipeline section