ntc-renderer fails to handle VK_SUBOPTIMAL_KHR correctly

[Saancreed]

While trying to launch binaries from https://github.com/NVIDIA-RTX/RTXNTC/releases/tag/v0.5.0-beta release, I've noticed that the renderer successfully launches, presents two images, then abruptly exits without any error on my system. Wine logs pointed me to one interesting event that happens just before the renderer goes into teardown sequence:

0024:trace:vulkan:thunk64_vkEndCommandBuffer 0x203140
0024:trace:vulkan:thunk64_vkQueueSubmit 0x207688, 1, 0x11e630, 0x0
0024:trace:vulkan:thunk64_vkQueueSubmit 0x207688, 1, 0x11ec20, 0x0
0024:trace:vulkan:thunk64_vkQueuePresentKHR 0x207688, 0x11ed50
0024:trace:vulkan:win32u_vkQueuePresentKHR queue 0x55557e5a49b8, present_info 0x11ed50
0024:warn:vulkan:win32u_vkQueuePresentKHR Present returned status 1000001003 for swapchain 0x55557f90b800
0024:trace:vulkan:thunk64_vkDeviceWaitIdle 0x207670
0024:trace:vulkan:thunk64_vkDestroySampler 0x207670, 0x55557f993ff0, (nil)
0024:trace:vulkan:thunk64_vkDestroySampler 0x207670, 0x55557fa34090, (nil)
0024:trace:vulkan:thunk64_vkDestroyDescriptorSetLayout 0x207670, 0x55557fd5b1b0, (nil)
0024:trace:vulkan:thunk64_vkDestroyPipelineLayout 0x207670, 0x55557fd865f0, (nil)
0024:trace:vulkan:thunk64_vkDestroyPipeline 0x207670, 0x55557fd62fb0, (nil)
0024:trace:vulkan:thunk64_vkDestroyCuFunctionNVX 0x207670, 0x555580c154f0, (nil)
0024:trace:vulkan:thunk64_vkDestroyCuModuleNVX 0x207670, 0x555580c8c4a0, (nil)
0024:trace:vulkan:thunk64_vkDestroyImageView 0x207670, 0x555580b243d0, (nil)
0024:trace:vulkan:thunk64_vkDestroyImage 0x207670, 0x555580b22d70, (nil)
0024:trace:vulkan:thunk64_vkFreeMemory 0x207670, 0x555580b23400, (nil)

It appears that when receiving VK_SUBOPTIMAL_KHR from vkQueuePresentKHR, the renderer doesn't know how to proceed and exits, treating this result as some kind of error that's impossible to recover from. I wrote and enabled a dumb Vulkan layer that prevents VK_SUBOPTIMAL_KHR from ever being returned to applications:

#include "vkroots.h"

struct VkDeviceOverrides
{
    static VkResult QueuePresentKHR(vkroots::VkDeviceDispatch const *pDispatch, VkQueue queue, VkPresentInfoKHR const *pPresentInfo)
    {
        auto result = pDispatch->QueuePresentKHR(queue, pPresentInfo);

        if (pPresentInfo->pResults)
        {
            for (auto i = 0u; i < pPresentInfo->swapchainCount; ++i)
            {
                if (pPresentInfo->pResults[i] == VkResult::VK_SUBOPTIMAL_KHR)
                    pPresentInfo->pResults[i] = VkResult::VK_SUCCESS;
            }
        }

        return result == VkResult::VK_SUBOPTIMAL_KHR ? VkResult::VK_SUCCESS : result;
    }
};

VKROOTS_DEFINE_LAYER_INTERFACES(vkroots::NoOverrides, vkroots::NoOverrides, VkDeviceOverrides);

… and with that I could launch the sample as presumably intended, but I suppose that's not something I should have been required to do in the first place.

This was tested on NVIDIA GeForce RTX 4080 Mobile with 570.86.16 driver.