[Nvidia] Black screen when HDR option is enabled

[Algorithm0]

Is there an existing issue for this?

• I have searched the existing issues

Are you using any gamescope patches or a forked version of gamescope?

• The issue occurs on upstream gamescope without any modifications

Current Behavior

When I activate the HDR option, I see that nothing is displayed on my TV in terms of images (there is sound, I hear Steam starting up and I return to the desktop by the sound of the keys).
To run Steam on my TV I use ChimeraOS/gamescope-session.
I will describe what I see in some steps of reproducing the problem:
3. I see Steam on my TV. I see the TV showing that it is currently outputting HDR. I see a bar flashing at the bottom of my screen.
4. The entire image is terribly distorted, image fragments are mixed up, the screen flickers.
6. I don't see anything. The TV shows a black screen. There is no HDR on/off sign either. There is only sound.

Steps To Reproduce

1. Specify in the gamescope-session-plus.conf parametr ENABLE_GAMESCOPE_HDR=1;
2. Log out of the desktop session;
3. Login to "Steam Big Picture" session;
4. In Steam, go to display settings and enable HDR;
5. Select "Exit" and "Go to Desktop" (maybe there are other names on steam, I use another language, sorry);
6. Login to "Steam Big Picture" session;

Hardware information

- Distro: Fedora Linux 40 (Workstation Edition)
- CPU: AMD Ryzen™ 7 7800X3D
- GPU: NVIDIA GeForce RTX™ 4070 Ti
- Driver Version: 560.35.03

Software information

- Desktop environment: GNOME 46
- Session type: wayland
- Gamescope version: gamescope version 3.15.13 (gcc 14.2.1)
- Gamescope launch command(s): /usr/local/bin/gamescope -W 3840 -H 2160 --hdr-enabled --hdr-itm-enable --prefer-output HDMI-A-2 --xwayland-count 2 --default-touch-mode 4 --hide-cursor-delay 3000 --fade-out-duration 200 --steam -R /run/user/1000/gamescope.Mn36AFM/startup.socket -T /run/user/1000/gamescope.Mn36AFM/stats.pipe --prefer-vk-device 10de:2782

Which gamescope backends have the issue you are reporting?

• Wayland (default for nested gamescope)
• DRM (default for embedded gamescope, i.e. gamescope-session)
• SDL
• OpenVR

Logging, screenshots, or anything else

Logs after entering a session after enabling HDR from the Steam settings

[Algorithm0]

I have uploaded some videos to my drive with the appropriate titles so you can see what is going on.

[matte-schwartz]

Your logs show it failing to modeset for the resolution it's picking, which is probably when you get a black screen. The other part, that flickering, looks exactly like an issue I reported a few months ago https://forums.developer.nvidia.com/t/display-modes-above-2560x1440p-120hz-with-hdr-enabled-cause-flickering-corruption-within-gamescope-session/295314 but I did not create a dedicated issue here at the time.

Seems like maybe there's some NV issue with HDR over a certain resolution and refresh rate, but I don't think it ever got logged for an internal NV bug report.

This is my 4090 with gamescope's DRM backend (gamescope-session) at 7680x2160p@120hz with HDR enabled with the same issue you recorded

[kisak-valve]

The first thing to check would be if the NVIDIA 565 release series behaves better.

[matte-schwartz]

already did, no change sadly https://forums.developer.nvidia.com/t/565-release-feedback-discussion/310777/16

[matte-schwartz]

@cubanismo sorry for the additional ping here but this seems like an issue you'd want to be aware of if there isn't an internal NV report filed already. here are some repro steps for this issue on upstream gamescope with DRM backend (without needing to install gamescope-session-plus):

Method One: Using the upstream Valve gamescope package from: https://steamdeck-packages.steamos.cloud/archlinux-mirror/sources/jupiter-main/?C=M&O=D

Prep: Launch Steam first with the following launch options, and log into your account if prompted: steam -steamdeck -steamos -steampal -gamepadui - this needs to be done to get around an issue where -steamos3 as a launch option will cause the OOBE to complain about network issues without extra services. using -steamos first and then subsequently using -steamos3, which Valve's gamescope-session script sets by default, works around this issue.

1. download the latest .src.tar.gz for gamescope
2. untar the folder and then use makepkg -si to build and install (you may need a couple additional dependencies from the jupiter-main repo like Powerbuttond). using the built package for gamescope would also work, but it will probably complain about differing dependencies at runtime since SteamOS uses older package versions
3. log out of your desktop session, and choose SteamOS (gamescope) from your display manager's splash screen
4. once logged in, try going to Settings -> Display -> turn off "automatic resolution" -> choose your display's highest modeset
5. From the right side menu in Steam: enable HDR with the toggle, and if it's already on, toggle it off and then on again

You should see the issue at this point.

Method Two: trying to repro without gamescope officially packaged by Valve. this method is a bit less reliable than the session, but it should still show the same issue

Prep: Same applies as above

1. From a VT, launch gamescope with gamescope -e --hdr-enabled --mangoapp -- steam -steamdeck -steamos3 -steampal -gamepadui
2. if it looks normal, shutdown with gamescopectl shutdown from a separate terminal and then try to relaunch with the same parameters, but without --hdr-enabled
3. Repeat step 2, except add --hdr-enabled back

You should (hopefully) see the issue at this point.

---

feel free to ping me with any questions, gamescope-session isn't the most well documented feature (although that's something I personally want to try and work on improving) and it can be a bit finnicky depending on setup. Method One is definitely my recommendation since the HDR toggle on the side menu will work without you needing to restart Steam/gamescope, although it's janky enough that I would only recommend it to devs

[matte-schwartz]

It's much rarer, but it seems like the Wayland backend also has the possibility to be affected by the same flickering, even when the WSI layer is disabled with ENABLE_GAMESCOPE_WSI=0 and HDR is not used.

I do have the argument to allow Nvidia colorspace in my KWin environment left over from Gamescope testing yesterday. The flickering went away when I alt-tabbed out of full screen Gamescope and started again once tabbing back in.

My specific launch command was gamescope -f -h 2160 -w 7680 -r 120 --adaptive-sync in Lutris and only started after entering and exiting the game (Star Citizen) a few times without quitting the launcher. Screenshots did not capture the flickering which makes me suspect maybe something at scanout?

plasma-artifact-drm_info.log