-
-
Notifications
You must be signed in to change notification settings - Fork 8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
win-capture: Avoid NULL deref when capture not initialized #11375
base: master
Are you sure you want to change the base?
Conversation
Removed myself as a reviewer since I'm not active enough to be holding things up. |
I take jpark37's points. This change does not really get at root causes. It's little challenging on my end because I don't have physical access to any of the machines where this crash is being reported. But with more effort, I can determine the root issue on the graphics-hook side. Or maybe I should just focus on modifying the game client to avoid putting graphics-hook in a position where it crashes and then this PR won't be needed at all? Lain-B, do you have any thoughts? |
For reference, I believe #11430 goes the "sprinkling a null check somewhere else" route by checking @confusionattack does that PR (already merged) resolve your observed crashes? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So this happens because two APIs can be used at the same time, so capture_active()
will return true even in other hooked APIs, which results in this issue. I think for the time being this is an acceptable change and works fine to alleviate this issue.
I'm just going to force-push a change to rebase and increment the graphics hook patch version on this PR's branch and then we can merge this.
Regardless, this will not cause a regression.
Description
Add a check to avoid a NULL dereference on backbuffer capture in the D3D10, D3D11, and D3D12 capture modules in case d3d10_data, d3d11_data, or d3d12_data has not yet been successfully configured. This is similar to NULL checks in other places in these modules, such as the d3d10_free, d3d11_free, and d3d12_free functions. This change is intended to fix a specific issue with D3D12 capture we are seeing the wild, but for completeness we added NULL checks to the D3D10 and D3D11 capture modules.
Motivation and Context
We were seeing an intermittent crash in graphics-hook when using OBS Studio to capture the Midnight Murder Club demo (https://store.steampowered.com/app/2698870/Midnight_Murder_Club/) on some laptops with multiple GPUs. We tracked it down to a NULL dereference when capturing the backbuffer in the D3D12 capture module.
How Has This Been Tested?
We were able to a user who could reliably replicate the crash on an Intel i7-11800H laptop running Windows 11 with an NVIDIA GeForce RTX 3060 Laptop GPU and a built-in Intel UHD GPU. The repro steps were as follows:
After applying implementing the NULL check fix and testing against a locally built OBS Studio, we were unable to reproduce any issue.
Types of changes
Checklist: