Material system: Implement GPU occlusion culling

[VReaperV]

Implement occlusion culling in compute shaders using a Hierarchical Depth-Buffer.

The depth buffer is processed before the post-processing pass, reducing it to the next mip level until a 1x1 resolution is reached.
Then it's used to find out if a surface that passed frustum culling is behind some opaque surfaces. This is done by getting the screen-space AABB from the bounding sphere of the surface, and then comparing its depth to the depth in the appropriate level of the depth-reduction image. The level of the depth image is chosen based on how much of the screen space it's taking up.

[VReaperV]

The sphere projection function is giving incorrect results apparently, since I'm pretty sure the world->view space conversion is correct. I'm also not seeing much of any difference in frame time between this change and frustum culling-only: the frame time that I win in having less draws is nullified by the hi-z pass. It could be improved a bit by making the depth buffer downsampling be done in one compute dispatch, but it might just not really be worth it. On the other hand the hi-z buffer could be used for some other effects too.

[VReaperV]

Well, I'm seeing some improvements in certain cases, especially with high graphics settings. The whole thing also isn't taking too much frame time in itself anyway (the indirect draws + cull compute).

[VReaperV]

So the lower framerate that illwieckz was getting in #1137 might have to do with something else. Ideally we could also use better synchronization, but I'm not sure that would be possible.

[VReaperV]

Possibly rasterizing the bounding boxes with colour/depth writes disabled to determine object visibility would be faster. The depth pre-pass is super fast wherever I looked at it here, so this should be fast too. It does, however, mean that object visibility would be lagging behind by a frame, but that's already happening with this hi-z algorithm.

It would also require a bunch of writes to a buffer from every rasterized fragment that passed the depth test, which is not ideal. Perhaps a custom rasterizer in a compute shader that would instead write the visible object ids once per triangle or a batch of triangles would be better.

[VReaperV]

There are a few more things that could be improved independent of occlusion culling that might make this work faster in every case anyway:

1. Use batches of e. g. 64 triangles instead of specifying surfaces, which would allow dropping the processSurfaces_cp shader and doing writes in the cull_cp shader instead.
2. That would also allow creating an index buffer instead of an indirect buffer, for the most part. There would still be one draw per material, but that draw would have all the triangles in it (that weren't culled).
3. Potentially with triangle batches culling at triangle batch level might speed things up even more: e. g. with cone culling.

[VReaperV]

Another potential advantage of compute rasterizer is assigning light sources to a tile/cluster buffer, though with low light counts it might not make much of a difference.

[VReaperV]

Doing hardware rasterization of BBoxes and writing to a buffer from the fragment shader with an early discard doesn't work well for some reason.

[VReaperV]

I believe this fixes the incorrect projection. However there is still some incorrect culling, which might be due to incorrect depth downsampling with even/odd dimensions, haven't checked yet.

[VReaperV]

This mostly works now, however there's still an error with AABB depth projection, and some things need to be un-hardcoded.

[VReaperV]

Added some comments and made r_lockPVS 1 work with occlusion culling.

[VReaperV]

Rebased.

[VReaperV]

Cleaned up a bit, and added a toggle through r_gpuOcclusionCulling.

[illwieckz]

I'm not sure to understand everything, but the overall looks good to me and I confirm it fixes many bugs.

I would appreciate some better spacing:

I also noticed some new GLSL files use spaces or seem to mix tabs and spaces, I would prefer tabs.

[illwieckz]

Lol I wanted to approve #1137 instead… 🤦‍♀️️

Wrong one.

[slipher]

This works by creating a depth-reduction image from the depth buffer, then using it on the next frame.

How can data generated on the previous frame be valid? Everything could be in a different position or viewed from a different angle.

[VReaperV]

How can data generated on the previous frame be valid? Everything could be in a different position or viewed from a different angle.

That's a mistake in pr description, what is meant is that the results of the culling are used on the next frame, i. e. double-buffered (the depth buffer is used from the current frame).

[VReaperV]

Fixed pr description.

[slipher]

The object won't always occupy 2 pixels in both dimensions, right? It could have some oblong shape and be 1 pixel wide in the x axis and 2 in the y axis for example. I understand this is not a bug as the worst thing that could happen here is a surface not being culled, but I wonder if there isn't another way to do the calculation.

[VReaperV]

That can happen sometimes, yes. I've tried wrapping the texel fetches in if statements so only the first one is guaranteed and the next ones are only done if the new coordinates don't match any of the previous ones, but some surfaces became invisible when they shouldn't be. Not sure why as I couldn't spot what the issue was. This can be improved later though.

[VReaperV]

There's also an extension that would allow doing this with just one regular texture sample, but the availability of that extension is not high enough to be the only option.

[VReaperV]

Ah wait, it wasn't working properly because I accidentally used min instead of max. Anyway though, with that change it seems to be a tiny bit slower (by <0.1ms). Any consecutive texel fetches with the same coordinates will likely just come from the same texture cache anyway.

[VReaperV]

I also noticed some new GLSL files use spaces or seem to mix tabs and spaces, I would prefer tabs.

Ah, yep, that should be fixed now.

[VReaperV]

I've added a portal readback as well: now instead of just calling R_MirrorViewBySurface() for every portal surface after sorting them by distance each frame, the cull_cp shader will cull portal surfaces in addition to regular ones, then the CPU will read the results back next frame and use that to determine which portals might be visible.

[VReaperV]

Adding R_SyncRenderThread() to GenerateWorldMaterials() also seems to have fixed crashes with r_smp 1, so I have disabled that restriction.

[slipher]

Could we get a comment somewhere with an overview of how the double buffering works and what is the benefit of it? Like is there an async API for the compute shaders? If world surfaces are double-buffered but models are not, how do we keep them in sync when rendering?

[slipher]

What do these represent? Portals directly visible from this one?

[VReaperV]

Yep. Some of it can probably be made simpler.

[VReaperV]

Could we get a comment somewhere with an overview of how the double buffering works and what is the benefit of it? Like is there an async API for the compute shaders? If world surfaces are double-buffered but models are not, how do we keep them in sync when rendering?

There's this graph from #1137 which holds true here as well:

The benefit is not introducing stalls with extra synchronisation, so they can keep executing alongside the regular pipeline (including swapBuffers()). Compute shaders are async by nature, so nothing extra needs to be done for that.
Double-buffering only applies to which surfaces are rendered, not how they are rendered. And since world and model surfaces are culled in completely different ways synchronisation between them does not make sense.

[slipher]

LGTM, but I would still like to understand the double buffering thing. I guess I never really understood the answer to this previous conversation:

How can data generated on the previous frame be valid? Everything could be in a different position or viewed from a different angle.

That's a mistake in pr description, what is meant is that the results of the culling are used on the next frame, i. e. double-buffered (the depth buffer is used from the current frame).

I don't get how surfaces can be double buffered without also buffering everything else that may be rendered (particle effects, 2D drawing, etc.). If you used the surface visibility from an earlier time then the player could have moved to a point that makes some of them newly visible.

[VReaperV]

(rebased)

[VReaperV]

If you used the surface visibility from an earlier time then the player could have moved to a point that makes some of them newly visible.

Technically yes, but practically the fact that it's only one frame of difference makes it more or less impossible, even when moving around fast. So far I've not seen any discrepancies when using the engine with this change.

It's also aided by the fact that the cull takes bounding spheres of the surfaces (either directly or projecting an AABB from them), so the culling somewhat coarse.