Implementing ReSTIR-GI in Mitsuba3

[DoeringChristian]

Hi,

For some time I have tried to implement the ReSTIR-GI integrator presented in the paper ReSTIR GI: Path Resampling for Real-Time Path Tracing.
I understood how the algorithm is supposed to work however, it was difficult to follow the implementations that currently exist as they are optimized for performance and not readability.
Now I do have an implementation that comes pretty close to the algorithm described in the paper while still being relatively readable and I would like to present my implementation here.
That said, unfortunately there are still some aspects which do not yet work in my implementation.
This includes the Jacobean bias correction (I could not experimentally verify it and the sample bias is higher than without).

I hope my implementation can help people who want to learn about advanced algorithms such as ReSTIR and also learn a bit more about Mitsuba.
I would also appreciate it if anyone wants to critique my implementation, since it is by far not perfect.

Finally, here are some images rendered using the integrator (1spp):

The Cornell Box:
First frame:

30th frame:

Reference (256spp):

And the Staircase scene:

First frame:

30th frame:

Reference (256spp):

[Mephisto405]

How does the code perform in terms of speed when compared to the code in the current paper?

[DoeringChristian]

Unfortunately it doesn't come close. They seem to achieve a frame time of 14ms (70fps) for some pretty complex scenes, whereas my implementation can only achieve about 15fps for the Cornell-box scene and 1fps for the staircase scene. Of course Mitsuba3 is not designed to be a real time renderer and this is expected.

[Mephisto405]

Also, (I'm not familiar with the original paper) the result at the 30th frame is quite noisy, while the result at 256spp is incredibly clean. It's a bit surprising to see such a clear result with about 10 times more samples (from the perspective of path tracing). Does 1 frame equal 1 spp?

[Mephisto405]

And most importantly:
Thank you for providing this excellent tutorial for a new rendering paper and Mitsuba 3!

[DoeringChristian]

Thanks

The reference is rendered using standard path tracing, the other images are rendered using the ReSTIR-GI algorithm with 1spp per frame. After some time the reservoirs "stabilize" and do not improve noticeably.
You can try to enable Spatial-Spatial reuse, this reduces the noise but with my implementation you get a biased image.

We can visually compare images with the option disabled and enabled.
Note, that a more complete implementation would resample neighboring spatial reservoirs instead of only the same.

Frame 30, with 1spp per frame:

No spatial-spatial reuse

With spatial-spatial reuse

And here is an image from the paper for comparison (I copied it directly out of the paper 😅)