Rendering with high sample count - Hitting DrJit memory limit

[orarugogerincufelpattano]

Hi!
I'm new to Mitsuba and wanted to test it to visualize CAD Data.
I exported a light guide (transparent tube with hundreds of optical out coupling surfaces on the side) with a LED on it's end from Blender and tried to render it.
It does bend the light correctly and the out coupling of light works as well.
Such a setup needs tons of samples though and if I try to increase the sample count, the renderer brakes down as Dr.Jit hits a memory limit.
Are there any workarounds to this problem?
Is it at all possible to render such sample intensive scenes with Mitsuba?
Any help is much appreciated!

[wjakob]

How are you calling Mitsuba? I.e. how do you invoke the render process, what are the parameters, etc.? It sounds to me like you are somehow forcing the renderer to work in wavefront mode, where memory usage is proportional to the sample count. Normally we prefer to work in megakernel mode where that doesn't happen. And this is also the default, which is why your message is surprising.

[orarugogerincufelpattano]

Hallo Wenzel, thanks for your reply.
I exported my model from Blender, edited the XML file and then rendered from the command line:
Mitsuba -m cuda_ad_rgb scene_name.xml

I'm completely new to Mitsuba and got very excited as it offers realistic materials and light distribution which would be perfect to visualize real engineering car headlight data.
I read some of the documentation and watched some of your videos and simply started to render. Unfortunately I don't understand many of the scientific terminology, so probably that's the reason why I do it wrong.
I can program a bit in Python if necessary, so if you could give me some hints on how to start the renderings properly, that would be a great help.
Or if you can point to the necessary part of the documentation, I would be happy to read and test...
Thanks.

[orarugogerincufelpattano]

I read some more and now I know what you mean. I did not started Mitsuba with the -W option, so it should be in megakernel mode. If I lower the resolution (from 1920x1080) to the half of that, I can go up with the sample count to 8192. That still doesn't produce enough illumination tough.
I'm trying to test something similar to this image:

It's a nightmare to render as it only has a bright LED as the light source but it illuminates the whole foreground.
(Think of the Daytime Running Lights of nowadays cars. Most of them are done with a light pipe like the image above and are capable of illuminating the road ahead. Millions of photons distributed all around from a single (actually 2-3) tiny LEDs.)
Would Mitsuba be capable of render something like this and if yes, how would such a setup look like?
I tried with the "path" tracer integrator as well as with the volpath integrator and a simple plane as the LED emitter. I'm not able to produce enough samples to get a reasonable image.

[Angom8]

Have you tried to set a fixed amount of samples per pass ? (2048 or 1024 for 1920x1080)

[orarugogerincufelpattano]

Yes, at that resolution I can only render with maximum 2048 spp. Anything higher and I'm hitting the memory limit.
With 2048 samples the image is rendered in only 2 minutes but it is far too coarse. There are simply not enough samples to cover all that light distribution.

[orarugogerincufelpattano]

I will give the ptracer a try next week. Thanks!

[Angom8]

There is a difference between samples per pass and total spps. Mitsuba will try to divide your render into multiple passes if it exceeds its limits (notably, 2^32 samples). The 'samples_per_pass' is a deprecated parameter that allows to set yourself the amount of spp that Mitsuba will use per pass (Ex : 32768 spps with a samples_per_pass of 2048 will lead to a 16 pass render). In your case, manually setting this parameter might help.
You may want to directly look into the base rendering integrator here

[orarugogerincufelpattano]

Please excuse me that I'm not good in programming and so I have some problems understanding some of your suggestions.
I tried to play around with the sample count but I still don't really understand what is happening.
When I create a defaul spp value and use this in the sampler:

	<default name="spp" value="2048"/> 
        ...
	<integrator type="volpath" id="elm__0" name="elm__0">
		<integer name="max_depth" value="32"/>
	</integrator> 
        ...
		<sampler type="independent" name="sampler">
			<integer name="sample_count" value="$spp"/>
		</sampler>

then I can't go higher than 2048 as drjit tells me that it hit the limit:
jit_flush_malloc_cache(): Dr.Jit exhausted the available memory and had to flush its allocation cache to free up additional memory. This is an expensive operation and will have a negative effect on performance. You may want to change your computation so that it uses less memory. This warning will only be displayed once.
After this message the rendering is stopped.

When I try to manually set sample sizes:

	<default name="spp" value="0"/>
        ...
	<integrator type="path" id="elm__0" name="elm__0">
		<integer name="max_depth" value="-1"/>
		<integer name="samples_per_pass" value="5112"/>
	</integrator>
        ...
		<sampler type="multijitter" name="sampler">
			<integer name="sample_count" value="20400"/>
		</sampler>

interesting things happen:

[orarugogerincufelpattano]

Whatever I set for the "sample_count" value, it gets increased at rendering and I get the message that it must be a multiple of the spp_per_pass value. (this is impossible to predict tough as the renderer increases the value at runtime by a value that I can't figure out.)

Caught a critical exception: [SamplingIntegrator] sample_count (20448) must be a multiple of spp_per_pass (1600).
When I stay below the limit (in both cases), the rendering is (much too) fast (2-3 minutes) but super noisy and therefore not usable.

I also tested the ptracer integrator and it produced a much smoother image but unfortunately the rays inside the light pipe have been not coupled out at the side optics, so it remained dark.

	<integrator type="ptracer" id="elm__0" name="elm__0">
		<integer name="max_depth" value="-1"/>
		<integer name="rr_depth" value="64"/>
		<integer name="samples_per_pass" value="4096"/>
	</integrator>

But the rendering time was about half an hour which would be what one would expect with such a trace intensive situation.
Now the question is: would it be possible to achieve a smooth image with the proper amount of bounces and with the physically correct outcoupling at the optics.
(It is not helping that I'm not allowed to post the resulting images, but try to imagine what I wrote in context of the above light pipe image.)

[njroussel]

Hi @orarugogerincufelpattano

Whatever I set for the "sample_count" value, it gets increased at rendering and I get the message that it must be a multiple of the spp_per_pass value. (this is impossible to predict tough as the renderer increases the value at runtime by a value that I can't figure out.)

For the example you provide, could you give us the exact sample_count, samples_per_pass and film resolution specified in the XML as well as the full CLI command you're using?

As mentioned in the other thread, by default, it should be impossible to run into memory issues unless you're using more than 2^31 samples in total (resolution_width x resolution_height x samples_count).

[orarugogerincufelpattano]

Mitsuba version 3.5.0 (master[4577713], Windows, 64bit, 16 threads, 8-wide SIMD)
I start the rendering like this: mitsuba -m cuda_ad_rgb -v Mitsuba_LigthGuide_Test.xml
With the following settings my render stops after drjit complains about not enough memory:

	<default name="resx" value="1920"/>
	<default name="resy" value="800"/>
	<integrator type="path" id="elm__0" name="elm__0">
		<integer name="max_depth" value="-1"/>
	</integrator>
	<sensor type="perspective" id="elm__1" name="elm__1">
		<string name="fov_axis" value="x"/>
		<float name="fov" value="39.597755"/>
		<float name="principal_point_offset_x" value="0.000000"/>
		<float name="principal_point_offset_y" value="-0.000000"/>
		<float name="near_clip" value="10.000000"/>
		<float name="far_clip" value="48000.000000"/>
		<transform name="to_world">
			<rotate x="1" angle="-179.88550371771024"/>
			<rotate y="1" angle="65.84427352354103"/>
			<rotate z="1" angle="177.96451301364417"/>
			<translate value="-932.493042 517.081604 848.539612"/>
		</transform>
		<sampler type="multijitter" name="sampler">
			<integer name="sample_count" value="4096"/>
		</sampler>		
		<film type="hdrfilm" name="film">
			<integer name="width" value="$resx"/>
			<integer name="height" value="$resy"/>
		</film>
	</sensor>

2024-02-07 16:59:17 INFO  main  [mitsuba.cpp:334] Mitsuba version 3.5.0 (master[4577713], Windows, 64bit, 16 threads, 8-wide SIMD)
2024-02-07 16:59:17 INFO  main  [mitsuba.cpp:335] Copyright 2022, Realistic Graphics Lab, EPFL
2024-02-07 16:59:17 INFO  main  [mitsuba.cpp:336] Enabled processor features: cuda llvm avx sse4.2 x86_64
2024-02-07 16:59:17 INFO  main  [xml.cpp:1380] Loading XML file "Mitsuba_LigthGuide_Test.xml" with variant "cuda_ad_rgb"..
2024-02-07 16:59:17 DEBUG main  [PluginManager] Loading plugin "plugins\srgb.dll" ..
2024-02-07 16:59:17 DEBUG main  [PluginManager] Loading plugin "plugins\d65.dll" ..
2024-02-07 16:59:17 DEBUG main  [PluginManager] Loading plugin "plugins\regular.dll" ..
2024-02-07 16:59:17 DEBUG wrk4  [PluginManager] Loading plugin "plugins\path.dll" ..
2024-02-07 16:59:17 DEBUG wrk13 [PluginManager] Loading plugin "plugins\measured.dll" ..
2024-02-07 16:59:17 DEBUG wrk2  [PluginManager] Loading plugin "plugins\hdrfilm.dll" ..
2024-02-07 16:59:17 INFO  wrk13 [TensorFile] Loading tensor data from "paper_white_rgb.bsdf" .. (643 KiB, 11 fields)
2024-02-07 16:59:17 INFO  wrk13 [Measured] Loaded material "White A4 paper (160 gsm)" (resolution 1 x 8 x 32 x 32 x 3)
2024-02-07 16:59:17 DEBUG wrk3  [PluginManager] Loading plugin "plugins\dielectric.dll" ..
2024-02-07 16:59:17 DEBUG wrk0  [PluginManager] Loading plugin "plugins\multijitter.dll" ..
2024-02-07 16:59:17 DEBUG wrk12 [PluginManager] Loading plugin "plugins\diffuse.dll" ..
2024-02-07 16:59:17 DEBUG wrk10 [PluginManager] Loading plugin "plugins\area.dll" ..
2024-02-07 16:59:17 DEBUG wrk5  [PluginManager] Loading plugin "plugins\plastic.dll" ..
2024-02-07 16:59:17 DEBUG wrk4  [PluginManager] Loading plugin "plugins\ply.dll" ..
2024-02-07 16:59:17 DEBUG wrk2  [PluginManager] Loading plugin "plugins\gaussian.dll" ..
2024-02-07 16:59:17 DEBUG wrk4  [PLYMesh] Loading mesh from "_Paper.ply" ..
2024-02-07 16:59:17 DEBUG wrk4  [PLYMesh] "_Paper.ply": read 23247 faces, 12475 vertices (662 KiB in 1ms)
2024-02-07 16:59:17 DEBUG wrk10 [PLYMesh] Loading mesh from "_LED_Boxes.ply" ..
2024-02-07 16:59:17 DEBUG wrk13 [PLYMesh] Loading mesh from "_Light_Guide.ply" ..
2024-02-07 16:59:17 DEBUG wrk3  [PLYMesh] Loading mesh from "_Box.ply" ..
2024-02-07 16:59:17 DEBUG main  [PluginManager] Loading plugin "plugins\perspective.dll" ..
2024-02-07 16:59:17 DEBUG wrk0  [PLYMesh] Loading mesh from "_LED_Emitters.ply" ..
2024-02-07 16:59:17 DEBUG wrk10 [PLYMesh] "_LED_Boxes.ply": read 78 faces, 132 vertices (5.04 KiB in 0ms)
2024-02-07 16:59:17 DEBUG wrk3  [PLYMesh] "_Box.ply": read 548 faces, 572 vertices (24.3 KiB in 0ms)
2024-02-07 16:59:17 DEBUG wrk0  [PLYMesh] "_LED_Emitters.ply": read 6 faces, 12 vertices (456 B in 0ms)
2024-02-07 16:59:17 DEBUG wrk13 [PLYMesh] "_Light_Guide.ply": read 117708 faces, 75669 vertices (3.66 MiB in 10ms)
2024-02-07 16:59:17 INFO  main  [Scene] Building scene in OptiX ..
2024-02-07 16:59:18 DEBUG main  [scene_optix.inl:80] Initialize Optix configuration (index=2)..
2024-02-07 16:59:18 INFO  main  [Scene] OptiX ready. (took 158ms)
2024-02-07 16:59:18 INFO  main  [xml.cpp:1398] Done loading XML file "Mitsuba_LigthGuide_Test.xml" (took 206ms).
2024-02-07 16:59:18 WARN  main  [SamplingIntegrator] The requested rendering task involves 3145728000 Monte Carlo samples, which exceeds the upper limit of 2^32 = 4294967296 for this variant. Mitsuba will instead split the rendering task into 2 smaller passes to avoid exceeding the limits.
2024-02-07 16:59:18 INFO  main  [SamplingIntegrator] Starting render job (1920x800, 4096 samples, 2 passes)
jit_flush_malloc_cache(): Dr.Jit exhausted the available memory and had to flush its allocation cache to free up additional memory. This is an expensive operation and will have a negative effect on performance. You may want to change your computation so that it uses less memory. This warning will only be displayed once.

[orarugogerincufelpattano]

To be honest, I'm getting really confused. What settings would you use if you would need to render such a light guide?

It needs a lot of samples as there are hundreds of light ray bounces before the ray (starting at the front of the pipe) reaches the outcoupling optical surface and leaves the pipe, hitting something outside. (I put a sheet of paper underneath the pipe).
As I wrote, in reality, such a setup (with 2-3 super bright LEDs/pipe) generates enough photons to exit the light pipe, go trough a grained transparent bezel in front of it (gets diffused), go trough the lamp outer lens and then light up the road ahead so bright, that some people forget to turn on their low beam as the road is lit brightly.
Would it be possible to simulate something like this with Mitsuba on a normal computer. (Regardless of rendering time.)
EDIT: as I understand, the render resolution plays also an important role here. HD would be minimum but 4k would be the preferred resolution.

[wjakob]

There some issues with the current version of Mitsuba that make such extremely high sample count renderings problematic. If you need to use the path tracer, then would do in your position is to render with the maximum # of samples that still fit in one pass (withheightsample count < 4 billion) and do the multi-pass rendering manually, by configuring HDRFilm to writing out EXR images with a component type of float32. You can average multiple images to reduce noise (this will require launching the render with different seed values for the random number generator to ensure that things are uncorrelated).

Given that your radiation comes from a light source and is coupled into a guide, rendering with a path tracer (which starts at the camera and has to rely on randomly encountering sources of illumination) generally seems like a bad idea to me though. You should better start from the other side, by using the particle tracer.

[orarugogerincufelpattano]

As I stated above, I tried ptracer but something is wrong there as the whole pipe remains dark except the front and end part.
There is no outcoupling at the side optics. I don't know why.
When I have some time, I will try to build a light pipe in blender on my private computer, roughly simulating the real CAD part (which is built by a supplier with special optical software). If I manage to do that, I will then share the file here if anyone of you would like to give it a try.
It would make more sense to discuss it when you also have the data.
But only if there is interest from your side...

[Angom8]

There some issues with the current version of Mitsuba that make such extremely high sample count renderings problematic. If you need to use the path tracer, then would do in your position is to render with the maximum # of samples that still fit in one pass (with_height_sample count < 4 billion) and do the multi-pass rendering manually, by configuring HDRFilm to writing out EXR images with a component type of float32. You can average multiple images to reduce noise (this will require launching the render with different seed values for the random number generator to ensure that things are uncorrelated).

I noticed that this problem seems to be caused by "limited" VRAM. My 16384MiB card sends the multipass render into CPU hell (100%usage indefinitely) while a card with 24GiB (usage of ~18GiB) works fine. It doesn't seem to be related to how Mitsuba handles multipass as I tried in other works to alter this system (setting the limit to 2^29 instead of 2^31 or resetting DrJIT tables for example) and landed on the same observation. Are there more documented comments on this issue ?

Tested with a 1920*1080*4096 render (2 passes of 2048)

[orarugogerincufelpattano]

Finally I had some time to build a fake light guide in Blender. If anyone would like to test it.
(This is not an optically tuned guide, just some fake optics but it works for testing.)
I attached both the Blender file as well as a Mitsuba XML and Meshes.
LightGuide_Test_MITSUBA.zip
Light_Guide_Test_BLENDER.zip
Blender's Cycles can't really render it properly:

Mitsuba is doing it right but would need a much higher sample count which is I could not get to work without hitting the memory limit and ending in a forever running rendering:

This noisy image was rendered in less than a minute.
In the Mitsuba XML file, you will find 2 integrators. A path and a ptracer (one is commented out).
The ptracer rendering is already running for 3 hours so I can't post it's output yet.

So, the question is: is it possible to render such a light guide with 4K resolution and sufficiently low noise level?
If yes, how?

[orarugogerincufelpattano]

That was a great information, thanks! (Reading the manual certainly helps most of the time... :-) )
It looks much better with 65536 samples:

Now I have to do some python to automate the rendering and find a good way to stick the images together automatically.
Thanks for your help!

[orarugogerincufelpattano]

I need a little more help, sorry. When I render from command line, there is the -D key=value parameter. How can I use this to from python? If I define two parameters in my XML file like:

	<default name="crop_x" value="0"/>
	<default name="crop_y" value="0"/>

for:

			<integer name="crop_offset_x" value="$crop_x"/>
			<integer name="crop_offset_y" value="$crop_y"/>

How can I access them from python?
if I try:
image = mi.render(scene, crop_x=i, crop_y=j)
I get an error.
I know it's a noob question, but I'm a CAD sculptor and not a programmer so please have mercy... :-)

[ziyi-zhang]

Hi,
The values you defined are in the xml file, which is parsed when constructing the scene. So we should override the values like

scene = mi.load_file(filename, crop_x=i, crop_y=j, ...)
image = mi.render(scene, ...)

Best

[orarugogerincufelpattano]

Thanks, make sense of course. I loaded the file once and rendered the tiles but was wondering why it's always the same tile...

[orarugogerincufelpattano]

Just to close the thread: with the use of tiles (cropping) I was able to render a much better image:

I will spend some more time to study the documents for further fine tuning but it already looks very realistic.
Thanks for all of your help!