Suggestion for Implementing Rain and Snow Effects with Particle System #82
Comments
|
Hello @meetSeparate ! Sorry for the delayed response - here in the US it is the holiday season and I have been on vacation the last week. Thank you for the kind words - I'm happy that you are able to use and learn from my path tracing project. 😊 A particle system for the renderer sounds great! I have played with the idea years ago, back when I was first working on the ray marching outdoor environment demos, like Terrain_Rendering.html and Arctic_Circle.html. These demos could benefit from some weather effects like light rain or snow. However, at the time those demos were made, I didn't have a good plan for how to implement particle systems, especially rain drops that would possibly interact with the terrain. Another area that a particle system would be useful is in the path traced games that I have links to here on my repo, as well as the latest game I'm working on, called Glider Ball 3D. My style of 3D games has always been fast and simple (and hopefully fun!) gameplay with small, limited game scope. It would be fun to add some game effects like sparks, smoke, and even small explosions to the game objects. It would be even cooler if the particles would be able to interact with the game objects and the surrounding static environment, as you suggested. As for my advice for actually implementing a particle system, in particular a rain and snow system, unfortunately I'm not experienced in this area of rendering, but I could make an educated guess as to how to begin at least. Before I got into ray tracing and path tracing, I was, and still am, a hobby 3D game programmer. Way back in the late 1990's/ early 2000's, I completed and released 2 games that needed particle effects - one was Virtual Bombard (a Scorched Earth / Artillery/ Tank Wars clone, but in 3D) and Asteroid Patrol (a classic Asteroids clone, but also in 3D). In these games, when the player's ship/ tank exploded, I wanted to have pieces of the vehicle explode outward from the player, and then in the case of the Asteroids game, float gently outward into space, unaffected by gravity, and in the tanks game, have the particles explode more upward and definitely affected by gravity, pulling the small pieces back down to Earth and even bouncing once or twice around the exploded tank. Even though I was using OpenGL 1.1 and the C programming language for a Windows 98 build (I know, I'm old!), I remember that everything had to be set up on the CPU side and the actual motion and life/death of the particles also had to come from the CPU side. In other words, the GPU, or renderer, didn't know about the particles until the last second, when it was time to send all the triangles down the pipeline to the rasterizer. I think I ended up making each particle a small object of 4 triangles each, a tetrahedron, which is the minimum number of triangles to have a closed 3D object. The 4 vertices of each tetrahedron particle were slightly randomized at creation, which had the effect of some particles being smaller or bigger, and some particles being more or less stretched out in a particular direction. Back then, this simple geometry allowed me to add around 1000 particles for each explosion, and still kept the framerate smooth at 60 fps. This worked well for the traditional rasterization pipeline many years ago, but I don't think this approach would work so well for our current ray tracing projects. The reason why is that any time you add triangles to a ray tracer, you're going to slow it way down, unless you have a sophisticated BVH acceleration structure to handle dynamic triangles that can move fast and change shape. So for each particle (rain drop or snow flake), something other than triangles would have to be used. In theory, you could have each drop be made from an oblong or elongated sphere or ellipsoid, but the problem remains of adding hundreds or thousands of these objects - it will slow the renderer down a lot, having to check through each drop's bounding box for a possible intersection, and then do a somewhat costly ray-deformed Ellipsoid intersection test for each positive hit. I think a better approach, at least for now in the experimental stage, would be to have each particle represented by a small quad with transparency and a texture. And each quad would always face the camera, no matter where it is located in the scene. I believe that's how modern particle systems are made, although I haven't been keeping up on how Unreal Engine's Niagara system handles particles - the UE developers probably offer way more options than just camera-facing transparent quads! However, innocent as this quads solution sounds, you are still adding many 'objects' to the ray traced scene. With rasterization, GPUs were designed to handle millions of triangles on screen without breaking a sweat, but with ray/path tracers, each triangle, each quad which is 2 triangles, hurts you a little - and this cost can add up real fast. This is why I have yet to implement any kind of particle system in my path tracing projects- I'm just not sure how to handle all of the additional required geometry that is dynamic, fast moving, and has randomized, unpredictable movements, (especially snow flakes). If you didn't want to deal with triangles or quads, another route you could take is rendering all of the particles on the GPU side only - the CPU (or JavaScript in our web case) would know nothing of the particles. Examples of this technique can be found on ShaderToy- just search particles, rain, snow, etc., and you should find some good examples of how each shader artist did it. Although this would be the more ray tracing and framerate friendly option, a major drawback of using GPU shader-only particle systems is that it would be very difficult, maybe impossible, to have the particles interact with the scene objects around them. I'm thinking of bright sparks shooting out, or falling snowflakes and raindrops that land on surfaces and need to splash or bounce correctly. If the CPU physics code can't 'see' all of the particles because they're GPU-side only, I'm not sure how the physics would work. These are just some thoughts and issues to keep in mind while developing your particle system. Again, this is an area where I have spent very little time in, and it was many years ago for a different kind of platform and traditional rasterization renderer. I seem to recall seeing particle effects on some of the RTX ray tracing demos from NVIDIA, but I can't say for sure how they go about implementing it - and their dynamic BVH acceleration structure builders and traversal code is not public, so I don't really know how they handle all of the additional particle objects/geometry in their RTX ray tracing system. Please let me know if you had any plans of your own moving forward, or if you had any ideas on how to start adding a particle system. My final piece of advice is, at first, just try the most simple, straight-forward solution - even if it is very un-optimized and even if it doesn't end up working in the end. The process by which you make your 1st attempts will reveal gaps and flaws in your understanding of the problem, and will empower you to choose better and more optimal solutions. Hope this helps, and please keep me posted with any progress or questions you may have! 🙂 -Erich |
|
@erichlof Every time I read your answers, I admire how detailed you explain complex things in simple language. This is very valuable. I want to thank you for this and wish you a Happy New Year! |
|
@vinkovsky Thank you, Arkady! Whenever I'm trying to implement something for the first time, I have to (at least for me) understand it from a simplified 'bird's-eye' view first. Unfortunately when working with ray tracing, and path tracing in particular, we live in a world filled with complex language and underlying math (optics, calculus, monte carlo, statistics, probability theory, etc) that is often assumed that the reader/programmer already knows, especially in academic path tracing papers. And often the complex language used to describe a certain technique is used for language sophistication-sake (to sound like you have strong credentials and are an authority on the subject, ha). So what ends up happening is that a topic that could be simplified or more easily-explained, becomes muddied and obscured with complex language for complexity's-sake and assumptions about the reader's previous knowledge of the underlying fundamentals. So I'm glad that someone out there like you is able to understand what I'm talking about! :-D Happy New Year! |
|
Hello @erichlof, Thank you so much for taking the time to provide such a detailed and thoughtful response, especially during the holiday season! I greatly appreciate your insights and advice—they are incredibly valuable to me as I work on this project. I want to share some updates and ideas based on your suggestions and experience. After some experimentation, I’ve successfully implemented a CPU-based particle system that includes features like particle emitters, customizable parameters, and physical collisions. For the particles themselves, I’ve used billboards—camera-facing quads with textures and transparency. Additionally, I’ve also explored using meshes as particles, which allowed for more complex visual effects, though, as you mentioned, the performance cost of this approach can be significant. However, I’m now at a crossroads where I want to explore GPU-based particle systems to improve performance and scalability. As you pointed out, the GPU-based approach could potentially handle thousands (or even millions) of particles more efficiently, but the challenge lies in enabling meaningful interactions between particles and the scene geometry (e.g., collisions, splashes, or reflections). Here are a few ideas I’ve been considering, and I’d love to hear your thoughts: Using Compute Shader-like Techniques for Particle Dynamics (WebGL Limitations) Use two textures to store particle data (e.g., position, velocity, lifetime, etc.). One texture represents the current frame's data, while the other represents the next frame's data. The particle physics simulation is performed in the fragment shader, and the results are written to the texture for the "next frame." Finally, I completely agree with your advice to start simple and iterate. The process of trial and error has already taught me a lot, and I’ve found that even early, unoptimized prototypes can spark new ideas for improvement. Your emphasis on learning through experimentation really resonates with me. Thank you again for your inspiring work and for sharing your wisdom so generously. If it’s okay, I’d love to keep you updated on my progress and seek your feedback as I explore these ideas further! Wishing you a Happy New Year and all the best in 2025! |
|
Happy New Year, Zhang! Wow, I didn't realize how far you've already come with implementing your particle system! I will be glad to offer any advice or suggestions that I can, as it relates to a path tracing shader renderer. Although from how you describe your implementation and your crossroads options, you've already put a lot of time and thought into this - much more than I have, especially with implementing a modern particle system (recall that the last time I implemented particles, OpenGL 1.1 was the new thing, ha). Something I like to do when deciding which 'fork in the road' to go down, is to list pros and cons to the options. Just so we're on the same page, let's list some pros and cons of your choices moving forward (feel free to correct or add to these): CPU-based particle system (with GPU shader ray tracing handling actual final drawing to screen) Pros In general, CPUs are able to efficiently handle 'for' loops, in particular, looping over thousands or even hundreds of thousands of particles to update position, velocity, lifetime, prune potential collisions, and handle basic collision response (bouncing, reflecting, etc) Collisions and other interactions between the particles and the scene objects in the surrounding environment would be the most simple and straightforward with this avenue (and possibly the more efficient, because of the CPU's efficient loop handling) Would help balance the computation load between GPU and CPU. Throughout the history of my games and rendering projects, I've always tried to let the CPU handle what it is good at, and let the GPU handle what it is good at - for instance, doing all input handling, object motion, matrix transforms, BVH building, physics collision and response, on the CPU, while letting the GPU handle going through each pixel (which it easily does in parallel), sending out a ray, traversing the BVH, looking up the intersection material BRDF, sampling the light source or material BRDF, bouncing around the scene, and reporting back the final pixel color. This ray/path tracing procedure is a lot for the GPU to handle, so we can ease its burden by letting the CPU (JavaScript side) handle all of the particle motion and physics. Cons As you mentioned, there is a bottleneck and WebGL limitations when sending large amounts of data from the CPU to the GPU on every animation frame. The 1st time I ran into this was when I started my project that remakes the classic 80's home PC game, The Sentinel (which I call The Sentinel, 2nd Look - repo on my bio's repo list). I had around 50 game objects that need to send their 2 Vector4 (so 8 float 32s) BVH data for the shader's TLAS (Top-Level Acceleration Structure) traversal, as well as their 4x4 matrix inverses (so another 16 float 32s) to tell the ray tracer how each game object is positioned, scaled and rotated. It worked ok for 10 or so game objects, but when I approached 50, it crashed the browser page's webgl context on my mobile device, because I had exceeded the allowed number of uniforms. Luckily with the arrival of WebGL2, it allowed uniform buffer objects (UBOs) for sending large arrays of data from CPU to GPU. But still, this is a challenge to be considered. Using the CPU for all updates will limit the total number of particles on screen, whereas GPU-only rendering would allow potentially millions of on-screen particles. Having a CPU based particle system lends itself to rendering the final particles with the GPU as billboard quads. Having rain drops as flat quads for instance, would make realistic refraction challenging, because the drops themselves don't have any depth or 3d geometry to them - they are essentially just 2d pictures. However, I imagine this could be worked around, by the shader switching modes - so when the ray caster intersects a particle quad, it calls a raycast function that then does a quick 2nd intersection test against a real drop, or elongated sphere (ellipsoid maybe?), and then the ray interacts the way a ray tracer would, with splitting the light into reflected rays and refracted rays, depending on the angle of the ray with the rain drop surface, according to the Fresnel equations. Versus
Pros As mentioned, almost-limitless amount of particles on screen at once, since the GPU works in parallel Would be possible with WebGPU and compute shaders, three.js is already working on this and there are several TSL examples on threejs.org (but WebGPU support is not nearly as wide as WebGL/WebGL2, and spec might change) Physics and updates for particles could be handled the same as rendering, in parallel Having actual particle shapes is possible when rendering with shaders, so effects like refraction through elongated spherical water drops would add to the realism of the effect Cons Even if the particles could be simulated and updated via fbo, ping pong frame buffers, or compute shaders, the problem remains of how to integrate the particles with the environment scene objects. These objects could theoretically live on the GPU, but the more you stick inside the scene, the more risk you take of running out of GPU memory. It would be even worse if the scene objects were created on the CPU side and fed to a sophisticated BVH builder - then the particles on the GPU would also have to somehow know how to traverse a complex BVH, which I think would slow down the path tracing shader even more. Having everything on the GPU (scene BVH, ray casting for each pixel, path tracing bounces loop with light/BRDF sampling, and now the entire particle system) would put a lot of stress on the GPU, especially older or mobile GPUs. Meanwhile, the CPU would be essentially starved for work, unless it had some physics to do with the main larger scene objects in the environment. There wouldn't be much balance between the GPU and CPU workloads. |
|
Hi again, part 2 (my last post was getting too long, ha) So looking at the pros and cons of the two different approaches, I would say that the CPU-based system has a better pro to con ratio. The main pro with CPU particles is that you already have something in place. Therefore my initial vote would be to try the CPU particle system first, even knowing that it has flaws and will present challenges near the end of the pipeline, such as CPU to GPU UBO data transfer, and requiring a possibly 2nd intersection test, if 2d billboard quads will not be sufficient for true raytraced refraction effects. A word of warning though as you go down this path - getting the particles (many individual 2d quads) into an acceleration structure that can be quickly traversed inside the fragment shader will be challenging, and possibly even deal-breaking, if it slows down the fragment shader's ray casting loop. But being faced with this challenge might open up other ways of thinking to a better, faster solution. Off the top of my head, one optimization that you could do right away is to have the particles, especially rain, visible only on the initial camera ray cast. If a camera ray hits a rain drop, great - then reflect or refract based on Fresnel (just like you'd do with the typical glass sphere). However, if the camera ray misses all water drops on the initial ray cast, then they cease to exist. So in that case, the ray would go out into the scene and bounce around the scene objects as normal, or render the background sky if absolutely nothing was hit. I don't think the user would notice that the rain drops don't cast shadows or appear in a mirror's surface (both of which would require an expensive check for ALL rain particles on the secondary rays during the second bounce loop). Little hacks and corner-cutting tricks like this could mean the difference between this system working, and crashing the webgl context. So this was my very long-winded way of saying, maybe take your existing particle system to the next stage - and see what happens. If it works, great, if it works slowly, we can try to think of ways to optimize it, and if it flat-out crashes, all is not lost - it just might require a slight course-correction, or slightly different approach. Yes, please do keep me posted on your progress, as well as further questions and issues. Hope this helped somewhat- sorry I can't offer more specific detailed advice on the GPU-only choice, as I haven't made one of those myself. But if it does come to that as the only option, I will do my best to offer help or advice, and I might try to do some small GPU tests and experiments alongside your project, in order to better understand the problem and where you're trying to go. Look forward to discussing further! Take care 🙂 -Erich |
|
Hi @erichlof, |
|
I am sorry I have another question. In my project, it is necessary to initialize lighting data of multiple types (such as point light sources, directional lights, rectangle lights, etc.) and with multiple quantities. I'm confused about how to conduct lighting sampling efficiently and accurately. |
|
@meetSeparate I'm glad that my pros and cons breakdown was helpful to you! Best of luck with your CPU particles approach - if anything comes up that halts your progress, let me know here in this thread, and maybe we can figure it out. 🙂 About your questions on diffuse and specular separation and more realistic materials, I will try to offer some advice and point you in the right direction. Unfortunately, material models (especially specular) is a complex topic, both in the design/planning phase and the implementation phase, but I will try to navigate it below. First we have to take a step back and define what our goals for our renderer are in terms of correctness, speed, and quality. I've mentioned this before on other issue threads here on my repo, but my first inspiration to dive into the world of path tracing was the (now old) Brigade 1,2, and 3 path tracers from Jacco Bikker and colleagues from around the years 2010-2014 (way before RTX was even an idea). In fact if you search YouTube under the channel name Sam Lapere, you'll find some great examples of what amazed me back then, and still does today, considering the hardware they had to work with. From watching their videos and reading their blog posts, their clear mission was to make path tracing interactive, and to bring it into real-time (10 to 30 fps, later with 60 fps goals for Brigade 3), and to do all this on consumer-grade hardware. Back then, denoising wasn't as prominent as it is today (maybe even non-existent), so the final results were grainy and 'salt and pepper'-ish. But if we look past that, wow, it was incredible having whole scenes, sometimes with dense triangular models like the Stanford Bunny and Dragon, Sponza, etc, all moving in real-time and interactive with user input, and path traced on a typical 2010 home gaming computer! After watching those videos many times, I was inspired to learn about path tracing, and had the dream of taking their vision further, by bringing real-time path tracing to the browser (which is even more accessible than the home pc), which meant a goal of at least 30-60 fps, on any device with a browser, even cell phones. I'm happy to say that I've achieved (or gotten close to) some of these goals, but over the last 10 years, it seems like the goal post keeps moving farther out ahead of me (ha), especially with the advent of BRDF material model research, BVH and other acceleration structures research, and most prominent - denoising (A.I.) and efficient sampling and reconstruction strategies research. As you probably already know, path tracing is a huge subject with a rich ray tracing history to discover, and cutting edge research being done at all major CS universities and corporations (NVIDIA, Intel, etc). Because of this, it's difficult to make a renderer that does all things well at the same time. Usually, trade-offs and compromises must be made, if we want to get something off the ground and working. This brings me to trade-offs that I've had to make with my own renderer here, which will tie in with your questions (I promise!). Since I went down the real-time path (no pun intended) from the very beginning of my project, that has been the unwavering focus of the renderer all of these years. One of the first things I added to the renderer was the little Stats box in the upper-left corner of the webpage, which constantly displays the FPS (not a typical thing to add to someone's first ray tracing progam, ha), but again, speed and efficiency is the major consideration and I wanted those honest stats to be constantly displayed for myself and the users of my ray tracers. While developing a new path tracing demo/game, if those stats drop below 30 fps, I know that there is either a problem to be fixed, or optimization to be made, or a whole new approach that must be taken, in order to keep everything running smoothly on all devices. Sometimes I can do all of the above right, but adding more objects or lights to the scene starts to slow it down. That's when I realize I've hit a limit for my renderer and a limit in my understanding of the problem, and I'll usually back off a little with the stress tests, in order to make it still 30-60 fps, and call it a day. Now bringing it around to your question specifically, one of the areas where I really have cut a lot of corners and made many trade-offs is in the specular material models and sampling of those models. This is partly by choice, but I must admit that also it is due to my own ignorance of modern BRDF material models and sampling strategies for those materials. The original Brigade renderer only had Lambert Diffuse and perfect specular mirror for their material options (no glass or transparent materials even) therefore, so did mine (in the beginning). Through examples on ShaderToy and many classic ray tracing books and the source code to Kevin Beason's brilliant SmallPT (a complete path tracer with diffuse, metal, and glass materials - all in 100 lines of C++ code!), I was able to add refraction and transparency to my material options, and then soon after that, a clearcoat material option (which has the Lambert diffuse underneath, I know the material options that I have here are very limited when compared to most modern path tracers and software rendering packages (Blender Cycles, V-Ray, etc). Not too long ago I even tried to include an actual modern specular BRDF (like GGX) while following an NVIDIA RTX tutorial series for RTX beginners, but it slowed the renderer down a little too much for my comfort, and the results where really noisy and sometimes filled with 'fireflies' (annoying very bright pixels in random locations). Not satisfied, I ended up scrapping the whole thing, and returning to my simple material roots - Lambert Diffuse, Metal with roughness, Transparent with roughness, and ClearCoat. On some of the demos like Switching_Materials.html and CSG_Museum_1.html, I added a sub-surface material with the option of having a clearcoat on top (like polished jade and marble or teeth with enamel). However, my understanding of the sampling of this complex material is very limited, so that's why it doesn't appear that often in my demos. I would like to improve the correctness and realism of this material, but I don't fully understand the science or ray tracing techniques behind these fascinating materials. Going back to goals for our renderers, I've been able to get away with only having 4 or 5 material options while keeping everything real-time on all devices, because my simple materials (which are from classic ray tracing historic times) are physically plausible and 'good enough' to render the kind of scenes that I'm interested in. But if you want to go beyond that and take it to the next level in terms of realism (like for your car scene), you pretty much have to separate the diffuse code path from the specular code path, and make the specular path an actual modern Microfacet specular BRDF. I won't go into the details of microfacet BRDFs, as there are a lot of online resources for learning about them (also under 'PBR' materials and shaders). If my understanding is correct, there are 2 parts to modern BRDFs - the evaluation functions, and the sampling functions. So for every material, let's say metal with 0.5 roughness applied, there would be an evaluation function just for metals (or non-diffuse and non-transparent materials), as well as a matching sampling function for that material. Where the traditional rasterization with PBR material shaders and actual path tracers with those same materials are very alike, is in the evaluation functions. When a triangle is rasterized (or intersected in a ray tracer), the renderer must look up the final color of the pixel that covers that material at that exact tiny location on its surface. And where traditional rasterization with PBR material shaders differs from actual path tracers with those same materials, is in the sampling functions - as traditional rasterizers do not employ sampling strategies- they quickly and efficiently look up, or 'evaluate' the material color at that point. This evaluation can be as simple as the 1-liner: albedo * N.dot(L), as it is for Lambert diffuse, or as complex as the full GGX 10-line (or more) function. This evaluation keeps things running at real-time framerates while giving a very physically plausible material look to objects, which is why it is pretty much the industry standard. If the story ended here for path tracers as well, all would be as simple and cut and dry. But because we're dealing with actual light rays that have physical interactions with real-world materials, and making them bounce around the scene, we must be able to sample these complex BRDFs, so that we know which direction the ray is supposed to go after interacting with one of these materials. Sampling functions for ray/path tracing must include the correct pdf (or probability that the ray would bounce in a certain direction) as well as the correct weighting for the final color/intensity contribution to the ray after it interacts with a material. If the correct weights for each interaction are not applied, the material will look either too dark or too bright, compared with the same material as viewed in the real world. This pdf weighting part is fairly sensitive and math-heavy, and if something doesn't end up looking right, it's not clear where to look for the problem (unless you're comfortable with sampling, importance sampling, probability, etc). Not to worry though - there are many examples with source code out there that demonstrate how to implement a Microfacet BRDF into your path tracer. You may have heard of this, but a very popular BRDF system (and an industry standard) is the Disney principaled BRDF. This library of materials, PBR parameters like roughness and sheen, evaluation functions, and sampling functions has been used on major motion pictures from Pixar. There are several implementations of path tracers on ShaderToy that use the complete Disney BRDF system. So that would be my vote, if you wanted to give your materials that extra realism and photo-realistic appearance. |
|
(Part 2) Sorry for the lengthy posts, but as you can probably tell by now, I love all things path tracing - even if I don't fully understand some areas, I could go on and on, ha. Anyway, to focus on your example (the car showroom scene), I would first take a look at some examples of path tracing on ShaderToy that use the Disney principaled BRDF - just do a search for that in ShaderToy's search field. See if you like the overall look and performance of it (in terms of convergence speed and noise). If so, scroll down to the evaluation functions - these are the heart of the BRDFs (they determine the color and intensity of the material at a precise point on its surface and with respect to the incoming ray's angle of approach). Then probably below those functions, you'll see several 'sample...()' functions - these are more specific to path tracers. They help determine the weights to apply for the ray's intensity as it leaves the surface, as well as determine the reflected or refracted ray's exact 3D direction after interacting with the surface. These functions rely on Monte Carlo methods as well as statistics and probability. These sampling functions will usually return an outgoing Vec3 ray direction (sampling direction), as well as a pdf weight (a simple float number) that will be later used when determining the intensity of the outgoing ray's contribution. Going back to goals and trade-offs, if you don't mind your final image taking a while longer to converge, or things possibly running below 60 fps (if final image quality matters most), then I don't see any reason not to add microfacet specular materials to your renderer. With cars for instance, their paint is sometimes complex, with a heavy clearcoat (some with very high IoR), and fine metallic flakes just below the clearcoat surface. And the car interior must be considered as well - leather, wood, plastic, might all be in place on the inside. So when rendering these complex materials, I believe that the Disney BRDF should be able to handle everything, both on the inside and the outside, faithfully reproducing the real-world appearance, while still being efficient enough to run smoothly on most hardware. Hope this helps, and tomorrow I will respond to your lights and light source sampling questions. Talk to you soon! 😊 |
|
Hi again, Zhang! Part 2 (Lights) Your question about how to deal with multiple lights is a really good one - but like all things in path tracing, there is never a simple answer, and we must dive deeper if we want to be satisfied with the answer(s), or at least get close to a complete answer. I should let you know up front that I myself do not fully understand the math and probability theory/techniques that are required to deal with this topic of sampling many/multiple types of lights. Most of what follows is paraphrasing from ray tracing books and articles that I have collected over the years. And, like the situation with your particles system, I haven't actually implemented a scene of multiple different types of lights for my own project here, so take the following with a grain of salt. Since we are working with a Monte Carlo path tracer, let's quickly summarize what it's doing. To truly get a physically-accurate rendering that was indistinguishable from real life, we would have to send out countless rays (as many as there are photons in the scene) at an atomic scale, and let them bounce a near-infinite number of times, in order to capture every last detail of the scene. Since computers have a limited processing power and memory, we must employ the Monte Carlo integration method to get the best approximation humanly (and digitally) possible. Instead of sending a near-infinite amount of rays for a near-infinite number of bounces, the Monte Carlo method relies on certain aspects of statistics like the Law of Large Numbers - basically you take random samples of the sample space, add them all up, and take the average at the end. And probability and statistics holds that if you take enough random samples that are unbiased (more on this soon), eventually it will converge to the expected outcome (or as accurate an answer as we can possibly get). This all sounds great in theory, but in practice, using Monte Carlo and uniform random sampling results in high amounts of noise. And if we are careless about how we sample the problem space, it could take an eternity for the computer to give us the correct image with all the noise removed. The Law of Large Numbers says that if the samples are unbiased and truly random, it eventually WILL converge, but it might take beyond our lifetimes - ouch! Here is where Importance sampling comes to the rescue. This technique states that if we are more thoughtful and strategic about how we take our "random" samples, so that we sample important areas more often and less important areas less often, then not only will we get the result much faster, it will Still be as unbiased and correct as the earlier one that took eons! This is one of the facts from statistics that blew me away when I first understood what it meant. Basically it says that we are allowed to skew or bias our samples towards areas that give brighter, stronger, more meaningful results. And as long as we account for this non-uniformity or bias in our sample selection, we can essentially remove this bias from the final results. And how we account for this bias (or unfairness) in sample selection is through correctly weighting each sample. In other words, if a random sample comes from an area of the problem space that would give a large, bright, or important result, we should down-weight this kind of sample, to avoid over-representing its contribution. But occasionally if a sample happens to be taken from an area of the problem space that returns something that is less bright, or not as important, then we must up-weight this sample accordingly, to avoid under-representing this sample's contribution to the final image. In my previous post I mentioned Jacco Bikker, who was the leader behind the Brigade series of path tracers. Not only has Jacco been a pioneer in the field of path tracing and computer graphics (with several published research papers), but he has also devoted himself to teaching at the university level, and offering free online tutorials on the subject of ray tracing and path tracing. One of his more recent tutorial series, as a warm-up, covers the basics of Monte Carlo and probability/statistics in rendering. Here's a link to 2 articles from this series: https://jacco.ompf2.com/2019/12/11/probability-theory-for-physically-based-rendering/ https://jacco.ompf2.com/2019/12/13/probability-theory-for-physically-based-rendering-part-2/ I highly recommend reading part 1 and 2, as Jacco really knows what he's talking about, and presents it in a fun, concise article form with pictures and diagrams. To tie this in with your lights sampling questions, I'll borrow some of his scenarios and explanations, and try to paraphrase some of the main ideas. |
|
Part 3 (Lights continued) I always like to start with a simple example that showcases the need for sample weights. Again, the examples I'm using come from Jacco's articles, but I'll put my own spin on them. 😁 Let's say we have a scene with 4 lights in it. All lights are equal power, equal shaped, and roughly equal distance from the other scene objects that are being lit by these 4 lights. When rendering a scene like this, path tracing requires that after you send out an initial camera ray from each pixel, if a ray hits an object other than a light - a diffuse sphere for example, that you must randomly select a direction (path) for that ray to continue bouncing around the scene, gathering light and color as it interacts with other objects. Often we care about sending rays towards the bright lights in the scene, as these give us important lighting information for all the objects. But we need all 4 lights' contributions in order to properly light the scene. Now a recursive 80's style ray tracer would simply loop over all 4 lights, one at a time, gathering and adding up all the contributions as it goes. But this tends to be slow, especially as the number of lights grows. Enter path tracing- now we can just randomly choose 1 out of the 4 possible lights, and then just keep randomly choosing for that pixel, over and over. This will produce quite a bit of noise, but it will eventually converge on a smooth result. There's only one problem though- our image is much darker than the ray tracer's correct result! The reason is that we only sampled 1 light's contribution at each intersection along the ray's journey. To the poor 3 other lights that didn't get chosen, it's like they don't even exist, and their contributions don't matter. To fix this problem, we need to weight the light that we Do sample, so that the total brightness output of all 4 lights is taken into consideration at each juncture of the ray's path. The most straightforward (and still correct) fix is to simply multiply each light sample by 4. But why multiply, and why by the number 4? Just to keep the math simple, let's say that every light has a brightness of 1.0. Since global light intensity is additive, we can say that the scene objects are being lit by a total global light power that equals: 1.0 + 1.0 + 1.0 +1.0 = 4.0, or 4. In probability, we like to have ranges like 0% to 100%, or more formally, 0.0 to 1.0 (which is 'normalized'). So to get each light's contribution to the global intensity, we simply divide each light's output (1.0) by the global total output (4.0), or 1/4, or 0.25. Now we can see why our naive implementation was too dark - because every time we took a light sample, we only considered 0.25 or 25% of the actual global total light output. Therefore our scene would be 1/4th of the brightness of the actual scene brightness. Will continue in part 4 soon (have to get some sleep, ha)... 😊 |
Hi @erichlof ,
I hope this message finds you well. I wanted to express my gratitude for your open-source path tracing project on GitHub. It has been incredibly helpful for my studies.
I have a specific idea I would like to explore: implementing rain and snow effects using a particle system within the path tracing framework. Additionally, I am interested in how these effects could interact with models, such as causing collisions and reflecting or refracting light as they intersect.
Do you have any thoughts on how to approach this implementation? Any guidance or insights you could provide would be greatly appreciated.
Thank you once again for your amazing work!
Best regards,
Zhang Ziyu
The text was updated successfully, but these errors were encountered: