WebGLRenderer: Add support for Node Materials

[gkjohnson]

Description

I wanted to open a discussion to talk about the possibility of adding Node Materials to WebGLRenderer - as far as I understand Node Materials are only supported with WebGPURenderer which poses a lot of issues for community adoption, practical testing, and library support:

WebGPURenderer looks like it's coming along great but until it's more fully featured and mature I cannot use it as a platform for any of my professional work. Likewise many tools I've developed and work on are already built on WebGLRenderer and cannot ported to WebGPURenderer. I assume many developers are in the same boat. This means I haven't been able to invest any time in learning and trying the new node material system despite being very interested in the benefits after years of trying to solve shader and material problems.

Adding support to WebGLRenderer would help afford the well-experienced three.js community the ability to test the system in large scale, complex projects and give practical feedback sooner rather than trying it to WebGPURenderer. This will also help ease the transition projects and developers that currently need to use WebGLRenderer. Currently all material development currently being done for WebGL will be rendered useless in WebGPU which this can help avoid.

And from the library perspective - I need to be able to develop something that will work in both WebGLRenderer and WebGPURenderer. Asking developers to maintain two versions of their materials or libraries isn't great and I think will otherwise just make this renderer transition significantly more difficult than it needs to be.

I appreciate all the work done on the new material systems - it looks like a large improvement over what we have now so it would be great to make this transition as smooth as possible.

cc @sunag @RenaudRohlinger

Solution

Add support for NodeMaterials to WebGLRenderer.

Alternatives

Community feedback is significantly delayed, developers and libraries are fragmented.

Additional context

No response

[DennisSmolek]

Correct me if I'm wrong, but doesn't switching to the WebGPURenderer (which supports nodes) work on WebGL only instances as the fallback?

Or are you wanting a way to mix the existing GLSL code with the outputs of the node system?

Are you just wanting the output of the visual side of things or are you wanting things like the compute and functional nodes as well?

[gkjohnson]

I understand that WebGPURenderer has a webgl fallback but this is about using Node Materials with WebGLRenderer. In this case I don't care about the underlying browser graphics API being used - I care about the compatibility with the WebGLRenderer class that many projects are heavily invested in already and cannot or are not ready to switch for a variety of reasons.

Or are you wanting a way to mix the existing GLSL code with the outputs of the node system?

I just want to use node materials as they are with WebGLRenderer and leverage all the benefits that the node system already promises and provide feedback. I'm not proposing new features otherwise.

[sunag]

Thanks for the analysis @gkjohnson, it's a very interesting subject. In these last months where WebGPURenderer is gaining more users, it became clear that we need a migration guide to TSL, not just the a specification. I believe that just as it was done in previous years to design the new Nodes System and TSL, now it will be to smooth the transition.

I fear that it will not be possible to migrate all the TSL features, except for the basic functions already known in the old node system, but I believe that this is not enough. We had to discontinue the new node system for WebGLRenderer because it was falling far behind in features compared to WebGPURenderer (#28167), the renderer ended up being re-architected not only because of WebGPU but also to align with the new node system, and bring new features like backdrop for example, things that would not be possible using GLSL with simple functions in WebGLRenderer without complex work involved, only in this specific case WebGPURenderer does not render the opaque materials twice when using transmission materials like WebGLRenderer, instead it copies the g-buffer to a texture, for this it is also necessary that the color transform is done in post-production internally, this backdrop effect can be accessed on any material using viewportTexture(). The event system like '.updateBefore()' that allows nodes to create pre-pass, at the time of rendering the object, which can also be shared with other related ones, in WebGPURenderer the light and shadow system can be extended using nodes, and this allows users create new shadows and shadow filters as well as the light, light model and the light system without needing to modify the core for this. Including other things like compute()... surely this is not all just what I remembered now.

Alternatively, I would like to know if there is anything else we can improve like the TSL transpiler, and a guide for this migration.

[Mugen87]

I'd like to understand in more detail why migrating to the new renderer is any issue. To me, the real migration task is to move from GSLS to TSL. We want to support this process with more documentation and guides and also technically with a TSL transpiler. But WebGPURenderer itself is not that different compared to WebGLRenderer, at least API-wise. Most public methods are identical, it is required in some cases to switch to an async/await pattern though. As long as you didn't use an extensively forked version of the renderer, exchanging the renderers should not require a lot of migration effort.

Generally speaking, TSL and WebGPURenderer are superior compared to WebGLRenderer with its hardwired materials and lights. As outlined by @sunag supporting node materials in WebGLRenderer is essentially developing a new WebGLRenderer. I've said it earlier but the project should not make the mistake to invest too many resources in maintaining things which are in fact legacy. From a strategic point of view, it's better to focus on the future by adding missing features to the new renderer, improving performance and fixing bugs. I understand when companies (or individuals) have built solutions around WebGLRenderer but it should be clear that a migration to TSL and WebGPURenderer is inevitable. I also understand there are some reservations about the maturity level of WebGPU but it's important to highlight that a WebGPU usage isn't mandatory when using the new renderer. WebGL 2 can always be forced with a renderer parameter.

TBH, I have expected this discussion sooner or later since we always had it when moving to new approaches. E.g. Geometry to BufferGeometry, WebGL 1 to WebGL 2 or legacy to physically correct lighting. There were always developers who wanted to retain the old structures for different reasons but the project consistently deprecated (and eventually removed) legacy code paths. Which was in retrospective the right decision. It should be clear that at some point in the future WebGLRenderer will be deprecated as well. To me, the renderer is now in some sort of pre-stage. We still provide bug fixes and smaller additions, but implement no big features or larger refactorings anymore. And I think that's the right direction.