Improve the drop-in-replacement engine aspect

[ToJans]

Context

This engine is intended to be used as a drop-in replacement to Three.js’s WebGLRenderer.

Problem

It tends not to work as a drop-in replacement out-of-the-box, because the renderer crashes when:

• Meshes contain a THREE.Geometry instead of a THREE.BufferGeometry.
• It uses materials for which the texture images are not loaded yet.

This might cause people who try to replace the WebGLRenderer with RayTracingRenderer to drop out early, because 'it does not work as promised' out-of-the box.
This would be a shame, because if you get it up and running, it works great.

Solution

In my engine I have provided a few simple workarounds for these issues by pre-processing the THREE.Object3D before I attach it to the scene; this is an excerpt from the code:

    public placeholderMaterial = new THREE.MeshStandardMaterial({color: 0xccffcc, emissive: 0xccffcc, emissiveIntensity: 0.5});

    materialHasUnloadedMaps(m:THREE.MeshStandardMaterial) {
        return (m.map && !m.map.image) ||
            (m.normalMap && !m.normalMap.image) ||
            (m.roughnessMap && !m.roughnessMap.image) ||
            (m.metalnessMap && ! m.metalnessMap.image);
    }

    adaptModelForRenderer(model: THREE.Object3D) {
        model.traverse(o => {
            const mesh = o as THREE.Mesh;
            const mm = mesh.material as THREE.MeshStandardMaterial;
            const mg = mesh.geometry as THREE.BufferGeometry;
            const mgg = mesh.geometry as THREE.Geometry;

            // Convert  THREE.Geometry to THREE.BufferGeometry
            if (mgg && mgg.vertices) {
                mesh.geometry = new THREE.BufferGeometry().fromGeometry(mgg)
            }

            // Convert non-MeshStandardMaterial and Materials that are still loading
            // images with the placeholderMaterial
            if (mm) {
                if (mm.type !== "MeshStandardMaterial" || this.materialHasUnloadedMaps(mm)) {
                    mesh.material = this.placeholderMaterial;
                }
            }

            // rtRenderer does not support Texture.repeat and -.offset, so update uvs instead

            // ... and lots of other adjustments, removed for brevity
        });
    }

Proposal

As I can imagine that I am not the only one experiencing these issues, and intercepting these cases is probably not rocket science, I wonder if you would be open to intercept any THREE.Geometry instances and convert these to THREE.BufferGeometry.
As for the invalid materials, I would suggest to replace invalid instances with a placeholderMaterial, which could be specified in the params upon startup.

[jaxry]

Wonderful suggestions! Converting Three.Geometry to THREE.BufferGeometry sounds like an obvious solution, and having a placeholder material makes sense to me. I can come up with a PR for it.

Textures are only loaded to the GPU on initialization of the renderer, so async loading of textures aren't easy to support with the current implementation. I think it's a good idea to get full async texture support. I'll look into it and see what it'll take, but I was having difficulties in the past because of how we're packing textures inside a WebGL Array Texture. At the very least we can prevent crashing.

Texture repeating also isn't supported yet (#59) since it's a little tricky to implement in the ray tracing shader. But there is a way to do it. I suppose this is a high priority?

[ToJans]

Texture repeating also isn't supported yet (#59) since it's a little tricky to implement in the ray tracing shader. But there is a way to do it. I suppose this is a high priority?

Not for me ATM; it would be nice, but it is not urgent; I can mirror textures myself and use these...

[jaxry]

In the meantime, as long as there are missing features from the renderer, I created a wiki page which lists what's supported.

https://github.com/hoverinc/ray-tracing-renderer/wiki/Three.js-Compatibility

This should serve as a reference to what should currently work, so there's less confusion about what's supported and what's not. We could link to this page in the readme in the Limitations section. Does it look okay to you? Is there something you'd change or add?

[jaxry]

And also, I'll be working on the PR in the next week that fixes anything that crashes the renderer. You mentioned

• Three.Geometry
• Non-loaded textures

Are there any other cases you discovered that flat-out crash the renderer?

[ToJans]

Great news!
If I remember it correctly, there were no other issues that caused a crash.

However, if you do make an update, I can imagine 2 things that might be easy to add as well:

• Support for uv offset & repeat
• Support for Object3D.visible and Object3D.scale

[ToJans]

In the meantime, as long as there are missing features from the renderer, I created a wiki page which lists what's supported.

Somehow I managed to miss this message; great idea! Looks good....