GPU / Game engine optimised textures

[onelsonic]

Fixing GPU memory bottleneck with GPU optimised Textures.

well the current workflow picks up textures out from blender as PNG, JPG or HDR
and use them directly for any target export .

However these textures are not optimised for the GPU. Having optimised GPU textures can really improve real-time performances and GPU available memory.
I think we need at least ASTC and DXT1/DXT5 textures.

Textures should be processed in a first time as it is now but the exported code should point to the optimised format and load it properly. (as Blob object in JS for example)
Then you should convert manually the original textures files with the appropriate tool:

ASTC:
https://github.com/ARM-software/astc-encoder

DXT:
https://developer.nvidia.com/nvidia-texture-tools-exporter

This will liberate most of the GPU memory for well other better things than textures....

Example :
how to load AST textures in HTML5:

var ext = gl.getExtension('WEBGL_compressed_texture_astc');

var texture = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, texture);

gl.compressedTexImage2D(gl.TEXTURE_2D, 0, ext.COMPRESSED_RGBA_ASTC_12x12_KHR, 512, 512, 0, textureData);

[MoritzBrueckner]

Hi, as far as I know Khamake expects .png textures and then converts them to another format depending on the target used. If you think that there should be more formats supported, maybe ask Robert on the Khamake repository. This repository is probably the wrong place to implement this.

I also wonder if DXT textures are already used on some targets, it could be difficult to implement it as the correct DXT format depends on the type of texture (diffuse, normal etc.).

[onelsonic]

I will check with Robert.

[MoritzBrueckner]

Link to Kode/Kha#1351. The quality setting mentioned in Robert's answer can be found in Armory at Render Properties > Armory Project > Flags > Texture Quality.

[onelsonic]

I looked into KHA and the compress texture format is not implemented in any Backend I have checked.
Seems to only have a compiled tool similar to the one I have mentioned above.
(the compression parameter only compress in the same format PNG > to compressed PNG,
ASTC format is not even supported.)

First the compress texture format will need to be added to KHA.
Second the new format will need to me added to the Armory code.

@MoritzBrueckner
Might need to reopen this one for the Armory part.

[MoritzBrueckner]

First the compress texture format will need to be added to KHA.
Second the new format will need to me added to the Armory code.

Can you elaborate? Where in the Armory/Iron code should this be implemented (apart from iron.RenderPath.getTextureFormat())? As far as I know there isn't much Armory specific code when it comes to texture handling.

[onelsonic]

changes that will be needed in Kha:
mainly the WebGLImage.hx in the HTML5 backend and the Backends/HTML5/kha/graphics4/CubeMap.hx
need to add another texture format : example ASTC5x5
something like that:

				case ASTC5x5:
					SystemImpl.gl.compressedTexImage2D(gl.TEXTURE_2D, 0, ext.COMPRESSED_RGBA_ASTC_5x5_KHR, myWidth, myHeight, 0, image);

more infos:
https://www.khronos.org/registry/OpenGL-Refpages/es2.0/xhtml/glCompressedTexImage2D.xml
https://developer.mozilla.org/en-US/docs/Web/API/WebGLRenderingContext/compressedTexImage2D

changes that will be needed in Iron (some of these):
need to add another texture format : example ASTC5x5
https://github.com/armory3d/iron/search?q=rgba64

changes that will be needed in Armory (some of these):
need to add another texture format : example ASTC5x5
mostly both renderPath and the Inc.hx file
https://github.com/armory3d/armory/search?q=rgba64

PNG's are 8bbp
ASTC5x5 are 5.12bbp with similar visual quality and pretty high PSNR

and there is also the part about the file extension but that will be in Kha I think.

[MoritzBrueckner]

There is a new similar issue on the Armorpaint repo, let's see what Lubos answers: armory3d/armortools#1067. If it should get implemented in Armorpaint only there might be ways to use it in Armory as well.

[Calinou]

As a reminder, keep in mind that ASTC is slow to encode and isn't well-supported on older hardware. Last time I checked, it's also not supported at all on macOS (the same goes for BPTC).

DXT*, as old as it may be, is almost universally supported on desktop platforms by now. On mobile platforms, ETC2 support is decent nowadays, but it can be slow to encode if you don't use something like etcpak. ETC1 can be used as a more widely supported alternative on mobile platforms, but it lacks support for storing an alpha channel and its quality is pretty bad by today's standards.

Finally, if you want to optimize file sizes but don't mind very long encoding times and not-great quality, you could use Basis Universal.

[onelsonic]

@Calinou thanks for the note. About Macs the question is more about supporting discontinued hardware since the ecosystem has moved toward Metal.

About ETC2 (Ericsson Texture Compression version 2 / 2016).

A note from Google comparing ASTC and ETC2 :
With Texture Compression Format Targeting, you can start using ASTC for devices that support it while falling back to ETC2/ETC1 to devices that don’t. The Adaptive Scalable Texture Compression (ASTC) format offers advantages, such as improved rendering performance, faster load times, a smaller in-memory footprint, better battery life, and improved visual quality. You can even dramatically reduce your download size and on-device footprint by optimizing the tradeoff between size and quality.
https://android-developers.googleblog.com/2020/09/improve-your-game-with-texture.html

Rather than falling back to an older format ETC2 - Older machines can use non-compressed textures we have now.

My tests shows that VRAM consumption increases by a factor of six when using ETC2 compare to ASTC.
Compression time is not noticeable on my tests a 1024x1024 texture takes 2.2383s on a 2015 laptop with an old intel GPU.

Also the latest ASTC (2020) is much faster regarding compression time (about x3 times).

Compressed textures requires hardware support and it seems their is a better support for them on PC and Mobile.

PC : DTX
Android / iOS / Html5 / Mac(Metal) : ASTC
Linux/Mac(older) : uncompressed

Other machine can fall back to the uncompressed format.
Also latest MacOS can even run iOS apps.

[TriVoxel]

I have found that Direct Draw Surface (DDS) textures work well with Armory. They are likely better than PNG or JPEG as far as performance goes.