[Android (C) and HTML5 (JS)] GPU skinning "too many uniforms" crash on android Adreno while fine on Mali GPU

[onelsonic]

Does anyone came across to this error with armory and know what is actually causing this issue?

I suspect my 3d model is too complex and I am searching after some directions in order to run it correctly in Armory and making it more simple.

What is a "uniform" and how does it translate in Blender?

Any input will be much appreciated.
Thanks

[MoritzBrueckner]

A uniform is like a shader variable that's uniform to all "executions" of that shader in a particular draw call (draw call = tell the GPU it should draw something). It is a bit like a global variable that can be used to pass parameters to a shader.

So it is probably caused by a too complex material or world shader. Maybe the only way to find the exact cause is to check all generated .glsl files in build/compiled/Shaders/ and search for a file with many uniforms at the top (they use the keyword uniform so they're easy to find). But (if I'm not mistaken) the amount of variables is not the crucial factor, it's their size. A vec4 will take 4 times the space a float takes, an array or a sampler (texture) may take even more space. Depending on the target and hardware there might be different limits to how many uniforms can be used.

To verify your findings you can hide all meshes with that particular material.

[onelsonic]

thanks @MoritzBrueckner so I removed one by one any complex elements, materials, etc, from my 3D model and looked at all the GLSL materials and else and tested every time and it turns out it to be the following :

An Armory3D project will just stop with a "too many uniforms" error only in specific cases:
It occurs ONLY with Adreno GPUs which all have a pretty low Vertex Uniform and Fragment Uniform counts in their hardware: 256 vs usually 1024 for Mali GPUs while being much faster in 3D than Mali GPUs.

When you export a 3D model with a complex Armature and lot of Bones it will simply crash with this error on Adreno GPUs but it will work on an old and 70 times slower Mali GPU.

I checked other engines, Unreal, Unity and they do have 3D models with complex Armature animation running correctly on Adreno GPU. (Ex: Genshin Impact which is running using Unity).

This seems to be a problem with Armory and Adreno GPU while handling Armatures and Bones.
The way Armatures and bones are processed using and here I guess "Uniforms" is causing the issue.

For now the best workaround I found is using over simplified Armatures with low bones counts. (so no moving hairs or cloths, etc...)

[QuantumCoderQC]

Hi @onelsonic, Thanks for the info.

Would you be able to do the following checks and see if there are any differences please:

1. Remove ALL textures from the model and use only a simple diffuse shader. Keep the high number of bones and the complex armature.

2. There is an option in the Object settings panel of Armory that says "Skinned object" or similar. Try it with your character and check again.

Also, does your "Character" object have more than one sub-meshes ?

Best Regards,
QC

[onelsonic]

1. is not working: crashing with "too many uniforms" on Adreno and running smooth even on old Mali GPU.

2. There is a "dynamic usage" settings for the object, I activated it, but same as above.

Then this 2013/2014 thread seems similar to what I am experiencing with modern high end Adreno GPU:
https://developer.qualcomm.com/comment/6972

Issue probably come from this :
The GLES shading language spec:
http://www.khronos.org/files/opengles_shading_language.pdf
Chapter 12, Appendix A, section 5 "Limitations for ES 2.0", page 115
Uniforms (excluding samplers):
In the vertex shader, support for all forms of array indexing is mandated. In the fragment shader, support
for indexing is only mandated for constant integral expressions.

I guess the same limitations applies to ES 3.0, my Adreno is running on 3.0

#Edit: also my model has only one simple mesh with not much faces but lot of vertex groups for animations but it doesn't seem to be a problem on Mali GPU.

[onelsonic]

Hi there, after further investigation and tests.
It looks like the problem comes from Animating the 3D model with complex geometry on Adreno GPU.
Loading a static skeleton and complex bones do work on Adreno.

So the way the animation is exported and then loaded on Adreno GPU is probably using too large uniforms that are not supported on these mobile GPUs.

Anyone know how animation are stored in Armory, do they use "uniforms"?
I think we would need to rewrite this piece using the info from this thread in order to fix the problem.
https://developer.qualcomm.com/comment/6972

[N8n5h]

Haven't delved too deep into armory's bone animations, but from what I could gather:
https://github.com/armory3d/iron/blob/master/Sources/iron/object/BoneAnimation.hx
The skinBuffer float array from the BoneAnimation class is passed under the link _skinBones here https://github.com/armory3d/iron/blob/a2d8e4280ae5ad2e50f017a2dc6d5fa3cf61301e/Sources/iron/object/Uniforms.hx#L989
which is used by this shader https://github.com/armory3d/armory/blob/master/Shaders/std/skinning.glsl
and how it's used when writing vertex shaders https://github.com/armory3d/armory/blob/master/blender/arm/material/make_skin.py

If too many uniforms may be tripped probably is because of that uniform.

[onelsonic]

thanks @N8n5h, I checked all codes you mentioned and run a few tests and succeed to localise the issue.
It comes from this line here :

armory/Shaders/std/skinning.glsl

Line 3 in 48952e0

uniform vec4 skinBones[skinMaxBones * 2];

On Adreno GPUs (skinMaxBones * 2) is too high for the hardware. (Or badly supported, not sure...)

Hard coding the 'skinMaxBones * 2' value to 245 partially fix the problem as we just limit it to how many uniforms Adreno GPU are supporting.
uniform vec4 skinBones[245];

not sure why 245 works, when I think it should technically be 256.

Also changing this line here has no effect whatsoever on the GLSL shader:
public static var skinMaxBones = 128;
https://github.com/armory3d/iron/blob/a2d8e4280ae5ad2e50f017a2dc6d5fa3cf61301e/Sources/iron/object/BoneAnimation.hx#L17

Looking at how Adreno's work :

• max vertex uniform vectors : 256 (> 1024 on Mali)
• max vertex uniform components : 1024 (> 4096 on Mali)
  [full adreno OpenGL specs link]
  (https://user-images.githubusercontent.com/53947594/100106245-9bc83d80-2e68-11eb-8981-2720a47f915e.png)

I read that Adreno GPU's OpenGL was bogus somehow and not sure if it is still the case now...
Also Adreno GPU's are currently the fastest GPU's on android and the Google Pixel's phones all have it.
Reported bogus drivers here:
godotengine/godot#12816 (comment)
https://developer.qualcomm.com/comment/6972

some proposals to fix this for Adreno:
1: should bones skinning use texture data instead of uniforms? Will it be as fast?
2 : is there a way to force the use of OpenGL ES2 instead of ES3 for android or webGL2.0 to test it? It might fix this issue.
3: then not sure if it is related but when I try to set the Max bones parameters manually, here : the export crashes with the below:

to

compilation crash logs:

Exporting Scene
Traceback (most recent call last):
  File "C:\armory/blender\arm\props_ui.py", line 870, in execute
    make.build(item.arm_project_target, is_publish=True, is_export=True)
  File "C:\armory/blender\arm\make.py", line 383, in build
    export_data(fp, sdk_path)
  File "C:\armory/blender\arm\make.py", line 125, in export_data
    ArmoryExporter.export_scene(bpy.context, asset_path, scene=scene, depsgraph=depsgraph)
  File "C:\armory/blender\arm\exporter.py", line 154, in export_scene
    cls(context, filepath, scene, depsgraph).execute()
  File "C:\armory/blender\arm\exporter.py", line 2139, in execute
    self.export_objects(self.scene)
  File "C:\armory/blender\arm\exporter.py", line 2001, in export_objects
    self.export_mesh(mesh_ref)
  File "C:\armory/blender\arm\exporter.py", line 1436, in export_mesh
    exporter_opt.export_skin(self, bobject, armature, vert_list, out_mesh)
  File "C:\armory/blender\arm\exporter_opt.py", line 301, in export_skin
    log.warn(bobject.name + ' - ' + str(bone_count) + ' bones found, exceeds maximum of ' + str(max_bones) + ' bones defined - raise the value in Camera Data - Armory Render Props - Max Bones')
NameError: name 'log' is not defined

[MoritzBrueckner]

not sure why 245 works, when I think it should technically be 256.

Not sure, but I guess it's because there are still other uniforms somewhere that add to this count.

then not sure if it is related but when I try to set the Max bones parameters manually, here : the export crashes with the below:

Added a fix, this was not related to your changes: #2021

[onelsonic]

Digging into this problem,
another possible solution will be to rewrite the GPU skinning shader to use "Uniform Buffer Object" instead of "Vertex Uniform".
https://github.com/armory3d/armory/blob/master/Shaders/std/skinning.glsl

[N8n5h]

The size of skinBuffer from BoneAnimation is computed here https://github.com/armory3d/iron/blob/a2d8e4280ae5ad2e50f017a2dc6d5fa3cf61301e/Sources/iron/object/BoneAnimation.hx#L73,

this.skinBuffer = new Float32Array(skinMaxBones * boneSize);

so it's weird that reducing the value of skinMaxBones doesn't change anything. Have you tried drastically reducing the value to see if it has the same effect still?
Also be sure to check to what value armory writes the const int skinMaxBones in compiled/Shaders/compiled.inc.

Armory seems to be using textures in some places to store data (other than images), maybe the same could be applied to store that skin data instead as an alternative?

[onelsonic]

setting skinMaxBones to 1 or 122 in BoneAnimation.hx has no effect, the animation will play correctly on Mali GPU but crash on Adreno. Yes strange, there is probably something overwriting its value when running.
https://github.com/armory3d/iron/blob/a2d8e4280ae5ad2e50f017a2dc6d5fa3cf61301e/Sources/iron/object/BoneAnimation.hx#L17

You can test the animation and run it with Google Chrome on Android (Adreno and Mali GPU)
https://onelsonic.github.io/animation-android-adreno-bug/index.html

This issue comes from how OpenGL GPU skinning is rendered on Android with Adreno GPU, so the same issue will occur natively on Android as well.

Solution:
Rewriting the "uniform vec4 skinBone" here :

armory/Shaders/std/skinning.glsl

Line 3 in 48952e0

uniform vec4 skinBones[skinMaxBones * 2];

using either Texture buffer object or Uniform Buffer not sure which one is bigger on Adreno.

full adreno OpenGL specs link
(https://user-images.githubusercontent.com/53947594/100106245-9bc83d80-2e68-11eb-8981-2720a47f915e.png)

[onelsonic]

This is what is overwriting the skinMaxBones value. If set manually to 122 (thanks to @MoritzBrueckner fix #2021 ) --> 122*2 = 244 we get the same as hard coding the value to 244. Stripping the extra bones. If set to Auto it will default to an automatic calculated value.

Then back to the Adreno skinning problem, I think Texture buffer objects might be the way to fix this.
(below is what we should be aware of using Texture buffer objects)
https://gamedev.stackexchange.com/questions/174019/instancing-and-gpu-skinning.

some useful references for this issue:
using just textures, here are other useful GLSL references:
https://webgl2fundamentals.org/webgl/lessons/webgl-skinning.html
Let's update the shader to get the matrices out of a texture.

Dual Quaternion Skinning using textures:
https://github.com/ConstantineRudenko/DQ-skinning-for-Unity/blob/2cd819b840a93086b29a641f20bdd423bef05873/Code/DQ%20skinning/Shaders/Material/HackedStandard/HackedStandard.shader#L135

Edits:
I am trying a few things but GLSL is new to me so it's a big WIP...couple of unsuccessful fix so far.

[QuantumCoderQC]

@onelsonic Thanks for the insights. I am currently working on improving the animation system for Armory3D. Would it help fix this issue if we use uniform Mat4 skinBones[maxBones] instead of uniform Vec4 skinBones[maxBones * 3]. (its maxBones * 3 now because I will be including the scale of the bones too. So, 2 for dual quat and 1 for scale.)

So, will going from Vec4 to Mat4 reduce the number of uniforms and maybe fix this issue?

[MoritzBrueckner]

According to this, the allowed number of uniforms does depend on the size of the used uniforms in OpenGL. So if I understand it correctly, using a 4x4 matrix would take away 4 "components" more than 3 vec4s, but I might be mistaken here and it actually might vary between different platforms and GPUs. The linked page states for example that on some (especially older) GPUs vectors always take 4 components independent of their size. Maybe try to set maxBones to a value close to the allowed treshold on your system and then check whether it gets worse or better when using matrices.

I think the same holds true for DirectX/HLSL, at least I had some issue with that in the past where data was always aligned in blocks of 4 components which Kha couldn't handle very well.

[onelsonic]

What @MoritzBrueckner says

then the above issue is Hardware specific on Adreno/qualcomm GPU but also tight to the way bones animations are handle in Armory which uses a too restrictive format : "vertex uniform" not really fit for the Adreno platform.

I found this issue occuring on Adreno 200/300/400 and 500 series so far and I have not tested the 600 series yet...
Someone with a Android Adreno GPU with the 600 series can test it here :

• Can you see the animation ? Yes/No
  https://onelsonic.github.io/animation-android-adreno-bug/

• If not please report the following :
  what is your OpenGL specs?
  (you can use this) https://play.google.com/store/apps/details?id=com.realtechvr.glview
  or this : webGL2 specs
  https://webglreport.com/?v=2

and report on these values:

• max vertex uniform vectors : ???
• max vertex uniform components : ????

To overcome this, a solution is to step away from "uniform" all together as being too restrictive on Adreno.

Not 100% sure but it seems like Unity has the same issue and fixed it in 2018 :
https://issuetracker.unity3d.com/issues/materialpropertyblock-adreno-devices-returning-wrong-result-from-materialpropertyblock

Most reliable workaround is to use unsigned integer variables for array indexing for the bones animations.

Also there is another issue which may be fixed by this,
if you have a long animation let's say 100/150 frames / Armory will just take an eternity to process it (45min to 1h).

[MoritzBrueckner]

It's indeed fine on a Mali GPU and Android 10, I can see the animation of a waving Link on a Mali-G72.

GL Version	Max. Vertex Uniform Vectors	Max. Vertex Uniform Components
OpenGL ES 3.2	1024	4096

I also tested it on a cheap crappy tablet with Android 9 and a PowerVR Rogue GE8100 GPU from Imagination Technologies (never heard of it), and the animation also plays but the arms are very buggy and twisted. Specs:

GL Version	Max. Vertex Uniform Vectors	Max. Vertex Uniform Components
OpenGL ES 3.2	512	2048

If you say that using unsigned integers for the array indexing could work, can you try if changing

armory/Shaders/std/skinning.glsl

Line 7 in 55cfead

ivec4 bonei = bone * 2;

from ivec4 to uvec4 works already?

[QuantumCoderQC]

Not 100% sure but it seems like Unity has the same issue and fixed it in 2018 :
https://issuetracker.unity3d.com/issues/materialpropertyblock-adreno-devices-returning-wrong-result-from-materialpropertyblock
Most reliable workaround is to use unsigned integer variables for array indexing for the bones animations.

From what I see this does not address the issue of exceeding the maximum supported uniforms, rather, how to access each uniform properly from an array.

I also looked up on texture based animations. Although it is faster and might solve the issues with the limited uniforms, it opens up a host of other issues.

1. Exporting these animations as textures: Note that each animation can have different number of frames, compression of this data and so on.
2. For a game engine, Blending animations, IK and FK actions, Bone constraints and Bone parenting are all very essential and must all be done in real-time. These cannot be pre-baked into textures as they have to be processed on the CPU when required.

So, if the engine only plays complex animations with many bones and does not require much of real-time modifications to these animations, having a texture based animation makes sense.

[onelsonic]

Mobile "uniform" are usually limited from 256 to 1024 Max Vertex Uniform Vectors per shader so far.

armory/Shaders/std/skinning.glsl

Line 3 in 48952e0

uniform vec4 skinBones[skinMaxBones * 2];

On a 1024 Vertex Uniform Vectors Mali mobile GPU the Skinbones limit is Max =122 Skinbones Available on Mali....
which gives 122 * 2 = 244 elements to handle in the array (Theoretical max is 256 elements in the array but some uniforms used elsewhere are already taken).
thus : vec4 * 244 = 976 uniforms max available for Skinbones on a 1024 system which is just enough for complex animation.
with 1024 theorical and only 976 available uniforms thus 48 uniforms already taken.

On a 256 Vertex Uniform Vectors Adreno mobile GPU
same 48 uniforms already taken leaving 256-48=208 uniforms left available in which we can encode that give us
208/4/2= 26 Skinbones Available on Adreno....

Then that's true textures are not realy realtime in our case so the only solution I see is
to rewrite the GPU skinning shader to use "Uniform Buffer Object" instead of "Vertex Uniform".
and probably with unsigned integers for positive and null only integers since this is also an issue on some mobiles.

---

A side not for HTML5 instancing (not sure if this will impact instancing)
there will be a need to store the outputs of the vertex shader to one or more buffers and use those as input to another shader.
in webGL2 "createTransformFeedback" should be able to help with that.
https://webgl2fundamentals.org/webgl/lessons/resources/webgl-state-diagram.html?exampleId=transform-feedback

[QuantumCoderQC]

yes, I think using uniform buffers here might be a better option. Not sure how one should link this though.

The current linking is defined here

armory/blender/arm/material/make_skin.py

Line 14 in 55cfead

vert.add_uniform('vec4 skinBones[skinMaxBones * 2]', link='_skinBones', included=True)

Should this remain the same even if we use uniform buffers?

[onelsonic]

yes, I think using uniform buffers here might be a better option. Not sure how one should link this though.

The current linking is defined here

armory/blender/arm/material/make_skin.py

Line 14 in 55cfead

vert.add_uniform('vec4 skinBones[skinMaxBones * 2]', link='_skinBones', included=True)

Should this remain the same even if we use uniform buffers?

this should help : https://www.geeks3d.com/20140704/gpu-buffers-introduction-to-opengl-3-1-uniform-buffers-objects/