meshopt_simplifyWithAttributes() on skinned meshes

[ccpisheden]

Hi.

I'm trying to simplify a skinned mesh with up to 8 bone indices and weights. It's clear to me that bone indices (and their corresponding weights) are not linearly interpolatable, so they cannot really be passed in to meshopt_simplifyWithAttributes() with the expectation of getting sane results.

This issue confirms that:

Giving the algorithm bone indices and weights is going to backfire. The indices aren't interpolatable or stable and don't even have a meaningful distance metric. Vertices nearby will have a different index set, sometimes in a benign fashion - eg switching from ABCD to ABDE, or worse from ABCD to BACE, and this will not end well. I suspect that in terms of representative power, the blend weight/index representation above is about equal, maybe worse, to just two floats - single dominant bone index and its weight. It will still be unstable etc but will be a little more stable.

Unfortunately I don't see the single dominant bone index/weight approach being viable, as it simply mitigates the issue. However, @zeux then said this:

Ideally you don't even want this - you probably want to skin the mesh to a different pose and feed the resulting vertex position as an attribute...

It's not clear to me what he's referring to here. I don't see how it would be possible to reconstruct skinning indices/weights after simplification without them.

The simplest brute-force solution to the problem would be to convert the bone indices/weights into a simple weight-per-bone list. I.e:

Indices: (1, 3, 5, 7)
Weights: (0.1, 0.2, 0.3, 0.4)
Number of bones: 10.
Bone weights: [0, 0.1, 0, 0.2, 0, 0.3, 0, 0.4, 0, 0]

This array of floats IS linearly interpolatable; you'd just need to convert it back to indices+weights, handling the potential case where there are too many non-zero bone weights and discarding the smallest ones.

However, this requires one float per bone, meaning that you'll quickly hit the 32 attribute limit of meshopt_simplifyWithAttributes(). It also seems like it would be slow due to the large number of attributes.

What is the standard approach to this?

[zeux]

Currently the most practical approach for simplifying skinned meshes is to use meshopt_SimplifyRegularize flag and not pass blend indices/weights to the simplifier. I've seen some people also feed blend weights as 4 attributes (without bone indices); I'm not sure that it improves the results much overall, but might be interesting if simplifyWithUpdate is used.

I haven't experimented further with dominant bone information after introducing regularization support; it still might be useful but is less critical now.

[ccpisheden]

Thank you for the quick response!

From what I can tell, meshopt_SimplifyRegularize() only helps producing geometry that is more friendly to deformation, correct?

I'm a little bit confused about how mesh_simplifyWithUpdate() and mesh_simplifyWithAttributes() would work if not all attributes are provided. Let's say I left out both the bone indices and blend weights during the simplification, how would I know what values to use for those attributes after the vertices have potentially been modified during the simplification?

I still believe that the highest quality simplification would be achieved by passing in the weights for ALL bones as I described above. I will run some tests on when I can.

[zeux]

Yes - regularization doesn't directly inspect bone attributes. When using meshopt_simplifyWithAttributes, you keep all values of positions & attributes (including those that aren't passed to the simplifier), as it only returns an updated index buffer. Of course, this may mean that simplification disregards the attribute variance over some edges if it doesn't know about these attributes; but with regularization it's generally speaking not a significant issue.

When using simplifyWithUpdate, the simplifier will update attributes that it does see; however, since it also produces indices that refer to the original buffer, all attributes that it doesn't see can also be retained as is. Using bone weights as an input can in this case slightly improve the weighting if the vertex positions move, but may not be required, strictly speaking.

I agree that in theory, passing all information to the simplifier is preferable. It's very expensive though, memory and compute-wise, which is why a hybrid scheme is interesting.