Investigate if we still need the second pass during the bit rate optimization

[nfrechette]

See: http://nfrechette.github.io/2023/02/26/dominance_based_error_estimation/

Now that we use the dominant rigid shell, do we need the second pass?

[nfrechette]

If the first local space pass is not enough, then the distance we tested was too short and it did not encompass the true movement of the joint once error is factored in. Most joints fit within the error threshold, and so it should be possible to perform a single pass for most clips (joints that fail are very rare).

Consider what happens when every joint in a chain is at the edge of its precision. Should we add the tolerance down the chain instead of adding it to the dominant distance?

[nfrechette]

When we evaluate the error in local space using the dominant shell distance, we take into account the children and their imprecision. However, we fail to account for the parents which introduce error of their own. We either need to add the current joint padding to the overall distance, its parents', or all the parent padding in the chain.

[nfrechette]

I found a number of semi-related issues that contribute to this.

The was a mistake in how the dominant shell distance is calculated that under estimated the distance. I will change that code to be more conservative and to calculate the distance in object space to account for zero scale discarding child joints.

This led me to find that the error metric isn't conservative enough and it isn't performing consistently. When I originally wrote it almost 10 years ago, I asked the question: how can I make sure that the error from rot/trans/scale isn't hidden and suitably approximated. A better question would be, which points on the rigid shell see the largest error. With uniform scale, these points depend on the rotation axis which currently isn't taken into account. With non-uniform scale, the picture is more complicated. This leads to subtle issues where measuring the error in local/object space even with the same shell distance yields different results that cannot be explained by floating point rounding alone. Similarly, different bit rates may yield inconsistent results (e.g. fewer bits appearing to be more accurate). That will also need fixing.

[nfrechette]

As it turns out, the dominant shell distance isn't being calculated correctly per segment. The bone transforms are incorrectly sampled which results in normalized values being used instead of proper ones.

[nfrechette]

After further investigation, I concluded that the dominant shell only really works for rotation. This is because for translation/scale, the error is the same regardless of the distance that we measure at. As such, there is no way to account for the error that joints might introduce in a chain. A dominant joint for translation purposes might be one that has the most intermediary joints. However, it is not clear how to leverage this. If we have a precision of 0.01 and each joint is allowed to reach it, we have no clear way to add conservative padding. We could perhaps add the precision instead: if joint A has 3 children, we could add all 3 precision values and use that for our joint A. This would give an upper bound.

[nfrechette]

Almost done with this, just a few more weeks at current pace. ETA July.

[nfrechette]

Most of the changes are done; only one remains: new error metric.
I've found an analytical solution for rotation/translation, and am now in the process of adding scale support to it. I'll have to rework some things around the error metric, should need another month or two.
The huge time sync of this issue means that the next release will likely be a lot smaller than I originally wanted. I'll revisit the milestone once this is done to see how much time is left before November which is when I'd like to do the next release.

[nfrechette]

Progress is ongoing for scale support, not quite there yet.

[nfrechette]

I've found a path forward for scale support but I'm not super happy with it. Still iterating on this.
I've also implemented VQM support in RTM and it will fit nicely into ACL. We can use it to ensure associativity during the optimization algorithm which is key to allow caching to meet our performance goals. This is closely related to this and something I've been noodling with for 5-8 years and so it is nice to finally find a nice solution.