Genesis AI clams it is 10-80x faster than mujoco mjx. Do you know is that true and why is that?

[junqingqiao]

Genesis AI clams it is 10-80x faster than mujoco mjx. Do you know is that true and why is that?

[yuvaltassa]

As far as we can tell there is no reason for it to be that much faster. It appears they are comparing to single threaded MuJoCo rather than multithreaded or MJX, so the comparison seems disingenuous. However, we have not looked very carefully.

[kassasin]

It is true, they don't compare with MJX. I think their idea is interesting about combine simulator with generative AI .I think mujoco can do better .

[yuvaltassa]

Agreed. That is the main thing that is interesting. However:

1. It is not yet released.
2. This feature (chatGPT looking at an image and writing MJCF?) seems like something effectively external to the simulator, so it would work with MuJoCo (or Drake or any simulator) just as well as Genesis.

I look forward to playing with it.

[kassasin]

Me too !!!

[junqingqiao]

Thanks a lot for the quick reply. I'm doing exoskeleton research at MIT and heavily rely on MJX for human&robot simulation and policy learning. I have lightly modified mujoco and mjx to make a muscle model with flexible tendon work with mjx. I'm rely on you. I als want to say thanks for the great contribution to both academia and the industrial.

[zhouxian]

Thank you for raising the discussion!! The 10-80x speed is in comparison with isaac, not mujoco. Single-threaded mujoco is much slower than that.

We didn't show comparison with MJX because

• MJX (as far as we know) still doesn't support mesh-based collision well, and need specially-designed mjcf file for better performance. In the regular scene we test, if we use the standard Franka xml, it's much much slower due to collision bodies using mesh instead of primitive
• When using primitive for collision, MJX gives similar speed as Isaac, so we picked isaac to represent existing GPU-based simulation for comparison

Let me know if you have any other questions!

[zhouxian]

Thank you for raising the discussion!! The 10-80x speed is in comparison with isaac, not mujoco. Single-threaded mujoco is much slower than that.

We didn't show comparison with MJX because

MJX (as far as we know) still doesn't support mesh-based collision well, and need specially-designed mjcf file for better performance. In the regular scene we test, if we use the standard Franka xml, it's much much slower due to collision bodies using mesh instead of primitive
When using primitive for collision, MJX gives similar speed as Isaac, so we picked isaac to represent existing GPU-based simulation for comparison
Let me know if you have any other questions!

[Vittorio-Caggiano]

Hey Btaba, Thanks for the detailed reply. I'm looking forward to the CUDA-powered collision detection solution. As a PhD student relying heavily on MJX, I hope to contribute to this project. I have modified MJX to have a muscle model with flexible tendons based on the Millard2012AccelerationMuscle from OpenSim. If you are interested in the flexible tendon model, I'm happy to share my repo with you.

Hi @junqingqiao I am very interested in the muscle tests you did with MJX. Did you by any chance test also any of the myosuite muscle models? What do you mean by 'flexible tendons'? elastic?

[junqingqiao]

Hey @Vittorio-Caggiano, thanks for being interested in my modified mjx! I have used myosuite and a lot of my colleagues are using myosuite. Myosuite is good for a lot of research and is a very well-integrated platform for reinforcement learning. But for my application, I need flexible tendons to simulate energy harvesting by tendons while walking and running. Also, tendons play an important role in jumping which my exoskeleton will augment.

I developed a stand-alone muscle-tendon unit solver for mjx. The tendon is elastic with nonlinear stiffness, not like the default muscle in mujoco&mjx with infinite stiffness. My model is also fully differentiable since it assumes that there is a small mass between muscle and tendon. Thus, the length of muscle and tendon can be solved through forward integration through time. It is not like OpenSim, which uses Newton iteration to solve the muscle and tendon length, which is hard to be differentiable.

The simulator is fully functional. I'm working on the accurate metabolic cost calculation part now. I feel very lonely working on such a huge project on my own. I'm happy if we can work together.

[yuvaltassa]

@junqingqiao, my preference would be for the kinematic/dynamic dof to be an actual dof rather than hidden in the muscle model. It sounds like the only thing missing for this to be possible would be nonlinear stiffness in MuJoCo, correct? If yes, what sort of nonlinearity do you need?

[junqingqiao]

Yes, the only missing part is the nonlinear stiffness on tendons. I used cubic splines to simulate the normalized tendon length-force curve. I put my code for the spline in the end.

Here is the comparison of the spline curve in my model vs Opensim's

Here is my reason for making the MTU simulation a sub-cycle in mjx: The mass between muscles and tendons is very small compared to the other parts of the body. So it requires a smaller dt to ensure the stability. In my modification, I add another integrator for mtu only with a smaller dt. So the forward looks like:

def forward(m: Model, d: Data) -> Data:
  """Forward dynamics."""
  d = fwd_position(m, d)
  d = sensor.sensor_pos(m, d)
  d = fwd_velocity(m, d)
  d = sensor.sensor_vel(m, d)
  d = fwd_actuation(m, d)
  # BUGMAN: Added the biomtu calculation, it will updates the force for the next step.
  d = named_scope(acceleration_mtu.biomtu_step)(m, d)
  d = fwd_acceleration(m, d)
  d = sensor.sensor_acc(m, d)

In addition, a solver is also needed to solve the muscle and tendon relative length at the beginning of the simulation.
Another reason that I made MTU simulation stand-alone is that there were no muscle and tendon simulations in MJX 8 months ago when I started my development. Also having a stand-alone MTU module also allows researchers to test different muscle models. It can benefit several new muscle models under development such as the "Winding Filament Muscle Model" (https://pmc.ncbi.nlm.nih.gov/articles/PMC3259925/) from Kissa Nishikawa's and Hugh Herr's labs.

Here is the code to generate the tendon length-force curve

# BUGMAN Aug 13 2024
# This file is used to generate curve function for muscle tendon force simulation.
# The Cubic Hermite Spline is used here for it's local control and c1 contunity.

from scipy.interpolate import CubicHermiteSpline
import jax
import jax.numpy as jp

class biospline():
    def __init__(self, x: jax.Array, y: jax.Array, dy: jax.Array):
        self.spline = CubicHermiteSpline(x, y, dy)
        
    @classmethod
    def from_control_points(cls, c: jax.Array):
        # Input is control points
        x = c.T[0]
        y = c.T[1]
        dy = c.T[2]
        return biospline(x, y, dy)
        
    def getJaxFunc(self):
        x = jp.array(self.spline.x)
        c = jp.array(self.spline.c).transpose()  # Transpose to match original indexing
        # device = jax.devices()[0]
        # x = jax.device_put(x, device=device)
        # c = jax.device_put(c, device=device)
        
        def evaluate_spline(t):
            # i = jp.searchsorted(x, t) - 1
            # i = jp.clip(i, 0, len(x) - 2)
            
            # Compute interval indices
            n = x.shape[0]
            # Create a condition array where each element is True if t >= x[i]
            conditions = t[..., None] >= x[:-1]  # Shape: (t.shape, n-1)
            # Sum over the last axis to get the interval indices
            i = jp.sum(conditions, axis=-1) - 1  # Shape: t.shape
            i = jp.clip(i, 0, n - 2)
            
            s = t - x[i]
            # Corrected coefficient indexing
            y = ((c[i, 0] * s + c[i, 1]) * s + c[i, 2]) * s + c[i, 3]
            return y

        return evaluate_spline

# BUGMAN Aug 13 2024
'''
This file is used to generate curves for muscle tendon force simulation.
The parameter of those curve should be the same as those used in the OpenSim.
The curves include:
1. Muscle Active Force-Length curve
2. Muscle Passive Force-Length curve
3. Muscle Active Force-Velocity curve
4. Tendon Passive Force-Length curve
5. Muscle Compressive Force Cos Pennation curve
6. Muscle Compressive Force Length curve

There will be another version of this factory function. 
The new version will generate the biospline segment function based on optimization.
It optimize the paremeters of the spline by minimizing the distance between the functions and opensim curves.
'''

from mujoco.mjx._src.biomtu.biospline import biospline
import jax.numpy as jnp
import math


def Muscle_Active_Force_Length_Curve():
    control_points = []
    control_points.append([0.0,   0.0,   0.0])
    control_points.append([0.44,  0.0,   0.0])
    control_points.append([0.48,  0.04,  3.1])
    control_points.append([0.67,  0.68,  3.1])
    control_points.append([0.72,  0.795,  1.17])
    control_points.append([0.80,  0.87,  0.87])
    control_points.append([0.90,  0.9568,  0.855])
    control_points.append([0.95,  0.995,  0.465])
    control_points.append([0.97,  0.9995,   0.06])
    control_points.append([1.03,  1.0,   -0.08])
    control_points.append([1.05,  0.996,   -0.27])
    control_points.append([1.07,  0.987,  -0.7])
    control_points.append([1.3,  0.685,  -1.4])
    control_points.append([1.5,  0.405,  -1.4])
    control_points.append([1.76,  0.03,  -1.0])
    control_points.append([1.81,  0.0,   0.0])
    control_points.append([2.0,   0.0,   0.0])
    
    control_points = jnp.array(control_points)
    spline = biospline.from_control_points(control_points)
    
    return spline.getJaxFunc()


def Muscle_Active_Force_Velocity_Curve():
    concentricSlopeAtVmax = 0.0
    concentricSlopeNearVmax = 0.25
    isometricSlope = 5.0
    eccentricSlopeAtVmax = 0.0
    eccentricSlopeNearVmax = 0.15
    maxEccentricVelocityForceMultiplier = 1.4
    concentricCurviness = 0.6
    eccentricCurviness = 0.9


    corner1_size = 0.1/concentricCurviness
    corner2_size = 0.3/eccentricCurviness
    corner3_size = 0.05

    control_points = []
    control_points.append([-1.5, -0.5*concentricSlopeAtVmax, concentricSlopeAtVmax])
    control_points.append([-1.0, 0, concentricSlopeAtVmax])
    control_points.append([-1.0+corner1_size, corner1_size*(concentricSlopeAtVmax+concentricSlopeNearVmax)/2, concentricSlopeNearVmax])
    control_points.append([-0.5, 1-0.5*isometricSlope*(0.34), isometricSlope*(0.12)])
    control_points.append([-0.3, 1-0.3*isometricSlope*(0.445), isometricSlope*(0.2)])
    control_points.append([0.0, 1.0, isometricSlope])
    control_points.append([corner2_size/3, maxEccentricVelocityForceMultiplier-(1)*eccentricSlopeNearVmax, eccentricSlopeNearVmax*3])
    control_points.append([corner2_size, maxEccentricVelocityForceMultiplier-(1-corner2_size)*eccentricSlopeNearVmax, eccentricSlopeNearVmax])
    control_points.append([1.0-corner3_size, maxEccentricVelocityForceMultiplier-corner3_size*(eccentricSlopeAtVmax+eccentricSlopeNearVmax)/2,eccentricSlopeNearVmax])
    control_points.append([1.0, maxEccentricVelocityForceMultiplier,eccentricSlopeAtVmax])
    control_points.append([1.5, maxEccentricVelocityForceMultiplier+(0.5-corner3_size)*eccentricSlopeAtVmax, eccentricSlopeAtVmax])

    control_points = jnp.array(control_points)
    spline = biospline.from_control_points(control_points)
        
    return spline.getJaxFunc()


def Muscle_Passive_Force_Length_Curve():
    strainAtZeroForce = 0.0
    strainAtOneNormForce = 0.7
    stiffnessAtLowForce = 0.2
    stiffnessAtOneNormForce = 2.0 / (strainAtOneNormForce-strainAtZeroForce)  # = 2.86
    curviness = 0.75
    control_points = []
    control_points.append([0.0,  strainAtZeroForce,  0.0])
    control_points.append([1.0,  strainAtZeroForce,  0.0])
    control_points.append([1.16, stiffnessAtLowForce*0.16/2, stiffnessAtLowForce])
    # control_points.append([])
    control_points.append([1+strainAtOneNormForce, 1, stiffnessAtOneNormForce])
    
    control_points = jnp.array(control_points)
    spline = biospline.from_control_points(control_points)
        
    return spline.getJaxFunc()
    

def Tendon_Passive_Force_Length_Curve():
    strainAtOneNormForce = 0.049
    stiffnessAtOneNormForce = 1.375/strainAtOneNormForce
    normForceAtToeEnd = 2.0/3.0
    curviness = 0.5
    
    control_points = []
    control_points.append([0.0,  0.0,  0.0])
    control_points.append([1.0,  0.0,  0.0])
    # control_points.append([1.01])
    control_points.append([1.0+strainAtOneNormForce-(1-normForceAtToeEnd)/stiffnessAtOneNormForce, normForceAtToeEnd, stiffnessAtOneNormForce ])
    control_points.append([1+strainAtOneNormForce, 1, stiffnessAtOneNormForce])
    
    control_points = jnp.array(control_points)
    spline = biospline.from_control_points(control_points)
        
    return spline.getJaxFunc()


def Muscle_Compressive_Force_Cos_Pennation_Curve(engagementAngle = 80):
    
    cos_engagementAngle = math.cos(engagementAngle/180*math.pi)
    stiffnessAtPerpendicularFiber = -1/cos_engagementAngle *2
    corner_size = 0.02

    control_points = []
    control_points.append([-0.1,  1 - stiffnessAtPerpendicularFiber*0.1,  stiffnessAtPerpendicularFiber])
    control_points.append([-corner_size, 1 - stiffnessAtPerpendicularFiber*corner_size, stiffnessAtPerpendicularFiber])
    control_points.append([0, 1, (stiffnessAtPerpendicularFiber-1/cos_engagementAngle)/2])
    control_points.append([corner_size, 1+corner_size*(stiffnessAtPerpendicularFiber-1/cos_engagementAngle)/2, -1/cos_engagementAngle])
    control_points.append([cos_engagementAngle - corner_size, 1+corner_size*(stiffnessAtPerpendicularFiber-1/cos_engagementAngle)/2-(cos_engagementAngle-corner_size*2)/cos_engagementAngle, -1/cos_engagementAngle])
    control_points.append([cos_engagementAngle + corner_size, 0, 0])
    control_points.append([cos_engagementAngle + 0.1, 0, 0])
    control_points = jnp.array(control_points)
    spline = biospline.from_control_points(control_points)
    return spline.getJaxFunc()


def Muscle_Compressive_Force_Length_Curve():
    normLengthAtZeroForce = 0.6
    stiffnessAtZeroLength = -8.7
    corner_size = 0.05

    control_points = []
    control_points.append([-0.2,  1 - stiffnessAtZeroLength*0.2,  stiffnessAtZeroLength])
    control_points.append([-corner_size, 1 - stiffnessAtZeroLength*corner_size, stiffnessAtZeroLength])
    control_points.append([0, 1, (stiffnessAtZeroLength-1/normLengthAtZeroForce)/2])
    control_points.append([corner_size, 1+corner_size*((stiffnessAtZeroLength-1/normLengthAtZeroForce)/2-1/normLengthAtZeroForce)/2, -1/normLengthAtZeroForce])
    control_points.append([normLengthAtZeroForce - corner_size, corner_size/normLengthAtZeroForce/4, -1/normLengthAtZeroForce/2])
    control_points.append([normLengthAtZeroForce + corner_size, 0, 0])
    control_points.append([normLengthAtZeroForce + corner_size + 0.2, 0, 0])
    control_points = jnp.array(control_points)
    spline = biospline.from_control_points(control_points)
    return spline.getJaxFunc()

  

[yuvaltassa]

That's cogent. Smaller timestep for MTU makes total sense.

That said, supporting nonlinear stiffness natively still sounds like a useful feature to me. I'm imagining stiffness = "a b c" where a is the regular linear term and b, c are optional cubic and quintic terms, respectively. WDYT?

Keep in mind that under my proposal, you'd get the goodness of MJ's implicit integration of the mechanical dof which gives you maybe a 2-3x stability in terms of timestep size.

Another argument in favour of my proposal is that you can use tendon length limits for your passive nonlinear stiffness. It's not as cleanly formulated as what you're doing since the solver's effective stiffness is a nontrivial function of the sol{imp/ref}limit params, but the timestep issue goes away entirely.

In fact, it would be really neat if you tried to fit your stiffness using the solparams... I can help you with that if you like.

I feel like this discussion has totally strayed from the OP, if you'd like to continue, please open a new discussion 🙂

[StoneT2000]

I dug around the Genesis benchmarking code and it is a bit strange.

1. They use the fastest physics setting available which is less accurate (eg can't grasp objects)
2. They take one action followed by 999 steps of no new actions meaning it's likely their physics solver can exit out earlier and "run faster". The scene is mostly static
3. Self collisions are disabled by default. Enabled and runs much slower. Most sims and benchmarks don't disable self collisions as far as I understand.

I have a longer explanation and report on things regarding actual benchmark speeds that you can check out https://stoneztao.substack.com/p/the-new-hyped-genesis-simulator-is

While one can discuss at length about how to measure manipulation simulation speeds it seems quite clear at least the original Genesis benchmark code needs correction (I provided some in my post). Seems the speed might be 150x slower than claimed.

RE the comment about not comparing against MJX, while MJX might not support convex mesh collisions the Genesis benchmark disables those collisions to begin with (except the one with the planar floor).

[zhouxian]

While I have been trying my best to respond in a principle and academic way, none these accusations are fairly grounded on facts and there's a series of mistakes in the benchmarks you performed :) So this is now escalating to nonsense and biased attack, and I feel sorry to see that things have evolved this way.

We will not keep engaging in these back-and-forth debate as they are exhausting and meaningless. Leaving this message here to whoever seeing the discussion: rest assured our benchmark is legit and at least at this moment, we are not seeing any needs for correction. However, we are working on and will release a more detailed report to the public in the coming days.