Improved Python interactive viewer

[saran-t]

TL;DR: Python bindings for version 2.3.3 (launched today) come with a new viewer.launch_passive function, which does not block user scripts (similar to mujoco_py's old viewer). On macOS, users must launch their Python script through a new mjpython command, which is installed as part of the mujoco package. Users can treat mjpython as a drop-in replacement for the python command (e.g. it accepts the same command line arguments).

---

UPDATE: The launch_passive API has been changed in version 2.3.4. The user now needs to call viewer.sync() in order for the viewer to pick up changes to the physics state. This is done to eliminate the race condition between user code and the viewer's render loop. See documentation for detail.

---

Back in December, as part of version 2.3.1, we launched the Python bindings for the MuJoCo interactive viewer (once again, this work was largely done by @aftersomemath). Since then, we've received quite a fair amount of feedback about the viewer, specifically its usability relative to the old mujoco_py viewer.

By far the biggest complaint is that our new viewer blocks the main thread that is executing the user's script, whereas mujoco_py's viewer returns immediately. Suggested workarounds so far included using the control callback and running user code in a separate thread, both of which are more cumbersome than simply being able to have the script continue running.

The obvious solution for this is to run the viewer itself in a separate thread. This works fine on Linux and Windows, macOS has a fairly fundamental limitation where Cocoa API functions must be called on the main thread. The mujoco_py viewer worked around this by relying on the user to repeated call its render function. This "render" call actually does rather a lot of heavy lifting behind the scenes: it polls UI event and runs multiple rendering iterations in an inner loop, where the inner loop tries to keep the script ticking in sync with wallclock time. The main downside of this approach is that if the user's script takes too long between calls to render, the entire GUI becomes either janky or entirely non-responsive.

With our 2.3.3 release today, we added a new viewer.launch_passive function which we hope will finally provide most users with a suitable alternative to the old mujoco_py viewer. We have not added this to our official documentation since we're treating this as a beta/experimental release for the feature. It is still somewhat rough around the edges (e.g. interrupting a running script via CTRL-C sometimes results in a segfault), but should function correctly in most cases.

To use the viewer,

from mujoco import viewer
viewer.launch_passive(m, d)

where m and d are instances of mujoco.MjModel and mujoco.MjData respectively. This new function returns immediately, allowing your script to continue to run. Any subsequent change to m or d will be updated live in the viewer.

The main caveat is that on macOS, the script needs to be launched via the mjpython command, which is installed as part of the mujoco package. In other words, if you'd normally launch python somescript.py, you'd now need to run mjpython somescript.py in order to be able to use launch_passive on macOS.

The mjpython launcher is a drop-in replacement that should behave in an identical manner to the python command. It accepts the same command line arguments and picks up whatever virtual environment you happen to be in. If you run mjpython without any further argument, it'll launch the default interactive Python session. If you run mjpython -m IPython it'll launch an IPython shell.

Please try this out and let us know how you get on with the new viewer API!

NOTE: The API may change if beta testing goes well. The current thinking is to eventually have viewer.launch work in this passive mode by default, and the blocking mode will be provided as a keyword argument option.