Allow running web runtime in background windows #793
Conversation
This commit reworks the frame timing of the runtime ensuring a regular update on every frame on 60 fps monitors, while still supporting other framerates, both higher and lower. It achieves this by measuring the framerate continuously, and performing updates in a requestAnimationFrame() callback when the framerate is close to 60, but switching to a setTimeout() timing scheme otherwise. This keeps animation smooth on a 60 fps screen, but keeps updates both full-speed and at a regular 60Hz on e.g. a 30 fps screen, important for keeping tick-based audio smooth. This also has the effect of allowing you to run the runtime in a minimised or background browser window. This really improves the audio experience, allowing you to run games or music carts in the background. This also improves the netplay experience, as the game won't lag or halt any time a player clicks out of the browser window or changes tab.
The web runtime doesn't run on NodeJS, but was using node types. This was causing issues because some node types override the DOM types, e.g. `setTimeout()`. Adding `"types": []` to tsconfig.json filters out the types in `node_modules/@types` and prevents them from being used, including `node_modules/@types/node`, which is installed because it is a dependency of our dependencies.
Setting these security headers grants us access to higher precision time. This improves our ability to pace frame timings and make accurate performance measurements. I hope to take advantage of this to improve the devtools in the future.
Previously favicon.ico was being requested from wasm4.org when developing locally. This won't work if the user is offline or the website goes down, etc. It also doesn't work with secure CORS headers, as it is served from another domain.
There was a problem hiding this comment.
This is really clever, thanks!
I think we should only keep the game running in the background when netplay is active as a special case. By default we should continue to respect requestAnimationFrame's preference to pause when the browser is in the background, to avoid hogging the system unnecessarily. What do you think?
Please also remove this bit from the docs now that this is resolved:
wasm4/site/src/components/NetplayCart.js
Lines 42 to 43 in 77352e0
Also fixes up the code that pauses audio, which was buggy and some of which was never called. Pausing the audio system also helps to hint to browsers that it's okay to throttle the update loop when the menu is open.
I disagree, I think we should rather be respecting the user's wishes. If the user wants to continue to listen to the music in the background, they should be able to do so. It's better that they have the choice to pause the game manually if they would like to. Halting updates like that also interacts poorly with the audio system, especially tick-based audio. Long notes keep playing, and then are missing when you tab back to the game, it sounds pretty clunky and bad. ( Allowing the game to continue running also makes development and hot-swapping nicer. You can imagine someone composing music or editing sound effects with the runtime running in the background. I've also included a patch I had lying around that pauses the audio properly when the menu is open. This stops that strange note-holdover effect I mentioned above happening when you open the menu, but it also hints to the browser that it's okay to throttle timers while the menu is open. |
This patch makes update timing more regular in various environments, including when the browser window is in the background. This allows audio and netplay to continue if the user e.g. minimises the browser window. This is achieved by switching to a setTimeout() based timing solution if the observed requestAnimationFrame rate is not a multiple of 60.
Also included are a few other relevant fixes and improvements.