v0.35: Improved error messaging

This is a small release that adds error messaging and an editor break when a called function is not found.

The Zig toolchain produces stripped binaries where functions you want to call need to be added to the public API. Without the public API, it's just an inscrutable binary. Calling a function that was not found resulted in error messages that didn't state exactly what was wrong. This has been fixed.

Full Changelog: v0.34...v0.35

v0.34: Minor fixes

This release is just a convenience release for those who are using CMake or SCons to build their projects. Docker is now completely unused for C++ projects that have their own build system in the project. We are still working out how to instantiate or manage a build system from the editor.

There is now also support for program libraries. In project settings one can add a library and a github account/repository, like so:
mylibrary with the value mygithub/myprograms. Once added, we can do Sandbox.download_program("mylibrary/hello_world"). There must then be a hello_world.zip in the latest release of that repo.

Other improvements:

• Record public API and properties for profiling, for better reporting even when programs are stripped
• Improve sandbox loading times slightly
• Improve best-effort symbol lookups

What's Changed

• For C++, check if CMake or Scons and disable version check by @fwsGonzo in #238

Full Changelog: v0.33...v0.34

v0.33: Zig toolchain support

There is now support for using Zig to build RISC-V programs that are then used by Godot Sandbox. Zig is a programming language that has a stand-alone program with no dependencies for all platforms. It can build full Linux RISC-V programs using the zig cc and zig c++ commands. The underlying toolchain is LLVM-based and the C++ standard library is libc++. There are now CMake examples using the Zig toolchain and the unittests are also built and run using Zig.

The big picture idea here is that people will be able to compile Godot Sandbox programs no matter which platform they are on, without having to download anything other than Zig. This is effectively a Docker alternative, but with some additional benefits. CMake and git is still required for CMake projects.

Programs built with the Zig toolchain are much smaller than when built with the standard RISC-V GNU toolchain. Have a look for yourself: https://github.com/libriscv/godot-sandbox-programs/releases . The v0.5 binaries built using Zig are nearly half the size from v0.4. Most notably, the reduced binary sizes have reduced binary translation compilation times heavily.

Documentation on how to use this on Windows and macOS will be coming. We've tested macOS and Windows!

Otherwise, mostly bugfixes:

• Fixed a bug with binary translations and multiple instances of the same program with different memory sizes
• Fix precise simulation not throwing exception after a timeout
• Added a new program property on Sandbox, exposing the current program in the editor
• Made heap allocations default 16-byte aligned, in order to match expectations by run-times
• Add initial native memalign() (+friends) support in the guests
• Add a pre-generated version of the run-time generated API to CMake builds (default enabled)
• Fetch the Godot Sandbox C++ API automatically in CMake builds, making it easier to use

Full Changelog: v0.32...v0.33

v0.32: Program libraries and Rust improvements

It's now possible to download pre-made programs using Sandbox.download_program("name"):

if !FileAccess.file_exists("res://luajit.elf"):
	var buffer = Sandbox.download_program("luajit")
	var fa = FileAccess.open("res://luajit.elf", FileAccess.WRITE)
	fa.store_buffer(buffer)
	fa.close()

var luajit = Node.new()
luajit.set_script(load("res://luajit.elf"))
luajit.execution_timeout = 0
luajit.add_function("test", func(name): return "Test " + str(name) + " called!")
luajit.add_function("add", func(a, b): return a + b)
luajit.run("""
print(test(1))
print(add(333, 666))
function fib(n, acc, prev)
	if (n < 1) then
		return acc
	else
		return fib(n - 1, prev + acc, acc)
	end
end
print("The 500th fibonacci number is " .. fib(500, 0, 1))
""")

Read more about this feature here.

Rust has gained a new GodotString and access to the public API:

extern "C"
fn test(a: i32, b: i32, c: GodotString) -> Variant {
	let sum = a + b + c.to_string().parse::<i32>().unwrap();
	Variant::new_integer(sum.into())
}

pub fn main() {
	register_public_api_func3("test", test, "int", "int a, int b, String c");
}

Also:

• Fixed a rare bug with shared execute segments in libriscv
• Removed old Public API method as it didn't work with all languages, nor with stripped binaries
• Added new Public API method that is accessible during int main() (startup).
• Implement Public API support for sandboxes loaded from byte arrays
• Fixed empty Public API argument list appearing as Object
• Add support for Zig cc/c++ RISC-V builds using riscv64-linux-musl
• Fix multiple build issues when LLVM's libc++ is the standard library
• Add alternative API paths for when std::string is not 32-byte with 16b SSO
• Fixed compiler flags for C sources when using the CMake API

What's Changed

• Allow creating public API at init through system call by @fwsGonzo in #230
• Add function to download a known program from programs repo by @fwsGonzo in #231

Full Changelog: v0.31...v0.32

v0.31: Minor bugfixes

This release is mostly maintenance. Small changes that removes some friction here and there.

• Add Sandbox::try_compiling_binary_translation() to automatically produce a shared object (only tested on Linux so far)
• Toggling generate run-time API and methods requires an editor restart (fixed)
• Cache StringName usage where possible, reducing overhead of getting/setting properties
• Add ArrayProxy::get_as_type()
• Add better support for detecting void return types in the generated run-time API and public APIs
• Enable C++ demangling and 128-bit integers again in WASM
  • Sidenote: Demangling can probably be enabled for other platforms too, but needs to be verified first

In the future we will try to get editor documentation into Godot Sandbox, however it is currently only available for debug builds which we don't produce. We already build a ton of artifacts in CI and we just cannot build more. For those of you who knows your way around CI and you really want documentation, it's currently sitting in an open PR.

Also, check out the newly written introduction in the documentation for Godot Sandbox.

Full Changelog: v0.30...v0.31

v0.30: Custom public APIs

Public API with auto-complete

It is now possible to craft a custom API for Sandbox programs, documented here. The public API is optional and overrides the old method when present.

The public API generates detailed auto-completion in GDScript.

Binary translations loadable at run-time

It is now also possible to load shared objects that contains binary translations for programs on Windows, macOS and Linux. Documented here. Binary translations are freestanding C99 and should be very easy to compile using any local compiler. Adding -DCALLBACK_INIT will make the dll self-register when loaded:

x86_64-w64-mingw32-gcc-win32 -shared -O2 -DCALLBACK_INIT bintr_test.cpp -o bintr_test.dll

An example of compiling a binary translation for Windows from a Linux terminal. The .dll can be loaded using Sandbox.load_binary_translation("res://bintr_test.dll") during initialization. Loading DLLs after init is disabled as a security measure.

This feature is especially useful during development as baking binary translations into Godot or an extension is time-consuming. Compiling a shared object can be done automatically, even from the Godot editor, but the project must be reloaded in order to load it during init. It can technically also be used as part of shipping (performant) updates to desktop systems.

Other changes

• Setting max instruction limit to 0 will now disable execution timeout and enable faster intepretation, resulting in up to ~15% performance gain.
• Improve error messages when failing to cast or unbox Variants
• Add Array::at_or(), ArrayProxy::get_or() and DictAccessor::value_or() to the public API
• Add Sandbox::reset() which quickly resets the loaded program
• Fix missing language icons in the editor
• The performance of freeing heap allocations has been improved slightly

PRs merged

• Add support for loading binary translation DLLs on Windows by @fwsGonzo in #221
• Separate languages into separate docker and project folders by @fwsGonzo in #222
• Allow programs to expose custom API by @fwsGonzo in #224
• Parse public API and generate method information by @fwsGonzo in #225

Full Changelog: v0.29...v0.30

v0.29: Various bugfixes, SConstruct custom builds

This release fixes quite a few bugs, further stabilizing the APIs and the extension.

• It's now possible to build custom programs using SConstruct
  • When a SCons file is present, saving a .cpp file will invoke scons instead
• Make it possible to disable execution timeout by setting it to 0, which yields up to 13% performance
  • If the sandbox program loops forever the game can still be closed from the editor or process explorer, like other hung processes
• Completely disable all socket calls, reducing size of all shipped binaries
• Add a work-around for an issue with icon registration in the Android editor
• Fix mishap with BCLR and BINV instruction bitmasks, leading to exception about unimplemented opcodes
• Fix PackedVector3Array not able to be constructed from a Variant
• Fix Sandbox monitoring properties being inadvertently stored in project file
• Many improvements made to Array and PackedArray<> interactions
• Improve support for double builds, add DOUBLE_PRECISION CMake option to enable in custom builds
• Add size() and is_empty() to packed arrays
• Add support for PackedVector4Array
• Add a new Sandbox property that limits maximum heap allocations
• Add monitoring that shows arena allocations, deallocations and current chunk usage
• Add EXPECT() macro to the API
• Increase the argument limit for vmcallv to 16 (unxboxing disabled)
• Increase the argument limit for vmcall to up to 11 arguments (unboxing enabled)
  • The limit is still 7 integral and 8 fp arguments, but can now be better spread out

PRs

• Update SCsub for linux by @Ughuuu in #208
• Fix windows target for module build 1/2 by @Ughuuu in #211
• Build using SConstruct, when present by @Ughuuu in #213

Full Changelog: v0.28...v0.29

v0.28: Live-profiling and Godot module support

Godot Sandbox can now be built as a module, thanks to @Ughuuu. There is now also a live-profiling feature that enables automatic sampling of Sandbox programs in order to show where time is most spent. The feature works on all platforms, including Web. Documentation here.

The image shows live-profiling being displayed in the demo project. The profiling information is automatically gathered over time.

PRs

• Add support for instruction-counter profiling by @fwsGonzo in #199
• Build as both addon and module. Part 1, integration. by @Ughuuu in #184
• Make static libs too by @Ughuuu in #202
• package everything for modules by @Ughuuu in #203

Full Changelog: v0.27...v0.28

v0.27: Hot-reloading support

This release adds hot-reloading support to the Sandboxes initialized with set_program, directly or indirectly.

It also adds method argument names with default values to the run-time API. The full run-time API has higher compilation times, but the argument names can be disabled in project settings. It's also possible to filter out classes in order to reduce the generated API down to a more manageable size for the compiler. There is a default filtering of things like OS, multiplayer and networking.

Screencast.from.26.okt.2024.kl.18.18.+0200.webm

Full Changelog: v0.26...v0.27

v0.26: Run-time generated C++ API

A new run-time generated C++ API is made available to Docker- and CMake-based projects. The API is auto-generated on first save, and contains all loaded classes including extra modules and GDextensions. This adds auto-complete to your external editor as well, which greatly increases productiveness.

Since it's a run-time generated API it will automatically follow Godot versions forward.

	JSON j = ClassDB::instantiate<JSON>("JSON");
	j.parse(
	R"({
		"pi": 3.141,
		"happy": true,
		"name": "Niels Nielsen",
		"nothing": null,
		"answer": {
			"everything": 42
		},
		"list": [1, 0, 2],
		"object": {
			"currency": "USD",
			"value": 42.99
		}
	})");
	print(j.get_data());

Even the Sandbox class can be auto-completed in your favorite editor, as it is a loaded extension:

extern "C" Variant test_sandbox_pass(Sandbox s) {
	Sandbox s2 = s.FromBuffer(PackedArray<uint8_t>((const uint8_t *)binary_data, binary_data_size));
	return s2;
}

In the future we may add better support for static functions in classes and also try to add inline documentation to classes, properties and methods.

A new load<Class> resource loading helper has been added:

Node2D scene = load<PackedScene>("res://scenes/mod/mod.tscn").instantiate();

You can iterate nodes by group:

    for (Node node : get_tree().get_nodes_in_group("Coins")) {
    }

Other changes include preventing access to restrictions while in an active VM call, even indirectly.

What's Changed

• Add new run-time generated C++ API by @fwsGonzo in #189
• Add macros for static class method calls by @fwsGonzo in #190

Full Changelog: v0.25...v0.26