Compatibility with Godot 4: time estimate?

[k8035]

Hi! Thank you for this project. Are you able to a give a rough estimate when it will be compatible with Godot 4? Regardless how long it takes, it's appreciated.

[touilleMan]

I can assure you this will be ready before Godot arrives

More seriously, I hope to have a first usable version before the end of this year ;-)

[GeorgeS2019]

[WIP] progress update July 11th 2023

https://github.com/touilleMan/godot-python/actions/runs/5601317261/job/15173489440
Windows part fail

[WIP] Progress Update June 4th 2023

Support new GDExtensionInterfaceGetProcAddress

[WIP] Progress Update May 22nd 2023

#385

Fix test run for windows x86

[JuliuszK]

Hi,
How's the porting of this plugin to Godot 4 going? Would you be able to provide any rough estimates?

I am currently considering moving to Godot engine with my Python based project. This plugin would be crucial for me, as I have extensive libraries written in Python, which I'd prefer not to re-write in another language.

I would be willing to provide help, if needed, but probably need to familiarize myself with the engine itself first.

[touilleMan]

Hi @JuliuszK !

The porting has been quite slow for last two years, my daily job have been very demanding for the last couple of years and I got a kid on top of that, so things are progressing very slowly.

My hope is to be able to slow down on my daily work this year and hopefully dedicate more time on this project, but in all honesty there is still a great deal of work remaining (the Godot 4 API is very powerful but also pretty low-level and complex to deal with).

I would be willing to provide help, if needed, but probably need to familiarize myself with the engine itself first.

Thanks for the offering, however the project is currently in a very messy state so I won't be able to accept contributions at the moment.

I am currently considering moving to Godot engine with my Python based project.

If you want to use Python with Godot 4, I would advise you to use the C++ bindings instead (Godot-Cpp) and embed Python using python-build-standalone.
The python-build-standalone works very well for embedding (that's what I use I Godot-Python), and you can create a python native module calling into C++ with Pybind11 (I used it before in Godot-Python, it works great).

(You can also ditch Godot-Cpp and directly use the Godot C API, but I would advise against it as the C API is hard to deal with: it is designed to be used by bindings, not by end user)

In the end, this approach is not as comfortable as having Python directly baked into Godot, as you'll have to write wrapper code anytime you want to use a new feature of Godot, but I think it's the best approach for the moment ;-)

[JuliuszK]

Hi!
Thanks a lot for reply. I totally understand that RL needs to take priority over hobbies.

If you want to use Python with Godot 4, I would advise you to use the C++ bindings instead (Godot-Cpp) and embed Python using python-build-standalone.

This suggestion sounds interesting although it's a new territory for me. Does use of a standalone Python build implies that it's not extendable? Part of my project (a level editor for some old game engine) was to provide a python scripting editor. And the expectation would be, that the users could import any python module they wanted to support the logic for their scripted tools. Would they be able to install any arbitrary packages and use the inside the application built with Godot to run their Python script? Or just the ones I provide in my standalone build? (I can see that pip.exe is not available for Windows for those builds...)

and you can create a python native module calling into C++ with Pybind11 (I used it before in Godot-Python, it works great).

OK, I guess I could peek at your code then to figure out the approach you took and then write the wrapping code only for the functionality of Godot 4 API that I need?

BTW. How would you compare performance of Python vs GDScript inside Godot?

[touilleMan]

Does use of a standalone Python build implies that it's not extendable?

No, think of python-build-standalone as libpython.so shared library (in practice it's more than that given it contains plenty of .py files for Python's battery included). This shared library is the same thing that is used for the regular python, so you can load any python code you want, including native module (for instance that's what I'm doing in Godot-Python: Godot loads my extension module which initialize the python interpreter, python interpreter that loads a _godot.so python native module that calls the Godot C API to register Python as a scripting language in Godot)

This key point of python-build-standalone is that it is hard to build CPython from source in a portable way, so instead you download an already existing build.

Would they be able to install any arbitrary packages and use the inside the application built with Godot to run their Python script?

godot-build-standalone comes with the regular python and pip commands so you can use them to install the Python packages you need.
There is one caveat though: you cannot install binary wheel from source. This is because when building a native wheel, python relies on the configuration that have been use to build itself (this is done this way to avoid incompatibilities between compiler, and to pass similar compilation flags).
So you end up with godot-build-standalone's compilation flags which don't correspond to what on your machine (typically the Python's include dir will be passed as something like -I /work/python, but you won't have a /work/python dir on your machine !)
(see astral-sh/python-build-standalone#152 for more info)

I can see that pip.exe is not available for Windows for those builds...

That's strange, I recall using pip on Windows. Have you tried using python.exe -m pip instead ?

BTW. How would you compare performance of Python vs GDScript inside Godot?

No idea, GDScript used to be very slow, but a great deal of effort have been made so it's no longer the case.
On the other hand CPython have become faster and faster (and it's only the beginning with the work on a JIT starting in Python 3.13 ^^)

I think the important part to take into account is the fact Godot internally works with a Variant type (it is equivalent to Python's PyObject, so basically a C enum that can store any type of data: string, int, float, Vector2 etc.).
On top of that there is two ways to call a Godot API functions (e.g. CharacterBody2D.is_on_floor):

1. passing Variant as parameter and receiving a Variant as result
2. passing parameters and return value as a void* pointer pointing on the exact type, this is called "ptrcall".

GDScript internally use the Variant, so it always uses the first way.
With the GDExtension API we can choose to use either method, the first one being simpler but requires to do conversions (because a Vector2 in Python will be represented as a PyObject, then must be converted into a Variant before using it to call a Godot function).
The second way is more complex (you get a segfault if you pass the wrong arguments) but faster, this is the way used in Godot-Cpp.

And finally you get some types such as string that are totally different between Python and Godot, so you must do conversions between the two. This can kill your performances if you don't pay attention ;-)

[JuliuszK]

Thanks a lot for this detailed explanation.

It does sound a little bit outside of my realm at the moment, but maybe I'd be able to get into it, when I familiarize myself with the engine a bit more or maybe my team mate will be happy to look into that integration.

[nickcollins]

I know this probably isn't the right place to ask, but I am trying to get godot-python to work with godot 3.5 on windows 10.
My python code requires python 3.10 + , but when I build godot-python using python3.12, it seems that the build winds up on python3.8 anyways. Is there a way to set the python version that godot-python will use? Maybe I missed a setting somewhere? Does godot-python use a baked in python or does it use the one that I build it with?

[touilleMan]

@nickcollins, the version of Python embedded is specified in platforms/{your-os}/SConscript, e.g. :

godot-python/platforms/windows-64/SConscript

Line 34 in b9757da

CPYTHON_PREBUILD_URL = "https://github.com/indygreg/python-build-standalone/releases/download/20200830/cpython-3.8.5-x86_64-pc-windows-msvc-shared-pgo-20200830T2254.tar.zst"

On top of that, you can see there is the generate_cpython_build functions that do some cooking on the cpython (typically it checks the version of the manifest in the python-build-standalone archive, which I think has changed recently)