feat: add support for free-threaded (no-gil) Python 3.13

[mayeut]

Support has been added for Linux & Windows awaiting macOS installers (IMHO, not blocking to ship this experimental feature)

Users have to opt-in this feature using the new free-threaded-support option of the config file (probably preferable over the environment variable which is supported as well).

[henryiii]

Nice! Are we doing to go ahead and ship experimental support for Linux & Windows early, or wait on macOS builds with beta 2? I also think macOS might be tricky, as I'm not what it will look like installing both, I guess they aren't supposed to collide? (Maybe that's why it's taking till beta 2!)

[mayeut]

Are we doing to go ahead and ship experimental support for Linux & Windows early, or wait on macOS builds with beta 2? I also think macOS might be tricky, as I'm not what it will look like installing both, I guess they aren't supposed to collide? (Maybe that's why it's taking till beta 2!)

I think we should ship experimental support without waiting on macOS.
The only thing that's missing is the opt-in mechanism & I think it should ship with the first version allowing free threaded builds (c.f. #1657 (comment))

As for how the macOS build will turn out, the only info I found in the cpython repo was python/cpython#112430 (comment) (didn't find anything on discourse, there might be some other threads out there).
If I understand correctly, it will be an installation option. This probably means we'll have to rework the macOS bit to force a re-install (only in CI of course) if the free-threaded build is missing in an existing installation. But it might get trickier:

Ideally, we'd want to allow both variants to be able to to be installed side-by-side; while that should mostly work today, there are some loose ends that would need to be taken care of involving path names and the like. If it turns out to be too disruptive to support co-existing installs, the installer package could simply inhibit the installation of the normal variant if the user selects the optional free-threaded variant.

[mayeut]

This does not work with call("python", "-m", "pip", "install", "pip>=24.1b1", env=virtualenv_env)
Maybe something to do with shell=True being used on Windows ?

[henryiii]

What if we swap the default, at least while it’s under a beta flag, then decide if we keep it opt-out instead of opt-in when the RC drops?

[henryiii]

@ned-deily, will it be possible to install both variants at the same time on macOS like on windows/linux?

[henryiii]

Should we call this 2.18.1 or 2.19.0? It's experimental and opt-in, so I'm slightly biased toward the first. Especially since we don't have macOS yet.

[mayeut]

with #1792, we might want to go with 2.19.0 anyway ?

[henryiii]

Yes, possibly. Though I think that's also only opt-in, something that was disallowed before is now allowed, but existing builds don't change.

It's sad that #1456 isn't quite ready (even if it was finished, still waiting on the pyodide release, which will likely be delayed I'd expect with some wheel tag changes planned - we were working on it at PyCon). GraalPy might not be too far away, etc. Minor version bumps require everyone to update their two-digit pins. In that sense, I'm not sure this is a minor bump, as nothing changed or is added without user changes.

[mayeut]

I'm not sure this is a minor bump, as nothing changed or is added without user changes.

Let's go with 2.18.1 then ! Maybe being too pedantic regarding semver is not a good idea here.

GraalPy might not be too far away

This one is waiting at least on pip 24.1

I haven't looked in the pyodide one in a long time but saw there were some commits.

[henryiii]

This one is waiting at least on pip 24.1

Well, technically free-threading was too. ;)

[henryiii]

Not going to bother with Travis, it's not even started the job.

[mayeut]

Well, technically free-threading was too. ;)

yes, that's true. but bumping pip to a beta version for cp 3.13 is definitely acceptable (all is in BETA) while bumping pip for python 3.10 is probably less so (and I don't want to add specific code for this, personal opinion)

[ned-deily]

install both variants at the same time on macOS

Yes

[ned-deily]

FYI: 3.13.0b2 is now released (https://discuss.python.org/t/3-13-0b2-now-available/55056) and the macOS installer provided for it on python.org now (finally!) includes an optional and experimental python3.13t free-threaded build along with the traditional build. We are using python/cpython#120098 to track macOS installer-specific issues.

[henryiii]

Thanks, great work!

Ohh, 10.13 minimum, now too! ❤️

[henryiii]

@ned-deily The builds are not visible at https://www.python.org/api/v2/downloads/release/?is_published=true yet (https://www.python.org/api/v2/downloads/release/?is_published=true&name=Python%203.13.0b2 is empty). It's also not showing up at https://www.python.org/downloads/macos/ for example. It does show up here: https://www.python.org/download/pre-releases/. We use the first link above to update our Python versions, so it would be nice if those update, is that planned? I'd rather not manually tinker on our update scripts if it's going to fix itself soon.

[ned-deily]

The builds are not visible at https://www.python.org/api/v2/downloads/release/?is_published=true yet (https://www.python.org/api/v2/downloads/release/?is_published=true&name=Python%203.13.0b2 is empty). It's also not showing up at https://www.python.org/downloads/macos/ for example. It does show up here: https://www.python.org/download/pre-releases/. We use the first link above to update our Python versions, so it would be nice if those update, is that planned? I'd rather not manually tinker on our update scripts if it's going to fix itself soon.

@Yhg1s ?

[Yhg1s]

@Yhg1s ?

Should be visible now.

[henryiii]

Superb, thanks!