gh-140189: Add CI job to test iOS builds.

[freakboy3742]

Adds a CI task to build and test an iOS XCframework.

The shape of this is almost identical to the Android task; the only notable difference is that it explicitly selects an Xcode version. Explicitly selecting an Xcode version is recommended by GitHub for all macOS activity, but changes rolled out to GitHub Actions during August made this a necessity, as the default Xcode configurations will not reliably start an iOS simulator (see actions/runner-images/issues/12541 and actions/runner-images/issues/12751).

• Issue: iOS CI and binary releases #140189

[freakboy3742]

Hrm... this works, but I get the impression something isn't quite working right with logging, because the group collapsing ins't working the same as it is on Android.

@mhsmith Any ideas? It looks like the output of the subprocesses is being piped entirely differently...

[hugovk]

This job takes about 34 minutes, making it the new bottleneck. The other slowest are free-threaded Windows on x64 (26m) and arm64 (24m):

Is there anything we can do to speed it up a bit?

It's running tests sequentially in a single process, which takes 17 minutes. Can we parallelise? Or cache any of the build process?

[StanFromIreland]

Is it possible to build without installing, maybe that would save some time? From the logs it looks like everything is installed:

/usr/bin/install -c -m 644 ../../Lib/__future__.py Apple/iOS/Frameworks/arm64-iphoneos/lib/python3.15

Also looking at the logs, I am not familiar with the iOS build system, but it looks as if configure is ran four or five times, is that intentional?

[mhsmith]

Is it possible to build without installing, maybe that would save some time? From the logs it looks like everything is installed

The Android CI also runs make install, so that doesn't take much time in itself. But it's possible that the sheer quantity of output is slowing things down, and it certainly makes the logs more difficult to navigate. The make install output is very verbose and very unlikely to be useful, so the Android script suppresses it by default.

[freakboy3742]

Is it possible to build without installing, maybe that would save some time? From the logs it looks like everything is installed:

/usr/bin/install -c -m 644 ../../Lib/__future__.py Apple/iOS/Frameworks/arm64-iphoneos/lib/python3.15

Yes - running make install is part of the build process for iOS, because files need to be in the "installed" location in a framework in order to be used at all. It's not possible to use the "build in-situ" location, because the testbed app requires a formal framework.

However, as @mhsmith notes - this is a fairly minor component of the overall build time.

Also looking at the logs, I am not familiar with the iOS build system, but it looks as if configure is ran four or five times, is that intentional?

Four builds are very much intentional:

1. A build for macOS (for cross build purposes)
2. A build for arm64 iOS devices
3. A build for arm64 simulators
4. A build for x86_64 simulators

The first of those could be shared between iOS and Android (and be used for any other cross builds), and could also be used for actual macOS testing - although reusing in that way would require a fairly substantial rework of existing build and test workflows.

[freakboy3742]

This job takes about 34 minutes, making it the new bottleneck. The other slowest are free-threaded Windows on x64 (26m) and arm64 (24m):

Is there anything we can do to speed it up a bit?

It's running tests sequentially in a single process, which takes 17 minutes. Can we parallelise? Or cache any of the build process?

As with Android, the test suite isn't parallelizable on iOS, because we can't use subprocesses.

I'm not sure what we'd cache to speed things up - did you have something specific in mind?

The other obvious option would be "run fewer tests" - but that's not exactly satisfying.

I presume the concern is wall clock end-to-end time, rather than CPU time - if that's the case, the build could be parallelized in 2 phases - the three platform builds could be done in parallel; that would trim maybe 8 minutes from the overall end-to-end test time.

The other option would be for a CI build to only build and test single architecture (arm64-simulator). That would entirely remove the 8 minutes needed to build arm64-device and x86-64 simulator. This is essentially what the buildbots are doing today; they're able to complete CI end-to-end in about 22 minutes. However, that would mean that a second workflow would be needed for release purposes, and the release artefact wouldn't be tested until release day.

The other interesting detail that is that disk performance seems to be a factor here. On my local (4 year old) machine, I can complete 4 builds, sequentially, in 11 minutes. The same step takes 16 minutes in CI. There's a bunch of known issues with simulator performance on the macOS-15 image; those issues all seem to stem from disk I/O performance in the macOS hypervisor. I'm wondering if the same thing might at play here. I'll do some experiments to see if the macOS-14 or macOS-26 image are any better in terms of performance.

Not sure which of those options are more palatable.

[freakboy3742]

It's also worth noting that I can do a full end-to-end ci build, with the same options, on my 4 year old MacBook, in about 22 minutes 24 minutes. I'm not sure what is causing the slowdown in CI...

[mhsmith]

the group collapsing ins't working the same as it is on Android. @mhsmith Any ideas?

The script's own output is appearing after the output of its subprocesses, which probably means it's being buffered. The Android script works around this as follows:

cpython/Android/android.py

Lines 868 to 872 in e4f6445

	# Under the buildbot, stdout is not a TTY, but we must still flush after
	# every line to make sure our output appears in the correct order relative
	# to the output of our subprocesses.
	for stream in [sys.stdout, sys.stderr]:
	stream.reconfigure(line_buffering=True)

the three platform builds could be done in parallel

I guess that would mean a pipeline of 3 jobs:

• macOS build
• iOS build (matrix x3)
• iOS package and test.

Each job would pass an artifact to the next, but the number of files in each artifact would be significant, which could slow things down. Since macOS runners are no longer scarce in this project, it may be worth doing the macOS build in the same job as the iOS build, even if that means it's done the same way 3 times.

The timings will be be easier to analyze once the output grouping is fixed (press Shift-T in the log view).