Skip to content

Add support for building Android wheels #2349

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 16 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,10 @@ repos:
- id: mypy
name: mypy 3.11 on cibuildwheel/
args: ["--python-version=3.11"]
exclude: ^cibuildwheel/resources/_cross_venv.py$ # Requires Python 3.13 or later
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this because platform.android_ver isn't defined, so it raises type errors? If so - # type: ignore[attr-defined] on the single affected line is the pattern I've seen used elsewhere (including packaging and meson-python); that way we don't lost type checking on the whole file because of a single 3.13 feature.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good idea, but when I tried to implement it there were complications. There are two mypy checks, one on 3.11 and one on 3.13. If I add the ignore[attr-defined], then it passes on 3.11 but fails on 3.13 with the error "Unused "type: ignore" comment".

I can make it pass both versions with # type: ignore[attr-defined, unused-ignore], but that's a lot of clutter, which has to be repeated on 3 affected lines. And on top of that there would also need to be a human-readable comment explaining why it's necessary and when it can be removed.

Given that this file will never be run on anything older than Python 3.13, the 3.11 type check doesn't actually add anything. So I think excluding it in the pre-commit-config is a simpler solution, and is far more likely to be cleaned up when 3.13 becomes cibuildwheel's minimum version.

additional_dependencies: &mypy-dependencies
- bracex
- build
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As with the changes to pyproject.toml - why this change?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See the discussion in pyproject.toml.

- dependency-groups>=1.2
- nox>=2025.2.9
- orjson
Expand All @@ -47,7 +49,6 @@ repos:
- validate-pyproject
- id: mypy
name: mypy 3.13
exclude: ^cibuildwheel/resources/.*py$
args: ["--python-version=3.13"]
additional_dependencies: *mypy-dependencies

Expand Down
48 changes: 24 additions & 24 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,19 +24,19 @@ What does it do?

While cibuildwheel itself requires a recent Python version to run (we support the last three releases), it can target the following versions to build wheels:

| | macOS Intel | macOS Apple Silicon | Windows 64bit | Windows 32bit | Windows Arm64 | manylinux<br/>musllinux x86_64 | manylinux<br/>musllinux i686 | manylinux<br/>musllinux aarch64 | manylinux<br/>musllinux ppc64le | manylinux<br/>musllinux s390x | manylinux<br/>musllinux armv7l | iOS | Pyodide |
|----------------|----|-----|-----|-----|-----|----|-----|----|-----|-----|---|-----|-----|
| CPython 3.8 | ✅ | ✅ | ✅ | ✅ | N/A | ✅ | ✅ | ✅ | ✅ | ✅ | ✅⁵ | N/A | N/A |
| CPython 3.9 | ✅ | ✅ | ✅ | ✅ | ✅² | ✅ | ✅ | ✅ | ✅ | ✅ | ✅⁵ | N/A | N/A |
| CPython 3.10 | ✅ | ✅ | ✅ | ✅ | ✅² | ✅ | ✅ | ✅ | ✅ | ✅ | ✅⁵ | N/A | N/A |
| CPython 3.11 | ✅ | ✅ | ✅ | ✅ | ✅² | ✅ | ✅ | ✅ | ✅ | ✅ | ✅⁵ | N/A | N/A |
| CPython 3.12 | ✅ | ✅ | ✅ | ✅ | ✅² | ✅ | ✅ | ✅ | ✅ | ✅ | ✅⁵ | N/A | ✅⁴ |
| CPython 3.13³ | ✅ | ✅ | ✅ | ✅ | ✅² | ✅ | ✅ | ✅ | ✅ | ✅ | ✅⁵ | ✅ | N/A |
| PyPy 3.8 v7.3 | ✅ | ✅ | ✅ | N/A | N/A | ✅¹ | ✅¹ | ✅¹ | N/A | N/A | N/A | N/A | N/A |
| PyPy 3.9 v7.3 | ✅ | ✅ | ✅ | N/A | N/A | ✅¹ | ✅¹ | ✅¹ | N/A | N/A | N/A | N/A | N/A |
| PyPy 3.10 v7.3 | ✅ | ✅ | ✅ | N/A | N/A | ✅¹ | ✅¹ | ✅¹ | N/A | N/A | N/A | N/A | N/A |
| PyPy 3.11 v7.3 | ✅ | ✅ | ✅ | N/A | N/A | ✅¹ | ✅¹ | ✅¹ | N/A | N/A | N/A | N/A | N/A |
| GraalPy 24.2 | ✅ | ✅ | ✅ | N/A | N/A | ✅¹ | N/A | ✅¹ | N/A | N/A | N/A | N/A | N/A |
| | macOS Intel | macOS Apple Silicon | Windows 64bit | Windows 32bit | Windows Arm64 | manylinux<br/>musllinux x86_64 | manylinux<br/>musllinux i686 | manylinux<br/>musllinux aarch64 | manylinux<br/>musllinux ppc64le | manylinux<br/>musllinux s390x | manylinux<br/>musllinux armv7l | Android | iOS | Pyodide |
|----------------|----|-----|----|-----|-----|----|-----|----|-----|-----|---|-----|-----|-----|
| CPython 3.8 | ✅ | ✅ | ✅ | ✅ | N/A | ✅ | ✅ | ✅ | ✅ | ✅ | ✅⁵ | N/A | N/A | N/A |
| CPython 3.9 | ✅ | ✅ | ✅ | ✅ | ✅² | ✅ | ✅ | ✅ | ✅ | ✅ | ✅⁵ | N/A | N/A | N/A |
| CPython 3.10 | ✅ | ✅ | ✅ | ✅ | ✅² | ✅ | ✅ | ✅ | ✅ | ✅ | ✅⁵ | N/A | N/A | N/A |
| CPython 3.11 | ✅ | ✅ | ✅ | ✅ | ✅² | ✅ | ✅ | ✅ | ✅ | ✅ | ✅⁵ | N/A | N/A | N/A |
| CPython 3.12 | ✅ | ✅ | ✅ | ✅ | ✅² | ✅ | ✅ | ✅ | ✅ | ✅ | ✅⁵ | N/A | N/A | ✅⁴ |
| CPython 3.13³ | ✅ | ✅ | ✅ | ✅ | ✅² | ✅ | ✅ | ✅ | ✅ | ✅ | ✅⁵ | ✅ | ✅ | N/A |
| PyPy 3.8 v7.3 | ✅ | ✅ | ✅ | N/A | N/A | ✅¹ | ✅¹ | ✅¹ | N/A | N/A | N/A | N/A | N/A | N/A |
| PyPy 3.9 v7.3 | ✅ | ✅ | ✅ | N/A | N/A | ✅¹ | ✅¹ | ✅¹ | N/A | N/A | N/A | N/A | N/A | N/A |
| PyPy 3.10 v7.3 | ✅ | ✅ | ✅ | N/A | N/A | ✅¹ | ✅¹ | ✅¹ | N/A | N/A | N/A | N/A | N/A | N/A |
| PyPy 3.11 v7.3 | ✅ | ✅ | ✅ | N/A | N/A | ✅¹ | ✅¹ | ✅¹ | N/A | N/A | N/A | N/A | N/A | N/A |
| GraalPy 24.2 | ✅ | ✅ | ✅ | N/A | N/A | ✅¹ | N/A | ✅¹ | N/A | N/A | N/A | N/A | N/A | N/A |

<sup>¹ PyPy & GraalPy are only supported for manylinux wheels.</sup><br>
<sup>² Windows arm64 support is experimental.</sup><br>
Expand All @@ -56,20 +56,20 @@ Usage

`cibuildwheel` runs inside a CI service. Supported platforms depend on which service you're using:

| | Linux | macOS | Windows | Linux ARM | macOS ARM | Windows ARM | iOS |
|-----------------|-------|-------|---------|-----------|-----------|-------------|-----|
| GitHub Actions | ✅ | ✅ | ✅ | ✅ | ✅ | ✅² | ✅³ |
| Azure Pipelines | ✅ | ✅ | ✅ | | ✅ | ✅² | ✅³ |
| Travis CI | ✅ | | ✅ | ✅ | | | |
| AppVeyor | ✅ | ✅ | ✅ | | ✅ | ✅² | ✅³ |
| CircleCI | ✅ | ✅ | | ✅ | ✅ | | ✅³ |
| Gitlab CI | ✅ | ✅ | ✅ | ✅¹ | ✅ | | ✅³ |
| Cirrus CI | ✅ | ✅ | ✅ | ✅ | ✅ | | ✅³ |
| | Linux | macOS | Windows | Linux ARM | macOS ARM | Windows ARM | Android | iOS |
|-----------------|-------|-------|---------|-----------|-----------|-------------|---------|-----|
| GitHub Actions | ✅ | ✅ | ✅ | ✅ | ✅ | ✅² | ✅⁴ | ✅³ |
| Azure Pipelines | ✅ | ✅ | ✅ | | ✅ | ✅² | ✅⁴ | ✅³ |
| Travis CI | ✅ | | ✅ | ✅ | | | ✅⁴ | |
| AppVeyor | ✅ | ✅ | ✅ | | ✅ | ✅² | ✅⁴ | ✅³ |
| CircleCI | ✅ | ✅ | | ✅ | ✅ | | ✅⁴ | ✅³ |
| Gitlab CI | ✅ | ✅ | ✅ | ✅¹ | ✅ | | ✅⁴ | ✅³ |
| Cirrus CI | ✅ | ✅ | ✅ | ✅ | ✅ | | ✅⁴ | ✅³ |

<sup>¹ [Requires emulation](https://cibuildwheel.pypa.io/en/stable/faq/#emulation), distributed separately. Other services may also support Linux ARM through emulation or third-party build hosts, but these are not tested in our CI.</sup><br>
<sup>² [Uses cross-compilation](https://cibuildwheel.pypa.io/en/stable/faq/#windows-arm64). It is not possible to test `arm64` on this CI platform.</sup><br>
<sup>³ Requires a macOS runner; runs tests on the simulator for the runner's architecture.</sup>

<sup>³ Requires a macOS runner; runs tests on the simulator for the runner's architecture.</sup><br>
<sup>⁴ Requires runner to be Linux x86_64, macOS ARM64 or macOS x86_64. Runs tests on the emulator for the runner's architecture.</sup><br>
<!--intro-end-->

Example setup
Expand Down
1 change: 1 addition & 0 deletions bin/generate_schema.py
Original file line number Diff line number Diff line change
Expand Up @@ -333,6 +333,7 @@ def as_object(d: dict[str, Any]) -> dict[str, Any]:
"windows": as_object(not_linux),
"macos": as_object(not_linux),
"pyodide": as_object(not_linux),
"android": as_object(not_linux),
"ios": as_object(not_linux),
}

Expand Down
57 changes: 48 additions & 9 deletions bin/update_pythons.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
from collections.abc import Mapping, MutableMapping
from pathlib import Path
from typing import Any, Final, Literal, TypedDict
from xml.etree import ElementTree as ET

import click
import requests
Expand All @@ -20,6 +21,7 @@
from rich.syntax import Syntax

from cibuildwheel.extra import dump_python_configurations
from cibuildwheel.platforms.android import android_triplet

log = logging.getLogger("cibw")

Expand Down Expand Up @@ -57,7 +59,13 @@ class ConfigApple(TypedDict):
url: str


AnyConfig = ConfigWinCP | ConfigWinPP | ConfigWinGP | ConfigApple
class ConfigAndroid(TypedDict):
identifier: str
version: str
url: str


AnyConfig = ConfigWinCP | ConfigWinPP | ConfigWinGP | ConfigApple | ConfigAndroid


# The following set of "Versions" classes allow the initial call to the APIs to
Expand Down Expand Up @@ -305,6 +313,39 @@ def update_version_macos(
return None


class AndroidVersions:
# This should be replaced with official python.org downloads once they're available.
MAVEN_URL = "https://repo.maven.apache.org/maven2/com/chaquo/python/python"

def __init__(self) -> None:
response = requests.get(f"{self.MAVEN_URL}/maven-metadata.xml")
response.raise_for_status()
root = ET.fromstring(response.text)

self.versions: list[Version] = []
for version_elem in root.findall("./versioning/versions/version"):
version_str = version_elem.text
assert isinstance(version_str, str), version_str
self.versions.append(Version(version_str))

def update_version_android(
self, identifier: str, version: Version, spec: Specifier
) -> ConfigAndroid | None:
sorted_versions = sorted(spec.filter(self.versions), reverse=True)

# Return a config using the highest version for the given specifier.
if sorted_versions:
max_version = sorted_versions[0]
triplet = android_triplet(identifier)
return ConfigAndroid(
identifier=identifier,
version=str(version),
url=f"{self.MAVEN_URL}/{max_version}/python-{max_version}-{triplet}.tar.gz",
)
else:
return None


class CPythonIOSVersions:
def __init__(self) -> None:
response = requests.get(
Expand Down Expand Up @@ -365,6 +406,7 @@ def __init__(self) -> None:
self.macos_pypy = PyPyVersions("64")
self.macos_pypy_arm64 = PyPyVersions("ARM64")

self.android = AndroidVersions()
self.ios_cpython = CPythonIOSVersions()

self.graalpy = GraalPyVersions()
Expand Down Expand Up @@ -405,6 +447,8 @@ def update_config(self, config: MutableMapping[str, str]) -> None:
config_update = self.windows_t_arm64.update_version_windows(spec)
elif "win_arm64" in identifier and identifier.startswith("cp"):
config_update = self.windows_arm64.update_version_windows(spec)
elif "android" in identifier:
config_update = self.android.update_version_android(identifier, version, spec)
elif "ios" in identifier:
config_update = self.ios_cpython.update_version_ios(identifier, version)

Expand Down Expand Up @@ -436,14 +480,9 @@ def update_pythons(force: bool, level: str) -> None:
with toml_file_path.open("rb") as f:
configs = tomllib.load(f)

for config in configs["windows"]["python_configurations"]:
all_versions.update_config(config)

for config in configs["macos"]["python_configurations"]:
all_versions.update_config(config)

for config in configs["ios"]["python_configurations"]:
all_versions.update_config(config)
for platform in ["windows", "macos", "android", "ios"]:
for config in configs[platform]["python_configurations"]:
all_versions.update_config(config)

result_toml = dump_python_configurations(configs)

Expand Down
28 changes: 7 additions & 21 deletions cibuildwheel/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
import cibuildwheel
import cibuildwheel.util
from cibuildwheel import errors
from cibuildwheel.architecture import Architecture, allowed_architectures_check
from cibuildwheel.architecture import Architecture, allowed_architectures_check, native_platform
from cibuildwheel.ci import CIProvider, detect_ci_provider, fix_ansi_codes_for_github_actions
from cibuildwheel.logger import log
from cibuildwheel.options import CommandLineArguments, Options, compute_options
Expand Down Expand Up @@ -90,14 +90,14 @@ def main_inner(global_options: GlobalOptions) -> None:

parser.add_argument(
"--platform",
choices=["auto", "linux", "macos", "windows", "pyodide", "ios"],
choices=["auto", "linux", "macos", "windows", "pyodide", "android", "ios"],
default=None,
help="""
Platform to build for. Use this option to override the auto-detected
platform. Specifying "macos" or "windows" only works on that
operating system. "linux" works on any desktop OS, as long as
Docker/Podman is installed. "pyodide" only works on linux and macOS.
"ios" only work on macOS. Default: auto.
Docker/Podman is installed. "pyodide" and "android" only work on
Linux and macOS. "ios" only works on macOS. Default: auto.
""",
)

Expand Down Expand Up @@ -238,28 +238,14 @@ def _compute_platform_only(only: str) -> PlatformName:
return "windows"
if "pyodide_" in only:
return "pyodide"
if "android_" in only:
return "android"
if "ios_" in only:
return "ios"
msg = f"Invalid --only='{only}', must be a build selector with a known platform"
raise errors.ConfigurationError(msg)


def _compute_platform_auto() -> PlatformName:
if sys.platform.startswith("linux"):
return "linux"
elif sys.platform == "darwin":
return "macos"
elif sys.platform == "win32":
return "windows"
else:
msg = (
'Unable to detect platform from "sys.platform". cibuildwheel doesn\'t '
"support building wheels for this platform. You might be able to build for a different "
"platform using the --platform argument. Check --help output for more information."
)
raise errors.ConfigurationError(msg)


def _compute_platform(args: CommandLineArguments) -> PlatformName:
platform_option_value = args.platform or os.environ.get("CIBW_PLATFORM", "") or "auto"

Expand All @@ -279,7 +265,7 @@ def _compute_platform(args: CommandLineArguments) -> PlatformName:
elif platform_option_value != "auto":
return typing.cast(PlatformName, platform_option_value)

return _compute_platform_auto()
return native_platform()


@contextlib.contextmanager
Expand Down
48 changes: 37 additions & 11 deletions cibuildwheel/architecture.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,16 +17,41 @@
"macos": "macOS",
"windows": "Windows",
"pyodide": "Pyodide",
"android": "Android",
"ios": "iOS",
}

ARCH_SYNONYMS: Final[list[dict[PlatformName, str | None]]] = [
{"linux": "x86_64", "macos": "x86_64", "windows": "AMD64"},
{"linux": "x86_64", "macos": "x86_64", "windows": "AMD64", "android": "x86_64"},
{"linux": "i686", "macos": None, "windows": "x86"},
{"linux": "aarch64", "macos": "arm64", "windows": "ARM64"},
{"linux": "aarch64", "macos": "arm64", "windows": "ARM64", "android": "arm64_v8a"},
]


def arch_synonym(arch: str, from_platform: PlatformName, to_platform: PlatformName) -> str | None:
for arch_synonym in ARCH_SYNONYMS:

Check warning on line 32 in cibuildwheel/architecture.py

View workflow job for this annotation

GitHub Actions / Linters (mypy, flake8, etc.)

W0621

Redefining name 'arch_synonym' from outer scope (line 31)
if arch == arch_synonym.get(from_platform):
return arch_synonym.get(to_platform, arch)

return arch


def native_platform() -> PlatformName:
if sys.platform.startswith("linux"):
return "linux"
elif sys.platform == "darwin":
return "macos"
elif sys.platform == "win32":
return "windows"
else:
msg = (
'Unable to detect platform from "sys.platform". cibuildwheel doesn\'t '
"support building wheels for this platform. You might be able to build for a different "
"platform using the --platform argument. Check --help output for more information."
)
raise errors.ConfigurationError(msg)


Comment on lines +39 to +54
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we move this to platform/__init__.py? or does that hit a circular import problem...

def _check_aarch32_el0() -> bool:
"""Check if running armv7l natively on aarch64 is supported"""
if not sys.platform.startswith("linux"):
Expand All @@ -42,7 +67,7 @@

@typing.final
class Architecture(StrEnum):
# mac/linux archs
# mac/linux/android archs
x86_64 = auto()

# linux archs
Expand All @@ -65,6 +90,9 @@
# WebAssembly
wasm32 = auto()

# android archs
arm64_v8a = auto()

# iOS "multiarch" architectures that include both
# the CPU architecture and the ABI.
arm64_iphoneos = auto()
Expand Down Expand Up @@ -123,15 +151,12 @@
# we might need to rename the native arch to the machine we're running
# on, as the same arch can have different names on different platforms
if host_platform != platform:
for arch_synonym in ARCH_SYNONYMS:
if native_machine == arch_synonym.get(host_platform):
synonym = arch_synonym[platform]

if synonym is None:
# can't build anything on this platform
return None
synonym = arch_synonym(native_machine, host_platform, platform)
if synonym is None:
# can't build anything on this platform
return None

native_architecture = Architecture(synonym)
native_architecture = Architecture(synonym)

return native_architecture

Expand Down Expand Up @@ -173,6 +198,7 @@
"macos": {Architecture.x86_64, Architecture.arm64, Architecture.universal2},
"windows": {Architecture.x86, Architecture.AMD64, Architecture.ARM64},
"pyodide": {Architecture.wasm32},
"android": {Architecture.x86_64, Architecture.arm64_v8a},
"ios": {
Architecture.x86_64_iphonesimulator,
Architecture.arm64_iphonesimulator,
Expand Down
16 changes: 16 additions & 0 deletions cibuildwheel/frontend.py
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,22 @@ def _split_config_settings(config_settings: str) -> list[str]:
return [f"-C{setting}" for setting in config_settings_list]


# Based on build.__main__.main.
def parse_config_settings(config_settings_str: str) -> dict[str, str | list[str]]:
config_settings: dict[str, str | list[str]] = {}
for arg in shlex.split(config_settings_str):
setting, _, value = arg.partition("=")
existing_value = config_settings.get(setting)
if existing_value is None:
config_settings[setting] = value
elif isinstance(existing_value, str):
config_settings[setting] = [existing_value]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it looks to me that value would be discarded in this branch?

Suggested change
config_settings[setting] = [existing_value]
config_settings[setting] = [existing_value, value]

else:
existing_value.append(value)

return config_settings


def get_build_frontend_extra_flags(
build_frontend: BuildFrontendConfig, verbosity_level: int, config_settings: str
) -> list[str]:
Expand Down
2 changes: 2 additions & 0 deletions cibuildwheel/logger.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,8 @@
"macosx_universal2": "macOS Universal 2 - x86_64 and arm64",
"macosx_arm64": "macOS arm64 - Apple Silicon",
"pyodide_wasm32": "Pyodide",
"android_arm64_v8a": "Android arm64_v8a",
"android_x86_64": "Android x86_64",
"ios_arm64_iphoneos": "iOS Device (ARM64)",
"ios_arm64_iphonesimulator": "iOS Simulator (ARM64)",
"ios_x86_64_iphonesimulator": "iOS Simulator (x86_64)",
Expand Down
Loading
Loading