Skip to content

Commit

Permalink
feat: add musllinux armv7l (#2017)
Browse files Browse the repository at this point in the history 
* feature: add musllinux armv7l

* fix: local armv7l images

* fix python 3.8 tests

* address review comments

---------

Co-authored-by: Henry Schreiner <[email protected]>
  • Loading branch information
@mayeut @henryiii
mayeut and henryiii authored Oct 1, 2024
1 parent dfd01af commit 9fad66f
Show file tree
Hide file tree
Showing 16 changed files with 126 additions and 63 deletions.
28 changes: 14 additions & 14 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,20 +22,20 @@ Python wheels are great. Building them across **Mac, Linux, Windows**, on **mult
What does it do?
----------------

| | macOS Intel | macOS Apple Silicon | Windows 64bit | Windows 32bit | Windows Arm64 | manylinux<br/>musllinux x86_64 | manylinux<br/>musllinux i686 | manylinux<br/>musllinux aarch64 | manylinux<br/>musllinux ppc64le | manylinux<br/>musllinux s390x | Pyodide |
|----------------|----|-----|-----|-----|-----|----|-----|----|-----|-----|-----|
| CPython 3.6 | ✅ | N/A | ✅ | ✅ | N/A | ✅ | ✅ | ✅ | ✅ | ✅ | N/A |
| CPython 3.7 | ✅ | N/A | ✅ | ✅ | N/A | ✅ | ✅ | ✅ | ✅ | ✅ | N/A |
| CPython 3.8 | ✅ | ✅ | ✅ | ✅ | N/A | ✅ | ✅ | ✅ | ✅ | ✅ | N/A |
| CPython 3.9 | ✅ | ✅ | ✅ | ✅ | ✅² | ✅ | ✅ | ✅ | ✅ | ✅ | N/A |
| CPython 3.10 | ✅ | ✅ | ✅ | ✅ | ✅² | ✅ | ✅ | ✅ | ✅ | ✅ | N/A |
| CPython 3.11 | ✅ | ✅ | ✅ | ✅ | ✅² | ✅ | ✅ | ✅ | ✅ | ✅ | N/A |
| CPython 3.12 | ✅ | ✅ | ✅ | ✅ | ✅² | ✅ | ✅ | ✅ | ✅ | ✅ | ✅⁴ |
| CPython 3.13³ | ✅ | ✅ | ✅ | ✅ | ✅² | ✅ | ✅ | ✅ | ✅ | ✅ | N/A |
| PyPy 3.7 v7.3 | ✅ | N/A | ✅ | N/A | N/A | ✅¹ | ✅¹ | ✅¹ | N/A | N/A | N/A |
| PyPy 3.8 v7.3 | ✅ | ✅ | ✅ | N/A | N/A | ✅¹ | ✅¹ | ✅¹ | N/A | N/A | N/A |
| PyPy 3.9 v7.3 | ✅ | ✅ | ✅ | N/A | N/A | ✅¹ | ✅¹ | ✅¹ | N/A | N/A | N/A |
| PyPy 3.10 v7.3 | ✅ | ✅ | ✅ | N/A | N/A | ✅¹ | ✅¹ | ✅¹ | N/A | N/A | N/A |
| | macOS Intel | macOS Apple Silicon | Windows 64bit | Windows 32bit | Windows Arm64 | manylinux<br/>musllinux x86_64 | manylinux<br/>musllinux i686 | manylinux<br/>musllinux aarch64 | manylinux<br/>musllinux ppc64le | manylinux<br/>musllinux s390x | musllinux armv7l | Pyodide |
|----------------|----|-----|-----|-----|-----|----|-----|----|-----|-----|---|-----|
| CPython 3.6 | ✅ | N/A | ✅ | ✅ | N/A | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | N/A |
| CPython 3.7 | ✅ | N/A | ✅ | ✅ | N/A | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | N/A |
| CPython 3.8 | ✅ | ✅ | ✅ | ✅ | N/A | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | N/A |
| CPython 3.9 | ✅ | ✅ | ✅ | ✅ | ✅² | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | N/A |
| CPython 3.10 | ✅ | ✅ | ✅ | ✅ | ✅² | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | N/A |
| CPython 3.11 | ✅ | ✅ | ✅ | ✅ | ✅² | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | N/A |
| CPython 3.12 | ✅ | ✅ | ✅ | ✅ | ✅² | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅⁴ |
| CPython 3.13³ | ✅ | ✅ | ✅ | ✅ | ✅² | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | N/A |
| PyPy 3.7 v7.3 | ✅ | N/A | ✅ | N/A | N/A | ✅¹ | ✅¹ | ✅¹ | N/A | N/A | N/A | N/A |
| PyPy 3.8 v7.3 | ✅ | ✅ | ✅ | N/A | N/A | ✅¹ | ✅¹ | ✅¹ | N/A | N/A | N/A | N/A |
| PyPy 3.9 v7.3 | ✅ | ✅ | ✅ | N/A | N/A | ✅¹ | ✅¹ | ✅¹ | N/A | N/A | N/A | N/A |
| PyPy 3.10 v7.3 | ✅ | ✅ | ✅ | N/A | N/A | ✅¹ | ✅¹ | ✅¹ | N/A | N/A | N/A | N/A |

<sup>¹ PyPy is only supported for manylinux wheels.</sup><br>
<sup>² Windows arm64 support is experimental.</sup><br>
Expand Down
3 changes: 3 additions & 0 deletions bin/generate_schema.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,9 @@
musllinux-aarch64-image:
type: string
description: Specify alternative manylinux / musllinux container images
musllinux-armv7l-image:
type: string
description: Specify alternative manylinux / musllinux container images
musllinux-i686-image:
type: string
description: Specify alternative manylinux / musllinux container images
Expand Down
1 change: 1 addition & 0 deletions bin/update_docker.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ class Image:
Image("musllinux_1_2", "aarch64", "quay.io/pypa/musllinux_1_2_aarch64", None),
Image("musllinux_1_2", "ppc64le", "quay.io/pypa/musllinux_1_2_ppc64le", None),
Image("musllinux_1_2", "s390x", "quay.io/pypa/musllinux_1_2_s390x", None),
Image("musllinux_1_2", "armv7l", "quay.io/pypa/musllinux_1_2_armv7l", None),
]

config = configparser.ConfigParser()
Expand Down
10 changes: 5 additions & 5 deletions cibuildwheel/architecture.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ class Architecture(Enum):
aarch64 = "aarch64"
ppc64le = "ppc64le"
s390x = "s390x"
armv7l = "armv7l"

# mac archs
universal2 = "universal2"
Expand Down Expand Up @@ -132,6 +133,7 @@ def all_archs(platform: PlatformName) -> set[Architecture]:
Architecture.aarch64,
Architecture.ppc64le,
Architecture.s390x,
Architecture.armv7l,
},
"macos": {Architecture.x86_64, Architecture.arm64, Architecture.universal2},
"windows": {Architecture.x86, Architecture.AMD64, Architecture.ARM64},
Expand All @@ -140,17 +142,15 @@ def all_archs(platform: PlatformName) -> set[Architecture]:
return all_archs_map[platform]

@staticmethod
# pylint: disable-next=inconsistent-return-statements
def bitness_archs(platform: PlatformName, bitness: Literal["64", "32"]) -> set[Architecture]:
archs_32 = {Architecture.i686, Architecture.x86}
archs_32 = {Architecture.i686, Architecture.x86, Architecture.armv7l}
auto_archs = Architecture.auto_archs(platform)

if bitness == "64":
return auto_archs - archs_32
elif bitness == "32":
if bitness == "32":
return auto_archs & archs_32
else:
assert_never(bitness)
assert_never(bitness)


def allowed_architectures_check(
Expand Down
1 change: 1 addition & 0 deletions cibuildwheel/linux.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@
Architecture.aarch64: OCIPlatform.ARM64,
Architecture.ppc64le: OCIPlatform.PPC64LE,
Architecture.s390x: OCIPlatform.S390X,
Architecture.armv7l: OCIPlatform.ARMV7,
}


Expand Down
3 changes: 2 additions & 1 deletion cibuildwheel/logger.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,8 @@
"musllinux_i686": "musllinux i686",
"musllinux_aarch64": "musllinux aarch64",
"musllinux_ppc64le": "musllinux ppc64le",
"musllinux_s390x": "manylinux s390x",
"musllinux_s390x": "musllinux s390x",
"musllinux_armv7l": "musllinux armv7l",
"win32": "Windows 32bit",
"win_amd64": "Windows 64bit",
"win_arm64": "Windows on ARM 64bit",
Expand Down
24 changes: 15 additions & 9 deletions cibuildwheel/oci_container.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@
class OCIPlatform(Enum):
i386 = "linux/386"
AMD64 = "linux/amd64"
ARMV7 = "linux/arm/v7"
ARM64 = "linux/arm64"
PPC64LE = "linux/ppc64le"
S390X = "linux/s390x"
Expand Down Expand Up @@ -208,7 +209,11 @@ def _get_platform_args(self, *, oci_platform: OCIPlatform | None = None) -> tupl
"inspect",
self.image,
"--format",
"{{.Os}}/{{.Architecture}}",
(
"{{.Os}}/{{.Architecture}}/{{.Variant}}"
if len(oci_platform.value.split("/")) == 3
else "{{.Os}}/{{.Architecture}}"
),
capture_stdout=True,
).strip()
if image_platform == oci_platform.value:
Expand All @@ -235,7 +240,7 @@ def __enter__(self) -> Self:
platform_args = self._get_platform_args()

simulate_32_bit = False
if self.oci_platform == OCIPlatform.i386:
if self.oci_platform in {OCIPlatform.i386, OCIPlatform.ARMV7}:
# If the architecture running the image is already the right one
# or the image entrypoint takes care of enforcing this, then we don't need to
# simulate this
Expand All @@ -246,13 +251,14 @@ def __enter__(self) -> Self:
*run_cmd, *platform_args, self.image, *ctr_cmd, capture_stdout=True
).strip()
except subprocess.CalledProcessError:
# The image might have been built with amd64 architecture
# Let's try that
platform_args = self._get_platform_args(oci_platform=OCIPlatform.AMD64)
container_machine = call(
*run_cmd, *platform_args, self.image, *ctr_cmd, capture_stdout=True
).strip()
simulate_32_bit = container_machine != "i686"
if self.oci_platform == OCIPlatform.i386:
# The image might have been built with amd64 architecture
# Let's try that
platform_args = self._get_platform_args(oci_platform=OCIPlatform.AMD64)
container_machine = call(
*run_cmd, *platform_args, self.image, *ctr_cmd, capture_stdout=True
).strip()
simulate_32_bit = container_machine not in {"i686", "armv7l", "armv8l"}

shell_args = ["linux32", "/bin/bash"] if simulate_32_bit else ["/bin/bash"]

Expand Down
9 changes: 9 additions & 0 deletions cibuildwheel/resources/build-platforms.toml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,15 @@ python_configurations = [
{ identifier = "cp312-musllinux_s390x", version = "3.12", path_str = "/opt/python/cp312-cp312" },
{ identifier = "cp313-musllinux_s390x", version = "3.13", path_str = "/opt/python/cp313-cp313" },
{ identifier = "cp313t-musllinux_s390x", version = "3.13", path_str = "/opt/python/cp313-cp313t" },
{ identifier = "cp36-musllinux_armv7l", version = "3.6", path_str = "/opt/python/cp36-cp36m" },
{ identifier = "cp37-musllinux_armv7l", version = "3.7", path_str = "/opt/python/cp37-cp37m" },
{ identifier = "cp38-musllinux_armv7l", version = "3.8", path_str = "/opt/python/cp38-cp38" },
{ identifier = "cp39-musllinux_armv7l", version = "3.9", path_str = "/opt/python/cp39-cp39" },
{ identifier = "cp310-musllinux_armv7l", version = "3.10", path_str = "/opt/python/cp310-cp310" },
{ identifier = "cp311-musllinux_armv7l", version = "3.11", path_str = "/opt/python/cp311-cp311" },
{ identifier = "cp312-musllinux_armv7l", version = "3.12", path_str = "/opt/python/cp312-cp312" },
{ identifier = "cp313-musllinux_armv7l", version = "3.13", path_str = "/opt/python/cp313-cp313" },
{ identifier = "cp313t-musllinux_armv7l", version = "3.13", path_str = "/opt/python/cp313-cp313t" },
]

[macos]
Expand Down
11 changes: 11 additions & 0 deletions cibuildwheel/resources/cibuildwheel.schema.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -312,6 +312,11 @@
"description": "Specify alternative manylinux / musllinux container images",
"title": "CIBW_MUSLLINUX_AARCH64_IMAGE"
},
"musllinux-armv7l-image": {
"type": "string",
"description": "Specify alternative manylinux / musllinux container images",
"title": "CIBW_MUSLLINUX_ARMV7L_IMAGE"
},
"musllinux-i686-image": {
"type": "string",
"description": "Specify alternative manylinux / musllinux container images",
Expand Down Expand Up @@ -542,6 +547,9 @@
"musllinux-aarch64-image": {
"$ref": "#/properties/musllinux-aarch64-image"
},
"musllinux-armv7l-image": {
"$ref": "#/properties/musllinux-armv7l-image"
},
"musllinux-i686-image": {
"$ref": "#/properties/musllinux-i686-image"
},
Expand Down Expand Up @@ -630,6 +638,9 @@
"musllinux-aarch64-image": {
"$ref": "#/properties/musllinux-aarch64-image"
},
"musllinux-armv7l-image": {
"$ref": "#/properties/musllinux-armv7l-image"
},
"musllinux-i686-image": {
"$ref": "#/properties/musllinux-i686-image"
},
Expand Down
1 change: 1 addition & 0 deletions cibuildwheel/resources/defaults.toml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ musllinux-i686-image = "musllinux_1_2"
musllinux-aarch64-image = "musllinux_1_2"
musllinux-ppc64le-image = "musllinux_1_2"
musllinux-s390x-image = "musllinux_1_2"
musllinux-armv7l-image = "musllinux_1_2"


[tool.cibuildwheel.linux]
Expand Down
3 changes: 3 additions & 0 deletions cibuildwheel/resources/pinned_docker_images.cfg
View file
Original file line number Diff line number Diff line change
Expand Up @@ -52,3 +52,6 @@ manylinux2014 = quay.io/pypa/manylinux2014_aarch64:2024.09.28-3
manylinux_2_24 = quay.io/pypa/manylinux_2_24_aarch64:2022-12-26-0d38463
manylinux_2_28 = quay.io/pypa/manylinux_2_28_aarch64:2024.09.28-3

[armv7l]
musllinux_1_2 = quay.io/pypa/musllinux_1_2_armv7l:2024.09.22-4

1 change: 1 addition & 0 deletions cibuildwheel/util.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,7 @@
"aarch64",
"ppc64le",
"s390x",
"armv7l",
)

DEFAULT_CIBW_CACHE_PATH: Final[Path] = user_cache_path(appname="cibuildwheel", appauthor="pypa")
Expand Down
Loading

0 comments on commit 9fad66f

Please sign in to comment.