Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CLIENT-2996] CI/CD: Use Github ARM runners for ARM jobs #648

Merged
merged 7 commits into from
Jul 11, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .github/actions/run-ee-server/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,9 @@ runs:
using: "composite"
steps:
- name: Install crudini to manipulate config.conf
run: pip3 install crudini -c requirements.txt
# This will only work on the Github hosted runners.
# TODO: self hosted runners do not have pipx installed by default
run: pipx install crudini --pip-args "-c ${{ github.workspace }}/.github/workflows/requirements.txt"
working-directory: .github/workflows
shell: bash

Expand Down
23 changes: 8 additions & 15 deletions .github/workflows/build-wheels.yml
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,6 @@ jobs:
context: "Build wheels (sdist)"

manylinux:
runs-on: ubuntu-22.04
strategy:
fail-fast: false
matrix:
Expand All @@ -139,30 +138,24 @@ jobs:
"cp312"
]
platform: [
"x86_64",
"aarch64"
["x86_64", "ubuntu-22.04"],
["aarch64", "aerospike_arm_runners_2"]
]

runs-on: ${{ matrix.platform[1] }}
steps:
- name: Show job status for commit
uses: myrotvorets/[email protected]
if: ${{ github.event_name != 'push' && github.event_name != 'pull_request' }}
with:
sha: ${{ github.sha }}
context: "Build wheels (${{ matrix.python }}-manylinux_${{ matrix.platform }})"
context: "Build wheels (${{ matrix.python }}-manylinux_${{ matrix.platform[0] }})"

- uses: actions/checkout@v4
with:
submodules: recursive
ref: ${{ inputs.ref }}
fetch-depth: 0

- name: Set up QEMU for cross compiling arm64
if: ${{ matrix.platform == 'aarch64' }}
uses: docker/setup-qemu-action@v2
with:
platforms: arm64

- uses: ./.github/actions/run-ee-server-for-ext-container
if: ${{ inputs.run_tests }}
with:
Expand All @@ -187,29 +180,29 @@ jobs:
uses: pypa/[email protected]
env:
CIBW_ENVIRONMENT_PASS_LINUX: ${{ inputs.apply-no-optimizations && 'UNOPTIMIZED' || '' }}
CIBW_BUILD: ${{ matrix.python }}-manylinux_${{ matrix.platform }}
CIBW_BUILD: ${{ matrix.python }}-manylinux_${{ matrix.platform[0] }}
CIBW_BUILD_FRONTEND: build
CIBW_BEFORE_ALL_LINUX: >
yum install openssl-devel -y &&
yum install python-devel -y &&
yum install python-setuptools -y
CIBW_ARCHS: "${{ matrix.platform }}"
CIBW_ARCHS: "${{ matrix.platform[0] }}"
CIBW_TEST_COMMAND: ${{ env.TEST_COMMAND }}

- name: Upload wheels to GitHub
uses: actions/upload-artifact@v4
if: ${{ always() }}
with:
path: ./wheelhouse/*.whl
name: ${{ matrix.python }}-manylinux_${{ matrix.platform }}.build
name: ${{ matrix.python }}-manylinux_${{ matrix.platform[0] }}.build

- name: Set final commit status
uses: myrotvorets/[email protected]
if: ${{ always() && github.event_name != 'push' && github.event_name != 'pull_request' }}
with:
sha: ${{ github.sha }}
status: ${{ job.status }}
context: "Build wheels (${{ matrix.python }}-manylinux_${{ matrix.platform }})"
context: "Build wheels (${{ matrix.python }}-manylinux_${{ matrix.platform[0] }})"

macOS-x86:
strategy:
Expand Down
8 changes: 1 addition & 7 deletions .github/workflows/stage-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ jobs:
fail-fast: false
env:
LINUX_DISTRO_CONTAINER_NAME: linux-distro
runs-on: ubuntu-22.04
runs-on: ${{ matrix.test-case[2] == 'linux/amd64' && 'ubuntu-22.04' || 'aerospike_arm_runners_2' }}
steps:
# TODO: the checkout code is also duplicated in the macOS stage tests
# But it's only a few lines of code so I didn't bother to create a composite action for it.
Expand Down Expand Up @@ -96,12 +96,6 @@ jobs:
JFROG_ACCESS_TOKEN: ${{ secrets.JFROG_ACCESS_TOKEN }}
JFROG_REPO_NAME: ${{ vars.JFROG_REPO_NAME }}

- name: Set up QEMU for cross compiling arm64
if: ${{ env.DISTRO_DOCKER_IMAGE_PLATFORM == 'linux/arm64' }}
uses: docker/setup-qemu-action@v2
with:
platforms: arm64

- uses: ./.github/actions/run-ee-server
with:
use-server-rc: ${{ inputs.use-server-rc }}
Expand Down
17 changes: 5 additions & 12 deletions .github/workflows/test-server-rc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@ on:

jobs:
manylinux:
runs-on: ubuntu-22.04
strategy:
fail-fast: false
matrix:
Expand All @@ -26,20 +25,14 @@ jobs:
["3.12", "cp312"]
]
platform: [
"x86_64",
"aarch64"
["x86_64", "ubuntu-22.04"],
["aarch64", "aerospike_arm_runners_2"]
]

runs-on: ${{ matrix.platform[1] }}
steps:
- uses: actions/checkout@v4

- name: Set up QEMU for cross compiling arm64
if: ${{ matrix.platform == 'aarch64' }}
uses: docker/setup-qemu-action@v2
with:
platforms: arm64

- run: docker run -d --name manylinux quay.io/pypa/manylinux2014_${{ matrix.platform }} tail -f /dev/null
- run: docker run -d --name manylinux quay.io/pypa/manylinux2014_${{ matrix.platform[0] }} tail -f /dev/null

- uses: ./.github/actions/run-ee-server-for-ext-container
with:
Expand All @@ -50,7 +43,7 @@ jobs:

- uses: actions/download-artifact@v4
with:
name: ${{ matrix.python[1] }}-manylinux_${{ matrix.platform }}.build
name: ${{ matrix.python[1] }}-manylinux_${{ matrix.platform[0] }}.build
path: ./

- name: Copy repo and wheel into manylinux container
Expand Down
Loading