Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add aarch64 base images, update to Debian 12.2 #58

Merged
merged 33 commits into from
Oct 17, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
f11460b
Add support for multiplatform to the Docker images
martin-g Apr 7, 2023
57ea1dc
Make busybox image multiarch
martin-g Apr 10, 2023
22d5364
Do not use ${{ steps.build.outputs.tags }} in 'for' loops
martin-g Apr 13, 2023
fc8d5b2
Add debugging (set -x) to `Check Tags` job
martin-g Apr 13, 2023
b95dc50
Update actions/checkout to v4
martin-g Oct 4, 2023
b1188fa
Use archive.debian.org for Debian:9
martin-g Oct 5, 2023
fe1b44f
Use ubuntu-22.04 Docker action image for all workflows
martin-g Oct 10, 2023
ede10eb
Use single line for pkg-/deb-list labels
mbargull Oct 14, 2023
8e34e37
Fix typo
mbargull Oct 14, 2023
805a158
Use Miniforge download/install for tests
mbargull Oct 14, 2023
d395185
Restructure busybox-build
mbargull Oct 16, 2023
ac21046
Use only main+securtiy updates for Debian 9
mbargull Oct 16, 2023
a75717f
Adjust package installs for Debian 12
mbargull Oct 16, 2023
730f425
Tidy up apt-get install calls
mbargull Oct 16, 2023
14f8bc3
Support usrmerge in install-pkgs
mbargull Oct 16, 2023
e06bbeb
Build directly on Debian base
mbargull Oct 16, 2023
581072a
Add pwck, grpck, ldd checks and fixes
mbargull Oct 16, 2023
6e7621b
Update to Debian 12.2, BusyBox 1.36.1
mbargull Oct 16, 2023
e403f06
Extend debian base image with libegl1 and libopengl0
mvdbeek Sep 27, 2023
faa660d
Avoid spamming logs with wget/curl
mbargull Oct 16, 2023
cbad512
Copy workflow changes from base-glibc-busybox-bash
mbargull Oct 16, 2023
7993ccc
Use Quay /tag/ API endpoint
mbargull Oct 16, 2023
2b9274f
Test built base images for all platforms
mbargull Oct 16, 2023
7a2e3d6
Fix busybox install
mbargull Oct 16, 2023
2b5d90d
Get built image IDs via manifest digests
mbargull Oct 16, 2023
8846fa2
Allow backports for Debian 9 (install EGL, OpenGL)
mbargull Oct 16, 2023
a783cac
Add pwck, grpck, ldd checks to Debian base image
mbargull Oct 16, 2023
8b473ba
Old Buildah does not support --filter=dangling
mbargull Oct 16, 2023
a57caae
Simplify base image version scheme
mbargull Oct 16, 2023
2e8ad19
Fix labels for arm64; stop building twice
mbargull Oct 16, 2023
d53f912
Parallelize ldd checks
mbargull Oct 17, 2023
c32537b
Build locales only ones (during apt-get install)
mbargull Oct 17, 2023
8bf149c
Cross-compile BusyBox
mbargull Oct 17, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
158 changes: 96 additions & 62 deletions .github/workflows/base-glibc-busybox-bash.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,74 +14,101 @@ on:
jobs:
build:
name: Build & Push
runs-on: ubuntu-18.04
runs-on: ubuntu-22.04
env:
# The base image is not intended to change often and should be used with
# version tags or checksum IDs, but not via "latest".
IMAGE_VERSION: '2.1.0'
MAJOR_VERSION: 3
MINOR_VERSION: 0
IMAGE_NAME: base-glibc-busybox-bash
BUSYBOX_VERSION: '1.32.1'
DEBIAN_VERSION: '10.9'
BUSYBOX_VERSION: '1.36.1'
DEBIAN_VERSION: '12.2'

steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4

- name: Set up QEMU
uses: docker/setup-qemu-action@v2
with:
platforms: arm64

- name: Build
id: buildah-build
id: build
run: |
set -xeu
cd 'images/${{ env.IMAGE_NAME }}'

image_name='${{ env.IMAGE_NAME }}'
tags='
${{ env.MAJOR_VERSION }}
${{ env.MAJOR_VERSION }}.${{ env.MINOR_VERSION }}
latest
'
printf %s\\n \
"image=${image_name}" \
"tags=$( echo ${tags} )" \
>> $GITHUB_OUTPUT

for tag in ${tags} ; do
buildah manifest create "${image_name}:${tag}"
done

iidfile="$( mktemp )"
buildah bud --layers \
buildah bud \
--iidfile="${iidfile}" \
--build-arg=busybox_version="${{ env.BUSYBOX_VERSION }}" \
--build-arg=debian_version="${{ env.DEBIAN_VERSION }}"
image_id="$( cat "${iidfile}" )"
--build-arg=busybox_version='${{ env.BUSYBOX_VERSION }}' \
--file=Dockerfile.busybox
busybox_image="$( cat "${iidfile}" )"
rm "${iidfile}"

container="$( buildah from "${image_id}" )"
run() { buildah run "${container}" "${@}" ; }
deb_list="$( run cat /.deb.lst )"
pkg_list="$( run cat /.pkg.lst )"
glibc="$( run sh -c 'exec "$( find /lib -name libc.so.6 -print -quit )"' | sed '1!d' )"
busybox="$( run busybox | sed '1!d' )"
bash="$( run bash --version | sed '1!d' )"
buildah rm "${container}"

container="$( buildah from "${image_id}" )"
buildah config --label=glibc="${glibc}" "${container}"
buildah config --label=busybox="${busybox}" "${container}"
buildah config --label=deb-list="${deb_list}" "${container}"
buildah config --label=pkg-list="${pkg_list}" "${container}"

glibc_version="$( printf %s "${glibc}" | sed -E 's/.*version ([0-9.]*[0-9]).*/\1/' )"
busybox_version="$( printf %s "${busybox}" | sed -E '1 s/.*v([0-9.]*[0-9]).*/\1/' )"
bash_version="$( printf %s "${bash}" | sed -E 's/.*version ([0-9.]*[0-9]).*/\1/' )"
tags="
${{ env.IMAGE_VERSION }}
${{ env.IMAGE_VERSION }}_${glibc_version}_${busybox_version}_${bash_version}
latest
"
for arch in amd64 arm64 ; do
iidfile="$( mktemp )"
buildah bud --layers \
--arch="${arch}" \
--iidfile="${iidfile}" \
--build-arg=busybox_image="${busybox_image}" \
--build-arg=debian_version='${{ env.DEBIAN_VERSION }}'
image_id="$( cat "${iidfile}" )"
rm "${iidfile}"

image_id="$( buildah commit "${container}" )"
buildah rm "${container}"
image_name='${{ env.IMAGE_NAME }}'
container="$( buildah from "${image_id}" )"
run() { buildah run "${container}" "${@}" ; }
deb_list="$( run cat /.deb.lst | tr '\n' '|' | sed 's/|$//' )"
pkg_list="$( run cat /.pkg.lst | tr '\n' '|' | sed 's/|$//' )"
glibc="$( run sh -c 'exec "$( find -xdev -name libc.so.6 -print -quit )"' | sed '1!d' )"
busybox="$( run busybox | sed '1!d' )"
bash="$( run bash --version | sed '1!d' )"
buildah rm "${container}"

for tag in ${tags} ; do
buildah tag "${image_id}" \
"${image_name}":"${tag}"
done
container="$( buildah from "${image_id}" )"
buildah config \
--label=glibc="${glibc}" \
--label=busybox="${busybox}" \
--label=bash="${bash}" \
--label=deb-list="${deb_list}" \
--label=pkg-list="${pkg_list}" \
"${container}"

echo "::set-output name=image::${image_name}"
echo "::set-output name=tags::$( echo ${tags} )"
image_id="$( buildah commit "${container}" )"
buildah rm "${container}"
for tag in ${tags} ; do
buildah manifest add \
"${image_name}:${tag}" \
"${image_id}"
done
done

- name: Test
run: |
image='${{ steps.buildah-build.outputs.image }}'
image='${{ steps.build.outputs.image }}'
ids="$(
for tag in ${{ steps.buildah-build.outputs.tags }} ; do
buildah images --quiet --no-trunc "${image}:${tag}"
for tag in ${{ steps.build.outputs.tags }} ; do
buildah manifest inspect "${image}:${tag}" \
| jq -r '.manifests[]|.digest' \
| while read id ; do
buildah images --format '{{.ID}}{{.Digest}}' \
| sed -n "s/${id}//p"
done
done
)"
ids="$( printf %s "${ids}" | sort -u )"
Expand All @@ -98,49 +125,56 @@ jobs:
run: |
# FIX upstream: Quay.io does not support immutable images currently.
# => Try to use the REST API to check for duplicate tags.
respone="$(
response="$(
curl -sL \
'https://quay.io/api/v1/repository/bioconda/${{ steps.buildah-build.outputs.image }}/image'
'https://quay.io/api/v1/repository/bioconda/${{ steps.build.outputs.image }}/tag/'
)"

existing_tags="$(
printf %s "${respone}" \
| jq -r '.images[].tags[]'
printf %s "${response}" \
| jq -r '.tags[]|.name'
)" \
|| {
printf %s\\n \
'Could not get list of image tags.' \
'Does the repository exist on Quay.io?' \
'Quay.io REST API response was:' \
"${respone}"
"${response}"
exit 1
}
for tag in ${{ steps.buildah-build.outputs.tags }} ; do
if [ \! "${tag}" = latest ] ; then
if printf %s "${existing_tags}" | grep -qxF "${tag}" ; then
printf 'Tag %s already exists!\n' "${tag}"
exit 1
fi
fi
for tag in ${{ steps.build.outputs.tags }} ; do
case "${tag}" in
latest | '${{ env.MAJOR_VERSION }}' ) ;;
* )
if printf %s "${existing_tags}" | grep -qxF "${tag}" ; then
printf 'Tag %s already exists!\n' "${tag}"
exit 1
fi
esac
done

- if: ${{ github.ref == 'refs/heads/main' }}
name: Push
uses: redhat-actions/push-to-registry@v2
with:
image: ${{ steps.buildah-build.outputs.image }}
tags: ${{ steps.buildah-build.outputs.tags }}
image: ${{ steps.build.outputs.image }}
tags: ${{ steps.build.outputs.tags }}
registry: ${{ secrets.QUAY_BIOCONDA_REPO }}
username: ${{ secrets.QUAY_BIOCONDA_USERNAME }}
password: ${{ secrets.QUAY_BIOCONDA_TOKEN }}

- if: ${{ github.ref == 'refs/heads/main' }}
name: Test Pushed
run: |
image='${{ steps.buildah-build.outputs.image }}'
image='${{ env.IMAGE_NAME }}'
ids="$(
for tag in ${{ steps.buildah-build.outputs.tags }} ; do
buildah images --quiet --no-trunc "${image}:${tag}"
for tag in ${{ steps.build.outputs.tags }} ; do
buildah manifest inspect "${image}:${tag}" \
| jq -r '.manifests[]|.digest' \
| while read id ; do
buildah images --format '{{.ID}}{{.Digest}}' \
| sed -n "s/${id}//p"
done
done
)"
ids="$( printf %s "${ids}" | sort -u )"
Expand Down
Loading