Build, test and deploy from GitHub Actions #26
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,125 @@ | ||
| name: Build and push | ||
|
|
||
| on: | ||
| push: | ||
| pull_request: | ||
|
Comment on lines
+3
to
+5
There was a problem hiding this comment. we could let the workflow run every week automatically to produce updated images (e.g. if Alpine updates). There was a problem hiding this comment. Cool, that sounds like a good idea. I'd been wondering how other projects handle that. 😄 There was a problem hiding this comment. addressed with 012887f. There was a problem hiding this comment.
with daily builds. to keep the base system up to date you have to build the container every day/week with updates in the dockerfile There was a problem hiding this comment. Cool. Sounds like we just need to set up some scheduler somewhere and then mostly forget about it. 😄 |
||
| schedule: | ||
| - cron: "0 0 * * 0" | ||
|
|
||
| jobs: | ||
| base-images: | ||
| # for security reason, we only build these images in our repository and on the main branch | ||
| if: github.repository == 'pgautoupgrade/docker-pgautoupgrade' && github.ref == 'refs/heads/main' | ||
| runs-on: ubuntu-latest | ||
| strategy: | ||
| matrix: | ||
| pg_version: | ||
| - "9.5" | ||
| - "9.6" | ||
| - "10" | ||
| - "11" | ||
| - "12" | ||
| - "13" | ||
| - "14" | ||
| - "15" | ||
| - "16" | ||
|
|
||
| steps: | ||
| - name: Set up QEMU | ||
| uses: docker/setup-qemu-action@v3 | ||
|
|
||
| - name: Set up Docker Buildx | ||
| uses: docker/setup-buildx-action@v3 | ||
|
|
||
| - name: Login to Docker Hub | ||
| uses: docker/login-action@v3 | ||
| with: | ||
| username: ${{ vars.DOCKERHUB_USERNAME }} | ||
| password: ${{ secrets.DOCKERHUB_PASSWORD }} | ||
|
|
||
| - name: Build and push image | ||
| uses: docker/build-push-action@v5 | ||
| with: | ||
| push: true | ||
| platforms: linux/amd64,linux/arm64 | ||
| build-args: | | ||
| "PGTARGET=16" | ||
| target: "build-${{ matrix.pg_version }}" | ||
| tags: "pgautoupgrade/pgautoupgrade:build-${{ matrix.pg_version }}" | ||
| cache-to: type=inline | ||
| cache-from: type=registry,ref=pgautoupgrade/pgautoupgrade:build-${{ matrix.pg_version }} | ||
|
|
||
| target-images: | ||
| runs-on: ubuntu-latest | ||
| needs: base-images | ||
| # otherwise, it would skip the build entirely (because the base step does not run) | ||
| if: always() | ||
| env: | ||
| # but still use our public caches in any case | ||
| # they might be outdated, in which case a full rebuild will be triggered | ||
| TARGET_TAG: ${{ github.ref == 'refs/heads/main' && 'alpine3.19' || 'dev-alpine3.19' }} | ||
| CACHE_FROM: | | ||
| type=registry,ref=pgautoupgrade/pgautoupgrade:build-9.5 | ||
| type=registry,ref=pgautoupgrade/pgautoupgrade:build-9.6 | ||
| type=registry,ref=pgautoupgrade/pgautoupgrade:build-10 | ||
| type=registry,ref=pgautoupgrade/pgautoupgrade:build-11 | ||
| type=registry,ref=pgautoupgrade/pgautoupgrade:build-12 | ||
| type=registry,ref=pgautoupgrade/pgautoupgrade:build-13 | ||
| type=registry,ref=pgautoupgrade/pgautoupgrade:build-14 | ||
| type=registry,ref=pgautoupgrade/pgautoupgrade:build-15 | ||
| type=registry,ref=pgautoupgrade/pgautoupgrade:build-16 | ||
| type=registry,ref=pgautoupgrade/pgautoupgrade:${{ matrix.pg_target }}-alpine3.19 | ||
| # we cannot access TARGET_TAG from env | ||
| # https://docs.github.com/en/actions/learn-github-actions/contexts#context-availability | ||
|
|
||
| strategy: | ||
| matrix: | ||
| pg_target: | ||
| - "12" | ||
| - "13" | ||
| - "14" | ||
| - "15" | ||
| - "16" | ||
|
|
||
| steps: | ||
| - name: Set up QEMU | ||
| uses: docker/setup-qemu-action@v3 | ||
|
|
||
| - name: Set up Docker Buildx | ||
| uses: docker/setup-buildx-action@v3 | ||
|
|
||
| - name: Login to Docker Hub | ||
| if: github.repository == 'pgautoupgrade/docker-pgautoupgrade' | ||
| uses: docker/login-action@v3 | ||
| with: | ||
| username: ${{ vars.DOCKERHUB_USERNAME }} | ||
| password: ${{ secrets.DOCKERHUB_PASSWORD }} | ||
|
|
||
| - name: Build image | ||
| uses: docker/build-push-action@v5 | ||
| with: | ||
| load: true | ||
| tags: "pgautoupgrade/pgautoupgrade:${{ matrix.pg_target }}-alpine3.19" | ||
| build-args: | | ||
| "PGTARGET=${{ matrix.pg_target }}" | ||
| cache-to: type=inline | ||
| cache-from: "${{ env.CACHE_FROM }}" | ||
|
|
||
| - name: Checkout code | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Test image | ||
| run: | | ||
| make test | ||
| env: | ||
| PGTARGET: ${{ matrix.pg_target }} | ||
|
|
||
| - name: Push image | ||
| if: github.repository == 'pgautoupgrade/docker-pgautoupgrade' && github.ref == 'refs/heads/main' | ||
| uses: docker/build-push-action@v5 | ||
| with: | ||
| platforms: linux/amd64,linux/arm64 | ||
| tags: "pgautoupgrade/pgautoupgrade:${{ matrix.pg_target }}-${{ env.TARGET_TAG }}" | ||
| push: true | ||
| cache-to: type=inline | ||
| cache-from: "${{ env.CACHE_FROM }}" | ||
justinclift marked this conversation as resolved.
Show resolved
Hide resolved
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,13 +1,7 @@ | ||
| ARG PGTARGET=16 | ||
|
|
||
| ### Things we need in all build containers | ||
| FROM alpine:3.19 as base-build | ||
|
|
||
| # Where we'll do all our compiling and similar | ||
| ENV BUILD_ROOT /buildroot | ||
|
|
@@ -16,76 +10,118 @@ ENV BUILD_ROOT /buildroot | |
| RUN mkdir ${BUILD_ROOT} | ||
| WORKDIR ${BUILD_ROOT} | ||
|
|
||
| # Install things needed for development | ||
| # We might want to install "alpine-sdk" instead of "build-base", if build-base | ||
| # doesn't have everything we need | ||
| RUN apk update && \ | ||
| apk upgrade && \ | ||
| apk add --update build-base icu-data-full icu-dev linux-headers lz4-dev musl musl-locales musl-utils tzdata zlib-dev zstd-dev && \ | ||
| apk cache clean | ||
|
|
||
| ### PostgreSQL 9.5 | ||
| FROM base-build as build-9.5 | ||
|
|
||
| RUN wget https://ftp.postgresql.org/pub/source/v9.5.25/postgresql-9.5.25.tar.bz2 && \ | ||
| tar -xf postgresql-9.5*.tar.bz2 | ||
|
|
||
| RUN cd postgresql-9.5.* && \ | ||
| ./configure --prefix=/usr/local-pg9.5 --with-openssl=no --without-readline --with-system-tzdata=/usr/share/zoneinfo --enable-debug=no CFLAGS="-Os" && \ | ||
| make -j $(nproc) && \ | ||
| make install && \ | ||
| rm -rf /usr/local-pg9.5/include | ||
|
|
||
| ### PostgreSQL 9.6 | ||
| FROM base-build as build-9.6 | ||
|
|
||
| RUN wget https://ftp.postgresql.org/pub/source/v9.6.24/postgresql-9.6.24.tar.bz2 && \ | ||
| tar -xf postgresql-9.6*.tar.bz2 | ||
|
|
||
| RUN cd postgresql-9.6.* && \ | ||
| ./configure --prefix=/usr/local-pg9.6 --with-openssl=no --without-readline --with-system-tzdata=/usr/share/zoneinfo --enable-debug=no CFLAGS="-Os" && \ | ||
| make -j $(nproc) && \ | ||
| make install && \ | ||
| rm -rf /usr/local-pg9.6/include | ||
|
|
||
| ### PostgreSQL 10 | ||
| FROM base-build as build-10 | ||
| RUN wget https://ftp.postgresql.org/pub/source/v10.23/postgresql-10.23.tar.bz2 && \ | ||
| tar -xf postgresql-10*.tar.bz2 | ||
|
|
||
| RUN cd postgresql-10.* && \ | ||
| ./configure --prefix=/usr/local-pg10 --with-openssl=no --without-readline --with-icu --with-system-tzdata=/usr/share/zoneinfo --enable-debug=no CFLAGS="-Os" && \ | ||
| make -j $(nproc) && \ | ||
| make install && \ | ||
| rm -rf /usr/local-pg10/include | ||
|
|
||
| ### PostgreSQL 11 | ||
| FROM base-build as build-11 | ||
| RUN wget https://ftp.postgresql.org/pub/source/v11.22/postgresql-11.22.tar.bz2 && \ | ||
| tar -xf postgresql-11*.tar.bz2 | ||
|
|
||
| RUN cd postgresql-11.* && \ | ||
| ./configure --prefix=/usr/local-pg11 --with-openssl=no --without-readline --with-icu --with-system-tzdata=/usr/share/zoneinfo --enable-debug=no CFLAGS="-Os" && \ | ||
| make -j $(nproc) && \ | ||
| make install && \ | ||
| rm -rf /usr/local-pg11/include | ||
|
|
||
| ### PostgreSQL 12 | ||
| FROM base-build as build-12 | ||
| RUN wget https://ftp.postgresql.org/pub/source/v12.18/postgresql-12.18.tar.bz2 && \ | ||
| tar -xf postgresql-12*.tar.bz2 | ||
|
|
||
| RUN cd postgresql-12.* && \ | ||
| ./configure --prefix=/usr/local-pg12 --with-openssl=no --without-readline --with-icu --with-system-tzdata=/usr/share/zoneinfo --enable-debug=no CFLAGS="-Os" && \ | ||
| make -j $(nproc) && \ | ||
| make install && \ | ||
| rm -rf /usr/local-pg12/include | ||
|
|
||
| ### PostgreSQL 13 | ||
| FROM base-build as build-13 | ||
|
|
||
| RUN wget https://ftp.postgresql.org/pub/source/v13.14/postgresql-13.14.tar.bz2 && \ | ||
| tar -xf postgresql-13*.tar.bz2 | ||
|
|
||
| RUN cd postgresql-13.* && \ | ||
| ./configure --prefix=/usr/local-pg13 --with-openssl=no --without-readline --with-icu --with-system-tzdata=/usr/share/zoneinfo --enable-debug=no CFLAGS="-Os" && \ | ||
| make -j $(nproc) && \ | ||
| make install && \ | ||
| rm -rf /usr/local-pg13/include | ||
|
|
||
| ### PostgreSQL 14 | ||
| FROM base-build as build-14 | ||
|
|
||
| RUN wget https://ftp.postgresql.org/pub/source/v14.11/postgresql-14.11.tar.bz2 && \ | ||
| tar -xf postgresql-14*.tar.bz2 | ||
|
|
||
| RUN cd postgresql-14.* && \ | ||
| ./configure --prefix=/usr/local-pg14 --with-openssl=no --without-readline --with-icu --with-lz4 --with-system-tzdata=/usr/share/zoneinfo --enable-debug=no CFLAGS="-Os" && \ | ||
| make -j $(nproc) && \ | ||
| make install && \ | ||
| rm -rf /usr/local-pg14/include | ||
|
|
||
| ### PostgreSQL 15 | ||
| FROM base-build as build-15 | ||
|
|
||
| RUN wget https://ftp.postgresql.org/pub/source/v15.6/postgresql-15.6.tar.bz2 && \ | ||
| tar -xf postgresql-15*.tar.bz2 | ||
|
|
||
| RUN cd postgresql-15.* && \ | ||
| ./configure --prefix=/usr/local-pg15 --with-openssl=no --without-readline --with-icu --with-lz4 --with-system-tzdata=/usr/share/zoneinfo --enable-debug=no CFLAGS="-Os" && \ | ||
| make -j $(nproc) && \ | ||
| make install && \ | ||
| rm -rf /usr/local-pg15/include | ||
|
|
||
| ### PostgreSQL 16 | ||
| FROM base-build as build-16 | ||
|
|
||
| RUN wget https://ftp.postgresql.org/pub/source/v16.2/postgresql-16.2.tar.gz && \ | ||
| tar -xf postgresql-16*.tar.gz | ||
|
|
||
| RUN cd postgresql-16.* && \ | ||
| ./configure --prefix=/usr/local-pg16 --with-openssl=no --without-readline --with-icu --with-lz4 --with-system-tzdata=/usr/share/zoneinfo --enable-debug=no CFLAGS="-Os" && \ | ||
| make -j $(nproc) && \ | ||
| make install && \ | ||
|
There was a problem hiding this comment. Needs to be The Thinking about that more, we might not need it for the early PG versions people are migrating away from, but it'll definitely be needed for the "target" PG versions people are upgrading to. There was a problem hiding this comment. I adjusted it for all PG versions now, just to keep it consistent, in bad495a. The build will likely take a while, since it will not be able to use the build images. good test in any case 😅 |
||
| rm -rf /usr/local-pg16/include | ||
|
|
||
| # Use the PostgreSQL Alpine image as our output image base | ||
| FROM postgres:${PGTARGET}-alpine3.19 | ||
|
|
@@ -94,25 +130,27 @@ FROM postgres:${PGTARGET}-alpine3.19 | |
| ARG PGTARGET | ||
|
|
||
| # Copy across our compiled files | ||
| COPY --from=build-9.5 /usr/local-pg9.5 /usr/local-pg9.5 | ||
| COPY --from=build-9.6 /usr/local-pg9.6 /usr/local-pg9.6 | ||
| COPY --from=build-10 /usr/local-pg10 /usr/local-pg10 | ||
| COPY --from=build-11 /usr/local-pg11 /usr/local-pg11 | ||
| COPY --from=build-12 /usr/local-pg12 /usr/local-pg12 | ||
| COPY --from=build-13 /usr/local-pg13 /usr/local-pg13 | ||
| COPY --from=build-14 /usr/local-pg14 /usr/local-pg14 | ||
| COPY --from=build-15 /usr/local-pg15 /usr/local-pg15 | ||
| COPY --from=build-16 /usr/local-pg16 /usr/local-pg16 | ||
|
|
||
| # Remove any left over PG directory stubs. Doesn't help with image size, just with clarity on what's in the image. | ||
| RUN if [ "${PGTARGET}" -eq 12 ]; then rm -rf /usr/local-pg12 /usr/local-pg13 /usr/local-pg14 /usr/local-pg15 /usr/local-pg16; fi | ||
andyundso marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| RUN if [ "${PGTARGET}" -eq 13 ]; then rm -rf /usr/local-pg13 /usr/local-pg14 /usr/local-pg15 /usr/local-pg16; fi | ||
| RUN if [ "${PGTARGET}" -eq 14 ]; then rm -rf /usr/local-pg14 /usr/local-pg15 /usr/local-pg16; fi | ||
| RUN if [ "${PGTARGET}" -eq 15 ]; then rm -rf /usr/local-pg15 /usr/local-pg16; fi | ||
| RUN if [ "${PGTARGET}" -eq 16 ]; then rm -rf /usr/local-pg16; fi | ||
|
|
||
| # Install locale | ||
| RUN apk update && \ | ||
| apk add --update icu-data-full musl musl-utils musl-locales tzdata && \ | ||
| apk cache clean | ||
|
|
||
| ## FIXME: Only useful while developing this Dockerfile | ||
| ##RUN apk add man-db man-pages-posix | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we should add some type of versioning to this.
i for example deploy this in a swarm cluster and run renovate against my repo to check for updates, that way i can monitor for updates and check if there are breaking changes.
i would suggest something like {semver}-{pg_version}+{build} = 1.2.3-pg16+20230315 when using daily builds
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wonder if we'll run into storage space problems with Docker Hub if we do that?
I'm guessing there's some limit to how much they'll let (free) users store.
Oh, the application I submitted to convert the Docker Hub repo to be "OSS Sponsored" never got a response.
I guess since we don't have a million downloads already we're too small or something. That's just a total guess though, could be totally wrong.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
as far as i can see is that the only limit a free account has is the pull rate limit and the number of private repos.
and i know other projects that have lots of daily builds
so i would say that shouldnt be a problem
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Awesome. Sounds like an approach we should try out then. 😄
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am a bit split for adding tags for daily builds.
On one hand, it is not much effort to implement, simply derive from the current date and adjust the tag. But we also might up with a lot of Docker tags that nobody really uses. Sure, there is not a storage limit, but still seems kind of wasteful.
Then there is the case of us running pushing a new build, maybe there was something in Alpine updated which could break something, but we do not catch with the automated tests. I have been working with containers in software development for the last five years without running any daily tags, and I would say there were only a couple of cases were this happened, compared to the hours we saved by not having to look what version we are now running exactly.
what I would support is adding a version to the script itself, because there I see we most likely implement a breaking feature. You would need to keep the Postgres version in front, otherwise Renovate will never update because v16 of Postgres > v1.0 of the script.
Ultimately, I do not mind if we happen to implement both, it would still be more "security" than there is now in regards to versions and builds.