Create new docker image for linux adapters. #8270
Conversation
✅ Deploy Preview for meta-velox canceled.
|
facebook-github-bot
added
the
CLA Signed
docker-compose.yml
Outdated
| # or | ||
| # docker-compose run -e NUM_THREADS=<NUMBER_OF_THREADS_TO_USE> --rm linux-adapters | ||
| # to set the number of threads used during compilation | ||
| image: ghcr.io/facebookincubator/velox-dev:amd64-centos-8-avx |
There was a problem hiding this comment.
the name needs to be updated as well (we can look at overhauling our images as well to make maintenance easier...)
| FROM quay.io/centos/centos:stream8 | ||
| ARG cpu_target | ||
| COPY scripts/setup-circleci.sh / | ||
| COPY scripts/setup-helper-functions.sh / | ||
| RUN mkdir build && ( cd build && CPU_TARGET="$cpu_target" bash /setup-circleci.sh ) && rm -rf build |
There was a problem hiding this comment.
Why build everything twice? ^^
| FROM quay.io/centos/centos:stream8 | |
| ARG cpu_target | |
| COPY scripts/setup-circleci.sh / | |
| COPY scripts/setup-helper-functions.sh / | |
| RUN mkdir build && ( cd build && CPU_TARGET="$cpu_target" bash /setup-circleci.sh ) && rm -rf build | |
| FROM ghcr.io/facebookincubator/velox-dev:circleci-avx |
We would also want to add this to the docker.yml workflow to automatically update it but probably as a second step with needs: to the matrix build so it uses the new image... not optimal but eh...
|
@kgpai It would be nice to create a GitHub issue to describe the problem being solved and attach this PR to it. |
|
@assignUser Can you have another look, I am not sure how to use the 'needs:' tag you were talking about , my very brief survey in the docker docs (https://docs.docker.com/compose/compose-file/05-services/#depends_on) . Thanks. |
|
Ah sorry for the confusion. |
|
@kgpai should we include DuckDB as well? |
|
Superseeded by 8734 |
Summary: This PR moves the following jobs to gha: - adapters - presto fuzzer run - linux build + short fuzzer I have integrated the short 60s normal and extended 1800s presto fuzzer into the existing workflow in `scheduled.yml` (which we probably want to rename to `fuzzer.yml` ^^). I have added new docker files for the adapters job so we can close #8270 kgpai . In a follow up pr I will add the signature check and bias fuzzer to the `ubuntu-debug` job. Additionally I will use the new centos docker file to create an image based on manylinux for #9014 . Pull Request resolved: #8734 Reviewed By: Yuhta Differential Revision: D55097698 Pulled By: kgpai fbshipit-source-id: 55fa23c23cde97dce6f035b8e8f229608f6df422
…r#8734) Summary: This PR moves the following jobs to gha: - adapters - presto fuzzer run - linux build + short fuzzer I have integrated the short 60s normal and extended 1800s presto fuzzer into the existing workflow in `scheduled.yml` (which we probably want to rename to `fuzzer.yml` ^^). I have added new docker files for the adapters job so we can close facebookincubator#8270 kgpai . In a follow up pr I will add the signature check and bias fuzzer to the `ubuntu-debug` job. Additionally I will use the new centos docker file to create an image based on manylinux for facebookincubator#9014 . Pull Request resolved: facebookincubator#8734 Reviewed By: Yuhta Differential Revision: D55097698 Pulled By: kgpai fbshipit-source-id: 55fa23c23cde97dce6f035b8e8f229608f6df422
This PR creates a docker image which uses our default circleci script as base but also adds minio and jdk on top. This will allow us to not download mino / install jdk for every build.