Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

use micromamba to create the python envs #190

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

use micromamba to create the python envs #190

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
85 changes: 10 additions & 75 deletions ci-conda.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,19 +1,18 @@
ARG CUDA_VER=notset
ARG LINUX_VER=notset
ARG PYTHON_VER=notset
ARG YQ_VER
ARG AWS_CLI_VER
ARG LOCKFILE
FROM mambaorg/micromamba:1.5.8 as conda_env_creator
ARG LOCKFILE
COPY --chown=$MAMBA_USER:$MAMBA_USER ${LOCKFILE} /tmp/env.lock
RUN cat /tmp/env.lock

FROM mikefarah/yq:${YQ_VER} as yq
RUN micromamba install --name base --yes conda-lock
RUN conda-lock install --prefix /opt/conda /tmp/env.lock

FROM amazon/aws-cli:${AWS_CLI_VER} as aws-cli

FROM rapidsai/miniforge-cuda:cuda${CUDA_VER}-base-${LINUX_VER}-py${PYTHON_VER}
FROM
COPY --from conda_env_creator /opt/conda /opt/conda

ARG TARGETPLATFORM
ARG CUDA_VER
ARG LINUX_VER
ARG PYTHON_VER

ARG DEBIAN_FRONTEND=noninteractive

Expand Down Expand Up @@ -103,77 +102,13 @@ EOF
RUN wget https://github.com/rapidsai/gha-tools/releases/latest/download/tools.tar.gz -O - \
| tar -xz -C /usr/local/bin

# Install prereq for envsubst
RUN <<EOF
rapids-mamba-retry install -y \
gettext
conda clean -aipty
EOF

# Create condarc file from env vars
ENV RAPIDS_CONDA_BLD_ROOT_DIR=/tmp/conda-bld-workspace
ENV RAPIDS_CONDA_BLD_OUTPUT_DIR=/tmp/conda-bld-output
COPY condarc.tmpl /tmp/condarc.tmpl
COPY context/condarc.tmpl /tmp/condarc.tmpl
RUN cat /tmp/condarc.tmpl | envsubst | tee /opt/conda/.condarc; \
rm -f /tmp/condarc.tmpl

# Install CI tools using mamba
RUN <<EOF
rapids-mamba-retry install -y \
anaconda-client \
boa \
dunamai \
git \
jq \
"python=${PYTHON_VERSION}.*=*_cpython" \
"rapids-dependency-file-generator==1.*"
conda clean -aipty
EOF

# Install sccache and gh cli
ARG SCCACHE_VER
ARG REAL_ARCH
ARG GH_CLI_VER=notset
ARG CPU_ARCH
RUN <<EOF
curl -o /tmp/sccache.tar.gz \
-L "https://github.com/mozilla/sccache/releases/download/v${SCCACHE_VER}/sccache-v${SCCACHE_VER}-"${REAL_ARCH}"-unknown-linux-musl.tar.gz"
tar -C /tmp -xvf /tmp/sccache.tar.gz
mv "/tmp/sccache-v${SCCACHE_VER}-"${REAL_ARCH}"-unknown-linux-musl/sccache" /usr/bin/sccache
chmod +x /usr/bin/sccache

wget https://github.com/cli/cli/releases/download/v${GH_CLI_VER}/gh_${GH_CLI_VER}_linux_${CPU_ARCH}.tar.gz
tar -xf gh_*.tar.gz
mv gh_*/bin/gh /usr/local/bin
rm -rf gh_*
EOF

# Install codecov binary
ARG CODECOV_VER
RUN <<EOF
curl https://uploader.codecov.io/verification.gpg --max-time 10 --retry 5 | gpg --no-default-keyring --keyring trustedkeys.gpg --import

case "${TARGETPLATFORM}" in
"linux/amd64") codecov_url="https://uploader.codecov.io/v${CODECOV_VER}/linux/codecov" ;;
"linux/arm64") codecov_url="https://uploader.codecov.io/v${CODECOV_VER}/aarch64/codecov" ;;
*) echo 'Unsupported platform' && exit 1 ;;
esac

curl -Os --max-time 10 --retry 5 ${codecov_url}
curl -Os --max-time 10 --retry 5 ${codecov_url}.SHA256SUM
curl -Os --max-time 10 --retry 5 ${codecov_url}.SHA256SUM.sig

gpgv codecov.SHA256SUM.sig codecov.SHA256SUM
shasum -a 256 -c codecov.SHA256SUM
chmod +x codecov
mv codecov /usr/local/bin
rm -f codecov.SHA256SUM codecov.SHA256SUM.sig
EOF

RUN /opt/conda/bin/git config --system --add safe.directory '*'

COPY --from=yq /usr/bin/yq /usr/local/bin/yq
COPY --from=aws-cli /usr/local/aws-cli/ /usr/local/aws-cli/
COPY --from=aws-cli /usr/local/bin/ /usr/local/bin/

CMD ["/bin/bash"]
Loading