Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify and optimize Docker image #264

Draft
wants to merge 13 commits into
base: main
Choose a base branch
from
Draft

Simplify and optimize Docker image #264

wants to merge 13 commits into from

Conversation

edoardob90
Copy link
Member

  • Create a simpler Docker image starting from official Jupyter Docker Stacks
  • Remove jupyter-repo2docker dependency
  • Add GitHub workflow to build & push the image (as before)
  • Adjust the Conda environment which is configured from docker/environment.yml

The old workflow has been renamed to .github/workflows/repo2docker.yml and Binder is still supported via the binder/ directory (unchanged).

Also: since Docker provides already the isolation, there's no need to have another virtual environment. All the tutorial dependencies are installed in the base environment, which is automatically configured by the base image.

@edoardob90 edoardob90 requested a review from baffelli November 20, 2024 14:39
@yakutovicha
Copy link
Member

@edoardob90 do you need a review on this? I could give a hand.

@edoardob90
Copy link
Member Author

@edoardob90 do you need a review on this? I could give a hand.

If you want, you can try to build the container from the Dockerfile. I tested the workflow in the main branch of my fork, and everything seems to work fine. But a double-check is always appreciated 😉

@yakutovicha yakutovicha self-requested a review November 25, 2024 15:56
Copy link
Member

@yakutovicha yakutovicha left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @edoardob90, I did a first pass and I would need some clarifications from your side.

.github/workflows/docker-build.yml Outdated Show resolved Hide resolved

# Copy the repository late in the build process
RUN mkdir -p ${REPO}
COPY --chown=${NB_UID}:${NB_GID} . ${REPO}/
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This part is a bit tricky. The user should either start the same container (so the readme needs to be updated), or we should mount the folder to the container.

In any case, the readme should be synchronized with the new approach.

Copy link
Member Author

@edoardob90 edoardob90 Nov 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the step is correct and makes sense only when we're building the image via GitHub Actions. The user can still mount the local folder with a local copy of the repository, and nothing changes if the are no mismatches. Otherwise they will have their local repository inside the container.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be good to create a docker volume, but I have to think about it carefully.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did a bit of research. It seems that those lines are not needed: I added them to make sure to have the repository material inside the container, but:

  1. If the user is mounting a local folder with the repository, that's redundant
  2. GitHub Codespaces, which can use a prebuilt Docker image to speed up the startup, also clones the repository by default, so the lines are again redundant

If you agree that this is the case, we can remove them and speed up building the image.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest we have a quick chat about this today during the meeting, and then we can decide.

docker/post-build.sh Outdated Show resolved Hide resolved
docker/setup_custom_env.py Outdated Show resolved Hide resolved
@edoardob90 edoardob90 marked this pull request as draft November 27, 2024 17:32
@edoardob90
Copy link
Member Author

edoardob90 commented Nov 30, 2024

Options:

Leave the COPY command

  • We will have two copies of the material when using Codespaces: one in /home/jovyan/python-tutorial and another in /workspaces/python-tutorial (clone by default by GitHub)
  • Docker image is larger and takes slightly longer to build

Remove COPY

  • Users must clone the repository to have the materials available inside the container. On Windows, this means downloading and installing git first (here)
  • Change the README accordingly: the bind-mount directory is not /home/jovyan/work but /home/jovyan/python-tutorial

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants