differences for PR #254

docker-image-examples.md

@@ -36,23 +36,7 @@ Ask @mkuzak to make a PR to add extra for <https://github.com/escience-academy/d
 
 ## Using Containers on an HPC Cluster
 
-It is possible to run containers on shared computing systems run by a university or national
-computing center. As a researcher, you can build container images and test containers on your own
-computer and then run your full-scale computing work on a shared computing
-system like a high performance cluster or high throughput grid.
-
-The catch? Most university and national computing centers do not support *running*
-containers with Docker commands, and instead use a similar tool called Singularity or
-Shifter. However, both of these programs can be used to run containers based on Docker container images,
-so often people create their container image as a Docker container image, so they can
-run it using either of Docker or Singularity.
-
-There isn't yet a working example of how to use Docker container images on a shared
-computing system, partially because each system is slightly different, but the
-following resources show what it can look like:
-
-- [Introduction to Singularity](https://carpentries-incubator.github.io/singularity-introduction/): See the episode titled "Running MPI parallel jobs using Singularity containers"
-- [Container Workflows at Pawsey](https://pawseysc.github.io/container-workflows/): See the episode titled "Run containers on HPC with Shifter (and Singularity)"
+[In this example](../instructors/singularity.md), you will learn about using running containers on HPC clusters.
 
 ## Seeking Examples
 

md5sum.txt

@@ -12,13 +12,14 @@
 "episodes/docker-hub.md" "430220bbc73531857a09eddfc6247b4c" "site/built/docker-hub.md" "2024-06-27"
 "episodes/creating-container-images.md" "1c4f5343cd4e6e32f49c7105b879cd46" "site/built/creating-container-images.md" "2024-08-16"
 "episodes/advanced-containers.md" "a7bce20bf3222a7ac60363800526990d" "site/built/advanced-containers.md" "2024-08-16"
-"episodes/docker-image-examples.md" "caddfa3f2785fee60367ae05d100920a" "site/built/docker-image-examples.md" "2024-08-16"
+"episodes/docker-image-examples.md" "6ae881404b2199ce9019baa58e17fe16" "site/built/docker-image-examples.md" "2024-09-18"
 "episodes/reproduciblity.md" "55087b4f3997a95e2a5c5d6f9fd8cb7a" "site/built/reproduciblity.md" "2024-08-16"
 "instructors/06-containers-on-the-cloud.md" "6838e441f1869570ec5313bc72e85eb4" "site/built/06-containers-on-the-cloud.md" "2024-06-27"
 "instructors/08-orchestration.md" "6f69af23a2cd48c8382e2573ec2907ad" "site/built/08-orchestration.md" "2024-06-27"
 "instructors/about.md" "1df29c85850c4e3a718d5fc3a361e846" "site/built/about.md" "2024-06-27"
 "instructors/e01-github-actions.md" "ae95c2390c400410b5708a9e5f4c29c1" "site/built/e01-github-actions.md" "2024-06-27"
 "instructors/instructor-notes.md" "6ccb557863cff40a02727a9b8729add7" "site/built/instructor-notes.md" "2024-06-27"
+"instructors/singularity.md" "f1383c7cc44a3fad38f482e7fba60e5b" "site/built/singularity.md" "2024-09-18"
 "learners/discuss.md" "2758e2e5abd231d82d25c6453d8abbc6" "site/built/discuss.md" "2024-06-27"
 "learners/reference.md" "bbb68ff9187bcebed81d18156df503cc" "site/built/reference.md" "2024-08-01"
 "learners/setup.md" "fd74bc2dd9538bf486391304cb6f6f7f" "site/built/setup.md" "2024-06-27"

singularity.md

@@ -0,0 +1,65 @@
+---
+title: Running Containers on HPC
+teaching: 30
+exercises: 0
+---
+
+::::::::::::::::::::::::::::::::::::::: objectives
+
+- Learn how to convert Docker images to SIF
+- Distinguish the `run` and `exec` subcommands
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+:::::::::::::::::::::::::::::::::::::::: questions
+
+- How is singularity different to Docker?
+- How do I use my Docker images on a shared HPC?
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+:::::::::::::::::::::::::::::::::::::::::  callout
+
+You can find more detail about using Singularity in the [singularity-introduction](https://carpentries-incubator.github.io/singularity-introduction/) Carpentries workshop.
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+Singularity is a container engine, like Docker.
+However, unlike Docker, container images are stored as single files called `.sif` (Singularity Image Format).
+For a number of reasons, Singularity suits shared High Performance Computing (HPC) environments much better than Docker, so is valuable to learn if you work in these environments.
+A related tool called singularity is a fork of Singularity that generally has the same command line interface.
+
+::: challenge
+## Singularity Command Line Interface
+
+Like we did with Docker, try to work out what commands Singularity has.
+Which one do you think is the equivalent of `docker run`?
+:::
+::: solution
+`singularity run` behaves similarly to `docker run`, but as we will see, the arguments are somewhat different.
+:::
+
+## Running Docker Containers
+
+Since Singularity containers have their own file format, if we have a Docker image we want to run, it first has to be converted.
+We can do this using `singularity pull`.
+For example, we can pull the container we previously pushed to Docker Hub:
+
+```bash
+singularity pull docker://alice/alpine-python
+```
+
+This creates a file called `alpine_python.sif` in our working directory.
+To run this container, we then use `singularity run`:
+
+```bash
+singularity run alpine_python.sif
+```
+
+## Singularity Exec
+
+If we want to modify the command run in the container, we have to use `singularity exec`.
+For example, to make Python add numbers like in our sum example, we could do:
+```bash
+apptainer exec alpine_python.sif python -c 'print(1 + 1)'
+```