Registering algorithms based on custom docker images

[emileten]

From the docs and slack conversations, it seems that commonly, ADE workspace images are used as base images for DPS algorithms. It makes sense to me since the flow is usually : explore and test code in the ADE -> clean it up and register it in an algorithm.

However, there may be use cases where users want to use simpler/lighter images of their own ? Has this happened in the past, and in which case, what's the usual practice ? Does it work if I pass to the DPS algorithm registration form the link to a public dockerhub image for example ?

[wildintellect]

Standard practice currently is to use a build_command.sh which is executed when the algorithm is registered. It's not fully explained yet in https://docs.maap-project.org/en/latest/getting_started/running_at_scale.html#Register-an-Algorithm

But the idea is the build_command handles customization of package or conda environments at registration time so that it's baked into the image, rather than run every time. Example from GEDI Subsetter https://github.com/MAAP-Project/gedi-subsetter/blob/main/build.sh

I believe arbitrary images is not supported because some libraries and environment variables must be controlled for the jobs to succeed.

[sujen1412]

We have in the past used custom images (public links from docker hub). As long as we have root in the image and are able to create a new user in the image it should work. Give it a go if you have something already and we can debug if any issues arise.

The usual practice is what you have described earlier that a user brings/develops their algorithm on the platform and uses provided base images as DPS images. If there are custom packages required that need root to install, they can be requested to be installed as @wildintellect described above.