Upgrade kubeflow container base image

[EveningStarlight]

PR: StatCan/aaw-kubeflow-containers#750

current base image is quay.io/jupyter/datascience-notebook:2024-06-17

An attempt was made to upgrade to quay.io/jupyter/datascience-notebook:2025-01-06

This was originally done to attempt to fix #2013 by using mamba 2.0, found in the newer base image.

Looking at the jupyter docker docs, there was an upgrade of Ubuntu to 24.04
The build fails in a couple places.

• upgrade OpenM++ version, new version needs updated glibc found in newer Ubuntu, and old version has a vulnerability. See OpenM++: web service API and UI no longer working #2016 and roll back OpenM++ version aaw-kubeflow-containers#744

Changes in the base image

2024-06-17 -> 2025-02-18

Ubuntu: 22.04 24.04
Mamba: 1.5.8 2.0.5
Conda: 24.5.0 25.1.1
R-Studio: 4.3.3
Python: 3.11.9 3.12.9

[EveningStarlight]

clean-layer.sh breaks the apt-get install due to the removal of the sources list.
rm -rf /var/lib/apt/lists/* /etc/apt/sources.list.d/*
removing the second part will let the build continue.

it then breaks on the pinned version list

RUN pip3 --no-cache-dir install --quiet \
      'wheel==0.40.0' \
      'setuptools==67.6.0' \
      'pyjwt==2.6.0' \
      'oauthlib==3.2.2' \
      'mpmath==1.3.0' \
      'lxml==4.9.2' \
      'pyarrow==14.0.1' \
      'cryptography==41.0.6' \
      && fix-permissions $CONDA_DIR && \
      fix-permissions /home/$NB_USER

Removing the pins lets the build continue further. {Further examination, only lxml needs to be unpinned. Others can be stay pinned for now.}

The build then fails at

RUN pip install \
    'git+https://github.com/betatim/vscode-binder' && \
    # jupyter_contrib_nbextensions likes to be installed with pip
    mamba install --quiet --yes -c conda-forge \
    'jupyter_contrib_nbextensions' \ 
    'dash' \
    'plotly' \
    'ipywidgets' \
    'markupsafe' \
    'ipympl' \
    'pexpect==4.9.0' \
    'jupyter-server-proxy==4.2.0' \
    'jupyterlab-language-pack-fr-fr' \
    'jupyterlab_execute_time' \
    'nb_conda_kernels' \
    'jupyterlab-lsp' \
    'jupyter-lsp'  && \
    jupyter server extension enable --py jupyter_server_proxy && \
    jupyter nbextension enable codefolding/main --sys-prefix && \
    jupyter labextension enable \
      '@jupyterlab/translation-extension' \
      '@jupyterlab/server-proxy' \
      'nbdime-jupyterlab' \
    && \
    jupyter lab build && \
    jupyter lab clean && \
  clean-layer.sh && \
  rm -rf /home/$NB_USER/.cache/yarn && \
  rm -rf /home/$NB_USER/.node-gyp && \
  fix-permissions $CONDA_DIR && \
  fix-permissions /home/$NB_USER

The pins were removed and it was retested, but the failure remained.

[EveningStarlight]

Add nodejs to build as it is required by jupyterlab, and is no longer in the base image.

error:

47.42 /opt/conda/lib/python3.12/site-packages/jupyterlab/debuglog.py:55: UserWarning: ValueError: Please install Node.js and npm before continuing installation. You may be able to install Node.js from your package manager, from conda, or directly from the Node.js website (https://nodejs.org).

[EveningStarlight]

fix for tensorflow image:

Upgrade CUDA code with the parts from https://gitlab.com/nvidia/container-images/cuda/-/tree/master/dist/12.3.2/ubuntu2204

[EveningStarlight]

fix for rstudio:

change jammy to noble in
add-apt-repository "deb https://cloud.r-project.org/bin/linux/ubuntu $(lsb_release -cs)-cran40/" && \
Using a version agnostic command that will support future changes.

[EveningStarlight]

Fix testing error for jupyterhub-singleuser,
The package is actually jupyterhub.singleuser in python,
so just set the mapping.

[EveningStarlight]

Two packages still being flagged are formatted irregularly

"notebook[version='>=7.2.2']",
"conda-forge::blas=[build=openblas]"

The first can be fixed by splitting on [ and taking the first section.

The second is already in the exclude, but it has an extra equals sign since last time

"conda-forge::blas[build=openblas]"

Note that the names are being parsed incorrectly by the interoperator, and are harder to manage as a result.

FAILED tests/general/test_packages.py::test_python_packages - AssertionError: ("Caught the following import error.  If you're adding new conda installs to this build that cannot be imported by python or R (eg: jupyterlab extensions, etc) see README.md instructions and add to test_packages.py's exclusion list", {'conda-forge::blas=': AssertionError("Package [conda-forge::blas=] import failed\nassert 1 == 0\n +  where 1 = <function check_import_python_package at 0x7fb465ec64d0>(<helpers.CondaPackageHelper object at 0x7fb466030100>, 'conda-forge::blas=')")})