Upgrade jupyterhub-singleuser custom image deps so it can build w/ Python 3.11

[themightychris]

After upgrading JupyterHub, we need to upgrade our custom docker image to incorporate the latest jupyterhub library. It appears that this dependency comes is as part of the jupyter/datascience-notebook base image we use. In #3350 I upgraded it from :python-3.9 to the latest :python-3.11.6 since the :python-3.9 tag hasn't been updated in over 2 years, but now the image fails to build:

 => [15/23] COPY ./pyproject.toml /reqs/pyproject.toml                                                                                                                                                                                 0.2s
 => [16/23] COPY ./poetry.lock /reqs/poetry.lock                                                                                                                                                                                       0.0s
 => [17/23] RUN poetry config virtualenvs.create false                                                                                                                                                                                 0.7s
 => ERROR [18/23] RUN cd /reqs && poetry install --with=shared_utils --with=portfolio                                                                                                                                                  0.8s
------
 > [18/23] RUN cd /reqs && poetry install --with=shared_utils --with=portfolio:
0.600 The currently activated Python version 3.11.6 is not supported by the project (~3.9).
0.600 Trying to find and use a compatible version. 
0.691 
0.691 Poetry was unable to find a compatible version. If you have one, you can explicitly use it via the "env use" command.
------
Dockerfile:45
--------------------
  43 |     COPY ./poetry.lock /reqs/poetry.lock
  44 |     RUN poetry config virtualenvs.create false
  45 | >>> RUN cd /reqs && poetry install --with=shared_utils --with=portfolio
  46 |     RUN poetry config virtualenvs.create true
  47 |     
--------------------
ERROR: failed to solve: process "/bin/bash -o pipefail -c cd /reqs && poetry install --with=shared_utils --with=portfolio" did not complete successfully: exit code: 1

To reproduce locally, check out the latest main branch and:

cd images/jupyter-singleuser/Dockerfile
docker build .

It appears that some poetry work will need to be done to upgrade dependencies to be compatible with Python 3.11

[mjumbewu]

@themightychris have you tried updating the Python version in the pyproject.toml file? That might be enough to fix, but there may also be additional package version changes needed with the Python upgrade.

[themightychris]

@mjumbewu good catch, after changing that line to python = "~3.11" I get:

[mjumbewu]

@themightychris Ooh, looks like the calitp-map-utils package is pinned to Python 3.9.x. Will have to take a look at that.

[mjumbewu]

Yep, it's pinned. That will have to be updated and pushed up to PyPI.

[themightychris]

@mjumbewu I might have fixed it in #3354

[mjumbewu]

We'll run into the same issue for calitp-data-analysis.

[themightychris]

@mjumbewu yep just spotted that, already on it :-)

[themightychris]

Now we're deeper into the deps failing:

This repo is also full of dependabot PRs relating to this and the calitp-* packages, so a more thorough sweep of dependencies might be needed @mjumbewu

Latest work is in #3359

[vevetron]

Yea i'm testing out different combos - building it on my local mac using docker.

Right now i'm running into a dask 2022 not working with python 3.11. Also dask seems to be both in the calitp-data-analysis package as well as the base jupyter-single-user-image. I'm not sure why.

Tiffany says dask wasn't working when it was just in the image, "dependencies needed weren't getting installed / weren't passing tests," they needed to install it to the package

I propopse we remove it from the caltip-data-analysis package and just trying to have the requirements in the jupyter-single-user-image for now. The higher dask version. Then assuming we get this image working, can test out dask, and install the packages back in and test later.

[vevetron]

OOf except calitp-data-analysis doesn't like that:
https://github.com/cal-itp/data-infra/actions/runs/9230746834/job/25399500299?pr=3359#step:6:176

[vevetron]

Okay a version of the image is complete:
#3359