upgrade to jupyterlab > 4

[emmaai]

Sorry I didn't look close enough when making that change. The conda version is not what I thought, and hence retrieve the installation.

[github-actions]

Installed packages:

absl-py==2.1.0
access==1.1.9
affine==2.4.0
aiobotocore==2.15.1
aiohappyeyeballs==2.4.3
aiohttp==3.10.9
aioitertools==0.12.0
aiosignal==1.3.1
alabaster==1.0.0
alembic==1.13.3
amply==0.1.6
annotated-types==0.7.0
anyio==4.6.0
apache-sedona==1.6.1
argon2-cffi==23.1.0
argon2-cffi-bindings==21.2.0
arrow==1.3.0
asciitree==0.3.3
astropy==6.1.4
astropy-iers-data==0.2024.10.7.0.32.46
asttokens==2.4.1
astunparse==1.6.3
async_generator==1.10
async-lru==2.0.4
async-timeout==4.0.3
attrs==24.2.0
autopep8==2.3.1
Babel==2.14.0
beautifulsoup4==4.12.3
black==24.10.0
bleach==6.1.0
blinker==1.8.2
bokeh==3.5.2
boltons==24.0.0
boto3==1.35.23
botocore==1.35.23
Bottleneck==1.4.0
branca==0.7.2
Brotli==1.1.0
build==1.2.2.post1
CacheControl==0.14.0
cached-property==1.5.2
cachetools==5.5.0
Cartopy==0.24.0
cattrs==24.1.2
certifi==2024.8.30
certipy==0.2.1
cffi==1.17.1
cftime==1.6.4
charset-normalizer==3.4.0
checksumdir==1.2.0
ciso8601==2.3.1
cleo==2.1.0
click==8.1.7
click-plugins==1.1.1
cligj==0.7.2
cloudpickle==3.0.0
cmocean==4.0.3
cognitojwt==1.4.1
colorama==0.4.6
colorcet==3.1.0
colorspacious==1.1.2
colour==0.1.5
comm==0.2.2
contextily==1.6.2
contourpy==1.3.0
coverage==7.6.2
crashtest==0.4.1
cryptography==43.0.1
cycler==0.12.1
Cython==3.0.11
cytoolz==1.0.0
dask==2024.9.1
dask-expr==1.1.15
dask-gateway==2024.1.0
dask-glm==0.3.2
dask-image==2024.5.3
dask_labextension==7.0.0
dask-ml==2024.4.4
dataclasses==0.8
datacube==1.8.19
datacube-ows==1.8.42
datacube-stats==1.0.2
datadog==0.50.1
datashader==0.16.3
dea-tools==0.3.5
debugpy==1.8.6
decorator==5.1.1
deepdiff==8.0.1
defusedxml==0.7.1
deprecat==2.1.3
deprecation==2.1.0
descartes==1.1.0
dill==0.3.9
distlib==0.3.8
distributed==2024.9.1
docutils==0.21.2
dulwich==0.21.7
earthengine-api==1.1.3
ecdsa==0.19.0
ee-extra==0.0.15
eemont==0.3.6
entrypoints==0.4
eodatasets3==0.30.6
ephem==4.1.6
esda==2.6.0
exceptiongroup==1.2.2
executing==2.1.0
fasteners==0.17.3
fastjsonschema==2.20.0
ffmpeg-python==0.2.0
filelock==3.16.1
fiona==1.10.1
Flask==3.0.3
flask-babel==4.0.0
flatbuffers==24.3.25
folium==0.17.0
fonttools==4.54.1
fqdn==1.5.1
fractional-cover==1.3.10
frozenlist==1.4.1
fsspec==2024.9.0
future==1.0.0
gast==0.5.5
gcsfs==2024.9.0.post1
GDAL==3.9.2
GeoAlchemy2==0.15.2
geographiclib==2.0
geohash2==1.1
geojson==3.1.0
geopandas==0.14.4
geopy==2.4.1
geoviews==1.13.0
giddy==2.3.5
gitdb==4.0.11
GitPython==3.1.43
gmpy2==2.1.5
google-api-core==2.20.0
google-api-python-client==2.149.0
google-auth==2.35.0
google-auth-httplib2==0.2.0
google-auth-oauthlib==1.2.1
google-cloud-core==2.4.1
google-cloud-storage==2.18.2
google-crc32c==1.1.2
google-pasta==0.2.0
google-resumable-media==2.7.2
googleapis-common-protos==1.65.0
greenlet==3.1.1
grpcio==1.62.2
h11==0.14.0
h2==4.1.0
h3==3.7.7
h5py==3.11.0
hdmedians==0.14.2
hdstats==0.1.8.post1
HeapDict==1.0.1
holoviews==1.19.1
hpack==4.0.0
httpcore==1.0.6
httplib2==0.22.0
httpx==0.27.2
hyperframe==6.0.1
idna==3.10
imagecodecs==2024.6.1
imageio==2.35.1
importlib_metadata==8.5.0
importlib_resources==6.4.5
inequality==1.0.1
iniconfig==2.0.0
installer==0.7.0
ipycanvas==0.13.3
ipycytoscape==1.3.3
ipyevents==2.0.2
ipyfilechooser==0.6.0
ipykernel==6.29.5
ipyleaflet==0.19.2
ipympl==0.9.4
ipython==8.28.0
ipython_genutils==0.2.0
ipywidgets==8.1.5
iso8601==2.1.0
isoduration==20.11.0
isort==5.13.2
itsdangerous==2.2.0
jaraco.classes==3.4.0
jedi==0.19.1
jeepney==0.8.0
Jinja2==3.1.4
jmespath==1.0.1
joblib==1.4.2
json5==0.9.25
jsonpointer==3.0.0
jsonschema==4.23.0
jsonschema-specifications==2024.10.1
jupyter_bokeh==4.0.5
jupyter_client==7.4.9
jupyter_contrib_core==0.4.2
jupyter_contrib_nbextensions==0.7.0
jupyter_core==5.7.2
jupyter-events==0.10.0
jupyter-highlight-selected-word==0.2.0
jupyter-leaflet==0.19.2
jupyter-lsp==2.2.5
jupyter_nbextensions_configurator==0.6.4
jupyter-resource-usage==0.7.1
jupyter_server==2.14.2
jupyter-server-mathjax==0.2.6
jupyter_server_proxy==4.4.0
jupyter_server_terminals==0.5.3
jupyter-telemetry==0.1.0
jupyter-ui-poll==0.2.2
jupyterhub==4.1.5
jupyterlab==4.2.5
jupyterlab_code_formatter==2.2.1
jupyterlab-code-snippets==2.2.1
jupyterlab_geojson==3.4.0
jupyterlab_git==0.50.1
jupyterlab_github==4.0.0
jupyterlab_iframe==0.5.0
jupyterlab-language-pack-fr-FR==4.2.post3
jupyterlab-logout==1.0.0
jupyterlab_pygments==0.3.0
jupyterlab_server==2.27.3
jupyterlab-spellchecker==0.8.4
jupyterlab-theme-toggler==1.0.0
jupyterlab-topbar-text==1.0.0
jupyterlab_widgets==3.0.13
jupytext==1.16.4
keras==3.6.0
keyring==24.3.1
kiwisolver==1.4.7
lark==1.2.2
lazy_loader==0.4
libpysal==4.12.1
line_profiler==4.1.3
linkify-it-py==2.0.3
llvmlite==0.43.0
lmdb==1.5.1
locket==1.0.0
looseversion==1.3.0
lxml==5.3.0
lz4==4.3.3
Mako==1.3.5
mapclassify==2.8.1
Markdown==3.6
markdown-it-py==3.0.0
MarkupSafe==3.0.1
matplotlib==3.9.2
matplotlib-inline==0.1.7
mdit-py-plugins==0.4.2
mdurl==0.1.2
mercantile==1.2.1
mgwr==2.2.1
mistune==3.0.2
ml-dtypes==0.4.0
momepy==0.8.1
more-itertools==10.5.0
morecantile==5.4.2
mpi4py==4.0.0
mpmath==1.3.0
msgpack==1.1.0
multidict==6.1.0
multipledispatch==0.6.0
multiprocess==0.70.16
munch==4.0.0
munkres==1.1.4
mypy==1.11.2
mypy-extensions==1.0.0
namex==0.0.8
nbclassic==1.1.0
nbclient==0.10.0
nbconvert==7.16.4
nbdime==4.0.2
nbformat==5.10.4
nbgitpuller==1.2.1
nbval==0.11.0
nest_asyncio==1.6.0
netCDF4==1.7.1
networkx==3.3
notebook==6.5.7
notebook_shim==0.2.4
numba==0.60.0
numcodecs==0.13.0
numexpr==2.10.2.dev0
numpy==1.26.4
oauthlib==3.2.2
odc-algo==0.2.3
odc-cloud==0.2.5
odc-dscache==0.2.3
odc-geo==0.4.8
odc-io==0.2.2
odc-stac==0.3.10
odc-stats==1.0.55
odc-ui==0.2.1
opencv-contrib-python-headless==4.10.0.84
opencv-python-headless==4.10.0.84
opt_einsum==3.4.0
optree==0.13.0
ordered-set==4.1.0
orderly-set==5.2.2
osmnet==0.1.7
otps==0.3.2
overrides==7.7.0
OWSLib==0.31.0
packaging==24.1
pamela==1.2.0
pandana==0.7
pandas==2.2.3
pandocfilters==1.5.0
panel==1.5.2
param==2.1.1
parso==0.8.4
partd==1.4.2
pathos==0.3.2
pathspec==0.12.1
patsy==0.5.6
pexpect==4.9.0
pickleshare==0.7.5
pillow==10.4.0
PIMS==0.7
pip==24.2
pkginfo==1.11.1
pkgutil_resolve_name==1.3.10
planetary-computer==1.0.0
platformdirs==4.3.6
plotly==5.24.1
pluggy==1.5.0
poetry==1.8.3
poetry-core==1.9.0
poetry-plugin-export==1.8.0
pointpats==2.5.1
pox==0.3.5
ppft==1.7.6.9
prometheus_client==0.21.0
prometheus_flask_exporter==0.23.1
prompt_toolkit==3.0.48
proto-plus==1.23.0
protobuf==4.25.3
psutil==6.0.0
psycopg2==2.9.9
ptyprocess==0.7.0
PuLP==2.8.0
pure_eval==0.2.3
py-cpuinfo==9.0.0
py4j==0.10.9.7
pyarrow==17.0.0
pyarrow-hotfix==0.6
pyasn1==0.6.1
pyasn1_modules==0.4.1
pycodestyle==2.12.1
pycparser==2.22
pyct==0.5.0
pycurl==7.45.3
pydantic==2.9.2
pydantic_core==2.23.4
pydash==8.0.3
pydotplus==2.0.2
pyerfa==2.0.1.4
Pygments==2.18.0
PyJWT==2.9.0
pyOpenSSL==24.2.1
pyows==0.3.1
pyparsing==2.4.7
pyproj==3.7.0
pyproject_hooks==1.2.0
pyrsistent==0.20.0
pysal==24.7
pysheds==0.4
pyshp==2.3.1
PySide6==6.7.3
PySocks==1.7.1
pyspark==3.5.1
pystac==1.11.0
pystac-client==0.8.3
pytest==8.3.3
python-box==7.2.0
python-dateutil==2.9.0
python-dotenv==1.0.1
python-geohash==0.8.5
python-jose==3.3.0
python-json-logger==2.0.7
python-rapidjson==1.20
python-slugify==8.0.4
pyTMD==2.1.6
pytz==2024.1
pyu2f==0.1.5
pyviz_comms==3.0.3
PyWavelets==1.7.0
PyYAML==6.0.2
pyzmq==26.2.0
quantecon==0.7.2
RapidFuzz==3.10.0
rasterio==1.4.1
rasterstats==0.20.0
referencing==0.35.1
regex==2024.9.11
requests==2.32.3
requests-aws4auth==1.1.1
requests-oauthlib==2.0.0
requests-toolbelt==1.0.0
rfc3339-validator==0.1.4
rfc3986-validator==0.1.1
rich==13.9.2
rio-cogeo==5.3.6
rio-stac==0.9.0
rios==2.0.4
rioxarray==0.17.0
rpds-py==0.20.0
rsa==4.9
Rtree==1.3.0
ruamel.yaml==0.18.6
ruamel.yaml.clib==0.2.8
s2cloudmask==0.1
s3fs==2024.9.0
s3transfer==0.10.3
scikit-image==0.24.0
scikit-learn==1.5.2
scipy==1.14.1
seaborn==0.13.2
SecretStorage==3.3.3
segregation==2.5
Send2Trash==1.8.3
sentry-sdk==2.16.0
setuptools==75.1.0
setuptools-scm==8.1.0
shapely==2.0.6
shellingham==1.5.4
shiboken6==6.7.3
sidecar==0.7.0
simpervisor==1.0.0
simplejson==3.19.3
six==1.16.0
slicerator==1.1.0
smmap==5.0.0
sniffio==1.3.1
snuggs==1.4.7
sortedcontainers==2.4.0
soupsieve==2.5
spaghetti==1.7.6
sparse==0.15.4
spectate==1.0.1
spglm==1.1.0
spint==1.0.7
splot==1.1.7
spopt==0.6.1
spreg==1.7
spvcm==0.3.0
spyndex==0.6.0
SQLAlchemy==1.4.49
stack-data==0.6.2
statsmodels==0.14.4
structlog==24.4.0
sympy==1.13.3
tables==3.10.1
tblib==3.0.0
tenacity==9.0.0
tensorboard==2.17.1
tensorboard-data-server==0.7.0
tensorflow==2.17.0
tensorflow_estimator==2.15.0
termcolor==2.5.0
terminado==0.18.1
text-unidecode==1.3
tflite-runtime==2.14.0
threadpoolctl==3.5.0
thredds-crawler==1.5.4
tifffile==2024.9.20
timescale==0.0.5
timezonefinder==6.5.3
tinycss2==1.3.0
tl2cgen==1.0.0
tobler==0.11.3
toml==0.10.2
tomli==2.0.2
tomlkit==0.13.2
toolz==1.0.0
tornado==6.4.1
tornado-proxy-handlers==0.0.6
tqdm==4.66.5
traitlets==5.14.3
traittypes==0.2.1
treelite==4.3.0
trove-classifiers==2024.9.12
types-python-dateutil==2.9.0.20241003
typing_extensions==4.12.2
typing-utils==0.1.0
tzdata==2024.2
uc-micro-py==1.0.3
unicodedata2==15.1.0
urbanaccess==0.2.2
uri-template==1.3.0
uritemplate==4.1.1
urllib3==2.2.3
urlpath==1.2.0
virtualenv==20.26.6
voluptuous==0.15.2
wcwidth==0.2.13
webcolors==24.8.0
webencodings==0.5.1
websocket-client==1.8.0
Werkzeug==3.0.4
wheel==0.44.0
widgetsnbextension==4.0.13
wrapt==1.16.0
xarray==2024.9.0
xarray-spatial==0.4.0
xgboost==2.1.1
xyzservices==2024.9.0
yapf==0.40.1
yarl==1.13.1
zarr==2.18.3
zict==3.0.0
zipp==3.20.2
zstandard==0.23.0

[omad]

Sorry I didn't look close enough when making that change. The conda version is not what I thought, and hence retrieve the installation.

What is the problem with the conda version?

It's important to know/record so we don't make the same (wrong) assumption in the future.

[emmaai]

Sorry I didn't look close enough when making that change. The conda version is not what I thought, and hence retrieve the installation.

What is the problem with the conda version?

It's important to know/record so we don't make the same (wrong) assumption in the future.

The awscliv2 on conda is a handler of invoking a pre-compiled image with awscli, not the tool itself.

[emmaai]

Sorry I didn't look close enough when making that change. The conda version is not what I thought, and hence retrieve the installation.

What is the problem with the conda version?
It's important to know/record so we don't make the same (wrong) assumption in the future.

The awscliv2 on conda is a handler of invoking a pre-compiled image with awscli, not the tool itself.

noted in docker file

[stacyhorton]

As I mentioned to Robbi, I noticed some new error messages in the pod logs when launching an unstable server:

2024-08-28 12:38:52.537 2024-08-28 02:38:52,536 - ServerApp - ERROR - Uncaught exception GET /user/[email protected]/github?1724812732141 (::ffff:10.95.1.179)
2024-08-28 12:38:52.537 HTTPServerRequest(protocol='http', host='app.sandbox.dea.ga.gov.au', method='GET', uri='/user/[email protected]/github?1724812732141', version='HTTP/1.1', remote_ip='::ffff:10.95.1.179')
2024-08-28 12:38:52.537 Traceback (most recent call last):
2024-08-28 12:38:52.537 File "/env/lib/python3.10/site-packages/tornado/web.py", line 1790, in _execute
2024-08-28 12:38:52.537 result = await result
2024-08-28 12:38:52.537 File "/env/lib/python3.10/site-packages/tornado/gen.py", line 234, in wrapper
2024-08-28 12:38:52.537 yielded = ctx_run(next, result)
2024-08-28 12:38:52.537 File "/env/lib/python3.10/site-packages/jupyterlab_github/init.py", line 95, in get
2024-08-28 12:38:52.537 headers={"Authorization": "token {}".format(token)}
2024-08-28 12:38:52.537 UnboundLocalError: local variable 'token' referenced before assignment
2024-08-28 12:38:52.537 [W 2024-08-28 02:38:52.537 ServerApp] Unhandled error
2024-08-28 12:51:07.153 [W 2024-08-28 02:51:07.153 LabApp] Failed validating settings (jupyterlab-code-snippets:snippets): ['import matplotlib.pyplot as plt', 'import numpy as np', '', 'x = np.linspace(0, 10, 100)', '', 'plt.plot(x, np.sin(x))', 'plt.plot(x, np.cos(x))', '', 'plt.show()'] is not of type 'string'
2024-08-28 12:51:07.154
2024-08-28 12:51:07.154 Failed validating 'type' in schema['properties']['snippets']['items']['properties']['code']:
2024-08-28 12:51:07.154 {'title': 'Code', 'type': 'string'}
2024-08-28 12:51:07.154
2024-08-28 12:51:07.154 On instance['snippets'][0]['code']:
2024-08-28 12:51:07.154 ['import matplotlib.pyplot as plt',
2024-08-28 12:51:07.154 'import numpy as np',
2024-08-28 12:51:07.154 '',
2024-08-28 12:51:07.154 'x = np.linspace(0, 10, 100)',
2024-08-28 12:51:07.154 '',
2024-08-28 12:51:07.154 'plt.plot(x, np.sin(x))',
2024-08-28 12:51:07.154 'plt.plot(x, np.cos(x))',
2024-08-28 12:51:07.154 '',
2024-08-28 12:51:07.154 'plt.show()']
2024-08-28 12:51:09.146 [I 2024-08-28 02:51:09.146 LabApp] Build is up to date
2024-08-28 12:51:56.575 [W 2024-08-28 02:51:56.574 LabApp] wrote error: 'Schema not found: /env/share/jupyter/lab/schemas/@jupyterlab/codemirror-extension/commands.json'
2024-08-28 12:51:56.575 Traceback (most recent call last):
2024-08-28 12:51:56.575 File "/env/lib/python3.10/site-packages/tornado/web.py", line 1788, in _execute
2024-08-28 12:51:56.575 result = method(*self.path_args, **self.path_kwargs)
2024-08-28 12:51:56.575 File "/env/lib/python3.10/site-packages/tornado/web.py", line 3301, in wrapper
2024-08-28 12:51:56.575 return method(self, *args, **kwargs)
2024-08-28 12:51:56.575 File "/env/lib/python3.10/site-packages/jupyterlab_server/settings_handler.py", line 58, in get
2024-08-28 12:51:56.575 result, warnings = get_settings(
2024-08-28 12:51:56.575 File "/env/lib/python3.10/site-packages/jupyterlab_server/settings_utils.py", line 385, in get_settings
2024-08-28 12:51:56.575 schema, version = _get_schema(schemas_dir, schema_name, overrides, labextensions_path)
2024-08-28 12:51:56.575 File "/env/lib/python3.10/site-packages/jupyterlab_server/settings_utils.py", line 61, in _get_schema
2024-08-28 12:51:56.575 raise web.HTTPError(404, notfound_error % path)
2024-08-28 12:51:56.575 tornado.web.HTTPError: HTTP 404: Not Found (Schema not found: /env/share/jupyter/lab/schemas/@jupyterlab/codemirror-extension/commands.json)
2024-08-28 12:51:56.583 [W 2024-08-28 02:51:56.582 LabApp] wrote error: 'Schema not found: /env/share/jupyter/lab/schemas/@jupyterlab/toc-extension/plugin.json'
2024-08-28 12:51:56.583 Traceback (most recent call last):
2024-08-28 12:51:56.583 File "/env/lib/python3.10/site-packages/tornado/web.py", line 1788, in _execute
2024-08-28 12:51:56.583 result = method(*self.path_args, **self.path_kwargs)
2024-08-28 12:51:56.583 File "/env/lib/python3.10/site-packages/tornado/web.py", line 3301, in wrapper
2024-08-28 12:51:56.583 return method(self, *args, **kwargs)
2024-08-28 12:51:56.583 File "/env/lib/python3.10/site-packages/jupyterlab_server/settings_handler.py", line 58, in get
2024-08-28 12:51:56.583 result, warnings = get_settings(
2024-08-28 12:51:56.583 File "/env/lib/python3.10/site-packages/jupyterlab_server/settings_utils.py", line 385, in get_settings
2024-08-28 12:51:56.583 schema, version = _get_schema(schemas_dir, schema_name, overrides, labextensions_path)
2024-08-28 12:51:56.583 File "/env/lib/python3.10/site-packages/jupyterlab_server/settings_utils.py", line 61, in _get_schema
2024-08-28 12:51:56.583 raise web.HTTPError(404, notfound_error % path)
2024-08-28 12:51:56.583 tornado.web.HTTPError: HTTP 404: Not Found (Schema not found: /env/share/jupyter/lab/schemas/@jupyterlab/toc-extension/plugin.json)
2024-08-28 12:51:57.517 [I 2024-08-28 02:51:57.517 ServerApp] New terminal with automatic name: 1
2024-08-28 12:51:58.165 [W 2024-08-28 02:51:58.164 ServerApp] Notebook DEA_Sandbox.ipynb is not trusted
2024-08-28 12:51:58.173 [W 2024-08-28 02:51:58.172 LabApp] wrote error: 'Schema not found: /env/share/jupyter/lab/schemas/@jupyterlab/application-extension/sidebar.json'
2024-08-28 12:51:58.173 Traceback (most recent call last):
2024-08-28 12:51:58.173 File "/env/lib/python3.10/site-packages/tornado/web.py", line 1788, in _execute
2024-08-28 12:51:58.173 result = method(*self.path_args, **self.path_kwargs)
2024-08-28 12:51:58.173 File "/env/lib/python3.10/site-packages/tornado/web.py", line 3301, in wrapper
2024-08-28 12:51:58.173 return method(self, *args, **kwargs)
2024-08-28 12:51:58.173 File "/env/lib/python3.10/site-packages/jupyterlab_server/settings_handler.py", line 58, in get
2024-08-28 12:51:58.173 result, warnings = get_settings(
2024-08-28 12:51:58.173 File "/env/lib/python3.10/site-packages/jupyterlab_server/settings_utils.py", line 385, in get_settings
2024-08-28 12:51:58.173 schema, version = _get_schema(schemas_dir, schema_name, overrides, labextensions_path)
2024-08-28 12:51:58.173 File "/env/lib/python3.10/site-packages/jupyterlab_server/settings_utils.py", line 61, in _get_schema
2024-08-28 12:51:58.173 raise web.HTTPError(404, notfound_error % path)
2024-08-28 12:51:58.173 tornado.web.HTTPError: HTTP 404: Not Found (Schema not found: /env/share/jupyter/lab/schemas/@jupyterlab/application-extension/sidebar.json)

[robbibt]

@emmaai Any thoughts on what might be causing/how we could resolve those issues in the logs that Stacy shared above?

[emmaai]

When you don't clean your cache...it may happen. The js of jupyterlab is different between 3 and 4. It'd be gone if the cache is cleared between stable and unstable profile switching.

[robbibt]

Looks great, let's merge this.