[sdk] kfp<=1.8.20 incompatible with urllib3>=2.0.0

[ccurro]

Environment

• KFP version:

• KFP SDK version: 1.8.20

• All dependencies version:

kfp                      1.8.20
kfp-pipeline-spec        0.1.16
kfp-server-api           1.8.5

Steps to reproduce

$ python3.10 -m venv venv
$ source venv/bin/activate
$ pip install --upgrade pip
$ pip install kfp==1.8.20 urllib3==2.0.0
$ python -c "import kfp"
Traceback (most recent call last):
  File "~/kfp-urllib/venv/lib/python3.10/site-packages/requests_toolbelt/_compat.py", line 48, in <module>
    from requests.packages.urllib3.contrib import appengine as gaecontrib
ImportError: cannot import name 'appengine' from 'requests.packages.urllib3.contrib' (~/kfp-urllib/venv/lib/python3.10/site-packages/urllib3/contrib/__init__.py)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "~/kfp-urllib/venv/lib/python3.10/site-packages/kfp/__init__.py", line 25, in <module>
    from ._client import Client
  File "~/kfp-urllib/venv/lib/python3.10/site-packages/kfp/_client.py", line 29, in <module>
    from kfp._auth import get_auth_token
  File "~/kfp-urllib/venv/lib/python3.10/site-packages/kfp/_auth.py", line 34, in <module>
    import requests_toolbelt.adapters.appengine
  File "~/kfp-urllib/venv/lib/python3.10/site-packages/requests_toolbelt/__init__.py", line 12, in <module>
    from .adapters import SSLAdapter, SourceAddressAdapter
  File "~/kfp-urllib/venv/lib/python3.10/site-packages/requests_toolbelt/adapters/__init__.py", line 12, in <module>
    from .ssl import SSLAdapter
  File "~/kfp-urllib/venv/lib/python3.10/site-packages/requests_toolbelt/adapters/ssl.py", line 16, in <module>
    from .._compat import poolmanager
  File "~/kfp-urllib/venv/lib/python3.10/site-packages/requests_toolbelt/_compat.py", line 50, in <module>
    from urllib3.contrib import appengine as gaecontrib
ImportError: cannot import name 'appengine' from 'urllib3.contrib' (~/kfp-urllib/venv/lib/python3.10/site-packages/urllib3/contrib/__init__.py)

Expected result

A successful import

Materials and Reference

Remove of that appengine from urllib3==2.0.0: https://github.com/urllib3/urllib3/releases/tag/2.0.0

---

Impacted by this bug? Give it a 👍.

[Davidnet]

This is related to: request issue I had to start upgrading to 2.0.0b15

[Davidnet]

Actually nevermind it also happens in the 2.0.x versions see issue

[connor-mccarthy]

This issue is resolved in the latest v1 and v2 releases: kfp==1.8.21 and kfp==2.0.0b16.

Preferred solution
Pip install and recompile your pipelines with kfp==2.0.0-beta.16 or kfp==1.8.21 to observe the fix. Recompilation is necessary because the KFP SDK compiler injects the KFP SDK version used for compilation into the pipeline YAML/JSON. This is the same version that will be installed and used at component runtime.

Alternative solutions
Alternatively, if you don't have the Python available for recompilation, you can search and replace

kfp==2.0.0-beta.14 (or whatever your current version is)

with

kfp==2.0.0-beta.16 in your compiled component/pipeline YAML/JSON.

If for some reason you cannot use kfp==2.0.0-beta.16 or kfp==1.8.21, you can manually pip install urllib3<2.0 in your local environment (the one used for authoring/compilation) and in your component at runtime:

@dsl.component(packages_to_install=['urllib3<2.0'])
def identity(string: str) -> str:
    return string