Python 3.11 [SSL: CERTIFICATE_VERIFY_FAILED] in download-dependency-if-necessary.py

[iancoralogix]

Python 3.11 has backwards incompatible CAs?

If you use Python 3.11 to run download-dependency-if-necessary.py, it will generate this error:

/tmp/test 0 $ /usr/local/bin/python3 download-dependency-if-necessary.py --executable kubergrunt --download-url https://github.com/gruntwork-io/kubergrunt/releases/download/v0.11.3/kubergrunt --append-os-arch --install-dir /tmp/kubergrunt-test
2023-07-12 14:26:25,314 - __main__ - INFO - Downloading from https://github.com/gruntwork-io/kubergrunt/releases/download/v0.11.3/kubergrunt_darwin_amd64 to /tmp/kubergrunt-test/kubergrunt
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 1348, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 1286, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 1332, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 1281, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 1041, in _send_output
    self.send(msg)
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 979, in send
    self.connect()
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 1458, in connect
    self.sock = self._context.wrap_socket(self.sock,
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/ssl.py", line 517, in wrap_socket
    return self.sslsocket_class._create(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/ssl.py", line 1075, in _create
    self.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/ssl.py", line 1346, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1002)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/private/tmp/test/download-dependency-if-necessary.py", line 110, in <module>
    main()
  File "/private/tmp/test/download-dependency-if-necessary.py", line 55, in main
    executable_path = download_executable(args.executable, args.download_url, args.install_dir, args.append_os_arch)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/private/tmp/test/download-dependency-if-necessary.py", line 83, in download_executable
    urlretrieve(download_url, executable_path)
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 241, in urlretrieve
    with contextlib.closing(urlopen(url, data)) as fp:
                            ^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 216, in urlopen
    return opener.open(url, data, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 519, in open
    response = self._open(req, data)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 536, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 496, in _call_chain
    result = func(*args)
             ^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 1391, in https_open
    return self.do_open(http.client.HTTPSConnection, req,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 1351, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1002)>
/tmp/test 1 $ /usr/local/bin/python3 --version
Python 3.11.4

This configuration is meant to mimic the call in the eks-cluster-control-plane module:

 /usr/local/bin/python3 download-dependency-if-necessary.py --executable kubergrunt --download-url https://github.com/gruntwork-io/kubergrunt/releases/download/v0.11.3/kubergrunt --append-os-arch --install-dir /tmp/kubergrunt-test

Python 3.9 still works

/tmp/test 0 $ python3 --version
Python 3.9.6
/tmp/test 0 $ python3 download-dependency-if-necessary.py --executable kubergrunt --download-url https://github.com/gruntwork-io/kubergrunt/releases/download/v0.11.3/kubergrunt --append-os-arch --install-dir /tmp/kubergrunt-test-python-3.9
2023-07-12 14:29:58,861 - __main__ - INFO - Downloading from https://github.com/gruntwork-io/kubergrunt/releases/download/v0.11.3/kubergrunt_darwin_amd64 to /tmp/kubergrunt-test-python-3.9/kubergrunt
{"path": "/tmp/kubergrunt-test-python-3.9/kubergrunt"}
/tmp/test 0 $ ls ../kubergrunt-test-python-3.9 
kubergrunt*