Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

requests.exceptions.ConnectionError,how to deal with it? #375

Open
RLJC0212 opened this issue Oct 25, 2024 · 3 comments
Open

requests.exceptions.ConnectionError,how to deal with it? #375

RLJC0212 opened this issue Oct 25, 2024 · 3 comments

Comments

@RLJC0212
Copy link

requests.exceptions.ConnectionError: HTTPSConnectionPool(host='ecmwf-forecasts.s3.eu-central-1.amazonaws.com', port=443): Max retries exceeded with url: /20240202/06z/0p25/oper/20240202060000-0h-oper-fc.index (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f551fb6fa90>: Failed to establish a new connection: [Errno 101] Network is unreachable'))

@blaylockbk
Copy link
Owner

Can you provide a reproducible example that gives this error?

@RLJC0212
Copy link
Author

Thank you for getting back to me! I've prepared a reproducible example to demonstrate the issue. Here is the code and the steps to reproduce it::

model='ifs'; time=0; step=0; date=dates[1]; proxy='socks5h://10.10.10.1:10959'
date
'2024-02-02'
os.environ['HTTP_PROXY'] = proxy # 设置当前线程使用的代理
date = datetime.strptime(date, '%Y-%m-%d')
date_num = datetime.strftime((date + timedelta(hours=step + time)), '%Y%m%d%H')
outdir = "/storage_new/home/stmap/cd_forecast/data/meteorology/"
f_start = date+timedelta(hours=time)
H = Herbie(f_start.strftime('%Y-%m-%d %H'), model=model, step=step, outdir=outdir)
Traceback (most recent call last):
File "/mnt/sdc1/home/juchengli/anaconda3/envs/herbie/lib/python3.11/site-packages/urllib3/connection.py", line 199, in _new_conn
sock = connection.create_connection(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/sdc1/home/juchengli/anaconda3/envs/herbie/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection
raise err
File "/mnt/sdc1/home/juchengli/anaconda3/envs/herbie/lib/python3.11/site-packages/urllib3/util/connection.py", line 73, in create_connection
sock.connect(sa)
OSError: [Errno 101] Network is unreachable

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/mnt/sdc1/home/juchengli/anaconda3/envs/herbie/lib/python3.11/site-packages/urllib3/connectionpool.py", line 789, in urlopen
response = self._make_request(
^^^^^^^^^^^^^^^^^^^
File "/mnt/sdc1/home/juchengli/anaconda3/envs/herbie/lib/python3.11/site-packages/urllib3/connectionpool.py", line 490, in _make_request
raise new_e
File "/mnt/sdc1/home/juchengli/anaconda3/envs/herbie/lib/python3.11/site-packages/urllib3/connectionpool.py", line 466, in _make_request
self._validate_conn(conn)
File "/mnt/sdc1/home/juchengli/anaconda3/envs/herbie/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1095, in _validate_conn
conn.connect()
File "/mnt/sdc1/home/juchengli/anaconda3/envs/herbie/lib/python3.11/site-packages/urllib3/connection.py", line 693, in connect
self.sock = sock = self._new_conn()
^^^^^^^^^^^^^^^^
File "/mnt/sdc1/home/juchengli/anaconda3/envs/herbie/lib/python3.11/site-packages/urllib3/connection.py", line 214, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f0396d402d0>: Failed to establish a new connection: [Errno 101] Network is unreachable

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/mnt/sdc1/home/juchengli/anaconda3/envs/herbie/lib/python3.11/site-packages/requests/adapters.py", line 667, in send
resp = conn.urlopen(
^^^^^^^^^^^^^
File "/mnt/sdc1/home/juchengli/anaconda3/envs/herbie/lib/python3.11/site-packages/urllib3/connectionpool.py", line 843, in urlopen
retries = retries.increment(
^^^^^^^^^^^^^^^^^^
File "/mnt/sdc1/home/juchengli/anaconda3/envs/herbie/lib/python3.11/site-packages/urllib3/util/retry.py", line 519, in increment
raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='ecmwf-forecasts.s3.eu-central-1.amazonaws.com', port=443): Max retries exceeded with url: /20240202/00z/0p25/oper/20240202000000-0h-oper-fc.grib2 (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f0396d402d0>: Failed to establish a new connection: [Errno 101] Network is unreachable'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "", line 1, in
File "/mnt/sdc1/home/juchengli/Downloads/Herbie/herbie/core.py", line 266, in init
self.grib, self.grib_source = self.find_grib()
^^^^^^^^^^^^^^^^
File "/mnt/sdc1/home/juchengli/Downloads/Herbie/herbie/core.py", line 474, in find_grib
elif self._check_grib(grib_url):
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/sdc1/home/juchengli/Downloads/Herbie/herbie/core.py", line 396, in _check_grib
head = requests.head(url)
^^^^^^^^^^^^^^^^^^
File "/mnt/sdc1/home/juchengli/anaconda3/envs/herbie/lib/python3.11/site-packages/requests/api.py", line 100, in head
return request("head", url, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/sdc1/home/juchengli/anaconda3/envs/herbie/lib/python3.11/site-packages/requests/api.py", line 59, in request
return session.request(method=method, url=url, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/sdc1/home/juchengli/anaconda3/envs/herbie/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/sdc1/home/juchengli/anaconda3/envs/herbie/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/sdc1/home/juchengli/anaconda3/envs/herbie/lib/python3.11/site-packages/requests/adapters.py", line 700, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='ecmwf-forecasts.s3.eu-central-1.amazonaws.com', port=443): Max retries exceeded with url: /20240202/00z/0p25/oper/20240202000000-0h-oper-fc.grib2 (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f0396d402d0>: Failed to establish a new connection: [Errno 101] Network is unreachable'))

Please let me know if additional details are needed or if there's anything else I should clarify. Thanks again for your help!

@ABCDZLG
Copy link

ABCDZLG commented Nov 1, 2024

I am running this code on the server and trying to run it at different times, but I still encounter the above error and cannot solve it yet. Can this problem be resolved?(Not using a proxy)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants