httpx.ReadError is raised with httpx async api when requesting certain insecure ssl website

[heroesm]

How to reproduce:

Environment

Python 3.9.1

Name: httpx
Version: 0.19.0

Name: anyio
Version: 3.2.0

Name: httpcore
Version: 0.13.6

Script

#!/usr/bin/env python3
# encoding=utf-8

import asyncio
import httpx

async def arun():
    async with httpx.AsyncClient(verify=False) as cli:
        res = await cli.get('https://103.95.25.22/')
    print(res)


def main():
    loop = asyncio.get_event_loop()
    loop.run_until_complete(arun())


if __name__ == '__main__':
    main()

Expected

SSL error is ignored; response got successfully.

Actual

Traceback (most recent call last):
  File ".../lib/python3.9/site-packages/anyio/streams/tls.py", line 102, in _call_sslobject_method
    result = func(*args)
  File ".../lib/python3.9/ssl.py", line 888, in read
    v = self._sslobj.read(len)
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:2622)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File ".../lib/python3.9/site-packages/httpcore/_backends/anyio.py", line 60, in read
    return await self.stream.receive(n)
  File ".../lib/python3.9/site-packages/anyio/streams/tls.py", line 152, in receive
    data = await self._call_sslobject_method(self._ssl_object.read, max_bytes)
  File ".../lib/python3.9/site-packages/anyio/streams/tls.py", line 121, in _call_sslobject_method
    raise BrokenResourceError from exc
anyio.BrokenResourceError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File ".../lib/python3.9/site-packages/httpx/_transports/default.py", line 61, in map_httpcore_exceptions
    yield
  File ".../lib/python3.9/site-packages/httpx/_transports/default.py", line 202, in __aiter__
    async for part in self._httpcore_stream:
  File ".../lib/python3.9/site-packages/httpcore/_async/connection_pool.py", line 57, in __aiter__
    async for chunk in self.stream:
  File ".../lib/python3.9/site-packages/httpcore/_bytestreams.py", line 91, in __aiter__
    async for chunk in self._aiterator:
  File ".../lib/python3.9/site-packages/httpcore/_async/http11.py", line 208, in _receive_response_data
    event = await self._receive_event(timeout)
  File ".../lib/python3.9/site-packages/httpcore/_async/http11.py", line 225, in _receive_event
    data = await self.socket.read(self.READ_NUM_BYTES, timeout)
  File ".../lib/python3.9/site-packages/httpcore/_backends/anyio.py", line 65, in read
    raise ReadError from exc
httpcore.ReadError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File ".../t.py", line 19, in <module>
    main()
  File ".../t.py", line 15, in main
    loop.run_until_complete(arun())
  File ".../lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
    return future.result()
  File ".../t.py", line 9, in arun
    res = await cli.get('https://103.95.25.22/')
  File ".../lib/python3.9/site-packages/httpx/_client.py", line 1740, in get
    return await self.request(
  File ".../lib/python3.9/site-packages/httpx/_client.py", line 1494, in request
    response = await self.send(
  File ".../lib/python3.9/site-packages/httpx/_client.py", line 1586, in send
    response = await self._send_handling_auth(
  File ".../lib/python3.9/site-packages/httpx/_client.py", line 1616, in _send_handling_auth
    response = await self._send_handling_redirects(
  File ".../lib/python3.9/site-packages/httpx/_client.py", line 1676, in _send_handling_redirects
    raise exc
  File ".../lib/python3.9/site-packages/httpx/_client.py", line 1669, in _send_handling_redirects
    await response.aread()
  File ".../lib/python3.9/site-packages/httpx/_models.py", line 1579, in aread
    self._content = b"".join([part async for part in self.aiter_bytes()])
  File ".../lib/python3.9/site-packages/httpx/_models.py", line 1579, in <listcomp>
    self._content = b"".join([part async for part in self.aiter_bytes()])
  File ".../lib/python3.9/site-packages/httpx/_models.py", line 1595, in aiter_bytes
    async for raw_bytes in self.aiter_raw():
  File ".../lib/python3.9/site-packages/httpx/_models.py", line 1649, in aiter_raw
    async for raw_stream_bytes in self.stream:
  File ".../lib/python3.9/site-packages/httpx/_client.py", line 150, in __aiter__
    async for chunk in self._stream:
  File ".../lib/python3.9/site-packages/httpx/_transports/default.py", line 203, in __aiter__
    yield part
  File ".../lib/python3.9/contextlib.py", line 135, in __exit__
    self.gen.throw(type, value, traceback)
  File ".../lib/python3.9/site-packages/httpx/_transports/default.py", line 78, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.ReadError

Comparison

With requests we can get response successfully:

>>> import requests
>>> requests.get('https://103.95.25.22/', verify=False)
/usr/lib/python3/dist-packages/urllib3/connectionpool.py:1015: InsecureRequestWarning: Unverified HTTPS request is being made to host '103.95.25.22'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  warnings.warn(
/usr/lib/python3/dist-packages/urllib3/connectionpool.py:1015: InsecureRequestWarning: Unverified HTTPS request is being made to host '103.95.25.22'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  warnings.warn(
<Response [200]>

With httpx sync api we can also get response successfully:

>>> import httpx
>>> httpx.get('https://103.95.25.22/', verify=False)
<Response [200 Ok]>

[heroesm]

Is it httpx issue or anyio issue? I'm not familiar with anyio.

[ghost]

I have met the same error, everything latest and stable.

anyio 3.3.1
httpcore 0.13.7
httpx 0.19.0

I could provide URLs or a Dockerfile to reproduce if needed.

[tomchristie]

If you could start with an example against some live URL that'd be fantastic, yup.

[ghost]

This should(n't) work

import httpx
import asyncio

class Api:
    url = 'https://www.comdirect.de/'
    def get(l: str)->dict:
        req = httpx.get(Api.url)
        return req.status_code
    async def get_async(url_list: list)->list:
        async with httpx.AsyncClient(http2=True, verify=False) as client:
            tasks = (client.get(Api.url) for l in url_list)
            reqs = await asyncio.gather(*tasks)
        return [req.status_code for req in reqs]

def main():
    print(Api.get('1'))
    list = ['1', '2']
    print(asyncio.run(Api.get_async(list)))

if __name__ == '__main__':
    main()

[ghost]

Hello @tomchristie,
do you maybe have any information on this?

[tomchristie]

Hiya - I've validated this and escalated to #1896 - needs some good looking into.
Thanks!

[heroesm]

#1896
agronholm/anyio#379

Resolved via pip install -U 'anyio>=3.3.4'