Cancellation tests are timing sensitive

[mweinelt]

Hi there!

After upgrading httpcore to 0.18.0 in nixpkgs we are seeing the new cancellation tests fail on our build infrastructure, which is generally pretty busy. I cannot reproduce these on my local desktop machine, which is mostly idle.

My assumption is that these tests are timing sensitive, as they wait for only wait 10ms for a reaction. This seems to affect all tests that use anyio.move_on_after(0.01).

We would appreciate it if they were made more stable, or marked as timing sensitive, so we and likely other distros, could skip these in bulk.

============================= test session starts ==============================
platform linux -- Python 3.10.13, pytest-7.4.2, pluggy-1.2.0
rootdir: /build/source
configfile: pyproject.toml
plugins: asyncio-0.21.1, httpbin-2.0.0, trio-0.8.0, anyio-4.0.0
asyncio: mode=strict
collected 213 items                                                            

tests/test_api.py 127.0.0.1 - - [21/Sep/2023 22:38:02] "GET / HTTP/1.1" 200 9509
.127.0.0.1 - - [21/Sep/2023 22:38:02] "GET / HTTP/1.1" 200 9509
..                                                    [  1%]
tests/test_cancellations.py .....FF.....FF                               [  7%]
tests/test_models.py .............                                       [ 14%]
tests/_async/test_connection.py ..................                       [ 22%]
tests/_async/test_connection_pool.py .................................   [ 38%]
tests/_async/test_http11.py ........................                     [ 49%]
tests/_async/test_http2.py ....................                          [ 58%]
tests/_async/test_http_proxy.py ..........                               [ 63%]
tests/_async/test_integration.py ......                                  [ 66%]
tests/_async/test_socks_proxy.py ..........                              [ 70%]
tests/_sync/test_connection.py .........                                 [ 75%]
tests/_sync/test_connection_pool.py ..................                   [ 83%]
tests/_sync/test_http11.py ............                                  [ 89%]
tests/_sync/test_http2.py ..........                                     [ 93%]
tests/_sync/test_http_proxy.py .....                                     [ 96%]
tests/_sync/test_integration.py ...                                      [ 97%]
tests/_sync/test_socks_proxy.py .....                                    [100%]

=================================== FAILURES ===================================
___________________ test_h2_timeout_during_request[asyncio] ____________________

    @pytest.mark.anyio
    async def test_h2_timeout_during_request():
        """
        An async timeout on an HTTP/2 during a request
        should leave the connection in a neatly idle state.
    
        The connection is not closed because it is multiplexed,
        and a timeout on one request does not require the entire
        connection be closed.
        """
        origin = httpcore.Origin(b"http", b"example.com", 80)
        stream = HandshakeThenSlowWriteStream()
        async with httpcore.AsyncHTTP2Connection(origin, stream) as conn:
            with anyio.move_on_after(0.01):
                await conn.request("GET", "http://example.com")
    
            assert not conn.is_closed()
>           assert conn.is_idle()
E           AssertionError: assert False
E            +  where False = <bound method AsyncHTTP2Connection.is_idle of <AsyncHTTP2Connection ['http://example.com:80', ACTIVE, Request Count: 1]>>()
E            +    where <bound method AsyncHTTP2Connection.is_idle of <AsyncHTTP2Connection ['http://example.com:80', ACTIVE, Request Count: 1]>> = <AsyncHTTP2Connection ['http://example.com:80', ACTIVE, Request Count: 1]>.is_idle

tests/test_cancellations.py:205: AssertionError
___________________ test_h2_timeout_during_response[asyncio] ___________________

    @pytest.mark.anyio
    async def test_h2_timeout_during_response():
        """
        An async timeout on an HTTP/2 during the response reading
        should leave the connection in a neatly idle state.
    
        The connection is not closed because it is multiplexed,
        and a timeout on one request does not require the entire
        connection be closed.
        """
        origin = httpcore.Origin(b"http", b"example.com", 80)
        stream = SlowReadStream(
            [
                hyperframe.frame.SettingsFrame().serialize(),
                hyperframe.frame.HeadersFrame(
                    stream_id=1,
                    data=hpack.Encoder().encode(
                        [
                            (b":status", b"200"),
                            (b"content-type", b"plain/text"),
                        ]
                    ),
                    flags=["END_HEADERS"],
                ).serialize(),
                hyperframe.frame.DataFrame(
                    stream_id=1, data=b"Hello, world!...", flags=[]
                ).serialize(),
            ]
        )
        async with httpcore.AsyncHTTP2Connection(origin, stream) as conn:
            with anyio.move_on_after(0.01):
                await conn.request("GET", "http://example.com")
    
            assert not conn.is_closed()
>           assert conn.is_idle()
E           AssertionError: assert False
E            +  where False = <bound method AsyncHTTP2Connection.is_idle of <AsyncHTTP2Connection ['http://example.com:80', ACTIVE, Request Count: 1]>>()
E            +    where <bound method AsyncHTTP2Connection.is_idle of <AsyncHTTP2Connection ['http://example.com:80', ACTIVE, Request Count: 1]>> = <AsyncHTTP2Connection ['http://example.com:80', ACTIVE, Request Count: 1]>.is_idle

tests/test_cancellations.py:242: AssertionError
_____________________ test_h2_timeout_during_request[trio] _____________________

    @pytest.mark.anyio
    async def test_h2_timeout_during_request():
        """
        An async timeout on an HTTP/2 during a request
        should leave the connection in a neatly idle state.
    
        The connection is not closed because it is multiplexed,
        and a timeout on one request does not require the entire
        connection be closed.
        """
        origin = httpcore.Origin(b"http", b"example.com", 80)
        stream = HandshakeThenSlowWriteStream()
        async with httpcore.AsyncHTTP2Connection(origin, stream) as conn:
            with anyio.move_on_after(0.01):
                await conn.request("GET", "http://example.com")
    
            assert not conn.is_closed()
>           assert conn.is_idle()
E           AssertionError: assert False
E            +  where False = <bound method AsyncHTTP2Connection.is_idle of <AsyncHTTP2Connection ['http://example.com:80', ACTIVE, Request Count: 1]>>()
E            +    where <bound method AsyncHTTP2Connection.is_idle of <AsyncHTTP2Connection ['http://example.com:80', ACTIVE, Request Count: 1]>> = <AsyncHTTP2Connection ['http://example.com:80', ACTIVE, Request Count: 1]>.is_idle

tests/test_cancellations.py:205: AssertionError
____________________ test_h2_timeout_during_response[trio] _____________________

    @pytest.mark.anyio
    async def test_h2_timeout_during_response():
        """
        An async timeout on an HTTP/2 during the response reading
        should leave the connection in a neatly idle state.
    
        The connection is not closed because it is multiplexed,
        and a timeout on one request does not require the entire
        connection be closed.
        """
        origin = httpcore.Origin(b"http", b"example.com", 80)
        stream = SlowReadStream(
            [
                hyperframe.frame.SettingsFrame().serialize(),
                hyperframe.frame.HeadersFrame(
                    stream_id=1,
                    data=hpack.Encoder().encode(
                        [
                            (b":status", b"200"),
                            (b"content-type", b"plain/text"),
                        ]
                    ),
                    flags=["END_HEADERS"],
                ).serialize(),
                hyperframe.frame.DataFrame(
                    stream_id=1, data=b"Hello, world!...", flags=[]
                ).serialize(),
            ]
        )
        async with httpcore.AsyncHTTP2Connection(origin, stream) as conn:
            with anyio.move_on_after(0.01):
                await conn.request("GET", "http://example.com")
    
            assert not conn.is_closed()
>           assert conn.is_idle()
E           AssertionError: assert False
E            +  where False = <bound method AsyncHTTP2Connection.is_idle of <AsyncHTTP2Connection ['http://example.com:80', ACTIVE, Request Count: 1]>>()
E            +    where <bound method AsyncHTTP2Connection.is_idle of <AsyncHTTP2Connection ['http://example.com:80', ACTIVE, Request Count: 1]>> = <AsyncHTTP2Connection ['http://example.com:80', ACTIVE, Request Count: 1]>.is_idle

tests/test_cancellations.py:242: AssertionError
======================== 4 failed, 209 passed in 50.99s ========================

[karpetrosyan]

Hi! I think #803 has already solved this problem.

[mweinelt]

Yes, looks like it. Thank you.