How do I know if a test function is async?

[ramnes]

Hey there, thanks for the awesome work. :)

I'm trying to check if a function is async or not within pytest_generate_tests. I'm using async_mode=auto and I would like to keep it that way.

My first bet was to simply use inspect.isawaitable(metafunc.function), but I quickly understood that pytest-asyncio wraps my async test functions into synchronous ones, which makes sense.

So the next thing I tried was:

def pytest_generate_tests(metafunc):
    if metafunc.definition.get_closest_marker("asyncio"):
        ...

I would have expected this one to work, because the README says it should:

In auto mode, the pytest.mark.asyncio marker can be omitted, the marker is added automatically to async test functions.

So either there's something obvious I'm missing here, or the README is wrong.

What I'm doing right now (but that's really just a hack):

def pytest_generate_tests(metafunc):
    if inspect.getsource(metafunc.function)[0:5] == "async":
        ...

Is there any better way right now? If not, do you want me to open a PR to automatically mark async test functions with pytest.mark.asyncio when using async_mode=auto?

Cheers!

[seifertm]

Is there any better way right now?

Unfortunately, there is currently no publicly supported way to inspect tests in the way you want.

I would have expected this one to work, because the README says it should:
In auto mode, the pytest.mark.asyncio marker can be omitted, the marker is added automatically to async test functions.
So either there's something obvious I'm missing here, or the README is wrong.

I agree this is ubiquitous. The asyncio marker is applied automatically during pytest_pycollect_makeitem. However, each marked coroutine is wrapped in a synchronous function in pytest_pyfunc_call. I'm not aware of the execution order for pytest hooks at the moment, sorry…

If not, do you want me to open a PR to automatically mark async test functions with pytest.mark.asyncio when using async_mode=auto?

I think pytest-asyncio should expose a public attribute for wrapped tests, which allows the user to query if this is a pytest-asyncio test. Hypothesis sets is_hypothesis_test=True, for example. It would be interesting what other libraries and pytest plugins use.

[ramnes]

I agree this is ubiquitous. The asyncio marker is applied automatically during pytest_pycollect_makeitem. However, each marked coroutine is wrapped in a synchronous function in pytest_pyfunc_call. I'm not aware of the execution order for pytest hooks at the moment, sorry…

AFAIK, markers aren't set at the function level but at the node level, so wrapping the coroutine should have no effect on the markers. Am I wrong?

[schlamar]

asyncio.iscoroutinefunction(metafunc.function) would be a better alternative to inspect.getsource.

[ramnes]

Yeah, that's pretty much what I ended up doing:

import inspect
from _pytest.compat import get_real_func

def pytest_generate_tests(metafunc):
    if inspect.iscoroutinefunction(get_real_func(metafunc.function)):
        ...

IIRC you can't just call inspect.iscoroutinefunction on metafunc.function, and have to use get_real_func from the internals.

[ramnes]

is_async_test has been released in 0.23; thanks!

#686

[ramnes]

Mhh, if I understand correctly, is_async_test is not usable in pytest_generate_tests(metafunc) because nodes haven't been generated yet, am I right @seifertm?

Edit: You can see the real code here: https://github.com/Refty/mongo-thingy/blob/879fd7cf7437c63d7bc1e7e37e4dcf0cdf7ac882/tests/conftest.py#L55-L76

[seifertm]

@ramnes You're right. is_async_test only determines if a pytest.Item is "managed" by pytest-asyncio. The pytest_generate_tests hook is responsible for generating test items, so _is_async_test is of no use there.
(I had the same misconception about this issue and tagged it in the PR)

Let's assume we want a similar function that works on Python functions/coroutines in pytest_generate_tests. That means the function would have to check whether strict mode or auto mode is used, whether an asyncio marker is present and perform some additional introspection of the Python function/coroutine to see if it's a Hypothesis test, for example. Technically, all this logic already exists, but it currently operates on pytest.Function items.

I'm a bit reluctant to refactor the code away from pytest items to work on plain Python objects. I'm not really sure how to proceed with this issue at the moment.

Your comment from July where you proposed using inspect.iscoroutinefunction in combination with _pytest.compat. get_real_func is probably a good way to go about this. The use of the internal pytest funtion is admittedly ugly, but your code also catches test coroutines from other async frameworks, e.g. trio. On the downside, it probably misses things like Hypothesis tests.