Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pytest-json-report makes pytest_report_teststatus hook to run twice for each step #96

Open
briceparmentier opened this issue Jun 26, 2024 · 0 comments

Comments

@briceparmentier
Copy link

Hi there!
I'm trying to understand to behavior of the pytest_report_teststatus hook, because in my environment (which does not seem to be that different from what I can see on the internet) I'm getting it called twice for each of the steps in setup, teardown and call.

I tried to run in another project written from scratch, so below is what I hope to be a clear explanation of what I have found so far.

I have the following code in my conftest.py file:

import pytest


def pytest_report_teststatus(report, config):
    print(report.when, report.outcome)


@pytest.fixture(scope="function")
def myfixture():
    print("################################ setup")
    yield
    print("################################ teardown")

Then I have 3 tests files as below:
tests_script1.py:

import pytest


@pytest.mark.skip(reason="reason")
def test_machin1(myfixture):
    print("something 1")

tests_script2.py:

def test_thing2():
    print("something 2")

tests_script3.py:

def test_thing3():
    print("something 3")

If I run the command pytest -s in this folder, I'm getting the following result:

================================================================================================================================================ test session starts ================================================================================================================================================
platform win32 -- Python 3.12.2, pytest-8.2.2, pluggy-1.5.0
rootdir: C:\workdir\pytest_try\pythonProject
configfile: pytest.ini
plugins: anyio-4.4.0, json-report-1.5.0, metadata-3.1.1
collected 3 items                                                                                                                                                                                                                                                                                                     

test_script.py setup skipped
steardown passed

test_script2.py ################################ setup
setup passed
something 2
call passed
.################################ teardown
teardown passed

test_script3.py ################################ setup
setup passed
something 3
call passed
.################################ teardown
teardown passed


=========================================================================================================================================== 2 passed, 1 skipped in 0.02s ============================================================================================================================================

Now if I edit the conftest.py file and add the following hook with the line that activates the json report plugin:

def pytest_configure(config):
    config.option.json_report = True

and then run the command again, I get the following output:

================================================================================================================================================ test session starts ================================================================================================================================================
platform win32 -- Python 3.12.2, pytest-8.2.2, pluggy-1.5.0
rootdir: C:\workdir\pytest_try\pythonProject
configfile: pytest.ini
plugins: anyio-4.4.0, json-report-1.5.0, metadata-3.1.1
collected 3 items                                                                                                                                                                                                                                                                                                     

test_script.py setup skipped
ssetup skipped
teardown passed
teardown passed

test_script2.py ################################ setup
setup passed
setup passed
something 2
call passed
.call passed
################################ teardown
teardown passed
teardown passed

test_script3.py ################################ setup
setup passed
setup passed
something 3
call passed
.call passed
################################ teardown
teardown passed
teardown passed


---------------------------------------------------------------------------------------------------------------------------------------------------- JSON report ---------------------------------------------------------------------------------------------------------------------------------------------------- 
report saved to: .report.json
=========================================================================================================================================== 2 passed, 1 skipped in 0.01s ============================================================================================================================================

As you can see, the pytest_report_teststatus is now called twice for each step.

Which causes me a problem as I want to submit tests results to another tool (using a HTTP request), so I'm getting two results for one test (whatever the result).

Thank you a lot in advance for the attention you can bring to this and help me fix it, or find a workaround.

Cheers,
Brice

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant