How to achieve assertion rewriting for a helper module loaded directly from file?

[Midnighter]

I'm using importlib to directly load a module from a file that would not (and should not) be discovered by pytest. However, currently I don't get any introspection of failed assertions within that module. Sadly, register_assert_rewrite does not seem to work in this case.

I have the following minimal example structure:

├── helpers
│       └── check.py
└── tests
    └── test_unit.py

Contents of check.py:

class TestAbstractBehaviour:

    def test_order(self):
        assert [1, 2, 3] == [3, 2, 1]

and of test_unit.py:

import importlib.util
from pathlib import Path

import pytest


pytest.register_assert_rewrite("check")
helpers = Path(__file__).resolve().parent.parent / "helpers" / "check.py"
spec = importlib.util.spec_from_file_location("check", helpers)
check = importlib.util.module_from_spec(spec)
spec.loader.exec_module(check)


class TestConcreteBehaviour(check.TestAbstractBehaviour):
    pass

Output when I run this:

❯ pytest tests
===================================== test session starts =====================================
platform linux -- Python 3.8.10, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
rootdir: ~/mre
plugins: anyio-3.2.0, cov-2.12.1, raises-0.11
collected 1 item

tests/test_unit.py F                                                                    [100%]

========================================== FAILURES ===========================================
______________________________ TestConcreteBehaviour.test_order _______________________________

self = <test_unit.TestConcreteBehaviour object at 0x7f381cd1a0d0>

    def test_order(self):
>       assert [1, 2, 3] == [3, 2, 1]
E       AssertionError

helpers/check.py:4: AssertionError
=================================== short test summary info ===================================
FAILED tests/test_unit.py::TestConcreteBehaviour::test_order - AssertionError
====================================== 1 failed in 0.06s ======================================

Can I force assertion writing in the exec_module call or afterward somehow? Thank you for your help.

[nicoddemus]

Hi @Midnighter,

Here's how we do it:

pytest/src/_pytest/pathlib.py

Lines 492 to 508 in 5016375

	if mode is ImportMode.importlib:
	module_name = module_name_from_path(path, root)
	for meta_importer in sys.meta_path:
	spec = meta_importer.find_spec(module_name, [str(path.parent)])
	if spec is not None:
	break
	else:
	spec = importlib.util.spec_from_file_location(module_name, str(path))
	if spec is None:
	raise ImportError(f"Can't find module {module_name} at location {path}")
	mod = importlib.util.module_from_spec(spec)
	sys.modules[module_name] = mod
	spec.loader.exec_module(mod) # type: ignore[union-attr]
	insert_missing_modules(sys.modules, module_name)
	return mod

Seems like you need to iterate over sys.meta_path yourself (I'm no expert in importlib).

[Midnighter]

No, iterating over sys.meta_path should not be necessary. My code is equivalent to the else branch lines 499 & 500 since the module will never be found through normal import mechanisms (it's not on any path).

The only difference I can see is line 507 which in my case should not be necessary since my module name is top level "check" rather than a submodule like "check.this.out". I also tried adding my module to sys.modules but as far as I can tell, it doesn't change anything.

So I don't see anything in the code that hints at how the module assertion rewriting is achieved. Possibly whatever code calls import_path itself has more clues.