feat(consume): add consume enginex simulator
#1765
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
danceratopz
added
type:feat
5 tasks
danceratopz
force-pushed
the
feat/consume-enginex
branch
from
a2411f5 to
b5a12fb
Compare
Plugins called 'conftest' are registered automatically if in a sub-path to a test. This rename requires an explicit registration, which is specified by defining the `pytest_plugins` variable in 'engine/conftest.py', respectively, 'rlp/conftest.py'.
These are fixtures that can also be shared by multi-client architecture simulators.
5 tasks
…ectory Consolidate test_via_engine.py and test_via_rlp.py into a shared hive_tests/ directory to improve organization and prepare for future multi-client architectures. - Create src/pytest_plugins/consume/simulators/hive_tests/ directory. - Move test_via_engine.py from engine/ to hive_tests/. - Move test_via_rlp.py from rlp/ to hive_tests/. - Update get_command_paths() to reference new locations. - Maintain separation: plugin configs (conftest.py) stay with their simulators. This provides better logical organization of test execution files while maintaining explicit pytest plugin structure.
… configs With the move to hive_tests/, the simulator-specific conftest.py files are no longer automatically discovered. Add explicit plugin registration to ensure _supported_fixture_formats and other simulator configs are properly loaded. - Add command_name parameter to HiveEnvironmentProcessor. - Register simulator-specific plugins based on command name. - Update ConsumeCommand to pass command_name to processors. - Fix AttributeError: 'Config' object has no attribute '_supported_fixture_formats'.
…ion of sub-groups
danceratopz
force-pushed
the
feat/consume-enginex
branch
from
b5a12fb to
8cb47da
Compare
…conftest files Move pytest_hive plugin registration from HiveEnvironmentProcessor to individual simulator conftest files to centralize all plugin configuration in one place per simulator.
Complete the plugin registration refactor by adding pytest_hive plugin to the enginex simulator conftest file for consistency with other simulators.
…ests Replace hardcoded function name check with parameter-based discovery for better maintainability. This avoids the design flaw of production code referencing specific test function names. Fixes unit test failures by checking for 'test_case' parameter presence instead of maintaining an allowlist of function names.
…zation Replace function name check with fixture format detection to properly distinguish between engine and enginex simulators. Both use the same test function name but process different fixture formats: - Engine simulator: "blockchain_test_engine" → standard parametrization - EngineX simulator: "blockchain_test_engine_x" → enhanced parametrization with xdist group splitting and load balancing This provides more robust detection and avoids the design flaw of hardcoding test function names.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
🗒️ Description
Requires refactor(fill,fixtures): rename "engine reorg" to "engine x"; improve pre-allocation group terminology #1760.rebased.Adds a simulator that runs
BlockchainEngineXFixtureagainst clients.Running The Simulator & Initial results
I've been testing locally with a subset of 1846 tests that create 29 groups from Cancun:
Then consume against a hive dev server:
Results for 1846 tests with 29 groups (=^ 29 client initializations):
reth: 85.37s (0:01:25).besu: 361.71s (0:06:01).(client versions at Pectra Fork).
FCU Behavior
Xdist Behavior
Tests get distributed to xdist worker by pre-allocation group using
loadgroup:🔗 Related Issues
BlockchainEngineReorgFixture#1706✅ Checklist
mkdocs servelocally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.