feat(tests) Precompile Checks #1120

reedsa · 2025-01-23T22:25:28Z

🗒️ Description

Add tests to check each precompile at specific addresses.

Tests from: https://github.com/ethereum/tests/blob/develop/src/GeneralStateTestsFiller/stPreCompiledContracts/idPrecompsFiller.yml

Removed in:
ethereum/tests#1444

🔗 Related Issues

✅ Checklist

All: Set appropriate labels for the changes.
All: Considered squashing commits to improve commit history.
All: Added an entry to CHANGELOG.md.
All: Considered updating the online docs in the ./docs/ directory.
Tests: All converted JSON/YML tests from ethereum/tests have been added to converted-ethereum-tests.txt.
Tests: A PR with removal of converted JSON/YML tests from ethereum/tests have been opened.
Tests: Included the type and version of evm t8n tool used to locally execute test cases: e.g., ref with commit hash or geth 1.13.1-stable-3f40e65.
Tests: Ran mkdocs serve locally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.

marioevz

Gave this one a quick look and I wrote one comment.

We also have this similar test over here: https://github.com/ethereum/execution-spec-tests/blob/main/tests/frontier/precompiles/test_precompile_absence.py

marioevz · 2025-01-24T16:47:08Z

tests/precompiles/test_precompiles.py

+    ["address", "exists"],
+    [
+        ("01", True),
+        ("02", True),
+        ("03", True),
+        ("04", True),
+        ("05", True),
+        ("06", True),
+        ("07", True),
+        ("08", True),
+        ("09", True),
+        ("0A", True),
+        ("0B", False),


This list would only be valid for Cancun, but we could use this marker @pytest.mark.parametrize_by_fork to dynamically generate this list by fork.

I would also reduce the amount of cases to the first exists=False fork.

reedsa · 2025-01-24T21:01:33Z

Gave this one a quick look and I wrote one comment.

We also have this similar test over here: https://github.com/ethereum/execution-spec-tests/blob/main/tests/frontier/precompiles/test_precompile_absence.py

Those tests are very similar. Do you think it's still worth migrating these tests?

reedsa · 2025-01-24T21:03:51Z

tests/frontier/precompiles/test_precompiles.py

+        + Op.MSTORE(gas_test, Op.GAS())
+        # Setup stack to CALL into precompile with the CALLDATA and CALL into it (+ pop value)
+        + Op.CALL(
+            address=Op.CALLDATALOAD(0),


Trying to pass in the precompile address in the data of the transaction for this call. It doesn't seem like I can pass the correct value from the bytes value passed, even if I try to convert the value from the CALLDATALOAD to an integer. Is there a recommendation for how to approach this?

here the data must be padded as EVM word of 32 bytes. so not enough just fromhex(1 byte) you need to padd it with zeros to the left. like Hash32(data, left_padding=true)

reedsa · 2025-01-30T20:34:43Z

@winsvega @marioevz now I have the data (precompile address) being sent into the transaction. It looks like all tests are returning 0 however. Is there something I may have missed with the logic in the contract? In the original test, the gas cost difference is found based on a condition that the precompile gas is greater than the gas for the call at the address 0x10000.

winsvega · 2025-02-03T09:45:49Z

tests/frontier/precompiles/test_precompiles.py

+
+    """
+    return [
+        ("01", True),


you can access fork class from the test if you add fixture in the parameter list.
fork: Fork

and then it has a function that returns all precompiles currently active on that fork as integer list.

winsvega · 2025-02-03T09:49:50Z

tests/frontier/precompiles/test_precompiles.py

+                if_false=Op.SUB(Op.MLOAD(gas_10000), Op.MLOAD(gas_test)),
+            ),
+        )
+        + Op.SSTORE(0, Op.LT(Op.SLOAD(0), 0x10))


here also a pattern we use. there is class Storage.
so you can call Op.SSTORE(storage.store_next(expected_value, "debug hint"), logic)

winsvega

added comments

reedsa · 2025-02-03T21:32:16Z

@marioevz @winsvega Latest revisions are creating parameterized tests for each fork with valid precompiles as well as a single invalid precompile. I noticed the gas cost is slightly higher than what the original tests were using. Instead of the gas being compared with 0x10 in the original test, it now checks that the cost is less than 0x1A4.

This however fails with Byzantium and Constantinople for precompile 5 and I'm not sure why.

winsvega · 2025-02-03T22:08:21Z

I think the tes was not designed for that forks. Check in ethereum/legacytests if it has a revision for other forks.

But yes, need to understand what's going on

Contract compares gas cost for each precompile. Only checks a single unsupported precompile address.

winsvega · 2025-02-06T09:06:37Z

tests/frontier/precompiles/test_precompiles.py

+    """
+    supported_precompiles = fork.precompiles()
+
+    for address in range(1, len(supported_precompiles) + 2):


why +2
and what if precompiles are ranges [1..to 11], [100 to 112]

winsvega · 2025-02-06T09:19:06Z

tests/frontier/precompiles/test_precompiles.py

+    gas_10000 = 0x20
+
+    account = pre.deploy_contract(
+        Op.MSTORE(gas_test, Op.GAS)


@marioevz it works Op.GAS as well as Op.GAS() ?

winsvega · 2025-02-06T09:19:24Z

tests/frontier/precompiles/test_precompiles.py

+            output_offset=output_offset,
+            output_size=output_size,
+        )
+        + Op.MSTORE(gas_10000, Op.SUB(Op.GAS, Op.MLOAD(gas_10000)))


you get negative value here

winsvega · 2025-02-06T09:20:03Z

.vscode/launch.json

+  // If the VS Code "Run and Debug" button, respecively launch selector are not visible, see this answer:
+  // https://stackoverflow.com/a/74245823
+  // 
+  "version": "0.2.0",


why the settings got exported?

winsvega · 2025-02-06T09:22:31Z

tests/frontier/precompiles/test_precompiles.py

+    """
+    env = Environment()
+
+    args_offset = 0x1000


I see you didn't want to copy, but then you can actually use default values of this args leaving it empty.
and it is always good when reading the code If it is visible what is the offset without scrolling up to the defenition (IMHO), then the call would fit in one line.

winsvega

when implementing test for the first time, we, actually take our understanding of vm for granted, but newcomers must use vmtrace. and it always good to use vmtrace to see what is actually happening. (yes by now we have little compiler in our head)
I remember the first year I was using vmtrace all the time, it is essential to get an idea of what is going on and how your code is executed.

saying that we might want to use/develop better tools for vmtrace visualizing and debugging. and manuals.

@marioevz @danceratopz

reedsa force-pushed the id-precompiles branch from 4ade831 to ccb6574 Compare January 23, 2025 22:27

marioevz reviewed Jan 24, 2025

View reviewed changes

reedsa force-pushed the id-precompiles branch from ccb6574 to 473e19f Compare January 24, 2025 21:00

reedsa commented Jan 24, 2025

View reviewed changes

reedsa changed the title ~~(feat) Precompile Checks~~ feat(tests) Precompile Checks Jan 28, 2025

reedsa force-pushed the id-precompiles branch 3 times, most recently from dfbf4c6 to f78c534 Compare January 30, 2025 20:30

winsvega reviewed Feb 3, 2025

View reviewed changes

reedsa force-pushed the id-precompiles branch 3 times, most recently from d2cab4a to e43edba Compare February 3, 2025 21:27

reedsa force-pushed the id-precompiles branch from e43edba to 9372ea0 Compare February 4, 2025 15:08

Check each fork for supported precompiles.

cc74bbb

Contract compares gas cost for each precompile. Only checks a single unsupported precompile address.

reedsa force-pushed the id-precompiles branch from 9372ea0 to cc74bbb Compare February 4, 2025 15:19

reedsa mentioned this pull request Feb 5, 2025

Remove migrated ID Precompile tests ethereum/tests#1444

Open

reedsa marked this pull request as ready for review February 5, 2025 16:41

reedsa requested review from marioevz, winsvega and danceratopz February 5, 2025 16:41

winsvega reviewed Feb 6, 2025

View reviewed changes

winsvega requested changes Feb 6, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(tests) Precompile Checks #1120

feat(tests) Precompile Checks #1120

reedsa commented Jan 23, 2025 •

edited

Loading

marioevz left a comment

marioevz Jan 24, 2025

reedsa commented Jan 24, 2025

reedsa Jan 24, 2025

winsvega Feb 3, 2025

reedsa commented Jan 30, 2025

winsvega Feb 3, 2025

winsvega Feb 3, 2025

winsvega left a comment

reedsa commented Feb 3, 2025

winsvega commented Feb 3, 2025

winsvega Feb 6, 2025

winsvega Feb 6, 2025

winsvega Feb 6, 2025

winsvega Feb 6, 2025

winsvega Feb 6, 2025 •

edited

Loading

winsvega left a comment

feat(tests) Precompile Checks #1120

Are you sure you want to change the base?

feat(tests) Precompile Checks #1120

Conversation

reedsa commented Jan 23, 2025 • edited Loading

🗒️ Description

🔗 Related Issues

✅ Checklist

marioevz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

reedsa commented Jan 24, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

reedsa commented Jan 30, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

winsvega left a comment

Choose a reason for hiding this comment

reedsa commented Feb 3, 2025

winsvega commented Feb 3, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

winsvega Feb 6, 2025 • edited Loading

Choose a reason for hiding this comment

winsvega left a comment

Choose a reason for hiding this comment

reedsa commented Jan 23, 2025 •

edited

Loading

winsvega Feb 6, 2025 •

edited

Loading