Merge branch 'main' into feat/directly-create-release-tarball

README.md

@@ -151,6 +151,10 @@ For further help with working with this codebase, see the [online documentation]
 
 The available test cases can be browsed in the [Test Case Reference doc](https://ethereum.github.io/execution-spec-tests/tests/).
 
+## Installation Troubleshooting
+
+If you encounter issues during the installation process, please refer to the [Installation Troubleshooting](https://ethereum.github.io/execution-spec-tests/main/getting_started/installation_troubleshooting/) page.
+
 ## Contributing
 
 Contributions and feedback are welcome. Please see the [online documentation](https://ethereum.github.io/execution-spec-tests/writing_tests/) for this repository's coding standards and help on implementing new tests.

docs/CHANGELOG.md

@@ -8,6 +8,7 @@ Test fixtures for use by clients are available for each release on the [Github r
 
 ### 🧪 Test Cases
 
+- ✨ Add tests for eof container's section bytes position smart fuzzing ([#592](https://github.com/ethereum/execution-spec-tests/pull/592)).
 - ✨ Add `test_create_selfdestruct_same_tx_increased_nonce` which tests self-destructing a contract with a nonce > 1 ([#478](https://github.com/ethereum/execution-spec-tests/pull/478)).
 - ✨ Add `test_double_kill` and `test_recreate` which test resurrection of accounts killed with `SELFDESTRUCT` ([#488](https://github.com/ethereum/execution-spec-tests/pull/488)).
 - ✨ Add eof example valid invalid tests from ori, fetch EOF Container implementation ([#535](https://github.com/ethereum/execution-spec-tests/pull/535)).
@@ -40,6 +41,7 @@ Test fixtures for use by clients are available for each release on the [Github r
 - ✨ Add "description" and "url" fields containing test case documentation and a source code permalink to fixtures during `fill` and use them in `consume`-generated Hive test reports ([#579](https://github.com/ethereum/execution-spec-tests/pull/579)).
 - ✨ Add git workflow evmone coverage script for any new lines mentioned in converted_ethereum_tests.txt ([#503](https://github.com/ethereum/execution-spec-tests/pull/503)).
 - ✨ Add a new covariant marker `with_all_contract_creating_tx_types` that allows automatic parametrization of a test with all contract-creating transaction types at the current executing fork ([#602](https://github.com/ethereum/execution-spec-tests/pull/602)).
+- ✨ Tests are now encouraged to declare a `pre: Alloc` parameter to get the pre-allocation object for the test, and use `pre.deploy_contract` and `pre.fund_eoa` to deploy contracts and fund accounts respectively, instead of declaring the `pre` as a dictionary or modifying its contents directly (see the [state test tutorial](https://ethereum.github.io/execution-spec-tests/main/tutorials/state_transition/) for an updated example) ([#584](https://github.com/ethereum/execution-spec-tests/pull/584)).
 - ✨ Enable loading of [ethereum/tests/BlockchainTests](https://github.com/ethereum/tests/tree/develop/BlockchainTests) ([#596](https://github.com/ethereum/execution-spec-tests/pull/596)).
 
 ### 🔧 EVM Tools

docs/getting_started/installation_troubleshooting.md

@@ -0,0 +1,35 @@
+# Installation Troubleshooting
+
+This page provides guidance on how to troubleshoot common issues that may arise when installing the Execution Spec Tests repository.
+
+## Pip Installation Issues
+
+### Coincurve Installation
+
+If you encounter an error when installing the `coincurve` package like the following:
+
+```bash
+Stored in directory: /tmp/...
+  Building wheel for coincurve (pyproject.toml) ... error
+  error: subprocess-exited-with-error
+
+  × Building wheel for coincurve (pyproject.toml) did not run successfully.
+  │ exit code: 1
+  ╰─> [27 lines of output]
+      ...
+        571 | #include <secp256k1_extrakeys.h>
+            |          ^~~~~~~~~~~~~~~~~~~~~~~
+      compilation terminated.
+      error: command '/usr/bin/gcc' failed with exit code 1
+      [end of output]
+
+  note: This error originates from a subprocess, and is likely not a problem with pip.
+  ERROR: Failed building wheel for coincurve
+```
+
+You may need to install the `libsecp256k1` library. On Ubuntu, you can install this library by running the following command:
+
+```bash
+sudo apt update
+sudo apt-get install libsecp256k1-dev
+```

docs/getting_started/quick_start.md

@@ -88,6 +88,10 @@ The following requires a Python 3.10, 3.11 or 3.12 installation.
             head fixtures/blockchain_tests/berlin/eip2930_access_list/acl/access_list.json
             ```
 
+## Installation Troubleshooting
+
+If you encounter issues during installation, see the [Installation Troubleshooting](./installation_troubleshooting.md) guide.
+
 ## Next Steps
 
 1. Learn [useful command-line flags](./executing_tests_command_line.md).

docs/navigation.md

@@ -4,6 +4,7 @@
     * [Quick Start](getting_started/quick_start.md)
     * [VS Code Setup](getting_started/setup_vs_code.md)
     * [Repository Overview](getting_started/repository_overview.md)
+    * [Installation Troubleshooting](getting_started/installation_troubleshooting.md)
     * [Executing Tests at a Prompt](getting_started/executing_tests_command_line.md)
     * [Executing Tests in VS Code](getting_started/executing_tests_vs_code.md)
     * [Executing Tests for Features Under Development](getting_started/executing_tests_dev_fork.md)

docs/tutorials/state_transition.md

@@ -31,26 +31,26 @@ Test Yul Source Code Examples
 In Python, multi-line strings are denoted using `"""`. As a convention, a file's purpose is often described in the opening string of the file.
 
 ```python
-from ethereum_test_forks import Fork
+from ethereum_test_forks import Fork, Frontier, Homestead
 from ethereum_test_tools import (
     Account,
+    Alloc,
     Environment,
     StateTestFiller,
-    TestAddress,
     Transaction,
-    Yul,
+    YulCompiler,
 )
 ```
 
 In this snippet the required constants, types and helper functions are imported from `ethereum_test_tools` and `ethereum_test_forks`. We will go over these as we come across them.
 
 ```python
-@pytest.mark.valid_from("Berlin")
+@pytest.mark.valid_from("Homestead")
 ```
 
 In Python this kind of definition is called a [*decorator*](https://docs.python.org/3/search.html?q=decorator).
 It modifies the action of the function after it.
-In this case, the decorator is a custom [pytest fixture](https://docs.pytest.org/en/latest/explanation/fixtures.html) defined by the execution-specs-test framework that specifies that the test is valid for the [Berlin fork](https://ethereum.org/en/history/#berlin) and all forks after it. The framework will then execute this test case for all forks in the fork range specified by the command-line arguments.
+In this case, the decorator is a custom [pytest fixture](https://docs.pytest.org/en/latest/explanation/fixtures.html) defined by the execution-specs-test framework that specifies that the test is valid for the [Homestead fork](https://ethereum.org/en/history/#homestead) and all forks after it. The framework will then execute this test case for all forks in the fork range specified by the command-line arguments.
 
 !!! info "Executing the test"
     To execute this test for all the specified forks, we can specify pytest's `-k` flag that [filters test cases by keyword expression](https://docs.pytest.org/en/latest/how-to/usage.html#specifying-tests-selecting-tests):
@@ -66,7 +66,7 @@ In this case, the decorator is a custom [pytest fixture](https://docs.pytest.org
     ```
 
 ```python
-def test_yul(state_test: StateTestFiller, fork: Fork):
+def test_yul(state_test: StateTestFiller, pre: Alloc, yul: YulCompiler, fork: Fork):
     """
     Test YUL compiled bytecode.
     """
@@ -79,31 +79,45 @@ The function definition ends when there is a line that is no longer indented. As
 !!! note "The `state_test` function argument"
     This test defines a state test and, as such, *must* include the `state_test` in its function arguments. This is a callable object (actually a wrapper class to the `StateTest`); we will see how it is called later.
 
+!!! note "The `pre` function argument"
+    For all types of tests, it is highly encouraged that we define the `pre` allocation as a function argument, which will be populated with the pre-state requirements during the execution of the test function (see below).
+
 ```python
     env = Environment()
 ```
 
-This line specifies that `env` is an [`Environment`](https://github.com/ethereum/execution-spec-tests/blob/main/src/ethereum_test_tools/common/types.py#L878) object, and that we just use the default parameters.
+This line specifies that `env` is an [`Environment`](https://github.com/ethereum/execution-spec-tests/blob/8b4504aaf6ae0b69c3e847a6c051e64fcefa4db0/src/ethereum_test_tools/common/types.py#L711) object, and that we just use the default parameters.
 If necessary we can modify the environment to have different block gas limits, block numbers, etc.
 In most tests the defaults are good enough.
 
-For more information, [see the static test documentation](https://ethereum-tests.readthedocs.io/en/latest/test_filler/state_filler.html#env).
+For more information, [see the static test documentation](../consuming_tests/state_test.md).
 
 #### Pre State
 
-```python
-    pre = {
-```
+For every test we need to define the pre-state requirements, so we are certain of what is on the "blockchain" before the test executes.
+It can be used as a [dictionary](https://docs.python.org/3/tutorial/datastructures.html#dictionaries), which is the Python term for an associative array, but the appropriate way to populate it is by using the methods `fund_eoa`, `deploy_contract` or `fund_address` from the `Alloc` object.
 
-Here we define the pre-state section, the one that tells us what is on the "blockchain" before the test.
-It is a [dictionary](https://docs.python.org/3/tutorial/datastructures.html#dictionaries), which is the Python term for an associative array.
+In this example we are using the `deploy_contract` method to deploy a contract to some address available in the pre-state.
 
 ```python
-        "0x1000000000000000000000000000000000000000": Account(
+    contract_address = pre.deploy_contract(
+        code=yul(
+            """
+            {
+                function f(a, b) -> c {
+                    c := add(a, b)
+                }
+
+                sstore(0, f(1, 2))
+                return(0, 32)
+            }
+            """
+        ),
+        balance=0x0BA1A9CE0BA1A9CE,
+    )
 ```
 
-The keys of the dictionary are addresses (as strings), and the values are [`Account`](https://github.com/ethereum/execution-spec-tests/blob/main/src/ethereum_test_tools/common/types.py#L517) objects.
-You can read more about address fields [in the static test documentation](https://ethereum-tests.readthedocs.io/en/latest/test_filler/state_filler.html#address-fields).
+Specifically we deploy a contract with yul code that adds two numbers and stores the result in storage.
 
 ```python
             balance=0x0BA1A9CE0BA1A9CE,
@@ -112,15 +126,29 @@ You can read more about address fields [in the static test documentation](https:
 This field is the balance: the amount of Wei that the account has. It usually doesn't matter what its value is in the case of state test contracts.
 
 ```python
-            code=Yul(
+    contract_address = pre.deploy_contract(
+```
+
+As return value of the `deploy_contract` method we get the address where the contract was deployed and put it in the `contract_address` variable, which will later be used in the transaction.
+
+```python
+        storage={
+            0x00: 0x00,
+        },
+```
+
+We could also specify a starting storage for the contract, which is done by adding a `storage` parameter to the `deploy_contract` method.
+
+```python
+            code=yul(
 ```
 
 Here we define the [Yul](https://docs.soliditylang.org/en/v0.8.17/yul.html) code for the contract. It is defined as a multi-line string and starts and ends with curly braces (`{ <yul> }`).
 
 When running the test filler `fill`, the solidity compiler `solc` will automatically translate the Yul to EVM opcode at runtime.
 
 !!! note
-    Currently Yul and direct EVM opcode are supported in execution spec tests. LLL and Solidity may be supported in the future.
+    Currently Yul and direct EVM opcode are supported in execution spec tests.
 
 ```python
                 """
@@ -132,45 +160,59 @@ When running the test filler `fill`, the solidity compiler `solc` will automatic
                     return(0, 32)
                 }
                 """
-            ),
-        ),
 ```
 
 Within this example test Yul code we have a function definition, and inside it we are using the Yul `add` instruction. When compiled with `solc` it translates the instruction directly to the `ADD` opcode. For further Yul instructions [see here](https://docs.soliditylang.org/en/latest/yul.html#evm-dialect). Notice that function is utilized with the Yul `sstore` instruction, which stores the result of `add(1, 2)` to the storage address `0x00`.
 
 Generally for execution spec tests the `sstore` instruction acts as a high-level assertion method to check pre to post-state changes. The test filler achieves this by verifying that the correct value is held within post-state storage, hence we can validate that the Yul code has run successfully.
 
 ```python
-        TestAddress: Account(balance=0x0BA1A9CE0BA1A9CE),
-    }
+    sender = pre.fund_eoa(amount=0x0BA1A9CE0BA1A9CE)
 ```
 
-[`TestAddress`](https://github.com/ethereum/execution-spec-tests/blob/main/src/ethereum_test_tools/common/constants.py#L7) is an address for which the test filler has the private key.
-This means that the test runner can issue a transaction as that contract.
-Of course, this address also needs a balance to be able to issue transactions.
+In this line we specify that we require a single externally owned account (EOA) with a balance of `0x0BA1A9CE0BA1A9CE` Wei.
+
+The returned object, which includes a private key, an address, and a nonce, is stored in the `sender` variable and will later be used as the sender of the transaction.
 
 #### Transactions
 
 ```python
     tx = Transaction(
         ty=0x0,
-        chain_id=0x0,
-        nonce=0,
-        to="0x1000000000000000000000000000000000000000",
+        chain_id=0x01,
+        sender=sender,
+        to=contract_address,
         gas_limit=500000,
         gas_price=10,
-        protected=False,
+        protected=False if fork in [Frontier, Homestead] else True,
     )
 ```
 
-With the pre-state specified, we can add a description for the [`Transaction`](https://github.com/ethereum/execution-spec-tests/blob/main/src/ethereum_test_tools/common/types.py#L1155).
-For more information, [see the static test documentation](https://ethereum-tests.readthedocs.io/en/latest/test_filler/state_filler.html#transaction)
+With the pre-state built, we can add a description for the [`Transaction`](https://github.com/ethereum/execution-spec-tests/blob/8b4504aaf6ae0b69c3e847a6c051e64fcefa4db0/src/ethereum_test_tools/common/types.py#L887).
+
+```python
+            sender=sender,
+```
+
+We use the sender variable from the pre-state to specify the sender of the transaction, which already has the necessary information to sign the transaction, and also contains the correct `nonce` for the transaction.
+
+The `nonce` is a protection mechanism to prevent replay attacks, and the current rules of Ethereum require that the nonce of a transaction is equal to the number of transactions sent from the sender's address, starting from zero. This means that the first transaction sent from an address must have a nonce of zero, the second transaction must have a nonce of one, and so on.
+
+The `nonce` field of the `sender` variable is automatically incremented for us by the `Transaction` object when the transaction is signed, so if we were to create another transaction with the same sender, the nonce would be incremented by one yet another time.
+
+```python
+            to=contract_address,
+```
+
+The `to` field specifies the address of the contract we want to call and, in this case, it is the address of the contract we deployed earlier.
+
+For more information, [see the static test documentation](../consuming_tests/state_test.md)
 
 #### Post State
 
 ```python
     post = {
-        "0x1000000000000000000000000000000000000000": Account(
+        contract_address: Account(
             storage={
                 0x00: 0x03,
             },

src/ethereum_test_tools/__init__.py

@@ -15,6 +15,7 @@
     YulCompiler,
 )
 from .common import (
+    EOA,
     AccessList,
     Account,
     Address,
@@ -99,6 +100,7 @@
     "ReferenceSpec",
     "ReferenceSpecTypes",
     "Removable",
+    "EOA",
     "StateTest",
     "StateTestFiller",
     "Storage",

src/ethereum_test_tools/common/__init__.py

@@ -36,6 +36,7 @@
 )
 from .json import to_json
 from .types import (
+    EOA,
     AccessList,
     Account,
     Alloc,
@@ -69,10 +70,11 @@
     "Number",
     "Removable",
     "Requests",
+    "EOA",
     "Storage",
+    "TestParameterGroup",
     "TestAddress",
     "TestAddress2",
-    "TestParameterGroup",
     "TestPrivateKey",
     "TestPrivateKey2",
     "Transaction",

src/ethereum_test_tools/common/base_types.py

@@ -115,6 +115,8 @@ def __new__(cls, input: BytesConvertible):
         """
         Creates a new Bytes object.
         """
+        if type(input) is cls:
+            return input
         return super(Bytes, cls).__new__(cls, to_bytes(input))
 
     def __hash__(self) -> int:
@@ -234,6 +236,8 @@ def __new__(cls, input: FixedSizeBytesConvertible | T):
         """
         Creates a new FixedSizeBytes object.
         """
+        if type(input) is cls:
+            return input
         return super(FixedSizeBytes, cls).__new__(cls, to_fixed_size_bytes(input, cls.byte_length))
 
     def __hash__(self) -> int:
@@ -277,7 +281,7 @@ class Address(FixedSizeBytes[20]):  # type: ignore
     Class that helps represent Ethereum addresses in tests.
     """
 
-    pass
+    label: str | None = None
 
 
 class Hash(FixedSizeBytes[32]):  # type: ignore

src/ethereum_test_tools/common/constants.py

@@ -9,8 +9,8 @@
 TestAddress = Address("0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b")
 TestAddress2 = Address("0x8a0a19589531694250d570040a0c4b74576919b8")
 
-TestPrivateKey = "0x45a915e4d060149eb4365960e6a7a45f334393093061116b197e3240065ff2d8"
-TestPrivateKey2 = "0x9e7645d0cfd9c3a04eb7a9db59a4eb7d359f2e75c9164a9d6b9a7d54e1b6a36f"
+TestPrivateKey = 0x45A915E4D060149EB4365960E6A7A45F334393093061116B197E3240065FF2D8
+TestPrivateKey2 = 0x9E7645D0CFD9C3A04EB7A9DB59A4EB7D359F2E75C9164A9D6B9A7D54E1B6A36F
 
 AddrAA = Address(0xAA)
 AddrBB = Address(0xBB)

src/ethereum_test_tools/common/helpers.py

@@ -10,6 +10,7 @@
 
 from .base_types import Address, Bytes, Hash
 from .conversions import BytesConvertible, FixedSizeBytesConvertible
+from .types import EOA
 
 """
 Helper functions
@@ -24,13 +25,17 @@ def ceiling_division(a: int, b: int) -> int:
     return -(a // -b)
 
 
-def compute_create_address(address: FixedSizeBytesConvertible, nonce: int) -> Address:
+def compute_create_address(address: FixedSizeBytesConvertible | EOA, nonce: int = 0) -> Address:
     """
     Compute address of the resulting contract created using a transaction
     or the `CREATE` opcode.
     """
+    if isinstance(address, EOA):
+        nonce = address.nonce
+    else:
+        address = Address(address)
     nonce_bytes = bytes() if nonce == 0 else nonce.to_bytes(length=1, byteorder="big")
-    hash = keccak256(encode([Address(address), nonce_bytes]))
+    hash = keccak256(encode([address, nonce_bytes]))
     return Address(hash[-20:])