Skip to content

Commit

Permalink
Add EIP: Forward compatible consensus data structures
Browse files Browse the repository at this point in the history 
Merged by EIP-Bot.
  • Loading branch information
@etan-status
etan-status authored May 5, 2024
1 parent 252cd9d commit a639237
Showing 1 changed file with 296 additions and 0 deletions.
296 changes: 296 additions & 0 deletions EIPS/eip-7688.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,296 @@
---
eip: 7688
title: Forward compatible consensus data structures
description: Transition consensus SSZ data structures to StableContainer
author: Etan Kissling (@etan-status), Cayman (@wemeetagain)
discussions-to: https://ethereum-magicians.org/t/eip-7688-forward-compatible-consensus-data-structures/19673
status: Draft
type: Standards Track
category: Core
created: 2024-04-15
requires: 7495
---

## Abstract

This EIP defines the changes needed to adopt `StableContainer` from [EIP-7495](./eip-7495.md) in consensus data structures.

## Motivation

Ethereum's consensus data structures make heavy use of [Simple Serialize (SSZ)](https://github.com/ethereum/consensus-specs/blob/ef434e87165e9a4c82a99f54ffd4974ae113f732/ssz/simple-serialize.md) `Container`, which defines how they are serialized and merkleized. The merkleization scheme allows application implementations to verify that individual fields (and partial fields) have not been tampered with. This is useful, for example, in smart contracts of decentralized staking pools that wish to verify that participating validators have not been slashed.

While SSZ `Container` defines how data structures are merkleized, the merkleization is prone to change across the different forks. When that happens, e.g., because new features are added or old features get removed, existing verifier implementations need to be updated to be able to continue processing proofs.

`StableContainer`, of [EIP-7495](./eip-7495.md), is a forward compatible alternative that guarantees a forward compatible merkleization scheme. By transitioning consensus data structures to use `StableContainer`, smart contracts that contain verifier logic no longer have to be maintained in lockstep with Ethereum's fork schedule as long as the underlying features that they verify don't change. For example, as long as the concept of slashing is represented using the boolean `slashed` field, existing verifiers will not break when unrelated features get added or removed. This is also true for off-chain verifiers, e.g., in hardware wallets or in operating systems for mobile devices that are on a different software update cadence than Ethereum.

## Specification

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 and RFC 8174.

### Conversion procedure

For each converted data structure, a new `StableContainer` type `S` is introduced that serves as the primary definition of each data structure.

- Each `StableContainer` is assigned a capacity to represent its potential design space that SHALL NOT change across future forks; if it is later determined that it is insufficient, a new field can be added to contain additional fields in a sub-container.
- The `StableContainer` starts as a copy of the latest fork's `Container` equivalent.
- To guarantee forward and backward compatibility, new fields from future forks MUST only be appended to the `StableContainer` definition. Existing fields MAY be converted to `Optional`.

Furthermore, a `Variant` type is defined that is specific to the fork at which the conversion is applied. This `Variant` is the equivalent of the legacy `Container` type, except that it inherits from `Variant[S]`. The SSZ serialization of `Variant` is compatible with `Container`, but the merkleization and `hash_tree_root` are computed differently. Furthermore, `Variant` MAY use fields of `Optional` type if necessary.

If subsequent forks add / remove fields, they specify a new `Variant`. This is similar to the previous approach of specifying a new `Container`. The `StableContainer` is the superset across all `Variant`; fields not present across all `Variant` use an `Optional` type.

The fork specific `Variant` definitions use the same serialization as the legacy `Container` type. Only merkleization and `hash_tree_root` are computed differently when switching to the new scheme.

### Limits

| Name | Value | Description |
| - | - | - |
| `MAX_EXECUTION_PAYLOAD_FIELDS` | `uint64(2**6)` (= 64) | Maximum number of fields to which `ExecutionPayload` can ever grow in the future |
| `MAX_BEACON_BLOCK_BODY_FIELDS` | `uint64(2**6)` (= 64) | Maximum number of fields to which `BeaconBlockBody` can ever grow in the future |
| `MAX_BEACON_STATE_FIELDS` | `uint64(2**7)` (= 128) | Maximum number of fields to which `BeaconState` can ever grow in the future |

### Fork-agnostic `StableContainer` definitions

These type definitions are fork independent and shared across all forks. They are not exchanged over libp2p.

```python
class StableExecutionPayload(StableContainer[MAX_EXECUTION_PAYLOAD_FIELDS]):
# Execution block header fields
parent_hash: Hash32
fee_recipient: ExecutionAddress # 'beneficiary' in the yellow paper
state_root: Bytes32
receipts_root: Bytes32
logs_bloom: ByteVector[BYTES_PER_LOGS_BLOOM]
prev_randao: Bytes32 # 'difficulty' in the yellow paper
block_number: uint64 # 'number' in the yellow paper
gas_limit: uint64
gas_used: uint64
timestamp: uint64
extra_data: ByteList[MAX_EXTRA_DATA_BYTES]
base_fee_per_gas: uint256
# Extra payload fields
block_hash: Hash32 # Hash of execution block
transactions: List[Transaction, MAX_TRANSACTIONS_PER_PAYLOAD]
withdrawals: List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD]
blob_gas_used: uint64
excess_blob_gas: uint64
... # Additional features from the fork that introduces this EIP

class StableExecutionPayloadHeader(StableContainer[MAX_EXECUTION_PAYLOAD_FIELDS]):
# Execution block header fields
parent_hash: Hash32
fee_recipient: ExecutionAddress
state_root: Bytes32
receipts_root: Bytes32
logs_bloom: ByteVector[BYTES_PER_LOGS_BLOOM]
prev_randao: Bytes32
block_number: uint64
gas_limit: uint64
gas_used: uint64
timestamp: uint64
extra_data: ByteList[MAX_EXTRA_DATA_BYTES]
base_fee_per_gas: uint256
# Extra payload fields
block_hash: Hash32 # Hash of execution block
transactions_root: Root
withdrawals_root: Root
blob_gas_used: uint64
excess_blob_gas: uint64
... # Additional features from the fork that introduces this EIP

class StableBeaconBlockBody(StableContainer[MAX_BEACON_BLOCK_BODY_FIELDS]):
randao_reveal: BLSSignature
eth1_data: Eth1Data # Eth1 data vote
graffiti: Bytes32 # Arbitrary data
# Operations
proposer_slashings: List[ProposerSlashing, MAX_PROPOSER_SLASHINGS]
attester_slashings: List[AttesterSlashing, MAX_ATTESTER_SLASHINGS]
attestations: List[Attestation, MAX_ATTESTATIONS]
deposits: List[Deposit, MAX_DEPOSITS]
voluntary_exits: List[SignedVoluntaryExit, MAX_VOLUNTARY_EXITS]
sync_aggregate: SyncAggregate
# Execution
execution_payload: StableExecutionPayload
bls_to_execution_changes: List[SignedBLSToExecutionChange, MAX_BLS_TO_EXECUTION_CHANGES]
blob_kzg_commitments: List[KZGCommitment, MAX_BLOB_COMMITMENTS_PER_BLOCK]
... # Additional features from the fork that introduces this EIP

class StableBeaconState(StableContainer[MAX_BEACON_STATE_FIELDS]):
# Versioning
genesis_time: uint64
genesis_validators_root: Root
slot: Slot
fork: Fork
# History
latest_block_header: BeaconBlockHeader
block_roots: Vector[Root, SLOTS_PER_HISTORICAL_ROOT]
state_roots: Vector[Root, SLOTS_PER_HISTORICAL_ROOT]
historical_roots: List[Root, HISTORICAL_ROOTS_LIMIT] # Frozen in Capella, replaced by historical_summaries
# Eth1
eth1_data: Eth1Data
eth1_data_votes: List[Eth1Data, EPOCHS_PER_ETH1_VOTING_PERIOD * SLOTS_PER_EPOCH]
eth1_deposit_index: uint64
# Registry
validators: List[Validator, VALIDATOR_REGISTRY_LIMIT]
balances: List[Gwei, VALIDATOR_REGISTRY_LIMIT]
# Randomness
randao_mixes: Vector[Bytes32, EPOCHS_PER_HISTORICAL_VECTOR]
# Slashings
slashings: Vector[Gwei, EPOCHS_PER_SLASHINGS_VECTOR] # Per-epoch sums of slashed effective balances
# Participation
previous_epoch_participation: List[ParticipationFlags, VALIDATOR_REGISTRY_LIMIT]
current_epoch_participation: List[ParticipationFlags, VALIDATOR_REGISTRY_LIMIT]
# Finality
justification_bits: Bitvector[JUSTIFICATION_BITS_LENGTH] # Bit set for every recent justified epoch
previous_justified_checkpoint: Checkpoint
current_justified_checkpoint: Checkpoint
finalized_checkpoint: Checkpoint
# Inactivity
inactivity_scores: List[uint64, VALIDATOR_REGISTRY_LIMIT]
# Sync
current_sync_committee: SyncCommittee
next_sync_committee: SyncCommittee
# Execution
latest_execution_payload_header: StableExecutionPayloadHeader
# Withdrawals
next_withdrawal_index: WithdrawalIndex
next_withdrawal_validator_index: ValidatorIndex
# Deep history valid from Capella onwards
historical_summaries: List[HistoricalSummary, HISTORICAL_ROOTS_LIMIT]
... # Additional features from the fork that introduces this EIP
```

### Fork-specific `Variant` definitions

These type definitions are specific to the fork that introduces this EIP. They are used in place of the legacy fork-specific `Container` definitions.

```python
class ExecutionPayload(Variant[StableExecutionPayload]):
# Execution block header fields
parent_hash: Hash32
fee_recipient: ExecutionAddress # 'beneficiary' in the yellow paper
state_root: Bytes32
receipts_root: Bytes32
logs_bloom: ByteVector[BYTES_PER_LOGS_BLOOM]
prev_randao: Bytes32 # 'difficulty' in the yellow paper
block_number: uint64 # 'number' in the yellow paper
gas_limit: uint64
gas_used: uint64
timestamp: uint64
extra_data: ByteList[MAX_EXTRA_DATA_BYTES]
base_fee_per_gas: uint256
# Extra payload fields
block_hash: Hash32 # Hash of execution block
transactions: List[Transaction, MAX_TRANSACTIONS_PER_PAYLOAD]
withdrawals: List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD]
blob_gas_used: uint64
excess_blob_gas: uint64
... # Additional features from the fork that introduces this EIP

class ExecutionPayloadHeader(Variant[StableExecutionPayloadHeader]):
# Execution block header fields
parent_hash: Hash32
fee_recipient: ExecutionAddress
state_root: Bytes32
receipts_root: Bytes32
logs_bloom: ByteVector[BYTES_PER_LOGS_BLOOM]
prev_randao: Bytes32
block_number: uint64
gas_limit: uint64
gas_used: uint64
timestamp: uint64
extra_data: ByteList[MAX_EXTRA_DATA_BYTES]
base_fee_per_gas: uint256
# Extra payload fields
block_hash: Hash32 # Hash of execution block
transactions_root: Root
withdrawals_root: Root
blob_gas_used: uint64
excess_blob_gas: uint64
... # Additional features from the fork that introduces this EIP

class BeaconBlockBody(Variant[StableBeaconBlockBody]):
randao_reveal: BLSSignature
eth1_data: Eth1Data # Eth1 data vote
graffiti: Bytes32 # Arbitrary data
# Operations
proposer_slashings: List[ProposerSlashing, MAX_PROPOSER_SLASHINGS]
attester_slashings: List[AttesterSlashing, MAX_ATTESTER_SLASHINGS]
attestations: List[Attestation, MAX_ATTESTATIONS]
deposits: List[Deposit, MAX_DEPOSITS]
voluntary_exits: List[SignedVoluntaryExit, MAX_VOLUNTARY_EXITS]
sync_aggregate: SyncAggregate
# Execution
execution_payload: ExecutionPayload
bls_to_execution_changes: List[SignedBLSToExecutionChange, MAX_BLS_TO_EXECUTION_CHANGES]
blob_kzg_commitments: List[KZGCommitment, MAX_BLOB_COMMITMENTS_PER_BLOCK]
... # Additional features from the fork that introduces this EIP

class BeaconState(Variant[StableBeaconState]):
# Versioning
genesis_time: uint64
genesis_validators_root: Root
slot: Slot
fork: Fork
# History
latest_block_header: BeaconBlockHeader
block_roots: Vector[Root, SLOTS_PER_HISTORICAL_ROOT]
state_roots: Vector[Root, SLOTS_PER_HISTORICAL_ROOT]
historical_roots: List[Root, HISTORICAL_ROOTS_LIMIT] # Frozen in Capella, replaced by historical_summaries
# Eth1
eth1_data: Eth1Data
eth1_data_votes: List[Eth1Data, EPOCHS_PER_ETH1_VOTING_PERIOD * SLOTS_PER_EPOCH]
eth1_deposit_index: uint64
# Registry
validators: List[Validator, VALIDATOR_REGISTRY_LIMIT]
balances: List[Gwei, VALIDATOR_REGISTRY_LIMIT]
# Randomness
randao_mixes: Vector[Bytes32, EPOCHS_PER_HISTORICAL_VECTOR]
# Slashings
slashings: Vector[Gwei, EPOCHS_PER_SLASHINGS_VECTOR] # Per-epoch sums of slashed effective balances
# Participation
previous_epoch_participation: List[ParticipationFlags, VALIDATOR_REGISTRY_LIMIT]
current_epoch_participation: List[ParticipationFlags, VALIDATOR_REGISTRY_LIMIT]
# Finality
justification_bits: Bitvector[JUSTIFICATION_BITS_LENGTH] # Bit set for every recent justified epoch
previous_justified_checkpoint: Checkpoint
current_justified_checkpoint: Checkpoint
finalized_checkpoint: Checkpoint
# Inactivity
inactivity_scores: List[uint64, VALIDATOR_REGISTRY_LIMIT]
# Sync
current_sync_committee: SyncCommittee
next_sync_committee: SyncCommittee
# Execution
latest_execution_payload_header: ExecutionPayloadHeader
# Withdrawals
next_withdrawal_index: WithdrawalIndex
next_withdrawal_validator_index: ValidatorIndex
# Deep history valid from Capella onwards
historical_summaries: List[HistoricalSummary, HISTORICAL_ROOTS_LIMIT]
... # Additional features from the fork that introduces this EIP
```

## Rationale

### Best timing?

Applying this EIP breaks `hash_tree_root` and Merkle tree verifiers a single time, while promising forward compatibility from the fork going forward. It is best to apply it before merkleization would be broken by different changes. Merkleization is broken by a `Container` reaching a new power of 2 in its number of fields.

### Can this be applied retroactively?

While `Variant` serializes in the same way as the legacy `Container`, the merkleization and `hash_tree_root` of affected data structures changes. Therefore, verifiers that wish to process Merkle proofs of legacy variants still need to support the corresponding legacy schemes.

## Backwards Compatibility

Existing Merkle proof verifiers need to be updated to support the new Merkle tree shape. This includes verifiers in smart contracts on different blockchains and verifiers in hardware wallets, if applicable.

Note that backwards compatibility is also broken when one of the converted `Container` data structures would reach a new power of 2 in its number of fields.

## Security Considerations

None

## Copyright

Copyright and related rights waived via [CC0](../LICENSE.md).

0 comments on commit a639237

Please sign in to comment.