feat: electra fork #6986
Merged
feat: electra fork #6986
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
g11tech
force-pushed
the
electra-fork-rebasejul30
branch
2 times, most recently
from
6e26706
to
db9bd27
Compare
Performance Report✔️ no performance regression detected Full benchmark results
|
g11tech
force-pushed
the
electra-fork-rebasejul30
branch
from
eb83cf7
to
1ffa8b5
Compare
wemeetagain
reviewed
Aug 1, 2024
| const cpEpoch = cp.epoch; | ||
| const electraEpoch = headState?.config.ELECTRA_FORK_EPOCH ?? Infinity; | ||
|
|
||
| if (headState === null) { |
There was a problem hiding this comment.
consider refactoring here. the head state should always be available here.
| @@ -40,7 +55,7 @@ export class PubkeyIndexMap { | |||
| return this.map.get(toMemoryEfficientHexStr(key)); | |||
| } | |||
|
|
|||
| set(key: Uint8Array, value: ValidatorIndex): void { | |||
| set(key: Uint8Array | PubkeyHex, value: ValidatorIndex): void { | |||
| @@ -8,6 +8,7 @@ export type BeaconStateAltair = CompositeViewDU<SSZTypesFor<ForkName.altair, "Be | |||
| export type BeaconStateBellatrix = CompositeViewDU<SSZTypesFor<ForkName.bellatrix, "BeaconState">>; | |||
| export type BeaconStateCapella = CompositeViewDU<SSZTypesFor<ForkName.capella, "BeaconState">>; | |||
| export type BeaconStateDeneb = CompositeViewDU<SSZTypesFor<ForkName.deneb, "BeaconState">>; | |||
| export type BeaconStateElectra = CompositeViewDU<SSZTypesFor<ForkName.electra, "BeaconState">>; | |||
There was a problem hiding this comment.
maybe make a TODO to clean up these types (like we've done with our basic forky ssz types)
This would involve having something like:
export type BeaconStateViewDU<F extends ForkName = ForkName> = CompositeViewDU<SSZTypesFor<F, "BeaconState">>as well as
export type CachedBeaconState<F extends ForkName = ForkName> = BeaconStateViewDU<F> & BeaconStateCache;
g11tech
force-pushed
the
electra-fork-rebasejul30
branch
from
91c6924
to
325086f
Compare
wemeetagain
reviewed
Aug 2, 2024
g11tech
force-pushed
the
electra-fork-rebasejul30
branch
3 times, most recently
from
f83cbf3
to
72b8074
Compare
ensi321
changed the title
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## unstable #6986 +/- ##
============================================
+ Coverage 49.24% 49.37% +0.12%
============================================
Files 578 589 +11
Lines 37443 39233 +1790
Branches 2168 2247 +79
============================================
+ Hits 18440 19371 +931
- Misses 18963 19821 +858
- Partials 40 41 +1 |
wemeetagain
added
the
status-do-not-merge
wemeetagain
changed the title
nflaig
reviewed
Aug 13, 2024
| @@ -39,6 +41,60 @@ export function computeActivationExitEpoch(epoch: Epoch): Epoch { | |||
| return epoch + 1 + MAX_SEED_LOOKAHEAD; | |||
| } | |||
|
|
|||
| export function computeExitEpochAndUpdateChurn(state: CachedBeaconStateElectra, exitBalance: Gwei): number { | |||
There was a problem hiding this comment.
might wanna move these functions to a different file as they don't really fit here. Maybe packages/state-transition/src/util/electra.ts?
| @@ -28,7 +28,8 @@ describe("epoch shufflings", () => { | |||
| id: `computeProposers - vc ${numValidators}`, | |||
| fn: () => { | |||
| const epochSeed = getSeed(state, state.epochCtx.nextShuffling.epoch, DOMAIN_BEACON_PROPOSER); | |||
| computeProposers(epochSeed, state.epochCtx.nextShuffling, state.epochCtx.effectiveBalanceIncrements); | |||
| const fork = state.config.getForkSeq(state.slot); | |||
There was a problem hiding this comment.
might wanna add state.fork, this pattern is quite frequent
| @@ -383,6 +402,13 @@ export const tekuHoleskyConfig = { | |||
| DOMAIN_AGGREGATE_AND_PROOF: "0x06000000", | |||
| CHURN_LIMIT_QUOTIENT: "65536", | |||
| BLS_WITHDRAWAL_PREFIX: "0x00", | |||
| MAX_EFFECTIVE_BALANCE_ELECTRA: "2048000000000", | |||
There was a problem hiding this comment.
are these updated based on what Teku actually returns or did we just add this to make tests pass? If those are not actual values from another client the interop config tests are pointless
nflaig
previously approved these changes
Aug 21, 2024
|
e2e consistently failed in https://github.com/ChainSafe/lodestar/actions/runs/10518956587/job/29145618824?pr=7043
one error I see in |
g11tech
force-pushed
the
electra-fork-rebasejul30
branch
from
1c73b3f
to
e35ff4a
Compare
nflaig
reviewed
Aug 23, 2024
There was a problem hiding this comment.
we should probably rename this file to processAttestation as it handles phase0 + electra attestations now
|
I deployed #7043 to beta, metrics look good to me, all are comparable to unstable |
add types stub and epoch config fix types
* Add immutable in the dependencies * Initial change to pubkeyCache * Added todos * Moved unfinalized cache to epochCache * Move populating finalized cache to afterProcessEpoch * Specify unfinalized cache during state cloning * Move from unfinalized to finalized cache in afterProcessEpoch * Confused myself * Clean up * Change logic * Fix cloning issue * Clean up redundant code * Add CarryoverData in epochCtx.createFromState * Fix typo * Update usage of pubkeyCache * Update pubkeyCache usage * Fix lint * Fix lint * Add 6110 to ChainConfig * Add 6110 to BeaconPreset * Define 6110 fork and container * Add V6110 api to execution engine * Update test * Add depositReceiptsRoot to process_execution_payload * State transitioning to EIP6110 * State transitioning to EIP6110 * Light client change in EIP-6110 * Update tests * produceBlock * Refactor processDeposit to match the spec * Implement processDepositReceipt * Implement 6110 fork guard for pubkeyCache * Handle changes in eth1 deposit * Update eth1 deposit test * Fix typo * Lint * Remove embarassing comments * Address comments * Modify applyDeposit signature * Update packages/state-transition/src/cache/epochCache.ts Co-authored-by: Lion - dapplion <[email protected]> * Update packages/state-transition/src/cache/epochCache.ts Co-authored-by: Lion - dapplion <[email protected]> * Update packages/state-transition/src/cache/pubkeyCache.ts Co-authored-by: Lion - dapplion <[email protected]> * Remove old code * Rename fields in epochCache and immutableData * Remove CarryoverData * Move isAfter6110 from var to method * Fix cyclic import * Fix operations spec runner * Fix for spec test * Fix spec test * state.depositReceiptsStartIndex to BigInt * getDeposit requires cached state * default depositReceiptsStartIndex value in genesis * Fix pubkeyCache bug * newUnfinalizedPubkeyIndexMap in createCachedBeaconState * Lint * Pass epochCache instead of pubkey2IndexFn in apis * Address comments * Add unit test on pubkey cache cloning * Add unfinalizedPubkeyCacheSize to metrics * Add unfinalizedPubkeyCacheSize to metrics * Clean up code * Add besu to el-interop * Add 6110 genesis file * Template for sim test * Add unit test for getEth1DepositCount * Update sim test * Update besudocker * Finish beacon api calls in sim test * Update epochCache.createFromState() * Fix bug unfinalized validators are not finalized * Add sim test to run a few blocks * Lint * Merge branch 'unstable' into 611 * Add more check to sim test * Update besu docker image instruction * Update sim test with correct tx * Address comment + cleanup * Clean up code * Properly handle promise rejection * Lint * Update packages/beacon-node/src/execution/engine/types.ts Co-authored-by: Lion - dapplion <[email protected]> * Update comments * Accept type undefined in ExecutionPayloadBodyRpc * Update comment and semantic * Remove if statement when adding finalized validator * Comment on repeated insert on finalized cache * rename createFromState * Add comment on getPubkey() * Stash change to reduce diffs * Stash change to reduce diffs * Lint * addFinalizedPubkey on finalized checkpoint * Update comment * Use OrderedMap for unfinalized cache * Pull out logic of deleting pubkeys for batch op * Add updateUnfinalizedPubkeys in regen * Update updateUnfinalizedPubkeys logic * Add comment * Add metrics for state context caches * Address comment * Address comment * Deprecate eth1Data polling when condition is reached * Fix conflicts * Fix sim test * Lint * Fix type * Fix test * Fix test * Lint * Update packages/light-client/src/spec/utils.ts Co-authored-by: Lion - dapplion <[email protected]> * Fix spec test * Address comments * Improve cache logic on checkpoint finalized * Update sim test according to new cache logic * Update comment * Lint * Finalized pubkey cache only update once per checkpoint * Add perf test for updateUnfinalizedPubkeys * Add perf test for updateUnfinalizedPubkeys * Tweak params for perf test * Freeze besu docker image version for 6110 * Add benchmark result * Use Map instead of OrderedMap. Update benchmark * Minor optimization * Minor optimization * Add memory test for immutable.js * Update test * Reduce code duplication * Lint * Remove try/catch in updateUnfinalizedPubkeys * Introduce EpochCache metric * Add historicalValidatorLengths * Polish code * Migrate state-transition unit tests to vitest * Fix calculation of pivot index * `historicalValidatorLengths` only activate post 6110 * Update sim test * Lint * Update packages/state-transition/src/cache/epochCache.ts Co-authored-by: Lion - dapplion <[email protected]> * Improve readability on historicalValidatorLengths * Update types * Fix calculation * Add eth1data poll todo * Add epochCache.getValidatorCountAtEpoch * Add todo * Add getStateIterator for state cache * Partial commit * Update perf test * updateUnfinalizedPubkeys directly modify states from regen * Update sim test. Lint * Add todo * some improvements and a fix for effectiveBalanceIncrements fork safeness * rename eip6110 to elctra * fix electra-interop.test.ts --------- Co-authored-by: Lion - dapplion <[email protected]> Co-authored-by: gajinder <[email protected]> lint and tsc small cleanup fix rebase issue
* fix: update cached balances in epoch transition * fix: more comments in processEffectiveBalanceUpdates()
* Initial commit * Update packages/beacon-node/src/chain/chain.ts Co-authored-by: Nico Flaig <[email protected]> * Update packages/beacon-node/src/api/impl/validator/index.ts Co-authored-by: Nico Flaig <[email protected]> * Update packages/beacon-node/src/chain/opPools/aggregatedAttestationPool.ts Co-authored-by: Nico Flaig <[email protected]> * address comment * Add unit test for attestation pool * fix: getSeenAttDataKey apis (#7009) * fix: getSeenAttDataKey apis * chore: use ForkName instead of ForkSeq * Update packages/beacon-node/src/util/sszBytes.ts Co-authored-by: Nico Flaig <[email protected]> * Update packages/beacon-node/src/chain/opPools/aggregatedAttestationPool.ts Co-authored-by: Nico Flaig <[email protected]> * address comment * Update error message * Update packages/beacon-node/src/chain/opPools/aggregatedAttestationPool.ts Co-authored-by: Nico Flaig <[email protected]> * address comment * Move determining post-electra fork out of loops --------- Co-authored-by: Nico Flaig <[email protected]> Co-authored-by: twoeths <[email protected]>
* fix: use attestation v1 endpoints pre-electra * Adapt block attestations error message with pool apis
* fix: work around for sliceFrom() api * Revert "fix: work around for sliceFrom() api" This reverts commit 7aa6678. * fix: upgrade ssz
g11tech
force-pushed
the
electra-fork-rebasejul30
branch
from
df82482
to
ed18ff0
Compare
nflaig
previously approved these changes
Aug 27, 2024
nflaig
removed
the
status-do-not-merge
wemeetagain
previously approved these changes
Aug 27, 2024
matthewkeil
force-pushed
the
electra-fork-rebasejul30
branch
2 times, most recently
from
aa653e2
to
ed18ff0
Compare
test: make import relative and add eslint-disable
nflaig
approved these changes
Aug 28, 2024
wemeetagain
approved these changes
Aug 28, 2024
|
🎉 This PR is included in v1.22.0 🎉 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
superseeding previous rebase :
superceeds :