Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(blockchain): introducing validator size cap size #2119

Merged
merged 102 commits into from
Dec 2, 2024
Merged
Show file tree
Hide file tree
Changes from 93 commits
Commits
Show all changes
102 commits
Select commit Hold shift + click to select a range
b4a1077
skip zero amount withdrawals to reduce number of withdrawals processed
abi87 Oct 28, 2024
f4a7d79
wip: adding UTs to state transition package
abi87 Oct 29, 2024
0f46172
wip: completed simple UT for state transition package
abi87 Oct 29, 2024
22c7717
wip: minimal execution engine stub
abi87 Oct 29, 2024
05cba80
extended asserts
abi87 Oct 29, 2024
443ac1b
added test case
abi87 Oct 29, 2024
10efd5e
nits
abi87 Oct 29, 2024
099716d
tests for helpers in state transition using mock
nidhi-singh02 Oct 30, 2024
151a533
Revert "tests for helpers in state transition using mock"
nidhi-singh02 Oct 30, 2024
9818c7e
tests with only mock for execution engine
nidhi-singh02 Oct 30, 2024
160cc88
removed test for VerifyAndNotifyNewPayload
nidhi-singh02 Oct 30, 2024
4a9fe1c
nit
abi87 Oct 31, 2024
e048be4
improved unit tests asserts
abi87 Oct 31, 2024
8bf34db
appease linter
abi87 Oct 31, 2024
d90a95a
fix(state-transition): fix deposit index upon genesis processing (#2116)
abi87 Oct 31, 2024
e17d29c
fixed bad merge
abi87 Oct 31, 2024
8dfe59d
Merge branch 'main' into fix-maximum-number-withdrawals
abi87 Nov 1, 2024
daa9262
Merge branch 'state-transition-add-UTs' into fix-maximum-number-withd…
abi87 Nov 1, 2024
92d494f
added UTs to show no withdrawals are made when unnecessary
abi87 Nov 1, 2024
6286b20
Merge branch 'main' into state-transition-add-UTs
abi87 Nov 1, 2024
a1a3def
Merge branch 'state-transition-add-UTs' into fix-maximum-number-withd…
abi87 Nov 1, 2024
b395ac2
added validator set cap size to chain specs
abi87 Nov 1, 2024
63e4db0
Merge branch 'fix-maximum-number-withdrawals' into validator-size-cap
abi87 Nov 1, 2024
36fe774
enforce validator set size cap on genesis
abi87 Nov 1, 2024
e64bc74
appease gosec
abi87 Nov 1, 2024
c6c68c3
mark extra validators as withdrawable
abi87 Nov 1, 2024
37542a0
nit
abi87 Nov 1, 2024
ea69a35
avoid double validators set cap checks on genesis
abi87 Nov 1, 2024
af43b93
add UT showing extra validator is added ready for withdrawal
abi87 Nov 1, 2024
a728bcb
extend UT to show extra validator deposit is withdrawn
abi87 Nov 1, 2024
97fba5c
fix withdrawal index update
abi87 Nov 1, 2024
826351d
Merge branch 'fix-maximum-number-withdrawals' into validator-size-cap
abi87 Nov 1, 2024
af8c5e0
fix(build): erigon repo
gummybera Nov 1, 2024
023ebfd
fix(build): bump erigon to recent version
gummybera Nov 1, 2024
d66b298
nits from code review
abi87 Nov 1, 2024
43b1eb4
Merge branch 'fix-erigon' into validator-size-cap
abi87 Nov 1, 2024
420f621
Merge branch 'state-transition-add-UTs' into fix-maximum-number-withd…
abi87 Nov 1, 2024
561f95f
Merge branch 'fix-maximum-number-withdrawals' into validator-size-cap
abi87 Nov 2, 2024
09eee8e
Merge branch 'fix-erigon' into fix-maximum-number-withdrawals
abi87 Nov 2, 2024
72539e0
Merge branch 'fix-maximum-number-withdrawals' into validator-size-cap
abi87 Nov 4, 2024
d8a267e
hacked fix for withdrawals SSZ serialization
abi87 Nov 4, 2024
3603f6b
Merge branch 'fix-maximum-number-withdrawals' into validator-size-cap
abi87 Nov 4, 2024
32fbbbc
nits
abi87 Nov 4, 2024
ebff958
nit
abi87 Nov 6, 2024
8c9cf2d
Merge branch 'main' into validator-size-cap
abi87 Nov 6, 2024
f51b157
renamed GetValidatorSetCapSize
abi87 Nov 6, 2024
5e38855
nits
abi87 Nov 6, 2024
bc26a85
wip: evicting smallest validator rathen than latest
abi87 Nov 6, 2024
401ab1a
improved UTs
abi87 Nov 6, 2024
9100d91
leftover
abi87 Nov 6, 2024
3699c20
Merge branch 'main' into validator-size-cap
abi87 Nov 6, 2024
9a7c057
consolidated UTs
abi87 Nov 7, 2024
0892421
minor test helpers renaming
abi87 Nov 7, 2024
1bd5f95
Merge branch 'main' into validator-size-cap
abi87 Nov 7, 2024
174229a
improving checks on validator set cap
abi87 Nov 7, 2024
3cca1f3
added processEffectiveBalanceUpdates
abi87 Nov 7, 2024
0ae0c7d
updated chain specs
abi87 Nov 7, 2024
3df7fc7
nits
abi87 Nov 7, 2024
4d3de74
nit
abi87 Nov 7, 2024
77f49c6
further UT extension
abi87 Nov 7, 2024
b96f776
fixed typo
abi87 Nov 7, 2024
882b91e
added processEffectiveBalanceUpdates
abi87 Nov 8, 2024
bca903b
Merge branch 'complete-validator-set-updates' into validator-size-cap
abi87 Nov 8, 2024
ebcf8b4
wip: update validator set returned to consensus
abi87 Nov 8, 2024
d4ec3e7
nit
abi87 Nov 8, 2024
3245f96
nit
abi87 Nov 8, 2024
bfca971
added cases for genesis initialization UTs
abi87 Nov 8, 2024
5d5a57d
Merge branch 'main' into complete-validator-set-updates
abi87 Nov 9, 2024
b898788
made process deposits closer to eth 2.0 specs
abi87 Nov 9, 2024
34ea814
nit
abi87 Nov 9, 2024
e2962ac
fixed expectations in UTs
abi87 Nov 9, 2024
0cdb569
fixed validators balance update
abi87 Nov 9, 2024
3f3d4eb
nits
abi87 Nov 9, 2024
3d9e3ab
fixed faulty assertion in UTs
abi87 Nov 9, 2024
0183914
added UT for validator creation
abi87 Nov 9, 2024
eda7909
minor processSyncCommitteeUpdates renaming
abi87 Nov 9, 2024
3e734e0
Merge branch 'complete-validator-set-updates' into validator-size-cap
abi87 Nov 9, 2024
c6b5586
send only validators set diffs
abi87 Nov 9, 2024
e9f01e2
minor UT code consolidation
abi87 Nov 10, 2024
3bc7678
appease linter
abi87 Nov 10, 2024
679da51
nits
abi87 Nov 10, 2024
b6d62c5
minor readme
abi87 Nov 10, 2024
f361fe4
Merge branch 'complete-validator-set-updates' into validator-set-upda…
abi87 Nov 10, 2024
260bddb
nit
abi87 Nov 10, 2024
f3c2b50
README nit
abi87 Nov 10, 2024
013aa95
Merge branch 'complete-validator-set-updates' into validator-size-cap
abi87 Nov 10, 2024
9b139f8
added UT for double eviction
abi87 Nov 10, 2024
0cbd981
fixed double evictions
abi87 Nov 10, 2024
52c4689
fixed eviction epoch
abi87 Nov 11, 2024
80af103
appease rabbit
abi87 Nov 11, 2024
976c27b
Merge branch 'validator-set-updates-optimizations' into validator-siz…
abi87 Nov 11, 2024
1cd96e8
cleanup UTs and improved asserts
abi87 Nov 11, 2024
e8db0d9
Merge branch 'main' into validator-size-cap
abi87 Nov 28, 2024
5ff81ac
Merge branch 'main' into validator-size-cap
abi87 Nov 28, 2024
4774160
Merge branch 'validator-size-cap' of https://github.com/berachain/bea…
abi87 Nov 29, 2024
78d6429
nit
abi87 Nov 29, 2024
e8b4dfd
Merge branch 'main' into validator-size-cap
abi87 Dec 2, 2024
4f90bfe
nit
abi87 Dec 2, 2024
f36db52
nit renaming + validation of valSetCap chain spec
calbera Dec 2, 2024
39b83ca
boonet TODOs for values
calbera Dec 2, 2024
355d99b
appease linter
abi87 Dec 2, 2024
0307a03
nit renaming error val set cap exceeded
calbera Dec 2, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions mod/chain-spec/pkg/chain/chain_spec.go
Original file line number Diff line number Diff line change
Expand Up @@ -199,6 +199,10 @@ type Spec[

// Berachain Values

// GetValidatorSetCap retrieves the maximum number of
// validators allowed in the active set.
GetValidatorSetCap() uint32
abi87 marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit can rename to ValidatorSetCap() for consistency with other value retrieval functions. Only the retrieval for the entire config of Comet GetCometBFTConfigForSlot has a Get prefix for some reason.


// EVMInflationAddress returns the address on the EVM which will receive
// the inflation amount of native EVM balance through a withdrawal every
// block.
Expand Down Expand Up @@ -534,6 +538,14 @@ func (c chainSpec[
return c.Data.CometValues
}

// GetValidatorSetCapS retrieves the maximum number of
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// GetValidatorSetCapS retrieves the maximum number of
// GetValidatorSetCap retrieves the maximum number of

// validators allowed in the active set.
func (c chainSpec[
DomainTypeT, EpochT, ExecutionAddressT, SlotT, CometBFTConfigT,
]) GetValidatorSetCap() uint32 {
return c.Data.ValidatorSetCap
}

// EVMInflationAddress returns the address on the EVM which will receive the
// inflation amount of native EVM balance through a withdrawal every block.
func (c chainSpec[
Expand Down
4 changes: 4 additions & 0 deletions mod/chain-spec/pkg/chain/data.go
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,10 @@ type SpecData[

// Berachain Values
//
// ValidatorSetCap is the maximum number of validators that can be active
// for a given epoch
// Note: ValidatorSetCap must be smaller than ValidatorRegistryLimit.
calbera marked this conversation as resolved.
Show resolved Hide resolved
ValidatorSetCap uint32 `mapstructure:"validator-set-cap-size"`
// EVMInflationAddress is the address on the EVM which will receive the
// inflation amount of native EVM balance through a withdrawal every block.
EVMInflationAddress ExecutionAddressT `mapstructure:"evm-inflation-address"`
Expand Down
3 changes: 3 additions & 0 deletions mod/config/pkg/spec/base.go
Original file line number Diff line number Diff line change
Expand Up @@ -123,5 +123,8 @@ func BaseSpec() chain.SpecData[
KZGCommitmentInclusionProofDepth: 17,
// Comet values.
CometValues: cmtConsensusParams,

// Berachain Values
ValidatorSetCap: 256,
calbera marked this conversation as resolved.
Show resolved Hide resolved
calbera marked this conversation as resolved.
Show resolved Hide resolved
}
}
5 changes: 5 additions & 0 deletions mod/consensus-types/pkg/types/validator.go
Original file line number Diff line number Diff line change
Expand Up @@ -325,6 +325,11 @@ func (v *Validator) SetEffectiveBalance(balance math.Gwei) {
v.EffectiveBalance = balance
}

// SetWithdrawableEpoch sets the epoch when the validator can withdraw.
func (v *Validator) SetWithdrawableEpoch(e math.Epoch) {
abi87 marked this conversation as resolved.
Show resolved Hide resolved
v.WithdrawableEpoch = e
}
calbera marked this conversation as resolved.
Show resolved Hide resolved

// GetWithdrawableEpoch returns the epoch when the validator can withdraw.
func (v Validator) GetWithdrawableEpoch() math.Epoch {
return v.WithdrawableEpoch
Expand Down
10 changes: 10 additions & 0 deletions mod/state-transition/pkg/core/core_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -301,3 +301,13 @@ func buildNextBlock(
Body: nextBlkBody,
}
}

var (
dummyExecutionPayload = &types.ExecutionPayload{
Timestamp: 0,
ExtraData: []byte("testing"),
Transactions: [][]byte{},
Withdrawals: []*engineprimitives.Withdrawal{},
BaseFeePerGas: math.NewU256(0),
}
abi87 marked this conversation as resolved.
Show resolved Hide resolved
)
abi87 marked this conversation as resolved.
Show resolved Hide resolved
4 changes: 4 additions & 0 deletions mod/state-transition/pkg/core/errors.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,10 @@ package core
import "github.com/berachain/beacon-kit/mod/errors"

var (
// ErrValidatorSetCapHit is returned when we try and add a validator
// that would breach validator set size cap.
ErrHitValidatorsSetCap = errors.New("hit validator set size cap")
calbera marked this conversation as resolved.
Show resolved Hide resolved

// ErrBlockSlotTooLow is returned when the block slot is too low.
ErrBlockSlotTooLow = errors.New("block slot too low")

Expand Down
120 changes: 119 additions & 1 deletion mod/state-transition/pkg/core/state_processor_staking.go
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,9 @@
package core

import (
"bytes"
"slices"

"github.com/berachain/beacon-kit/mod/config/pkg/spec"
"github.com/berachain/beacon-kit/mod/errors"
"github.com/berachain/beacon-kit/mod/primitives/pkg/common"
Expand Down Expand Up @@ -154,6 +157,121 @@ func (sp *StateProcessor[
math.Gwei(sp.cs.MaxEffectiveBalance()),
)

// BeaconKit enforces a cap on the validator set size. If the deposit
// breaches the cap, we find the validator with the smallest stake and
// mark it as withdrawable so that it will be evicted next epoch and
// its deposits returned.

nextEpochVals, err := sp.nextEpochValidatorSet(st)
if err != nil {
return err
}
//#nosec:G701 // no overflow risk here
if uint32(len(nextEpochVals)) < sp.cs.GetValidatorSetCap() {
abi87 marked this conversation as resolved.
Show resolved Hide resolved
// cap not hit, just add the validator
return sp.addValidatorInternal(st, val, dep.GetAmount())
}

// Adding the validator would breach the cap. Find the validator
// with the smallest stake among current and candidate validators
// and kick it out.
calbera marked this conversation as resolved.
Show resolved Hide resolved
smallestVal, err := sp.smallest(nextEpochVals)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: rename to lowestStakeVal

if err != nil {
return err
}

slot, err := st.GetSlot()
if err != nil {
return err
}
nextEpoch := sp.cs.SlotToEpoch(slot) + 1

if val.GetEffectiveBalance() <= smallestVal.GetEffectiveBalance() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: rename val to newVal

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or candidateVal

// in case of tie-break among candidate validator we prefer
// existing one so we mark candidate as withdrawable
// We wait next epoch to return funds, as a way to curb spamming
val.SetWithdrawableEpoch(nextEpoch)
return sp.addValidatorInternal(st, val, dep.GetAmount())
}

// mark exiting validator for eviction and add candidate
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// mark exiting validator for eviction and add candidate
// mark existing validator for eviction and add candidate

smallestVal.SetWithdrawableEpoch(nextEpoch)
idx, err := st.ValidatorIndexByPubkey(smallestVal.GetPubkey())
if err != nil {
return err
}
if err = st.UpdateValidatorAtIndex(idx, smallestVal); err != nil {
return err
}
return sp.addValidatorInternal(st, val, dep.GetAmount())
}

// nextEpochValidatorSet returns the current estimation of what next epoch
// validator set would be.
func (sp *StateProcessor[
_, _, _, BeaconStateT, _, _, _, _, _, _, _, _, ValidatorT, _, _, _, _,
]) nextEpochValidatorSet(st BeaconStateT) ([]ValidatorT, error) {
slot, err := st.GetSlot()
if err != nil {
return nil, err
}
withdrawEpoch := sp.cs.SlotToEpoch(slot) + 1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: rename to nextEpoch


vals, err := st.GetValidators()
if err != nil {
return nil, err
}
activeVals := make([]ValidatorT, 0, len(vals))
for _, val := range vals {
if val.GetEffectiveBalance() <= math.U64(sp.cs.EjectionBalance()) {
continue
}
if val.GetWithdrawableEpoch() == withdrawEpoch {
continue
}
activeVals = append(activeVals, val)
}

return activeVals, nil
}

// TODO: consider moving this to BeaconState directly
func (sp *StateProcessor[
_, _, _, _, _, _, _, _, _, _, _, _, ValidatorT, _, _, _, _,
]) smallest(currentVals []ValidatorT) (
Copy link
Contributor

@calbera calbera Nov 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rename to lowestStakeVal(vals []ValidatorT) or smth similar and keep as helper (no receiver StateProcessor)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tricky to get rid of the receiver (which I agree is not needed) and get the typing right. We have other methods here that could be free functions and are not for this reason.
I would rather defer everything to a general cleanup of the typing system

ValidatorT,
error,
) {
// TODO: consider heapifying slice instead. We only care about the smallest
abi87 marked this conversation as resolved.
Show resolved Hide resolved
slices.SortFunc(currentVals, func(lhs, rhs ValidatorT) int {
var (
val1Stake = lhs.GetEffectiveBalance()
val2Stake = rhs.GetEffectiveBalance()
)
switch {
case val1Stake < val2Stake:
return -1
case val1Stake > val2Stake:
return 1
default:
// validators pks are guaranteed to be different
var (
val1Pk = lhs.GetPubkey()
val2Pk = rhs.GetPubkey()
)
return bytes.Compare(val1Pk[:], val2Pk[:])
}
})
return currentVals[0], nil
}

func (sp *StateProcessor[
_, _, _, BeaconStateT, _, _, _, _, _, _, _, _, ValidatorT, _, _, _, _,
]) addValidatorInternal(
st BeaconStateT,
val ValidatorT,
depositAmount math.Gwei,
) error {
// TODO: This is a bug that lives on bArtio. Delete this eventually.
if sp.cs.DepositEth1ChainID() == spec.BartioChainID {
// Note in AddValidatorBartio we implicitly increase
Expand All @@ -168,5 +286,5 @@ func (sp *StateProcessor[
if err != nil {
return err
}
return st.IncreaseBalance(idx, dep.GetAmount())
return st.IncreaseBalance(idx, depositAmount)
}
Loading
Loading