Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: implement slashing functionality on the provider chain (ADR-013) #1275

Merged
merged 31 commits into from
Sep 27, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
93a0db5
first version
insumity Sep 7, 2023
ea87bdc
fix mocks
insumity Sep 8, 2023
177e1db
move slashing to other file and check tombstoning
insumity Sep 8, 2023
e1bdfff
add tests that checks that Slash is called
insumity Sep 11, 2023
bbff2ee
add FIXME msg
insumity Sep 11, 2023
dfef410
small changes
insumity Sep 11, 2023
8b49af1
add fixme
insumity Sep 11, 2023
bf89509
fix test
insumity Sep 11, 2023
868e0d2
modified E2E tests and general cleaning up
insumity Sep 12, 2023
9ba3a3d
clean up
insumity Sep 12, 2023
b1a6f31
feat!: Cryptographic verification of equivocation (#1287)
mpoke Sep 14, 2023
d92a4c8
undelegations are getting slashed integraiton test
insumity Sep 14, 2023
72abcf4
Merge branch 'release/v2.1.x-lsm' into insumity/adr-013-impl-on-feat
insumity Sep 14, 2023
b46ace4
fix merge issues
insumity Sep 14, 2023
59d8192
fix mocks
insumity Sep 14, 2023
31c090d
fix lint issue
insumity Sep 14, 2023
79f8f18
Merge branch 'feat/ics-misbehaviour-handling' into insumity/adr-013-i…
insumity Sep 22, 2023
ed35638
took into account Simon's comments
insumity Sep 22, 2023
62212c7
go.sum changes
insumity Sep 22, 2023
e5b46dd
gosec fix
insumity Sep 22, 2023
4cb6283
fix linter issue
insumity Sep 22, 2023
6f15a61
cherry-picked ADR-05 so markdown link checker does not complain
insumity Sep 22, 2023
93c2cdb
lint
sainoe Sep 22, 2023
1d3ee78
Use cached context to get tokens in undelegations and redelegations.
insumity Sep 25, 2023
5c053bb
return the error
insumity Sep 26, 2023
f8633b5
lint issue
insumity Sep 26, 2023
f96f1bc
take into account Philip's comments
insumity Sep 26, 2023
d3dc0bc
clean up
insumity Sep 26, 2023
39bbcf5
fix flakey test
insumity Sep 27, 2023
694dce7
lint issue
insumity Sep 27, 2023
366e3b1
fix error returns and fix flaky test in a better way
insumity Sep 27, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
105 changes: 105 additions & 0 deletions docs/docs/adrs/adr-005-cryptographic-equivocation-verification.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
---
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cherry picked this file so markdown link checker doesn't complain.

sidebar_position: 4
title: Cryptographic verification of equivocation evidence
---
# ADR 005: Cryptographic verification of equivocation evidence

## Changelog
* 5/1/2023: First draft
* 7/23/23: Add light client attacks handling

## Status

Accepted

## Context

Currently, we use a governance proposal to slash validators for equivocation (double signing and light client attacks).
Every proposal needs to go through a (two weeks) voting period before it can be approved.
Given a three-week unbonding period, this means that an equivocation proposal needs to be submitted within one week since the infraction occurred.

This ADR proposes a system to slash validators automatically for equivocation, immediately upon the provider chain's receipt of the evidence. Another thing to note is that we intend to introduce this system in stages, since even the partial ability to slash and/or tombstone is a strict improvement in security.
For the first stage of this work, we will only handle light client attacks.

### Light Client Attack

In a nutshell, the light client is a process that solely verifies a specific state machine's
consensus without executing the transactions. The light clients get new headers by querying
multiple nodes, called primary and witness nodes.

Light clients download new headers committed on chain from a primary. Headers can be verified in two ways: sequentially,
where the block height of headers is serial, or using skipping. This second verification method allows light clients to download headers
with nonconsecutive block height, where some intermediate headers are skipped (see [Tendermint Light Client, Figure 1 and Figure 3](https://arxiv.org/pdf/2010.07031.pdf)).
Additionally, light clients are cross-checking new headers obtained from a primary with witnesses to ensure all nodes share the same state.

A light client attack occurs when a Byzantine validator sends invalid headers to a light client.
As the light client doesn't execute transactions, it can be deceived into trusting corrupted application state transitions.
For instance, if a light client receives header `A` from the primary and header `B` from a witness for the same block height `H`,
and both headers are successfully verified, it indicates a light client attack.
Note that in this case, either the primary or the witness or both are malicious.

The types of light client attacks are defined by analyzing the differences between the conflicting headers.
There are three types of light client attacks: lunatic attack, equivocation attack, and amnesia attack.
For details, see the [CometBFT specification](https://github.com/cometbft/cometbft/blob/main/spec/light-client/attacks/notes-on-evidence-handling.md#evidence-handling).

When a light client agent detects two conflicting headers, it will initially verify their traces (see [cometBFT detector](https://github.com/cometbft/cometbft/blob/2af25aea6cfe6ac4ddac40ceddfb8c8eee17d0e6/light/detector.go#L28)) using its primary and witness nodes.
If these headers pass successful verification, the Byzantine validators will be identified based on the header's commit signatures
and the type of light client attack. The agent will then transmit this information to its nodes using a [`LightClientAttackEvidence`](https://github.com/cometbft/cometbft/blob/feed0ddf564e113a840c4678505601256b93a8bc/docs/architecture/adr-047-handling-evidence-from-light-client.md) to be eventually voted on and added to a block.
Note that from a light client agent perspective, it is not possible to establish whether a primary or a witness node, or both, are malicious.
Therefore, it will create and send two `LightClientAttackEvidence`: one against the primary (sent to the witness), and one against the witness (sent to the primary).
Both nodes will then verify it before broadcasting it and adding it to the [evidence pool](https://github.com/cometbft/cometbft/blob/2af25aea6cfe6ac4ddac40ceddfb8c8eee17d0e6/evidence/pool.go#L28).
If a `LightClientAttackEvidence` is finally committed to a block, the chain's evidence module will execute it, resulting in the jailing and the slashing of the validators responsible for the light client attack.


Light clients are a core component of IBC. In the event of a light client attack, IBC relayers notify the affected chains by submitting an [IBC misbehavior message](https://github.com/cosmos/ibc-go/blob/2b7c969066fbcb18f90c7f5bd256439ca12535c7/proto/ibc/lightclients/tendermint/v1/tendermint.proto#L79).
A misbehavior message includes the conflicting headers that constitute a `LightClientAttackEvidence`. Upon receiving such a message,
a chain will first verify whether these headers would have convinced its light client. This verification is achieved by checking
the header states against the light client consensus states (see [IBC misbehaviour handler](https://github.com/cosmos/ibc-go/blob/2b7c969066fbcb18f90c7f5bd256439ca12535c7/modules/light-clients/07-tendermint/types/misbehaviour_handle.go#L101)). If the misbehaviour is successfully verified, the chain will then "freeze" the
light client, halting any further trust in or updating of its states.


## Decision

In the first iteration of the feature, we will introduce a new endpoint: `HandleConsumerMisbehaviour(ctx sdk.Context, misbehaviour ibctmtypes.Misbehaviour)`.
The main idea is to leverage the current IBC misbehaviour handling and update it to solely jail and slash the validators that
performed a light client attack. This update will be made under the assumption that the chain connected via this light client
share the same validator set, as it is the case with Replicated Security.

This endpoint will reuse the IBC client libraries to verify that the misbehaviour headers would have fooled the light client.
Additionally, it’s crucial that the endpoint logic result in the slashing and jailing of validators under the same conditions
as a light client agent detector. Therefore, the endpoint will ensure that the two conditions are met:
the headers in the misbehaviour message have the same block height, and
the light client isn’t expired.

After having successfully verified a misbehaviour, the endpoint will execute the jailing and slashing of the malicious validators similarly as in the evidence module.

### Current limitations:

- This only handles light client attacks, not double signing. In the future, we will add the code to also verify double signing.

- We cannot derive an infraction height from the evidence, so it is only possible to tombstone validators, not actually slash them.
To explain the technical reasons behind this limitation, let's recap the initial consumer initiated slashing logic.
In a nutshell, consumer heights are mapped to provider heights through VSCPackets, namely through the so called vscIDs.
When an infraction occurs on the consumer, a SlashPacket containing the vscID obtained from mapping the consumer infraction height
is sent to the provider. Upon receiving the packet, the provider maps the consumer infraction height to a local infraction height,
which is used to slash the misbehaving validator. In the context of untrusted consumer chains, all their states, including vscIDs,
could be corrupted and therefore cannot be used for slashing purposes.

- Currently, the endpoint can only handle "equivocation" light client attacks. This is because the "lunatic" attacks require the endpoint to possess the ability to dissociate which header is conflicted or trusted upon receiving a misbehavior message. Without this information, it's not possible to define the Byzantine validators from the conflicting headers (see [comment](https://github.com/cosmos/interchain-security/pull/826#discussion_r1268668684)).


## Consequences

### Positive

- After this ADR is applied, it will be possible for the provider chain to tombstone validators who committed a light client attack.

### Negative

- N/A


## References

* [ICS misbehaviour handling PR](https://github.com/cosmos/interchain-security/pull/826)
* [Architectural diagrams](https://docs.google.com/document/d/1fe1uSJl1ZIYWXoME3Yf4Aodvz7V597Ric875JH-rigM/edit#heading=h.rv4t8i6d6jfn)
10 changes: 10 additions & 0 deletions tests/e2e/steps_consumer_misbehaviour.go
Original file line number Diff line number Diff line change
Expand Up @@ -230,6 +230,10 @@ func stepsCauseConsumerMisbehaviour(consumerName string) []Step {
validatorID("alice"): 511,
validatorID("bob"): 20,
},
RepresentativePowers: &map[validatorID]uint{
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using RepresentativePowers was the only way I found to verify that slashing indeed takes place.
Using ValPowers was not useful because after a validator gets jailed, the validator's voting power is 0. I also could not unjail and then check ValPowers similarly to what is done in steps_multi_consumer_downtime.go because in contrast to a downtime test, for misbehaviour & equivocation we also we also tombstone the validator.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems ok to me. Could rename it to something like StakedTokens instead of RepresentativePowers (looking at the code, I think that's what this actually is, right?)

validatorID("alice"): 511000000,
validatorID("bob"): 20000000,
},
},
chainID(consumerName): ChainState{
ValPowers: &map[validatorID]uint{
Expand All @@ -255,6 +259,12 @@ func stepsCauseConsumerMisbehaviour(consumerName string) []Step {
validatorID("alice"): 0,
validatorID("bob"): 20,
},
// "alice" should be slashed on the provider, hence representative
// power is 511000000 - 0.05 * 511000000 = 485450000
RepresentativePowers: &map[validatorID]uint{
validatorID("alice"): 485450000,
validatorID("bob"): 20000000,
},
// The consumer light client should be frozen on the provider
ClientsFrozenHeights: &map[string]clienttypes.Height{
consumerClientID: {
Expand Down
19 changes: 18 additions & 1 deletion tests/e2e/steps_double_sign.go
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,11 @@ func stepsCauseDoubleSignOnConsumer(consumerName, providerName string) []Step {
validatorID("bob"): 500,
validatorID("carol"): 500,
},
RepresentativePowers: &map[validatorID]uint{
validatorID("alice"): 500000000,
validatorID("bob"): 500000000,
validatorID("carol"): 500000000,
},
},
chainID(consumerName): ChainState{
ValPowers: &map[validatorID]uint{
Expand All @@ -155,7 +160,7 @@ func stepsCauseDoubleSignOnConsumer(consumerName, providerName string) []Step {
},
},
// detect the double voting infraction
// and jail bob on the provider
// and jail and slashing of bob on the provider
{
action: startConsumerEvidenceDetectorAction{
chain: chainID(consumerName),
Expand All @@ -167,6 +172,13 @@ func stepsCauseDoubleSignOnConsumer(consumerName, providerName string) []Step {
validatorID("bob"): 0,
validatorID("carol"): 500,
},
// "bob" gets slashed on the provider chain, hence representative
// power is 500000000 - 0.05 * 500000000 = 475000000
RepresentativePowers: &map[validatorID]uint{
validatorID("alice"): 500000000,
validatorID("bob"): 475000000,
validatorID("carol"): 500000000,
},
},
chainID(consumerName): ChainState{
ValPowers: &map[validatorID]uint{
Expand All @@ -192,6 +204,11 @@ func stepsCauseDoubleSignOnConsumer(consumerName, providerName string) []Step {
validatorID("bob"): 0,
validatorID("carol"): 500,
},
RepresentativePowers: &map[validatorID]uint{
validatorID("alice"): 500000000,
validatorID("bob"): 475000000,
validatorID("carol"): 500000000,
},
},
chainID(consumerName): ChainState{
ValPowers: &map[validatorID]uint{
Expand Down
134 changes: 126 additions & 8 deletions tests/integration/double_vote.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import (
)

// TestHandleConsumerDoubleVoting verifies that handling a double voting evidence
// of a consumer chain results in the expected jailing of the malicious validator
// of a consumer chain results in the expected tombstoning and jailing the misbehaved validator
func (s *CCVTestSuite) TestHandleConsumerDoubleVoting() {
s.SetupCCVChannel(s.path)
// required to have the consumer client revision height greater than 0
Expand All @@ -24,11 +24,11 @@ func (s *CCVTestSuite) TestHandleConsumerDoubleVoting() {
consuValSet, err := tmtypes.ValidatorSetFromProto(s.consumerChain.LastHeader.ValidatorSet)
s.Require().NoError(err)
consuVal := consuValSet.Validators[0]
s.Require().NoError(err)
consuSigner := s.consumerChain.Signers[consuVal.Address.String()]

provValSet, err := tmtypes.ValidatorSetFromProto(s.providerChain.LastHeader.ValidatorSet)
s.Require().NoError(err)

provVal := provValSet.Validators[0]
provSigner := s.providerChain.Signers[provVal.Address.String()]

Expand Down Expand Up @@ -156,13 +156,16 @@ func (s *CCVTestSuite) TestHandleConsumerDoubleVoting() {
},
}

consuAddr := types.NewConsumerConsAddress(sdk.ConsAddress(consuVal.Address.Bytes()))
provAddr := s.providerApp.GetProviderKeeper().GetProviderAddrFromConsumerAddr(s.providerCtx(), s.consumerChain.ChainID, consuAddr)

for _, tc := range testCases {
s.Run(tc.name, func() {
consuAddr := types.NewConsumerConsAddress(sdk.ConsAddress(tc.ev.VoteA.ValidatorAddress.Bytes()))
provAddr := s.providerApp.GetProviderKeeper().GetProviderAddrFromConsumerAddr(s.providerCtx(), s.consumerChain.ChainID, consuAddr)

validator, _ := s.providerApp.GetTestStakingKeeper().GetValidator(s.providerCtx(), provAddr.ToSdkConsAddr().Bytes())
initialTokens := validator.GetTokens().ToDec()

// reset context for each run
provCtx := s.providerCtx()
provCtx, _ := s.providerCtx().CacheContext()

// if the evidence was built using the validator provider address and key,
// we remove the consumer key assigned to the validator otherwise
Expand All @@ -185,14 +188,129 @@ func (s *CCVTestSuite) TestHandleConsumerDoubleVoting() {
if tc.expPass {
s.Require().NoError(err)

// verifies that the jailing has occurred
// verifies that the jailing and tombstoning has occurred
s.Require().True(s.providerApp.GetTestStakingKeeper().IsValidatorJailed(provCtx, provAddr.ToSdkConsAddr()))
s.Require().True(s.providerApp.GetTestSlashingKeeper().IsTombstoned(provCtx, provAddr.ToSdkConsAddr()))

// verifies that the val gets slashed and has fewer tokens after the slashing
val, _ := s.providerApp.GetTestStakingKeeper().GetValidator(provCtx, provAddr.ToSdkConsAddr().Bytes())
slashFraction := s.providerApp.GetTestSlashingKeeper().SlashFractionDoubleSign(provCtx)
actualTokens := val.GetTokens().ToDec()
s.Require().True(initialTokens.Sub(initialTokens.Mul(slashFraction)).Equal(actualTokens))
} else {
s.Require().Error(err)

// verifies that no jailing and has occurred
// verifies that no jailing and no tombstoning has occurred
s.Require().False(s.providerApp.GetTestStakingKeeper().IsValidatorJailed(provCtx, provAddr.ToSdkConsAddr()))
s.Require().False(s.providerApp.GetTestSlashingKeeper().IsTombstoned(provCtx, provAddr.ToSdkConsAddr()))
}
})
}
}

// TestHandleConsumerDoubleVotingSlashesUndelegations verifies that handling a successful double voting
// evidence of a consumer chain results in the expected slashing of the misbehave validator undelegations
func (s *CCVTestSuite) TestHandleConsumerDoubleVotingSlashesUndelegations() {
s.SetupCCVChannel(s.path)
// required to have the consumer client revision height greater than 0
s.SendEmptyVSCPacket()

// create signing info for all validators
for _, v := range s.providerChain.Vals.Validators {
s.setDefaultValSigningInfo(*v)
}

consuValSet, err := tmtypes.ValidatorSetFromProto(s.consumerChain.LastHeader.ValidatorSet)
s.Require().NoError(err)
consuVal := consuValSet.Validators[0]
consuSigner := s.consumerChain.Signers[consuVal.Address.String()]

blockID1 := testutil.MakeBlockID([]byte("blockhash"), 1000, []byte("partshash"))
blockID2 := testutil.MakeBlockID([]byte("blockhash2"), 1000, []byte("partshash"))

// create two votes using the consumer validator key
consuVote := testutil.MakeAndSignVote(
blockID1,
s.consumerCtx().BlockHeight(),
s.consumerCtx().BlockTime(),
consuValSet,
consuSigner,
s.consumerChain.ChainID,
)

consuBadVote := testutil.MakeAndSignVote(
blockID2,
s.consumerCtx().BlockHeight(),
s.consumerCtx().BlockTime(),
consuValSet,
consuSigner,
s.consumerChain.ChainID,
)

// In order to create an evidence for a consumer chain,
// we create two votes that only differ by their Block IDs and
// signed them using the same validator private key and chain ID
// of the consumer chain
evidence := &tmtypes.DuplicateVoteEvidence{
VoteA: consuVote,
VoteB: consuBadVote,
ValidatorPower: consuVal.VotingPower,
TotalVotingPower: consuVal.VotingPower,
Timestamp: s.consumerCtx().BlockTime(),
}

chainID := s.consumerChain.ChainID
pubKey := consuVal.PubKey

consuAddr := types.NewConsumerConsAddress(sdk.ConsAddress(consuVal.Address.Bytes()))
provAddr := s.providerApp.GetProviderKeeper().GetProviderAddrFromConsumerAddr(s.providerCtx(), s.consumerChain.ChainID, consuAddr)

validator, found := s.providerApp.GetTestStakingKeeper().GetValidator(s.providerCtx(), provAddr.ToSdkConsAddr().Bytes())
s.Require().True(found)

s.Run("slash undelegations when getting double voting evidence", func() {
// convert validator public key
pk, err := cryptocodec.FromTmPubKeyInterface(pubKey)
s.Require().NoError(err)

// perform a delegation and an undelegation of the whole amount
bondAmt := sdk.NewInt(10000000)
delAddr := s.providerChain.SenderAccount.GetAddress()

// in order to perform a delegation we need to know the validator's `idx` (that might not be 0)
// loop through all validators to find the right `idx`
idx := 0
for i := 0; i <= len(s.providerChain.Vals.Validators); i++ {
_, valAddr := s.getValByIdx(i)
if validator.OperatorAddress == valAddr.String() {
idx = i
break
}
}

_, shares, valAddr := delegateByIdx(s, delAddr, bondAmt, idx)
_ = undelegate(s, delAddr, valAddr, shares)

_, shares, _ = delegateByIdx(s, delAddr, sdk.NewInt(50000000), idx)
_ = undelegate(s, delAddr, valAddr, shares)

err = s.providerApp.GetProviderKeeper().HandleConsumerDoubleVoting(
s.providerCtx(),
evidence,
chainID,
pk,
)
s.Require().NoError(err)

slashFraction := s.providerApp.GetTestSlashingKeeper().SlashFractionDoubleSign(s.providerCtx())

// check undelegations are slashed
ubds, _ := s.providerApp.GetTestStakingKeeper().GetUnbondingDelegation(s.providerCtx(), delAddr, validator.GetOperator())
s.Require().True(len(ubds.Entries) > 0)
for _, unb := range ubds.Entries {
initialBalance := unb.InitialBalance.ToDec()
currentBalance := unb.Balance.ToDec()
s.Require().True(initialBalance.Sub(initialBalance.Mul(slashFraction)).Equal(currentBalance))
}
})
}
12 changes: 11 additions & 1 deletion tests/integration/misbehaviour.go
Original file line number Diff line number Diff line change
Expand Up @@ -53,16 +53,26 @@ func (s *CCVTestSuite) TestHandleConsumerMisbehaviour() {
),
}

// we assume that all validators have the same number of initial tokens
validator, _ := s.getValByIdx(0)
initialTokens := validator.GetTokens().ToDec()

err := s.providerApp.GetProviderKeeper().HandleConsumerMisbehaviour(s.providerCtx(), *misb)
s.NoError(err)

// verify that validators are jailed and tombstoned
// verify that validators are jailed, tombstoned, and slashed
for _, v := range clientTMValset.Validators {
consuAddr := sdk.ConsAddress(v.Address.Bytes())
provAddr := s.providerApp.GetProviderKeeper().GetProviderAddrFromConsumerAddr(s.providerCtx(), s.consumerChain.ChainID, types.NewConsumerConsAddress(consuAddr))
val, ok := s.providerApp.GetTestStakingKeeper().GetValidatorByConsAddr(s.providerCtx(), provAddr.Address)
s.Require().True(ok)
s.Require().True(val.Jailed)
s.Require().True(s.providerApp.GetTestSlashingKeeper().IsTombstoned(s.providerCtx(), provAddr.ToSdkConsAddr()))

validator, _ := s.providerApp.GetTestStakingKeeper().GetValidator(s.providerCtx(), provAddr.ToSdkConsAddr().Bytes())
slashFraction := s.providerApp.GetTestSlashingKeeper().SlashFractionDoubleSign(s.providerCtx())
actualTokens := validator.GetTokens().ToDec()
s.Require().True(initialTokens.Sub(initialTokens.Mul(slashFraction)).Equal(actualTokens))
}
}

Expand Down
Loading
Loading