Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prune EthereumSignatures and EthereumEventVoteRecords #517

Merged
merged 85 commits into from
Aug 10, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
85 commits
Select commit Hold shift + click to select a range
203b02f
Prune EthereumSignatures and EthereumEventVoteRecords
cbrit Apr 26, 2023
3b7e663
Prune signatures for all OutgoingTx types on tx deletion
cbrit Apr 27, 2023
9e7219b
Update event claim unit test to expect pruning
cbrit Apr 27, 2023
6e6d225
Change recordEventVote to only record for new events. Still updates s…
cbrit Apr 27, 2023
9576296
Review changes
cbrit May 9, 2023
5f290f5
Delete signature before outgoing tx
cbrit May 11, 2023
61c2f84
Addition of CompletedOutgoingTx, and windows for persisting event vot…
cbrit May 16, 2023
798bb28
Add queries for ethereum events and signatures, adjust pruning
cbrit May 18, 2023
8eac7c9
Remove TxHistoryWindow param recently added
cbrit May 18, 2023
3384780
Change stale signer set pruning to pattern used for other OutgoingTx
cbrit May 18, 2023
d879f93
Use completed outgoing tx store data for slashing, add pruning after …
cbrit May 18, 2023
016b9d8
Undo that signer set pruning rework because it broke everything
cbrit May 19, 2023
b59b95f
Set completed outgoing tx for observed signer set updates
cbrit May 19, 2023
18297d3
Fix slashing and pruning, conditionally add signer txs for to Complet…
cbrit May 22, 2023
97a7cfc
Ignore unit tests in /migrations in test-cov recipe
cbrit May 29, 2023
f11ae2f
Merge branch 'main' into collin/pruning
cbrit May 29, 2023
9b2fa69
Fix bugs found while getting validator_out working, test tweaks
cbrit May 30, 2023
7774350
Add completed signer set txs to CompletedOutgoingTxs query
cbrit May 30, 2023
93898a6
Tweaks around pruning. Comment for important param context
cbrit May 30, 2023
cee01a5
Happy path improvements
cbrit May 30, 2023
ad2e1ed
Bump target tx timeout so that transaction stress can pass
cbrit May 30, 2023
1a60aa4
Check pruning in batch/contract call slashing unit tests
cbrit May 30, 2023
c9e7b46
Check if validator is jailed before jailing to avoid panic
cbrit May 30, 2023
d094333
Fix double jailing, combine batch/contract call slashing and pruning …
cbrit May 30, 2023
10ba8e8
Merge branch 'main' into collin/pruning
cbrit May 30, 2023
62052b0
Remove coverage.txt from index
cbrit May 31, 2023
8aec8a4
Include CompletedOutgoingTxs in unsigned tx queries, + unit tests
cbrit May 31, 2023
e612af8
Tweak comment for pruning method
cbrit Jun 1, 2023
944db59
Refactor validator_out test jailing check
cbrit Jun 2, 2023
fa739d7
Review items - queries and signature -> confirmation term consistency
cbrit Jun 5, 2023
2b4f813
Refactor slashing
cbrit Jun 5, 2023
bddc200
Slash based on OutgoingTx (not completed), and new slashing window param
cbrit Jun 5, 2023
45f3b66
Slash for both Outgoing and Completed, make the Outgoing->Completed s…
cbrit Jun 6, 2023
71176eb
Move GetUnconfirmed*Txs methods into keeper, add tests
cbrit Jun 6, 2023
cfb424d
Refactor signature deletion method,tweak unit test
cbrit Jun 6, 2023
751b4f8
Revert "Review items - queries and signature -> confirmation term con…
cbrit Jun 9, 2023
fdfee5f
Update rust proto bindings
cbrit Jun 9, 2023
aabb8de
Correct mistakes in git revert
cbrit Jun 12, 2023
32d2a87
Rename Complete -> CompleteOutgoingTx, and use it for batch timeouts
cbrit Jun 12, 2023
1b53075
Fix condition ordering for slashing
cbrit Jun 12, 2023
bfe6f10
Add clarity to event votes command name
cbrit Jun 12, 2023
71bacf4
Remove unneeded comments
cbrit Jun 12, 2023
1bbf5c0
Fix thing in test_runner preventing rust tests from finishing
cbrit Jun 14, 2023
bb2df29
Fix minor proto inconsistency
cbrit Jun 29, 2023
2259b01
Add arg to cli command usage string
cbrit Jun 29, 2023
7160287
Correct function name in abci comment
cbrit Jun 29, 2023
4f10dd4
Err check on casting in executed tx handlers
cbrit Jun 29, 2023
6fd891b
Remove remaining references to unused param
cbrit Jun 29, 2023
49fe53c
Slashing reason event key/value
cbrit Jun 29, 2023
92b7c88
Only prune signer sets in abci
cbrit Jun 29, 2023
8f974ee
Ordering of outgoing txs returned by unslashed/unsigned methods
cbrit Jun 29, 2023
386d151
Move general outgoing tx methods to keeper.go
cbrit Jun 29, 2023
43e7eb9
Remove top level module, make integration-tests module
cbrit Jun 30, 2023
f525d8a
Make integration-tests module replacements consistent with previous m…
cbrit Jun 30, 2023
5ee3eee
Conditional delete in CompleteOutgoingTx
cbrit Jul 6, 2023
5fa34d1
Fix unit tests broken by review changes
cbrit Jul 10, 2023
cb3a8f2
Fix test file name
cbrit Jul 10, 2023
8bfdcdd
Update workflow file with test file fix
cbrit Jul 10, 2023
3b5532f
Only prune accepted event votes
cbrit Aug 2, 2023
c063451
Fix key name typo
cbrit Aug 2, 2023
dea66cd
Get height outside of loop in slashing
cbrit Aug 2, 2023
e1fb6b3
Fix efficiency issues in slashing refactor
cbrit Aug 3, 2023
a3e1a77
Fix signer set pruning bug
cbrit Aug 3, 2023
daa37bb
Add casting check for signer set
cbrit Aug 3, 2023
a1a73c8
Improve event vote cli command names
cbrit Aug 3, 2023
3255a88
Rename param to clarify it's an Ethereum height
cbrit Aug 3, 2023
2dc35c4
Update signerSetTxExecuted unit test
cbrit Aug 3, 2023
54f8996
Unit test fixes
cbrit Aug 3, 2023
fee0d33
Guarantee ascening batch nonce order when querying for unsigned
cbrit Aug 4, 2023
8fbe3de
Sign signer sets in ascending order
cbrit Aug 4, 2023
c2d78eb
Relay batches in ascending order
cbrit Aug 4, 2023
5ccfc53
Order contract call signing
cbrit Aug 4, 2023
9e4ef50
Clean up unsigned batches test
cbrit Aug 9, 2023
18d89aa
Remove unneeded ordering by scope for contract calls
cbrit Aug 9, 2023
1db56ab
Clarify comment forr param
cbrit Aug 9, 2023
9d0c767
Remove unnecessary contract call sorting from orchestrator
cbrit Aug 9, 2023
f8618ee
Revert "Fix key name typo"
cbrit Aug 9, 2023
c2a34f9
Remove duplicate height extraction from slashing loop
cbrit Aug 9, 2023
39c8888
Format slashing.go
cbrit Aug 9, 2023
3258760
Refactor GetUnbondingValidators method
cbrit Aug 9, 2023
5402aa8
Remove GetUnbondingValidators keeper method
cbrit Aug 9, 2023
dbb3f9c
Move relayer batch sorting into latest batches query
cbrit Aug 9, 2023
5b2cb3e
Remove unnecessary sort in BatchTxs query
cbrit Aug 10, 2023
b98ada6
Cover signer set tx pruning/slashing in unit test
cbrit Aug 10, 2023
64f09b2
Undo mistaken comment change
cbrit Aug 10, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 0 additions & 3 deletions module/x/gravity/abci.go
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,6 @@ func BeginBlocker(ctx sdk.Context, k keeper.Keeper) {
// EndBlocker is called at the end of every block
func EndBlocker(ctx sdk.Context, k keeper.Keeper) {
outgoingTxSlashing(ctx, k)
// Do we need to concern ourselves with future slashing windows for this pruning?
pruneEventVoteRecords(ctx, k)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we move this to BeginBlocker with the other pruning functions?

eventVoteRecordTally(ctx, k)
updateObservedEthereumHeight(ctx, k)
Expand Down Expand Up @@ -102,7 +101,6 @@ func pruneSignerSetTxs(ctx sdk.Context, k keeper.Keeper) {
earliestToPrune := currentBlock - params.SignedSignerSetTxsWindow
for _, set := range k.GetSignerSetTxs(ctx) {
if set.Nonce < lastObserved.Nonce && set.Height < earliestToPrune {
k.DeleteEthereumSignatures(ctx, set.GetStoreIndex())
k.DeleteOutgoingTx(ctx, set.GetStoreIndex())
}
}
Expand Down Expand Up @@ -287,7 +285,6 @@ func cleanupTimedOutContractCallTxs(ctx sdk.Context, k keeper.Keeper) {
k.IterateOutgoingTxsByType(ctx, types.ContractCallTxPrefixByte, func(_ []byte, otx types.OutgoingTx) bool {
cctx, _ := otx.(*types.ContractCallTx)
if cctx.Timeout < ethereumHeight {
k.DeleteEthereumSignatures(ctx, cctx.GetStoreIndex())
k.DeleteOutgoingTx(ctx, cctx.GetStoreIndex())
}
return true
Expand Down
2 changes: 0 additions & 2 deletions module/x/gravity/keeper/batch.go
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,6 @@ func (k Keeper) batchTxExecuted(ctx sdk.Context, tokenContract common.Address, n
}
return false
})
k.DeleteEthereumSignatures(ctx, batchTx.GetStoreIndex())
k.DeleteOutgoingTx(ctx, batchTx.GetStoreIndex())
}

Expand Down Expand Up @@ -124,7 +123,6 @@ func (k Keeper) CancelBatchTx(ctx sdk.Context, batch *types.BatchTx) {
}

// Delete batch since it is finished
k.DeleteEthereumSignatures(ctx, batch.GetStoreIndex())
k.DeleteOutgoingTx(ctx, batch.GetStoreIndex())

ctx.EventManager().EmitEvent(
Expand Down
2 changes: 0 additions & 2 deletions module/x/gravity/keeper/contract_call.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,12 +23,10 @@ func (k Keeper) contractCallExecuted(ctx sdk.Context, invalidationScope []byte,
cctx, _ := otx.(*types.ContractCallTx)
if (cctx.InvalidationNonce < completedCallTx.InvalidationNonce) &&
bytes.Equal(cctx.InvalidationScope, completedCallTx.InvalidationScope) {
k.DeleteEthereumSignatures(ctx, cctx.GetStoreIndex())
k.DeleteOutgoingTx(ctx, cctx.GetStoreIndex())
}
return false
})

k.DeleteEthereumSignatures(ctx, completedCallTx.GetStoreIndex())
k.DeleteOutgoingTx(ctx, completedCallTx.GetStoreIndex())
}
1 change: 1 addition & 0 deletions module/x/gravity/keeper/keeper.go
Original file line number Diff line number Diff line change
Expand Up @@ -464,6 +464,7 @@ func (k Keeper) SetOutgoingTx(ctx sdk.Context, outgoing types.OutgoingTx) {
// DeleteOutgoingTx deletes a given outgoingtx
func (k Keeper) DeleteOutgoingTx(ctx sdk.Context, storeIndex []byte) {
ctx.KVStore(k.storeKey).Delete(types.MakeOutgoingTxKey(storeIndex))
k.DeleteEthereumSignatures(ctx, storeIndex)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not certain if it matters in practice here, but I would suggest cleaning up the "child" records (the signatures) before deleting the "parent" record (the outgoing TX).

}

func (k Keeper) PaginateOutgoingTxsByType(ctx sdk.Context, pageReq *query.PageRequest, prefixByte byte, cb func(key []byte, outgoing types.OutgoingTx) bool) (*query.PageResponse, error) {
Expand Down