diff --git a/CHANGELOG.md b/CHANGELOG.md index 276edd4d12..1c017fc5b1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,8 @@ Add an entry to the unreleased section whenever merging a PR to main that is not targeted at a specific release. These entries will eventually be included in a release. +* (feat!) optimize pending packets storage on consumer, with migration! [#1037](https://github.com/cosmos/interchain-security/pull/1037) + ## v3.1.0 Date July 11th, 2023 diff --git a/docs/docs/adrs/adr-008-throttle-retries.md b/docs/docs/adrs/adr-008-throttle-retries.md index a8f0d250ce..134214fffb 100644 --- a/docs/docs/adrs/adr-008-throttle-retries.md +++ b/docs/docs/adrs/adr-008-throttle-retries.md @@ -8,6 +8,7 @@ title: Throttle with retries ## Changelog * 6/9/23: Initial draft +* 6/22/23: added note on consumer pending packets storage optimization ## Status @@ -46,6 +47,21 @@ With the behavior described, we maintain very similar behavior to the current th In the normal case, when no or a few slash packets are being sent, the VSCMaturedPackets will not be delayed, and hence unbonding will not be delayed. +### Consumer pending packets storage optimization + +In addition to the mentioned consumer changes above. An optimization will need to be made to the consumer's pending packets storage to properly implement the feature from this ADR. + +The consumer ccv module previously queued "pending packets" to be sent on each endblocker in [SendPackets](https://github.com/cosmos/interchain-security/blob/3bc4e7135066d848aac60b0787364c07157fd36d/x/ccv/consumer/keeper/relay.go#L178). These packets are queued in state with a protobuf list of `ConsumerPacketData`. For a single append operation, the entire list is deserialized, then a packet is appended to that list, and the list is serialized again. See older version of [AppendPendingPacket](https://github.com/cosmos/interchain-security/blob/05c2dae7c6372b1252b9e97215d07c6aa7618f33/x/ccv/consumer/keeper/keeper.go#L606). That is, a single append operation has O(N) complexity, where N is the size of the list. + +This poor append performance isn't a problem when the pending packets list is small. But with this ADR being implemented, the pending packets list could potentially grow to the order of thousands of entries, in the scenario that a slash packet is bouncing. + +We can improve the append time for this queue by converting it from a protobuf-esq list, to a queue implemented with sdk-esq code. The idea is to persist an uint64 index that will be incremented each time you queue up a packet. You can think of this as storing the tail of the queue. Then, packet data will be keyed by that index, making the data naturally ordered byte-wise for sdk's iterator. The index will also be stored in the packet data value bytes, so that the index can later be used to delete certain packets from the queue. + +Two things are achieved with this approach: + +* More efficient packet append/enqueue times +* The ability to delete select packets from the queue (previously all packets were deleted at once) + ### Provider changes The main change needed for the provider is the removal of queuing logic for slash and vsc matured packets upon being received. diff --git a/docs/docs/features/proposals.md b/docs/docs/features/proposals.md index a34160ecf4..b68fd343d6 100644 --- a/docs/docs/features/proposals.md +++ b/docs/docs/features/proposals.md @@ -48,7 +48,7 @@ Minimal example: "distribution_transmission_channel": "channel-123" } ``` -More examples can be found in the replicated security testnet repository [here](https://github.com/cosmos/testnets/blob/master/replicated-security/baryon-1/proposal-baryon-1.json) and [here](https://github.com/cosmos/testnets/blob/master/replicated-security/noble-1/start-proposal-noble-1.json). +More examples can be found in the replicated security testnet repository [here](https://github.com/cosmos/testnets/blob/master/replicated-security/stopped/baryon-1/proposal-baryon-1.json) and [here](https://github.com/cosmos/testnets/blob/master/replicated-security/stopped/noble-1/start-proposal-noble-1.json). ## `ConsumerRemovalProposal` Proposal type used to suggest removing an existing consumer chain. diff --git a/docs/docs/validators/joining-testnet.md b/docs/docs/validators/joining-testnet.md index 0e99c3710b..5ae9007416 100644 --- a/docs/docs/validators/joining-testnet.md +++ b/docs/docs/validators/joining-testnet.md @@ -180,9 +180,7 @@ gaiad tx provider assign-consensus-key consumer-1 '' --from = 64 { + return ErrIntOverflowCcv + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + m.Idx |= uint64(b&0x7F) << shift + if b < 0x80 { + break + } + } default: iNdEx = preIndex skippy, err := skipCcv(dAtA[iNdEx:])