kvserver: add rac2 v1 integration tests #130728

kvoli · 2024-09-13T23:12:36Z

1st commit from #130619.
2nd-3rd commits from #131106.
4th-5th commits from #131107.
6th-7th commits from #131108.
8th commit from #131109.

Introduce several tests in flow_control_integration_test.go, mirroring
the existing tests but applied to the replication flow control v2
machinery.

The tests largely follow an identical pattern to the existing v1 tests,
swapping in rac2 metrics and vtables.

The following tests are added:

TestFlowControlBasicV2
TestFlowControlRangeSplitMergeV2
TestFlowControlBlockedAdmissionV2
TestFlowControlAdmissionPostSplitMergeV2
TestFlowControlCrashedNodeV2
TestFlowControlRaftSnapshotV2
TestFlowControlRaftMembershipV2
TestFlowControlRaftMembershipRemoveSelfV2
TestFlowControlClassPrioritizationV2
TestFlowControlQuiescedRangeV2
TestFlowControlUnquiescedRangeV2
TestFlowControlTransferLeaseV2
TestFlowControlLeaderNotLeaseholderV2
TestFlowControlGranterAdmitOneByOneV2

These tests all have at least two variants:

V2EnabledWhenLeaderV1Encoding
V2EnabledWhenLeaderV2Encoding

When V2EnabledWhenLeaderV1Encoding is run, the tests use a different
testdata file, which has a _v1_encoding suffix. A separate file is
necessary because when the protocol enablement level is
V2EnabledWhenLeaderV1Encoding, all entries which are subject to
admission control are encoded as raftpb.LowPri, regardless of their
original priority, as we don't want to pay the cost to deserialize the
raft admission meta.

The v1 encoding variants retain the same comments as the v2 encoding,
however any comments referring to regular tokens should be interpreted
as elastic tokens instead, due to the above.

Two v1 tests are not ported over to v2:

TestFlowControlRaftTransportBreak
TestFlowControlRaftTransportCulled

These omitted tests behave identically to TestFlowControlCrashedNodeV2
as rac2 is less tightly coupled to the raft transport, instead operating
on replication states (e.g., StateProbe, StateReplicate).

Add TestFlowControlV1ToV2Transition, which ratchets up the enabled
version of replication flow control v2:

v1 protocol with v1 encoding =>
v2 protocol with v1 encoding =>
v2 protocol with v2 encoding

The test is structured to issue writes and wait for returned tokens
whenever the protocol transitions from v1 to v2, or a leader changes.

More specifically, the test takes the following steps:

(1) Start n1, n2, n3 with v1 protocol and v1 encoding.
(2) Upgrade n1 to v2 protocol with v1 encoding.
(3) Transfer the range lease to n2.
(4) Upgrade n2 to v2 protocol with v1 encoding.
(5) Upgrade n3 to v2 protocol with v1 encoding.
(6) Upgrade n1 to v2 protocol with v2 encoding.
(7) Transfer the range lease to n1.
(8) Upgrade n2,n3 to v2 protocol with v2 encoding.
(9) Transfer the range lease to n3.

Between each step, we issue writes, (un)block admission and observe the
flow control metrics and vtables.

Resolves: #130431
Resolves: #129276
Release note: None

cockroach-teamcity · 2024-09-13T23:12:45Z

This change is

blathers-crl · 2024-09-16T19:17:48Z

Your pull request contains more than 1000 changes. It is strongly encouraged to split big PRs into smaller chunks.

_{🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.}

kvoli · 2024-09-18T14:52:49Z

Split out the setup / fix commits into #130937. This PR can't merge until we enable the admitted vector protocol and resolve the hack (second last) commit.

sumeerbhola

Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @kvoli)

pkg/kv/kvserver/replica.go line 2535 at r16 (raw file):

	// TODO(kvoli,sumeerbhola): Should this be a cluster setting, or are we
	// ratcheting it up via cluster version? Either way, we need a cluster
	// setting somewhere.

Cluster version. We need two. We may as well add them now.

The cluster setting will only be used when on the latest cluster version to switch between push and pull mode.
As described in https://docs.google.com/document/d/1ROE1lpRVhfLxP39rs8J5mKoUfgsoiJyEkUkoNPhBJUk/edit#heading=h.pyzo8j6p2cb8

kvoli · 2024-09-20T17:36:51Z

Here are the diffed test outputs for flow_control_integration and flow_control_integration_v2 testdata files.

There are three notable diffs:

leader_not_leaseholder: Discovered a bug in the existing v1 implementation where admission is skipped entirely if a handle cannot be found, where it should instead enqueue the write into the store work queue. Tracked in kvflowcontrol: leaseholder not leader doesn't admit via store work queue #130948.
raft_membership_remove_self: Immaterial total tokens metrics diff, we create a stream token counter whenever a previously unseen replication stream is created. The v2 test output sees 4 whilst the v1 sees 3.
raft_transport_break and raft_transport_culled: The v2 versions of these tests were not added, see the commit / PR description.

Note that the outside the already mentioned tests, the diffs are substantial at a glance but immaterial because upon closer inspection; either a metric naming diff, or due to the v2 tests reporting additional metric values, such as send tokens, or opting to print both (regular|elastic) token metrics, while v1 only printed one or the other.

admission_post_split_merge

10c10
<    WHERE name LIKE '%kvadmission%tokens%'
---
>    WHERE name LIKE '%kvflowcontrol%tokens%'
13,20c13,28
<   kvadmission.flow_controller.elastic_tokens_available   | 18 MiB   
<   kvadmission.flow_controller.elastic_tokens_deducted    | 6.0 MiB  
<   kvadmission.flow_controller.elastic_tokens_returned    | 0 B      
<   kvadmission.flow_controller.elastic_tokens_unaccounted | 0 B      
<   kvadmission.flow_controller.regular_tokens_available   | 42 MiB   
<   kvadmission.flow_controller.regular_tokens_deducted    | 6.0 MiB  
<   kvadmission.flow_controller.regular_tokens_returned    | 0 B      
<   kvadmission.flow_controller.regular_tokens_unaccounted | 0 B      
---
>   kvflowcontrol.tokens.eval.elastic.available   | 18 MiB   
>   kvflowcontrol.tokens.eval.elastic.deducted    | 6.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.returned    | 0 B      
>   kvflowcontrol.tokens.eval.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.eval.regular.available   | 42 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 6.0 MiB  
>   kvflowcontrol.tokens.eval.regular.returned    | 0 B      
>   kvflowcontrol.tokens.eval.regular.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.elastic.available   | 18 MiB   
>   kvflowcontrol.tokens.send.elastic.deducted    | 6.0 MiB  
>   kvflowcontrol.tokens.send.elastic.returned    | 0 B      
>   kvflowcontrol.tokens.send.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.regular.available   | 42 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 6.0 MiB  
>   kvflowcontrol.tokens.send.regular.returned    | 0 B      
>   kvflowcontrol.tokens.send.regular.unaccounted | 0 B      
32c40
<    WHERE name LIKE '%kvadmission%tokens%'
---
>    WHERE name LIKE '%kvflowcontrol%tokens%'
35,42c43,58
<   kvadmission.flow_controller.elastic_tokens_available   | 3.0 MiB  
<   kvadmission.flow_controller.elastic_tokens_deducted    | 21 MiB   
<   kvadmission.flow_controller.elastic_tokens_returned    | 0 B      
<   kvadmission.flow_controller.elastic_tokens_unaccounted | 0 B      
<   kvadmission.flow_controller.regular_tokens_available   | 27 MiB   
<   kvadmission.flow_controller.regular_tokens_deducted    | 21 MiB   
<   kvadmission.flow_controller.regular_tokens_returned    | 0 B      
<   kvadmission.flow_controller.regular_tokens_unaccounted | 0 B      
---
>   kvflowcontrol.tokens.eval.elastic.available   | 3.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.deducted    | 21 MiB   
>   kvflowcontrol.tokens.eval.elastic.returned    | 0 B      
>   kvflowcontrol.tokens.eval.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.eval.regular.available   | 27 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 21 MiB   
>   kvflowcontrol.tokens.eval.regular.returned    | 0 B      
>   kvflowcontrol.tokens.eval.regular.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.elastic.available   | 3.0 MiB  
>   kvflowcontrol.tokens.send.elastic.deducted    | 21 MiB   
>   kvflowcontrol.tokens.send.elastic.returned    | 0 B      
>   kvflowcontrol.tokens.send.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.regular.available   | 27 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 21 MiB   
>   kvflowcontrol.tokens.send.regular.returned    | 0 B      
>   kvflowcontrol.tokens.send.regular.unaccounted | 0 B      
47c63
<     FROM crdb_internal.kv_flow_control_handles
---
>     FROM crdb_internal.kv_flow_control_handles_v2
67c83
<    WHERE name LIKE '%kvadmission%regular_tokens%'
---
>    WHERE name LIKE '%kvflowcontrol%tokens%'
70,73c86,101
<   kvadmission.flow_controller.regular_tokens_available   | 24 MiB   
<   kvadmission.flow_controller.regular_tokens_deducted    | 33 MiB   
<   kvadmission.flow_controller.regular_tokens_returned    | 9.0 MiB  
<   kvadmission.flow_controller.regular_tokens_unaccounted | 0 B      
---
>   kvflowcontrol.tokens.eval.elastic.available   | 0 B      
>   kvflowcontrol.tokens.eval.elastic.deducted    | 33 MiB   
>   kvflowcontrol.tokens.eval.elastic.returned    | 9.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.eval.regular.available   | 24 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 33 MiB   
>   kvflowcontrol.tokens.eval.regular.returned    | 9.0 MiB  
>   kvflowcontrol.tokens.eval.regular.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.elastic.available   | 0 B      
>   kvflowcontrol.tokens.send.elastic.deducted    | 33 MiB   
>   kvflowcontrol.tokens.send.elastic.returned    | 9.0 MiB  
>   kvflowcontrol.tokens.send.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.regular.available   | 24 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 33 MiB   
>   kvflowcontrol.tokens.send.regular.returned    | 9.0 MiB  
>   kvflowcontrol.tokens.send.regular.unaccounted | 0 B      
78c106
<     FROM crdb_internal.kv_flow_control_handles
---
>     FROM crdb_internal.kv_flow_control_handles_v2
96c124
<    WHERE name LIKE '%kvadmission%tokens%'
---
>    WHERE name LIKE '%kvflowcontrol%tokens%'
99,106c127,142
<   kvadmission.flow_controller.elastic_tokens_available   | 24 MiB  
<   kvadmission.flow_controller.elastic_tokens_deducted    | 33 MiB  
<   kvadmission.flow_controller.elastic_tokens_returned    | 33 MiB  
<   kvadmission.flow_controller.elastic_tokens_unaccounted | 0 B     
<   kvadmission.flow_controller.regular_tokens_available   | 48 MiB  
<   kvadmission.flow_controller.regular_tokens_deducted    | 33 MiB  
<   kvadmission.flow_controller.regular_tokens_returned    | 33 MiB  
<   kvadmission.flow_controller.regular_tokens_unaccounted | 0 B     
---
>   kvflowcontrol.tokens.eval.elastic.available   | 24 MiB  
>   kvflowcontrol.tokens.eval.elastic.deducted    | 33 MiB  
>   kvflowcontrol.tokens.eval.elastic.returned    | 33 MiB  
>   kvflowcontrol.tokens.eval.elastic.unaccounted | 0 B     
>   kvflowcontrol.tokens.eval.regular.available   | 48 MiB  
>   kvflowcontrol.tokens.eval.regular.deducted    | 33 MiB  
>   kvflowcontrol.tokens.eval.regular.returned    | 33 MiB  
>   kvflowcontrol.tokens.eval.regular.unaccounted | 0 B     
>   kvflowcontrol.tokens.send.elastic.available   | 24 MiB  
>   kvflowcontrol.tokens.send.elastic.deducted    | 33 MiB  
>   kvflowcontrol.tokens.send.elastic.returned    | 33 MiB  
>   kvflowcontrol.tokens.send.elastic.unaccounted | 0 B     
>   kvflowcontrol.tokens.send.regular.available   | 48 MiB  
>   kvflowcontrol.tokens.send.regular.deducted    | 33 MiB  
>   kvflowcontrol.tokens.send.regular.returned    | 33 MiB  
>   kvflowcontrol.tokens.send.regular.unaccounted | 0 B

basic

7c7
<    WHERE name LIKE '%kvadmission%tokens%'
---
>    WHERE name LIKE '%kvflowcontrol%tokens%'
10,17c10,25
<   kvadmission.flow_controller.elastic_tokens_available   | 24 MiB  
<   kvadmission.flow_controller.elastic_tokens_deducted    | 0 B     
<   kvadmission.flow_controller.elastic_tokens_returned    | 0 B     
<   kvadmission.flow_controller.elastic_tokens_unaccounted | 0 B     
<   kvadmission.flow_controller.regular_tokens_available   | 48 MiB  
<   kvadmission.flow_controller.regular_tokens_deducted    | 0 B     
<   kvadmission.flow_controller.regular_tokens_returned    | 0 B     
<   kvadmission.flow_controller.regular_tokens_unaccounted | 0 B     
---
>   kvflowcontrol.tokens.eval.elastic.available   | 24 MiB  
>   kvflowcontrol.tokens.eval.elastic.deducted    | 0 B     
>   kvflowcontrol.tokens.eval.elastic.returned    | 0 B     
>   kvflowcontrol.tokens.eval.elastic.unaccounted | 0 B     
>   kvflowcontrol.tokens.eval.regular.available   | 48 MiB  
>   kvflowcontrol.tokens.eval.regular.deducted    | 0 B     
>   kvflowcontrol.tokens.eval.regular.returned    | 0 B     
>   kvflowcontrol.tokens.eval.regular.unaccounted | 0 B     
>   kvflowcontrol.tokens.send.elastic.available   | 24 MiB  
>   kvflowcontrol.tokens.send.elastic.deducted    | 0 B     
>   kvflowcontrol.tokens.send.elastic.returned    | 0 B     
>   kvflowcontrol.tokens.send.elastic.unaccounted | 0 B     
>   kvflowcontrol.tokens.send.regular.available   | 48 MiB  
>   kvflowcontrol.tokens.send.regular.deducted    | 0 B     
>   kvflowcontrol.tokens.send.regular.returned    | 0 B     
>   kvflowcontrol.tokens.send.regular.unaccounted | 0 B     
28c36
<    WHERE name LIKE '%kvadmission.flow_controller%stream%'
---
>    WHERE name LIKE '%kvflowcontrol%stream%'
31,34c39,46
<   kvadmission.flow_controller.elastic_blocked_stream_count | 0  
<   kvadmission.flow_controller.elastic_stream_count         | 3  
<   kvadmission.flow_controller.regular_blocked_stream_count | 0  
<   kvadmission.flow_controller.regular_stream_count         | 3  
---
>   kvflowcontrol.streams.eval.elastic.blocked_count | 0  
>   kvflowcontrol.streams.eval.elastic.total_count   | 3  
>   kvflowcontrol.streams.eval.regular.blocked_count | 0  
>   kvflowcontrol.streams.eval.regular.total_count   | 3  
>   kvflowcontrol.streams.send.elastic.blocked_count | 0  
>   kvflowcontrol.streams.send.elastic.total_count   | 3  
>   kvflowcontrol.streams.send.regular.blocked_count | 0  
>   kvflowcontrol.streams.send.regular.total_count   | 3  
39c51
<     FROM crdb_internal.kv_flow_control_handles
---
>     FROM crdb_internal.kv_flow_control_handles_v2
54c66
<    WHERE name LIKE '%kvadmission%tokens%'
---
>    WHERE name LIKE '%kvflowcontrol%tokens%'
57,64c69,84
<   kvadmission.flow_controller.elastic_tokens_available   | 24 MiB   
<   kvadmission.flow_controller.elastic_tokens_deducted    | 3.0 MiB  
<   kvadmission.flow_controller.elastic_tokens_returned    | 3.0 MiB  
<   kvadmission.flow_controller.elastic_tokens_unaccounted | 0 B      
<   kvadmission.flow_controller.regular_tokens_available   | 48 MiB   
<   kvadmission.flow_controller.regular_tokens_deducted    | 3.0 MiB  
<   kvadmission.flow_controller.regular_tokens_returned    | 3.0 MiB  
<   kvadmission.flow_controller.regular_tokens_unaccounted | 0 B      
---
>   kvflowcontrol.tokens.eval.elastic.available   | 24 MiB   
>   kvflowcontrol.tokens.eval.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.returned    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.eval.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.regular.returned    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.regular.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.elastic.available   | 24 MiB   
>   kvflowcontrol.tokens.send.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.send.elastic.returned    | 3.0 MiB  
>   kvflowcontrol.tokens.send.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.send.regular.returned    | 3.0 MiB  
>   kvflowcontrol.tokens.send.regular.unaccounted | 0 B

blocked_admission

4c4
< -- (Issuing regular 1MiB, 3x replicated write that's not admitted.)
---
> -- (Issuing 5 regular 1MiB, 3x replicated write that's not admitted.)
12c12
<    WHERE name LIKE '%kvadmission%tokens%'
---
>    WHERE name LIKE '%kvflowcontrol%tokens%'
15,22c15,30
<   kvadmission.flow_controller.elastic_tokens_available   | 9.0 MiB  
<   kvadmission.flow_controller.elastic_tokens_deducted    | 15 MiB   
<   kvadmission.flow_controller.elastic_tokens_returned    | 0 B      
<   kvadmission.flow_controller.elastic_tokens_unaccounted | 0 B      
<   kvadmission.flow_controller.regular_tokens_available   | 33 MiB   
<   kvadmission.flow_controller.regular_tokens_deducted    | 15 MiB   
<   kvadmission.flow_controller.regular_tokens_returned    | 0 B      
<   kvadmission.flow_controller.regular_tokens_unaccounted | 0 B      
---
>   kvflowcontrol.tokens.eval.elastic.available   | 9.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.deducted    | 15 MiB   
>   kvflowcontrol.tokens.eval.elastic.returned    | 0 B      
>   kvflowcontrol.tokens.eval.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.eval.regular.available   | 33 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 15 MiB   
>   kvflowcontrol.tokens.eval.regular.returned    | 0 B      
>   kvflowcontrol.tokens.eval.regular.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.elastic.available   | 9.0 MiB  
>   kvflowcontrol.tokens.send.elastic.deducted    | 15 MiB   
>   kvflowcontrol.tokens.send.elastic.returned    | 0 B      
>   kvflowcontrol.tokens.send.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.regular.available   | 33 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 15 MiB   
>   kvflowcontrol.tokens.send.regular.returned    | 0 B      
>   kvflowcontrol.tokens.send.regular.unaccounted | 0 B      
27c35
<     FROM crdb_internal.kv_flow_control_handles
---
>     FROM crdb_internal.kv_flow_control_handles_v2
38c46
<     FROM crdb_internal.kv_flow_token_deductions
---
>     FROM crdb_internal.kv_flow_token_deductions_v2
62,79d69
< -- Observe flow token dispatch metrics from n1.
< SELECT name, value
<     FROM crdb_internal.node_metrics
<    WHERE name LIKE '%kvadmission.flow_token_dispatch.local_regular%'
< ORDER BY name ASC;
< 
<   kvadmission.flow_token_dispatch.local_regular | 5  
< 
< 
< -- Observe flow token dispatch metrics from n2.
< SELECT name, value
<     FROM crdb_internal.node_metrics
<    WHERE name LIKE '%kvadmission.flow_token_dispatch.remote_regular%'
< ORDER BY name ASC;
< 
<   kvadmission.flow_token_dispatch.remote_regular | 5  
< 
< 
85c75
<    WHERE name LIKE '%kvadmission%tokens%'
---
>    WHERE name LIKE '%kvflowcontrol%tokens%'
88,95c78,93
<   kvadmission.flow_controller.elastic_tokens_available   | 24 MiB  
<   kvadmission.flow_controller.elastic_tokens_deducted    | 15 MiB  
<   kvadmission.flow_controller.elastic_tokens_returned    | 15 MiB  
<   kvadmission.flow_controller.elastic_tokens_unaccounted | 0 B     
<   kvadmission.flow_controller.regular_tokens_available   | 48 MiB  
<   kvadmission.flow_controller.regular_tokens_deducted    | 15 MiB  
<   kvadmission.flow_controller.regular_tokens_returned    | 15 MiB  
<   kvadmission.flow_controller.regular_tokens_unaccounted | 0 B     
---
>   kvflowcontrol.tokens.eval.elastic.available   | 24 MiB  
>   kvflowcontrol.tokens.eval.elastic.deducted    | 15 MiB  
>   kvflowcontrol.tokens.eval.elastic.returned    | 15 MiB  
>   kvflowcontrol.tokens.eval.elastic.unaccounted | 0 B     
>   kvflowcontrol.tokens.eval.regular.available   | 48 MiB  
>   kvflowcontrol.tokens.eval.regular.deducted    | 15 MiB  
>   kvflowcontrol.tokens.eval.regular.returned    | 15 MiB  
>   kvflowcontrol.tokens.eval.regular.unaccounted | 0 B     
>   kvflowcontrol.tokens.send.elastic.available   | 24 MiB  
>   kvflowcontrol.tokens.send.elastic.deducted    | 15 MiB  
>   kvflowcontrol.tokens.send.elastic.returned    | 15 MiB  
>   kvflowcontrol.tokens.send.elastic.unaccounted | 0 B     
>   kvflowcontrol.tokens.send.regular.available   | 48 MiB  
>   kvflowcontrol.tokens.send.regular.deducted    | 15 MiB  
>   kvflowcontrol.tokens.send.regular.returned    | 15 MiB  
>   kvflowcontrol.tokens.send.regular.unaccounted | 0 B

class_prioritization

12c12
<    WHERE name LIKE '%kvadmission%tokens%'
---
>    WHERE name LIKE '%kvflowcontrol%tokens%'
15,22c15,30
<   kvadmission.flow_controller.elastic_tokens_available   | 21 MiB   
<   kvadmission.flow_controller.elastic_tokens_deducted    | 3.0 MiB  
<   kvadmission.flow_controller.elastic_tokens_returned    | 0 B      
<   kvadmission.flow_controller.elastic_tokens_unaccounted | 0 B      
<   kvadmission.flow_controller.regular_tokens_available   | 48 MiB   
<   kvadmission.flow_controller.regular_tokens_deducted    | 0 B      
<   kvadmission.flow_controller.regular_tokens_returned    | 0 B      
<   kvadmission.flow_controller.regular_tokens_unaccounted | 0 B      
---
>   kvflowcontrol.tokens.eval.elastic.available   | 21 MiB   
>   kvflowcontrol.tokens.eval.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.returned    | 0 B      
>   kvflowcontrol.tokens.eval.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.eval.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 0 B      
>   kvflowcontrol.tokens.eval.regular.returned    | 0 B      
>   kvflowcontrol.tokens.eval.regular.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.elastic.available   | 21 MiB   
>   kvflowcontrol.tokens.send.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.send.elastic.returned    | 0 B      
>   kvflowcontrol.tokens.send.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 0 B      
>   kvflowcontrol.tokens.send.regular.returned    | 0 B      
>   kvflowcontrol.tokens.send.regular.unaccounted | 0 B      
33c41
<    WHERE name LIKE '%kvadmission%tokens%'
---
>    WHERE name LIKE '%kvflowcontrol%tokens%'
36,43c44,59
<   kvadmission.flow_controller.elastic_tokens_available   | 18 MiB   
<   kvadmission.flow_controller.elastic_tokens_deducted    | 6.0 MiB  
<   kvadmission.flow_controller.elastic_tokens_returned    | 0 B      
<   kvadmission.flow_controller.elastic_tokens_unaccounted | 0 B      
<   kvadmission.flow_controller.regular_tokens_available   | 45 MiB   
<   kvadmission.flow_controller.regular_tokens_deducted    | 3.0 MiB  
<   kvadmission.flow_controller.regular_tokens_returned    | 0 B      
<   kvadmission.flow_controller.regular_tokens_unaccounted | 0 B      
---
>   kvflowcontrol.tokens.eval.elastic.available   | 18 MiB   
>   kvflowcontrol.tokens.eval.elastic.deducted    | 6.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.returned    | 0 B      
>   kvflowcontrol.tokens.eval.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.eval.regular.available   | 45 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.regular.returned    | 0 B      
>   kvflowcontrol.tokens.eval.regular.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.elastic.available   | 18 MiB   
>   kvflowcontrol.tokens.send.elastic.deducted    | 6.0 MiB  
>   kvflowcontrol.tokens.send.elastic.returned    | 0 B      
>   kvflowcontrol.tokens.send.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.regular.available   | 45 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.send.regular.returned    | 0 B      
>   kvflowcontrol.tokens.send.regular.unaccounted | 0 B      
53c69
<    WHERE name LIKE '%kvadmission%tokens%'
---
>    WHERE name LIKE '%kvflowcontrol%tokens%'
56,63c72,87
<   kvadmission.flow_controller.elastic_tokens_available   | 24 MiB   
<   kvadmission.flow_controller.elastic_tokens_deducted    | 6.0 MiB  
<   kvadmission.flow_controller.elastic_tokens_returned    | 6.0 MiB  
<   kvadmission.flow_controller.elastic_tokens_unaccounted | 0 B      
<   kvadmission.flow_controller.regular_tokens_available   | 48 MiB   
<   kvadmission.flow_controller.regular_tokens_deducted    | 3.0 MiB  
<   kvadmission.flow_controller.regular_tokens_returned    | 3.0 MiB  
<   kvadmission.flow_controller.regular_tokens_unaccounted | 0 B      
---
>   kvflowcontrol.tokens.eval.elastic.available   | 24 MiB   
>   kvflowcontrol.tokens.eval.elastic.deducted    | 6.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.returned    | 6.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.eval.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.regular.returned    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.regular.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.elastic.available   | 24 MiB   
>   kvflowcontrol.tokens.send.elastic.deducted    | 6.0 MiB  
>   kvflowcontrol.tokens.send.elastic.returned    | 6.0 MiB  
>   kvflowcontrol.tokens.send.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.send.regular.returned    | 3.0 MiB  
>   kvflowcontrol.tokens.send.regular.unaccounted | 0 B

crashed_node

12c12
<    WHERE name LIKE '%kvadmission%tokens%'
---
>    WHERE name LIKE '%kvflowcontrol%tokens%'
15,22c15,30
<   kvadmission.flow_controller.elastic_tokens_available   | 6.0 MiB  
<   kvadmission.flow_controller.elastic_tokens_deducted    | 10 MiB   
<   kvadmission.flow_controller.elastic_tokens_returned    | 0 B      
<   kvadmission.flow_controller.elastic_tokens_unaccounted | 0 B      
<   kvadmission.flow_controller.regular_tokens_available   | 22 MiB   
<   kvadmission.flow_controller.regular_tokens_deducted    | 10 MiB   
<   kvadmission.flow_controller.regular_tokens_returned    | 0 B      
<   kvadmission.flow_controller.regular_tokens_unaccounted | 0 B      
---
>   kvflowcontrol.tokens.eval.elastic.available   | 6.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.deducted    | 10 MiB   
>   kvflowcontrol.tokens.eval.elastic.returned    | 0 B      
>   kvflowcontrol.tokens.eval.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.eval.regular.available   | 22 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 10 MiB   
>   kvflowcontrol.tokens.eval.regular.returned    | 0 B      
>   kvflowcontrol.tokens.eval.regular.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.elastic.available   | 6.0 MiB  
>   kvflowcontrol.tokens.send.elastic.deducted    | 10 MiB   
>   kvflowcontrol.tokens.send.elastic.returned    | 0 B      
>   kvflowcontrol.tokens.send.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.regular.available   | 22 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 10 MiB   
>   kvflowcontrol.tokens.send.regular.returned    | 0 B      
>   kvflowcontrol.tokens.send.regular.unaccounted | 0 B      
27c35
<     FROM crdb_internal.kv_flow_control_handles
---
>     FROM crdb_internal.kv_flow_control_handles_v2
35c43
< -- (Crashing n2 but disabling the raft-transport-break token return mechanism.)
---
> -- (Crashing n2)
41c49
<     FROM crdb_internal.kv_flow_control_handles
---
>     FROM crdb_internal.kv_flow_control_handles_v2
52c60
<    WHERE name LIKE '%kvadmission%tokens%'
---
>    WHERE name LIKE '%kvflowcontrol%tokens%'
55,62c63,78
<   kvadmission.flow_controller.elastic_tokens_available   | 11 MiB   
<   kvadmission.flow_controller.elastic_tokens_deducted    | 10 MiB   
<   kvadmission.flow_controller.elastic_tokens_returned    | 5.0 MiB  
<   kvadmission.flow_controller.elastic_tokens_unaccounted | 0 B      
<   kvadmission.flow_controller.regular_tokens_available   | 27 MiB   
<   kvadmission.flow_controller.regular_tokens_deducted    | 10 MiB   
<   kvadmission.flow_controller.regular_tokens_returned    | 5.0 MiB  
<   kvadmission.flow_controller.regular_tokens_unaccounted | 0 B      
---
>   kvflowcontrol.tokens.eval.elastic.available   | 11 MiB   
>   kvflowcontrol.tokens.eval.elastic.deducted    | 10 MiB   
>   kvflowcontrol.tokens.eval.elastic.returned    | 5.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.eval.regular.available   | 27 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 10 MiB   
>   kvflowcontrol.tokens.eval.regular.returned    | 5.0 MiB  
>   kvflowcontrol.tokens.eval.regular.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.elastic.available   | 11 MiB   
>   kvflowcontrol.tokens.send.elastic.deducted    | 10 MiB   
>   kvflowcontrol.tokens.send.elastic.returned    | 5.0 MiB  
>   kvflowcontrol.tokens.send.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.regular.available   | 27 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 10 MiB   
>   kvflowcontrol.tokens.send.regular.returned    | 5.0 MiB  
>   kvflowcontrol.tokens.send.regular.unaccounted | 0 B

granter_admit_one_by_one

12c12
<    WHERE name LIKE '%kvadmission%tokens%'
---
>    WHERE name LIKE '%kvflowcontrol%tokens%'
15,22c15,30
<   kvadmission.flow_controller.elastic_tokens_available   | 21 MiB   
<   kvadmission.flow_controller.elastic_tokens_deducted    | 3.0 MiB  
<   kvadmission.flow_controller.elastic_tokens_returned    | 0 B      
<   kvadmission.flow_controller.elastic_tokens_unaccounted | 0 B      
<   kvadmission.flow_controller.regular_tokens_available   | 45 MiB   
<   kvadmission.flow_controller.regular_tokens_deducted    | 3.0 MiB  
<   kvadmission.flow_controller.regular_tokens_returned    | 0 B      
<   kvadmission.flow_controller.regular_tokens_unaccounted | 0 B      
---
>   kvflowcontrol.tokens.eval.elastic.available   | 21 MiB   
>   kvflowcontrol.tokens.eval.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.returned    | 0 B      
>   kvflowcontrol.tokens.eval.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.eval.regular.available   | 45 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.regular.returned    | 0 B      
>   kvflowcontrol.tokens.eval.regular.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.elastic.available   | 21 MiB   
>   kvflowcontrol.tokens.send.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.send.elastic.returned    | 0 B      
>   kvflowcontrol.tokens.send.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.regular.available   | 45 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.send.regular.returned    | 0 B      
>   kvflowcontrol.tokens.send.regular.unaccounted | 0 B      
27c35
<     FROM crdb_internal.kv_flow_control_handles
---
>     FROM crdb_internal.kv_flow_control_handles_v2
44c52
<    WHERE name LIKE '%kvadmission%tokens%'
---
>    WHERE name LIKE '%kvflowcontrol%tokens%'
47,54c55,70
<   kvadmission.flow_controller.elastic_tokens_available   | 24 MiB   
<   kvadmission.flow_controller.elastic_tokens_deducted    | 3.0 MiB  
<   kvadmission.flow_controller.elastic_tokens_returned    | 3.0 MiB  
<   kvadmission.flow_controller.elastic_tokens_unaccounted | 0 B      
<   kvadmission.flow_controller.regular_tokens_available   | 48 MiB   
<   kvadmission.flow_controller.regular_tokens_deducted    | 3.0 MiB  
<   kvadmission.flow_controller.regular_tokens_returned    | 3.0 MiB  
<   kvadmission.flow_controller.regular_tokens_unaccounted | 0 B      
---
>   kvflowcontrol.tokens.eval.elastic.available   | 24 MiB   
>   kvflowcontrol.tokens.eval.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.returned    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.eval.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.regular.returned    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.regular.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.elastic.available   | 24 MiB   
>   kvflowcontrol.tokens.send.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.send.elastic.returned    | 3.0 MiB  
>   kvflowcontrol.tokens.send.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.send.regular.returned    | 3.0 MiB  
>   kvflowcontrol.tokens.send.regular.unaccounted | 0 B

leader_not_leaseholder

12c12
<    WHERE name LIKE '%kvadmission%regular_tokens%'
---
>    WHERE name LIKE '%kvflowcontrol%tokens%'
15,18c15,30
<   kvadmission.flow_controller.regular_tokens_available   | 45 MiB   
<   kvadmission.flow_controller.regular_tokens_deducted    | 3.0 MiB  
<   kvadmission.flow_controller.regular_tokens_returned    | 0 B      
<   kvadmission.flow_controller.regular_tokens_unaccounted | 0 B      
---
>   kvflowcontrol.tokens.eval.elastic.available   | 21 MiB   
>   kvflowcontrol.tokens.eval.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.returned    | 0 B      
>   kvflowcontrol.tokens.eval.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.eval.regular.available   | 45 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.regular.returned    | 0 B      
>   kvflowcontrol.tokens.eval.regular.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.elastic.available   | 21 MiB   
>   kvflowcontrol.tokens.send.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.send.elastic.returned    | 0 B      
>   kvflowcontrol.tokens.send.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.regular.available   | 45 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.send.regular.returned    | 0 B      
>   kvflowcontrol.tokens.send.regular.unaccounted | 0 B      
28c40
<    WHERE name LIKE '%kvadmission%regular_tokens%'
---
>    WHERE name LIKE '%kvflowcontrol%tokens%'
31,34c43,58
<   kvadmission.flow_controller.regular_tokens_available   | 45 MiB   
<   kvadmission.flow_controller.regular_tokens_deducted    | 3.0 MiB  
<   kvadmission.flow_controller.regular_tokens_returned    | 0 B      
<   kvadmission.flow_controller.regular_tokens_unaccounted | 0 B      
---
>   kvflowcontrol.tokens.eval.elastic.available   | 21 MiB   
>   kvflowcontrol.tokens.eval.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.returned    | 0 B      
>   kvflowcontrol.tokens.eval.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.eval.regular.available   | 45 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.regular.returned    | 0 B      
>   kvflowcontrol.tokens.eval.regular.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.elastic.available   | 21 MiB   
>   kvflowcontrol.tokens.send.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.send.elastic.returned    | 0 B      
>   kvflowcontrol.tokens.send.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.regular.available   | 45 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.send.regular.returned    | 0 B      
>   kvflowcontrol.tokens.send.regular.unaccounted | 0 B      
37,38c61,65
< -- (Issuing another 1x1MiB, 3x replicated write that's not admitted while in
< -- this leader != leaseholder state.)
---
> -- (Allow below-raft admission to proceed. All tokens should be returned.)
> SELECT name, crdb_internal.humanize_bytes(value::INT8)
>     FROM crdb_internal.node_metrics
>    WHERE name LIKE '%kvflowcontrol%tokens%'
> ORDER BY name ASC;
39a67,82
>   kvflowcontrol.tokens.eval.elastic.available   | 24 MiB   
>   kvflowcontrol.tokens.eval.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.returned    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.eval.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.regular.returned    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.regular.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.elastic.available   | 24 MiB   
>   kvflowcontrol.tokens.send.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.send.elastic.returned    | 3.0 MiB  
>   kvflowcontrol.tokens.send.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.send.regular.returned    | 3.0 MiB  
>   kvflowcontrol.tokens.send.regular.unaccounted | 0 B      
40a84,88
> 
> -- (Issuing another 1x1MiB, 3x replicated write that's admitted via 
> -- the work queue on the leaseholder. It shouldn't deduct any tokens.)
> 
> 
45c93
<    WHERE name LIKE '%kvadmission%regular_tokens%'
---
>    WHERE name LIKE '%kvflowcontrol%tokens%'
48,51c96,111
<   kvadmission.flow_controller.regular_tokens_available   | 45 MiB   
<   kvadmission.flow_controller.regular_tokens_deducted    | 3.0 MiB  
<   kvadmission.flow_controller.regular_tokens_returned    | 0 B      
<   kvadmission.flow_controller.regular_tokens_unaccounted | 0 B      
---
>   kvflowcontrol.tokens.eval.elastic.available   | 24 MiB   
>   kvflowcontrol.tokens.eval.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.returned    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.eval.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.regular.returned    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.regular.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.elastic.available   | 24 MiB   
>   kvflowcontrol.tokens.send.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.send.elastic.returned    | 3.0 MiB  
>   kvflowcontrol.tokens.send.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.send.regular.returned    | 3.0 MiB  
>   kvflowcontrol.tokens.send.regular.unaccounted | 0 B      
58c118
<    WHERE name LIKE '%kvadmission%regular_tokens%'
---
>    WHERE name LIKE '%kvflowcontrol%tokens%'
61,79c121,136
<   kvadmission.flow_controller.regular_tokens_available   | 0 B  
<   kvadmission.flow_controller.regular_tokens_deducted    | 0 B  
<   kvadmission.flow_controller.regular_tokens_returned    | 0 B  
<   kvadmission.flow_controller.regular_tokens_unaccounted | 0 B  
< 
< 
< -- (Allow below-raft admission to proceed.)
< 
< 
< -- All deducted flow tokens are returned back to where the raft leader is.
< SELECT name, crdb_internal.humanize_bytes(value::INT8)
<     FROM crdb_internal.node_metrics
<    WHERE name LIKE '%kvadmission%regular_tokens%'
< ORDER BY name ASC;
< 
<   kvadmission.flow_controller.regular_tokens_available   | 48 MiB   
<   kvadmission.flow_controller.regular_tokens_deducted    | 3.0 MiB  
<   kvadmission.flow_controller.regular_tokens_returned    | 3.0 MiB  
<   kvadmission.flow_controller.regular_tokens_unaccounted | 0 B      
---
>   kvflowcontrol.tokens.eval.elastic.available   | 0 B  
>   kvflowcontrol.tokens.eval.elastic.deducted    | 0 B  
>   kvflowcontrol.tokens.eval.elastic.returned    | 0 B  
>   kvflowcontrol.tokens.eval.elastic.unaccounted | 0 B  
>   kvflowcontrol.tokens.eval.regular.available   | 0 B  
>   kvflowcontrol.tokens.eval.regular.deducted    | 0 B  
>   kvflowcontrol.tokens.eval.regular.returned    | 0 B  
>   kvflowcontrol.tokens.eval.regular.unaccounted | 0 B  
>   kvflowcontrol.tokens.send.elastic.available   | 0 B  
>   kvflowcontrol.tokens.send.elastic.deducted    | 0 B  
>   kvflowcontrol.tokens.send.elastic.returned    | 0 B  
>   kvflowcontrol.tokens.send.elastic.unaccounted | 0 B  
>   kvflowcontrol.tokens.send.regular.available   | 0 B  
>   kvflowcontrol.tokens.send.regular.deducted    | 0 B  
>   kvflowcontrol.tokens.send.regular.returned    | 0 B  
>   kvflowcontrol.tokens.send.regular.unaccounted | 0 B

quiesced_range

12c12
<    WHERE name LIKE '%kvadmission%elastic_tokens%'
---
>    WHERE name LIKE '%kvflowcontrol%tokens%'
15,18c15,30
<   kvadmission.flow_controller.elastic_tokens_available   | 21 MiB   
<   kvadmission.flow_controller.elastic_tokens_deducted    | 3.0 MiB  
<   kvadmission.flow_controller.elastic_tokens_returned    | 0 B      
<   kvadmission.flow_controller.elastic_tokens_unaccounted | 0 B      
---
>   kvflowcontrol.tokens.eval.elastic.available   | 21 MiB   
>   kvflowcontrol.tokens.eval.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.returned    | 0 B      
>   kvflowcontrol.tokens.eval.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.eval.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 0 B      
>   kvflowcontrol.tokens.eval.regular.returned    | 0 B      
>   kvflowcontrol.tokens.eval.regular.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.elastic.available   | 21 MiB   
>   kvflowcontrol.tokens.send.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.send.elastic.returned    | 0 B      
>   kvflowcontrol.tokens.send.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 0 B      
>   kvflowcontrol.tokens.send.regular.returned    | 0 B      
>   kvflowcontrol.tokens.send.regular.unaccounted | 0 B      
34c46
<    WHERE name LIKE '%kvadmission%elastic_tokens%'
---
>    WHERE name LIKE '%kvflowcontrol%tokens%'
37,40c49,64
<   kvadmission.flow_controller.elastic_tokens_available   | 22 MiB   
<   kvadmission.flow_controller.elastic_tokens_deducted    | 3.0 MiB  
<   kvadmission.flow_controller.elastic_tokens_returned    | 1.0 MiB  
<   kvadmission.flow_controller.elastic_tokens_unaccounted | 0 B      
---
>   kvflowcontrol.tokens.eval.elastic.available   | 22 MiB   
>   kvflowcontrol.tokens.eval.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.returned    | 1.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.eval.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 0 B      
>   kvflowcontrol.tokens.eval.regular.returned    | 0 B      
>   kvflowcontrol.tokens.eval.regular.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.elastic.available   | 22 MiB   
>   kvflowcontrol.tokens.send.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.send.elastic.returned    | 1.0 MiB  
>   kvflowcontrol.tokens.send.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 0 B      
>   kvflowcontrol.tokens.send.regular.returned    | 0 B      
>   kvflowcontrol.tokens.send.regular.unaccounted | 0 B      
50c74
<    WHERE name LIKE '%kvadmission%elastic_tokens%'
---
>    WHERE name LIKE '%kvflowcontrol%tokens%'
53,56c77,92
<   kvadmission.flow_controller.elastic_tokens_available   | 24 MiB   
<   kvadmission.flow_controller.elastic_tokens_deducted    | 3.0 MiB  
<   kvadmission.flow_controller.elastic_tokens_returned    | 3.0 MiB  
<   kvadmission.flow_controller.elastic_tokens_unaccounted | 0 B      
---
>   kvflowcontrol.tokens.eval.elastic.available   | 24 MiB   
>   kvflowcontrol.tokens.eval.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.returned    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.eval.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 0 B      
>   kvflowcontrol.tokens.eval.regular.returned    | 0 B      
>   kvflowcontrol.tokens.eval.regular.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.elastic.available   | 24 MiB   
>   kvflowcontrol.tokens.send.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.send.elastic.returned    | 3.0 MiB  
>   kvflowcontrol.tokens.send.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 0 B      
>   kvflowcontrol.tokens.send.regular.returned    | 0 B      
>   kvflowcontrol.tokens.send.regular.unaccounted | 0 B

raft_membership

12c12
<    WHERE name LIKE '%kvadmission%regular_tokens%'
---
>    WHERE name LIKE '%kvflowcontrol%tokens%'
15,18c15,30
<   kvadmission.flow_controller.regular_tokens_available   | 45 MiB   
<   kvadmission.flow_controller.regular_tokens_deducted    | 3.0 MiB  
<   kvadmission.flow_controller.regular_tokens_returned    | 0 B      
<   kvadmission.flow_controller.regular_tokens_unaccounted | 0 B      
---
>   kvflowcontrol.tokens.eval.elastic.available   | 21 MiB   
>   kvflowcontrol.tokens.eval.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.returned    | 0 B      
>   kvflowcontrol.tokens.eval.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.eval.regular.available   | 45 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.regular.returned    | 0 B      
>   kvflowcontrol.tokens.eval.regular.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.elastic.available   | 21 MiB   
>   kvflowcontrol.tokens.send.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.send.elastic.returned    | 0 B      
>   kvflowcontrol.tokens.send.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.regular.available   | 45 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.send.regular.returned    | 0 B      
>   kvflowcontrol.tokens.send.regular.unaccounted | 0 B      
27c39
<     FROM crdb_internal.kv_flow_control_handles
---
>     FROM crdb_internal.kv_flow_control_handles_v2
44c56
<     FROM crdb_internal.kv_flow_control_handles
---
>     FROM crdb_internal.kv_flow_control_handles_v2
68c80
<     FROM crdb_internal.kv_flow_control_handles
---
>     FROM crdb_internal.kv_flow_control_handles_v2
83c95
<     FROM crdb_internal.kv_flow_control_handles
---
>     FROM crdb_internal.kv_flow_control_handles_v2
98c110
<    WHERE name LIKE '%kvadmission%tokens%'
---
>    WHERE name LIKE '%kvflowcontrol%tokens%'
101,108c113,128
<   kvadmission.flow_controller.elastic_tokens_available   | 40 MiB  
<   kvadmission.flow_controller.elastic_tokens_deducted    | 11 MiB  
<   kvadmission.flow_controller.elastic_tokens_returned    | 11 MiB  
<   kvadmission.flow_controller.elastic_tokens_unaccounted | 0 B     
<   kvadmission.flow_controller.regular_tokens_available   | 80 MiB  
<   kvadmission.flow_controller.regular_tokens_deducted    | 11 MiB  
<   kvadmission.flow_controller.regular_tokens_returned    | 11 MiB  
<   kvadmission.flow_controller.regular_tokens_unaccounted | 0 B     
---
>   kvflowcontrol.tokens.eval.elastic.available   | 40 MiB  
>   kvflowcontrol.tokens.eval.elastic.deducted    | 11 MiB  
>   kvflowcontrol.tokens.eval.elastic.returned    | 11 MiB  
>   kvflowcontrol.tokens.eval.elastic.unaccounted | 0 B     
>   kvflowcontrol.tokens.eval.regular.available   | 80 MiB  
>   kvflowcontrol.tokens.eval.regular.deducted    | 11 MiB  
>   kvflowcontrol.tokens.eval.regular.returned    | 11 MiB  
>   kvflowcontrol.tokens.eval.regular.unaccounted | 0 B     
>   kvflowcontrol.tokens.send.elastic.available   | 40 MiB  
>   kvflowcontrol.tokens.send.elastic.deducted    | 11 MiB  
>   kvflowcontrol.tokens.send.elastic.returned    | 11 MiB  
>   kvflowcontrol.tokens.send.elastic.unaccounted | 0 B     
>   kvflowcontrol.tokens.send.regular.available   | 80 MiB  
>   kvflowcontrol.tokens.send.regular.deducted    | 11 MiB  
>   kvflowcontrol.tokens.send.regular.returned    | 11 MiB  
>   kvflowcontrol.tokens.send.regular.unaccounted | 0 B

raft_membership_remove_self

12c12
<    WHERE name LIKE '%kvadmission%regular_tokens%'
---
>    WHERE name LIKE '%kvflowcontrol%tokens%'
15,18c15,30
<   kvadmission.flow_controller.regular_tokens_available   | 45 MiB   
<   kvadmission.flow_controller.regular_tokens_deducted    | 3.0 MiB  
<   kvadmission.flow_controller.regular_tokens_returned    | 0 B      
<   kvadmission.flow_controller.regular_tokens_unaccounted | 0 B      
---
>   kvflowcontrol.tokens.eval.elastic.available   | 21 MiB   
>   kvflowcontrol.tokens.eval.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.returned    | 0 B      
>   kvflowcontrol.tokens.eval.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.eval.regular.available   | 45 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.regular.returned    | 0 B      
>   kvflowcontrol.tokens.eval.regular.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.elastic.available   | 21 MiB   
>   kvflowcontrol.tokens.send.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.send.elastic.returned    | 0 B      
>   kvflowcontrol.tokens.send.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.regular.available   | 45 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.send.regular.returned    | 0 B      
>   kvflowcontrol.tokens.send.regular.unaccounted | 0 B      
25c37,38
< -- All {regular,elastic} tokens deducted are returned.
---
> -- All {regular,elastic} tokens deducted are returned. Note that the available
> -- tokens increases, as n1 has seen 4 replication streams, s1,s2,s3,s4.
28c41
<    WHERE name LIKE '%kvadmission%tokens%'
---
>    WHERE name LIKE '%kvflowcontrol%tokens%'
31,38c44,59
<   kvadmission.flow_controller.elastic_tokens_available   | 24 MiB   
<   kvadmission.flow_controller.elastic_tokens_deducted    | 3.0 MiB  
<   kvadmission.flow_controller.elastic_tokens_returned    | 3.0 MiB  
<   kvadmission.flow_controller.elastic_tokens_unaccounted | 0 B      
<   kvadmission.flow_controller.regular_tokens_available   | 48 MiB   
<   kvadmission.flow_controller.regular_tokens_deducted    | 3.0 MiB  
<   kvadmission.flow_controller.regular_tokens_returned    | 3.0 MiB  
<   kvadmission.flow_controller.regular_tokens_unaccounted | 0 B      
---
>   kvflowcontrol.tokens.eval.elastic.available   | 32 MiB   
>   kvflowcontrol.tokens.eval.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.returned    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.eval.regular.available   | 64 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.regular.returned    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.regular.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.elastic.available   | 32 MiB   
>   kvflowcontrol.tokens.send.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.send.elastic.returned    | 3.0 MiB  
>   kvflowcontrol.tokens.send.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.regular.available   | 64 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.send.regular.returned    | 3.0 MiB  
>   kvflowcontrol.tokens.send.regular.unaccounted | 0 B      
40a62,73
> -- n1 should have no connected streams now after transferring the lease to n2.
> -- While, n2 should have 3 connected streams to s2,s3,s4. Query the stream count
> -- on n1, then on n2.
> SELECT range_id, count(*) AS streams
>     FROM crdb_internal.kv_flow_control_handles_v2
> GROUP BY (range_id)
> ORDER BY streams DESC;
> 
>   range_id | stream_count  
> -----------+---------------
> 
> 
48c81
<    WHERE name LIKE '%kvadmission%tokens%'
---
>    WHERE name LIKE '%kvflowcontrol%tokens%'
51,58c84,99
<   kvadmission.flow_controller.elastic_tokens_available   | 24 MiB   
<   kvadmission.flow_controller.elastic_tokens_deducted    | 3.0 MiB  
<   kvadmission.flow_controller.elastic_tokens_returned    | 3.0 MiB  
<   kvadmission.flow_controller.elastic_tokens_unaccounted | 0 B      
<   kvadmission.flow_controller.regular_tokens_available   | 48 MiB   
<   kvadmission.flow_controller.regular_tokens_deducted    | 3.0 MiB  
<   kvadmission.flow_controller.regular_tokens_returned    | 3.0 MiB  
<   kvadmission.flow_controller.regular_tokens_unaccounted | 0 B      
---
>   kvflowcontrol.tokens.eval.elastic.available   | 32 MiB   
>   kvflowcontrol.tokens.eval.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.returned    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.eval.regular.available   | 64 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.regular.returned    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.regular.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.elastic.available   | 32 MiB   
>   kvflowcontrol.tokens.send.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.send.elastic.returned    | 3.0 MiB  
>   kvflowcontrol.tokens.send.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.regular.available   | 64 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.send.regular.returned    | 3.0 MiB  
>   kvflowcontrol.tokens.send.regular.unaccounted | 0 B

raft_snapshot

9c9
<    WHERE name LIKE '%kvadmission%tokens%'
---
>    WHERE name LIKE '%kvflowcontrol%tokens%'
12,19c12,27
<   kvadmission.flow_controller.elastic_tokens_available   | 35 MiB   
<   kvadmission.flow_controller.elastic_tokens_deducted    | 5.0 MiB  
<   kvadmission.flow_controller.elastic_tokens_returned    | 0 B      
<   kvadmission.flow_controller.elastic_tokens_unaccounted | 0 B      
<   kvadmission.flow_controller.regular_tokens_available   | 75 MiB   
<   kvadmission.flow_controller.regular_tokens_deducted    | 5.0 MiB  
<   kvadmission.flow_controller.regular_tokens_returned    | 0 B      
<   kvadmission.flow_controller.regular_tokens_unaccounted | 0 B      
---
>   kvflowcontrol.tokens.eval.elastic.available   | 35 MiB   
>   kvflowcontrol.tokens.eval.elastic.deducted    | 5.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.returned    | 0 B      
>   kvflowcontrol.tokens.eval.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.eval.regular.available   | 75 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 5.0 MiB  
>   kvflowcontrol.tokens.eval.regular.returned    | 0 B      
>   kvflowcontrol.tokens.eval.regular.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.elastic.available   | 35 MiB   
>   kvflowcontrol.tokens.send.elastic.deducted    | 5.0 MiB  
>   kvflowcontrol.tokens.send.elastic.returned    | 0 B      
>   kvflowcontrol.tokens.send.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.regular.available   | 75 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 5.0 MiB  
>   kvflowcontrol.tokens.send.regular.returned    | 0 B      
>   kvflowcontrol.tokens.send.regular.unaccounted | 0 B      
24c32
< 	   FROM crdb_internal.kv_flow_control_handles
---
> 	   FROM crdb_internal.kv_flow_control_handles_v2
40,42c48,49
< -- for n1-n5. Typically n2, n3 would release their tokens, but this test is
< -- intentionally suppressing that behavior to observe token returns only once
< -- issuing a raft snapshot.
---
> -- for n1-n5, because they are not in StateSnapshot yet and it have likely been
> -- in StateProbe for less than the close timer.
44c51
< 	   FROM crdb_internal.kv_flow_control_handles
---
> 	   FROM crdb_internal.kv_flow_control_handles_v2
61,64d67
< -- Normally we wouldn't deduct tokens for n2 and n3 since they're dead (both
< -- according to the per-replica last-updated map, and according broken
< -- RaftTransport streams). But this test is intentionally suppressing that
< -- behavior to observe token returns when sending raft snapshots.
67c70
<    WHERE name LIKE '%kvadmission%tokens%'
---
>    WHERE name LIKE '%kvflowcontrol%tokens%'
70,77c73,88
<   kvadmission.flow_controller.elastic_tokens_available   | 30 MiB  
<   kvadmission.flow_controller.elastic_tokens_deducted    | 10 MiB  
<   kvadmission.flow_controller.elastic_tokens_returned    | 0 B     
<   kvadmission.flow_controller.elastic_tokens_unaccounted | 0 B     
<   kvadmission.flow_controller.regular_tokens_available   | 70 MiB  
<   kvadmission.flow_controller.regular_tokens_deducted    | 10 MiB  
<   kvadmission.flow_controller.regular_tokens_returned    | 0 B     
<   kvadmission.flow_controller.regular_tokens_unaccounted | 0 B     
---
>   kvflowcontrol.tokens.eval.elastic.available   | 30 MiB  
>   kvflowcontrol.tokens.eval.elastic.deducted    | 10 MiB  
>   kvflowcontrol.tokens.eval.elastic.returned    | 0 B     
>   kvflowcontrol.tokens.eval.elastic.unaccounted | 0 B     
>   kvflowcontrol.tokens.eval.regular.available   | 70 MiB  
>   kvflowcontrol.tokens.eval.regular.deducted    | 10 MiB  
>   kvflowcontrol.tokens.eval.regular.returned    | 0 B     
>   kvflowcontrol.tokens.eval.regular.unaccounted | 0 B     
>   kvflowcontrol.tokens.send.elastic.available   | 30 MiB  
>   kvflowcontrol.tokens.send.elastic.deducted    | 10 MiB  
>   kvflowcontrol.tokens.send.elastic.returned    | 0 B     
>   kvflowcontrol.tokens.send.elastic.unaccounted | 0 B     
>   kvflowcontrol.tokens.send.regular.available   | 70 MiB  
>   kvflowcontrol.tokens.send.regular.deducted    | 10 MiB  
>   kvflowcontrol.tokens.send.regular.returned    | 0 B     
>   kvflowcontrol.tokens.send.regular.unaccounted | 0 B     
83c94
< 	   FROM crdb_internal.kv_flow_control_handles
---
> 	   FROM crdb_internal.kv_flow_control_handles_v2
106c117
<    WHERE name LIKE '%kvadmission%tokens%'
---
>    WHERE name LIKE '%kvflowcontrol%tokens%'
109,116c120,135
<   kvadmission.flow_controller.elastic_tokens_available   | 34 MiB   
<   kvadmission.flow_controller.elastic_tokens_deducted    | 10 MiB   
<   kvadmission.flow_controller.elastic_tokens_returned    | 4.0 MiB  
<   kvadmission.flow_controller.elastic_tokens_unaccounted | 0 B      
<   kvadmission.flow_controller.regular_tokens_available   | 74 MiB   
<   kvadmission.flow_controller.regular_tokens_deducted    | 10 MiB   
<   kvadmission.flow_controller.regular_tokens_returned    | 4.0 MiB  
<   kvadmission.flow_controller.regular_tokens_unaccounted | 0 B      
---
>   kvflowcontrol.tokens.eval.elastic.available   | 34 MiB   
>   kvflowcontrol.tokens.eval.elastic.deducted    | 10 MiB   
>   kvflowcontrol.tokens.eval.elastic.returned    | 4.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.eval.regular.available   | 74 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 10 MiB   
>   kvflowcontrol.tokens.eval.regular.returned    | 4.0 MiB  
>   kvflowcontrol.tokens.eval.regular.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.elastic.available   | 34 MiB   
>   kvflowcontrol.tokens.send.elastic.deducted    | 10 MiB   
>   kvflowcontrol.tokens.send.elastic.returned    | 4.0 MiB  
>   kvflowcontrol.tokens.send.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.regular.available   | 74 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 10 MiB   
>   kvflowcontrol.tokens.send.regular.returned    | 4.0 MiB  
>   kvflowcontrol.tokens.send.regular.unaccounted | 0 B      
122c141
<    FROM crdb_internal.kv_flow_control_handles
---
>    FROM crdb_internal.kv_flow_control_handles_v2
135,143d153
< -- Observe flow token dispatch metrics from n4.
< SELECT name, value
<     FROM crdb_internal.node_metrics
<    WHERE name LIKE '%kvadmission.flow_token_dispatch.pending_nodes%'
< ORDER BY name ASC;
< 
<   kvadmission.flow_token_dispatch.pending_nodes | 0  
< 
< 
148c158
<    WHERE name LIKE '%kvadmission%tokens%'
---
>    WHERE name LIKE '%kvflowcontrol%tokens%'
151,158c161,176
<   kvadmission.flow_controller.elastic_tokens_available   | 40 MiB  
<   kvadmission.flow_controller.elastic_tokens_deducted    | 10 MiB  
<   kvadmission.flow_controller.elastic_tokens_returned    | 10 MiB  
<   kvadmission.flow_controller.elastic_tokens_unaccounted | 0 B     
<   kvadmission.flow_controller.regular_tokens_available   | 80 MiB  
<   kvadmission.flow_controller.regular_tokens_deducted    | 10 MiB  
<   kvadmission.flow_controller.regular_tokens_returned    | 10 MiB  
<   kvadmission.flow_controller.regular_tokens_unaccounted | 0 B     
---
>   kvflowcontrol.tokens.eval.elastic.available   | 40 MiB  
>   kvflowcontrol.tokens.eval.elastic.deducted    | 10 MiB  
>   kvflowcontrol.tokens.eval.elastic.returned    | 10 MiB  
>   kvflowcontrol.tokens.eval.elastic.unaccounted | 0 B     
>   kvflowcontrol.tokens.eval.regular.available   | 80 MiB  
>   kvflowcontrol.tokens.eval.regular.deducted    | 10 MiB  
>   kvflowcontrol.tokens.eval.regular.returned    | 10 MiB  
>   kvflowcontrol.tokens.eval.regular.unaccounted | 0 B     
>   kvflowcontrol.tokens.send.elastic.available   | 40 MiB  
>   kvflowcontrol.tokens.send.elastic.deducted    | 10 MiB  
>   kvflowcontrol.tokens.send.elastic.returned    | 10 MiB  
>   kvflowcontrol.tokens.send.elastic.unaccounted | 0 B     
>   kvflowcontrol.tokens.send.regular.available   | 80 MiB  
>   kvflowcontrol.tokens.send.regular.deducted    | 10 MiB  
>   kvflowcontrol.tokens.send.regular.returned    | 10 MiB  
>   kvflowcontrol.tokens.send.regular.unaccounted | 0 B     
163c181
<    FROM crdb_internal.kv_flow_control_handles
---
>    FROM crdb_internal.kv_flow_control_handles_v2
176,178c194,196
< 	   crdb_internal.humanize_bytes(available_regular_tokens),
< 	   crdb_internal.humanize_bytes(available_elastic_tokens)
<   FROM crdb_internal.kv_flow_controller
---
> 	   crdb_internal.humanize_bytes(available_eval_regular_tokens),
> 	   crdb_internal.humanize_bytes(available_eval_elastic_tokens)
>   FROM crdb_internal.kv_flow_controller_v2
181,187c199,205
<   range_id | regular_available | elastic_available  
< -----------+-------------------+--------------------
<   1        | 16 MiB            | 8.0 MiB            
<   2        | 16 MiB            | 8.0 MiB            
<   3        | 16 MiB            | 8.0 MiB            
<   4        | 16 MiB            | 8.0 MiB            
<   5        | 16 MiB            | 8.0 MiB            
---
>   range_id | eval_regular_available | eval_elastic_available  
> -----------+------------------------+-------------------------
>   1        | 16 MiB                 | 8.0 MiB                 
>   2        | 16 MiB                 | 8.0 MiB                 
>   3        | 16 MiB                 | 8.0 MiB                 
>   4        | 16 MiB                 | 8.0 MiB                 
>   5        | 16 MiB                 | 8.0 MiB

raft_transport_break

File not found in comparison directory

raft_transport_culled

File not found in comparison directory

split_merge

9c9
<    WHERE name LIKE '%kvadmission%tokens%'
---
>    WHERE name LIKE '%kvflowcontrol%tokens%'
12,19c12,27
<   kvadmission.flow_controller.elastic_tokens_available   | 24 MiB   
<   kvadmission.flow_controller.elastic_tokens_deducted    | 3.0 MiB  
<   kvadmission.flow_controller.elastic_tokens_returned    | 3.0 MiB  
<   kvadmission.flow_controller.elastic_tokens_unaccounted | 0 B      
<   kvadmission.flow_controller.regular_tokens_available   | 48 MiB   
<   kvadmission.flow_controller.regular_tokens_deducted    | 3.0 MiB  
<   kvadmission.flow_controller.regular_tokens_returned    | 3.0 MiB  
<   kvadmission.flow_controller.regular_tokens_unaccounted | 0 B      
---
>   kvflowcontrol.tokens.eval.elastic.available   | 24 MiB   
>   kvflowcontrol.tokens.eval.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.returned    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.eval.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.regular.returned    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.regular.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.elastic.available   | 24 MiB   
>   kvflowcontrol.tokens.send.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.send.elastic.returned    | 3.0 MiB  
>   kvflowcontrol.tokens.send.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.send.regular.returned    | 3.0 MiB  
>   kvflowcontrol.tokens.send.regular.unaccounted | 0 B      
31c39
<    WHERE name LIKE '%kvadmission%tokens%'
---
>    WHERE name LIKE '%kvflowcontrol%tokens%'
34,41c42,57
<   kvadmission.flow_controller.elastic_tokens_available   | 24 MiB  
<   kvadmission.flow_controller.elastic_tokens_deducted    | 18 MiB  
<   kvadmission.flow_controller.elastic_tokens_returned    | 18 MiB  
<   kvadmission.flow_controller.elastic_tokens_unaccounted | 0 B     
<   kvadmission.flow_controller.regular_tokens_available   | 48 MiB  
<   kvadmission.flow_controller.regular_tokens_deducted    | 18 MiB  
<   kvadmission.flow_controller.regular_tokens_returned    | 18 MiB  
<   kvadmission.flow_controller.regular_tokens_unaccounted | 0 B     
---
>   kvflowcontrol.tokens.eval.elastic.available   | 24 MiB  
>   kvflowcontrol.tokens.eval.elastic.deducted    | 18 MiB  
>   kvflowcontrol.tokens.eval.elastic.returned    | 18 MiB  
>   kvflowcontrol.tokens.eval.elastic.unaccounted | 0 B     
>   kvflowcontrol.tokens.eval.regular.available   | 48 MiB  
>   kvflowcontrol.tokens.eval.regular.deducted    | 18 MiB  
>   kvflowcontrol.tokens.eval.regular.returned    | 18 MiB  
>   kvflowcontrol.tokens.eval.regular.unaccounted | 0 B     
>   kvflowcontrol.tokens.send.elastic.available   | 24 MiB  
>   kvflowcontrol.tokens.send.elastic.deducted    | 18 MiB  
>   kvflowcontrol.tokens.send.elastic.returned    | 18 MiB  
>   kvflowcontrol.tokens.send.elastic.unaccounted | 0 B     
>   kvflowcontrol.tokens.send.regular.available   | 48 MiB  
>   kvflowcontrol.tokens.send.regular.deducted    | 18 MiB  
>   kvflowcontrol.tokens.send.regular.returned    | 18 MiB  
>   kvflowcontrol.tokens.send.regular.unaccounted | 0 B     
46c62
<     FROM crdb_internal.kv_flow_control_handles
---
>     FROM crdb_internal.kv_flow_control_handles_v2
64c80
<    WHERE name LIKE '%kvadmission%tokens%'
---
>    WHERE name LIKE '%kvflowcontrol%tokens%'
67,74c83,98
<   kvadmission.flow_controller.elastic_tokens_available   | 24 MiB  
<   kvadmission.flow_controller.elastic_tokens_deducted    | 30 MiB  
<   kvadmission.flow_controller.elastic_tokens_returned    | 30 MiB  
<   kvadmission.flow_controller.elastic_tokens_unaccounted | 0 B     
<   kvadmission.flow_controller.regular_tokens_available   | 48 MiB  
<   kvadmission.flow_controller.regular_tokens_deducted    | 30 MiB  
<   kvadmission.flow_controller.regular_tokens_returned    | 30 MiB  
<   kvadmission.flow_controller.regular_tokens_unaccounted | 0 B     
---
>   kvflowcontrol.tokens.eval.elastic.available   | 24 MiB  
>   kvflowcontrol.tokens.eval.elastic.deducted    | 30 MiB  
>   kvflowcontrol.tokens.eval.elastic.returned    | 30 MiB  
>   kvflowcontrol.tokens.eval.elastic.unaccounted | 0 B     
>   kvflowcontrol.tokens.eval.regular.available   | 48 MiB  
>   kvflowcontrol.tokens.eval.regular.deducted    | 30 MiB  
>   kvflowcontrol.tokens.eval.regular.returned    | 30 MiB  
>   kvflowcontrol.tokens.eval.regular.unaccounted | 0 B     
>   kvflowcontrol.tokens.send.elastic.available   | 24 MiB  
>   kvflowcontrol.tokens.send.elastic.deducted    | 30 MiB  
>   kvflowcontrol.tokens.send.elastic.returned    | 30 MiB  
>   kvflowcontrol.tokens.send.elastic.unaccounted | 0 B     
>   kvflowcontrol.tokens.send.regular.available   | 48 MiB  
>   kvflowcontrol.tokens.send.regular.deducted    | 30 MiB  
>   kvflowcontrol.tokens.send.regular.returned    | 30 MiB  
>   kvflowcontrol.tokens.send.regular.unaccounted | 0 B     
79c103
<     FROM crdb_internal.kv_flow_control_handles
---
>     FROM crdb_internal.kv_flow_control_handles_v2

transfer_lease

12c12
<    WHERE name LIKE '%kvadmission%regular_tokens%'
---
>    WHERE name LIKE '%kvflowcontrol%tokens%'
15,18c15,30
<   kvadmission.flow_controller.regular_tokens_available   | 45 MiB   
<   kvadmission.flow_controller.regular_tokens_deducted    | 3.0 MiB  
<   kvadmission.flow_controller.regular_tokens_returned    | 0 B      
<   kvadmission.flow_controller.regular_tokens_unaccounted | 0 B      
---
>   kvflowcontrol.tokens.eval.elastic.available   | 21 MiB   
>   kvflowcontrol.tokens.eval.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.returned    | 0 B      
>   kvflowcontrol.tokens.eval.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.eval.regular.available   | 45 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.regular.returned    | 0 B      
>   kvflowcontrol.tokens.eval.regular.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.elastic.available   | 21 MiB   
>   kvflowcontrol.tokens.send.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.send.elastic.returned    | 0 B      
>   kvflowcontrol.tokens.send.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.regular.available   | 45 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.send.regular.returned    | 0 B      
>   kvflowcontrol.tokens.send.regular.unaccounted | 0 B      
28c40
<    WHERE name LIKE '%kvadmission%regular_tokens%'
---
>    WHERE name LIKE '%kvflowcontrol%tokens%'
31,34c43,58
<   kvadmission.flow_controller.regular_tokens_available   | 48 MiB   
<   kvadmission.flow_controller.regular_tokens_deducted    | 3.0 MiB  
<   kvadmission.flow_controller.regular_tokens_returned    | 3.0 MiB  
<   kvadmission.flow_controller.regular_tokens_unaccounted | 0 B      
---
>   kvflowcontrol.tokens.eval.elastic.available   | 24 MiB   
>   kvflowcontrol.tokens.eval.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.returned    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.eval.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.regular.returned    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.regular.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.elastic.available   | 24 MiB   
>   kvflowcontrol.tokens.send.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.send.elastic.returned    | 3.0 MiB  
>   kvflowcontrol.tokens.send.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.send.regular.returned    | 3.0 MiB  
>   kvflowcontrol.tokens.send.regular.unaccounted | 0 B

unquiesced_range

12c12
<    WHERE name LIKE '%kvadmission%elastic_tokens%'
---
>    WHERE name LIKE '%kvflowcontrol%tokens%'
15,18c15,30
<   kvadmission.flow_controller.elastic_tokens_available   | 21 MiB   
<   kvadmission.flow_controller.elastic_tokens_deducted    | 3.0 MiB  
<   kvadmission.flow_controller.elastic_tokens_returned    | 0 B      
<   kvadmission.flow_controller.elastic_tokens_unaccounted | 0 B      
---
>   kvflowcontrol.tokens.eval.elastic.available   | 21 MiB   
>   kvflowcontrol.tokens.eval.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.returned    | 0 B      
>   kvflowcontrol.tokens.eval.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.eval.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 0 B      
>   kvflowcontrol.tokens.eval.regular.returned    | 0 B      
>   kvflowcontrol.tokens.eval.regular.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.elastic.available   | 21 MiB   
>   kvflowcontrol.tokens.send.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.send.elastic.returned    | 0 B      
>   kvflowcontrol.tokens.send.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 0 B      
>   kvflowcontrol.tokens.send.regular.returned    | 0 B      
>   kvflowcontrol.tokens.send.regular.unaccounted | 0 B      
34c46
<    WHERE name LIKE '%kvadmission%elastic_tokens%'
---
>    WHERE name LIKE '%kvflowcontrol%tokens%'
37,40c49,64
<   kvadmission.flow_controller.elastic_tokens_available   | 22 MiB   
<   kvadmission.flow_controller.elastic_tokens_deducted    | 3.0 MiB  
<   kvadmission.flow_controller.elastic_tokens_returned    | 1.0 MiB  
<   kvadmission.flow_controller.elastic_tokens_unaccounted | 0 B      
---
>   kvflowcontrol.tokens.eval.elastic.available   | 22 MiB   
>   kvflowcontrol.tokens.eval.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.returned    | 1.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.eval.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 0 B      
>   kvflowcontrol.tokens.eval.regular.returned    | 0 B      
>   kvflowcontrol.tokens.eval.regular.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.elastic.available   | 22 MiB   
>   kvflowcontrol.tokens.send.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.send.elastic.returned    | 1.0 MiB  
>   kvflowcontrol.tokens.send.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 0 B      
>   kvflowcontrol.tokens.send.regular.returned    | 0 B      
>   kvflowcontrol.tokens.send.regular.unaccounted | 0 B      
53c77
<    WHERE name LIKE '%kvadmission%elastic_tokens%'
---
>    WHERE name LIKE '%kvflowcontrol%tokens%'
56,59c80,95
<   kvadmission.flow_controller.elastic_tokens_available   | 24 MiB   
<   kvadmission.flow_controller.elastic_tokens_deducted    | 3.0 MiB  
<   kvadmission.flow_controller.elastic_tokens_returned    | 3.0 MiB  
<   kvadmission.flow_controller.elastic_tokens_unaccounted | 0 B      
---
>   kvflowcontrol.tokens.eval.elastic.available   | 24 MiB   
>   kvflowcontrol.tokens.eval.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.returned    | 3.0 MiB  
>   kvflowcontrol.tokens.eval.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.eval.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 0 B      
>   kvflowcontrol.tokens.eval.regular.returned    | 0 B      
>   kvflowcontrol.tokens.eval.regular.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.elastic.available   | 24 MiB   
>   kvflowcontrol.tokens.send.elastic.deducted    | 3.0 MiB  
>   kvflowcontrol.tokens.send.elastic.returned    | 3.0 MiB  
>   kvflowcontrol.tokens.send.elastic.unaccounted | 0 B      
>   kvflowcontrol.tokens.send.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 0 B      
>   kvflowcontrol.tokens.send.regular.returned    | 0 B      
>   kvflowcontrol.tokens.send.regular.unaccounted | 0 B

kvoli · 2024-09-20T17:51:47Z

Here are the diffs for v1 vs v2 encoding testdata files added in this PR. There are no notable diffs. The diffs shown reflect rac2 only deducting elastic tokens, regardless of the original priority when v1 encoding is enabled.

admission_post_split_merge

0a1,7
> # -- When using v1 encoding (V2EnabledWhenLeaderV1Encoding), all entries which
> # -- are subject to admission control are encoded as `raftpb.LowPri`,
> # -- regardless of their original priority; to avoid the overhead of
> # -- deserializing the raft admission metadata. Therefore, as the underlying
> # -- test is shared between the v1 and v2 encoding testdata files, the reader
> # -- should interpret any comments referring to regular tokens as referring to
> # -- elastic token.
17,18c24,25
<   kvflowcontrol.tokens.eval.regular.available   | 42 MiB   
<   kvflowcontrol.tokens.eval.regular.deducted    | 6.0 MiB  
---
>   kvflowcontrol.tokens.eval.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 0 B      
25,26c32,33
<   kvflowcontrol.tokens.send.regular.available   | 42 MiB   
<   kvflowcontrol.tokens.send.regular.deducted    | 6.0 MiB  
---
>   kvflowcontrol.tokens.send.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 0 B      
47,48c54,55
<   kvflowcontrol.tokens.eval.regular.available   | 27 MiB   
<   kvflowcontrol.tokens.eval.regular.deducted    | 21 MiB   
---
>   kvflowcontrol.tokens.eval.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 0 B      
55,56c62,63
<   kvflowcontrol.tokens.send.regular.available   | 27 MiB   
<   kvflowcontrol.tokens.send.regular.deducted    | 21 MiB   
---
>   kvflowcontrol.tokens.send.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 0 B      
90,92c97,99
<   kvflowcontrol.tokens.eval.regular.available   | 24 MiB   
<   kvflowcontrol.tokens.eval.regular.deducted    | 33 MiB   
<   kvflowcontrol.tokens.eval.regular.returned    | 9.0 MiB  
---
>   kvflowcontrol.tokens.eval.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 0 B      
>   kvflowcontrol.tokens.eval.regular.returned    | 0 B      
98,100c105,107
<   kvflowcontrol.tokens.send.regular.available   | 24 MiB   
<   kvflowcontrol.tokens.send.regular.deducted    | 33 MiB   
<   kvflowcontrol.tokens.send.regular.returned    | 9.0 MiB  
---
>   kvflowcontrol.tokens.send.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 0 B      
>   kvflowcontrol.tokens.send.regular.returned    | 0 B      
132,133c139,140
<   kvflowcontrol.tokens.eval.regular.deducted    | 33 MiB  
<   kvflowcontrol.tokens.eval.regular.returned    | 33 MiB  
---
>   kvflowcontrol.tokens.eval.regular.deducted    | 0 B     
>   kvflowcontrol.tokens.eval.regular.returned    | 0 B     
140,141c147,148
<   kvflowcontrol.tokens.send.regular.deducted    | 33 MiB  
<   kvflowcontrol.tokens.send.regular.returned    | 33 MiB  
---
>   kvflowcontrol.tokens.send.regular.deducted    | 0 B     
>   kvflowcontrol.tokens.send.regular.returned    | 0 B

basic

0a1,7
> # -- When using v1 encoding (V2EnabledWhenLeaderV1Encoding), all entries which
> # -- are subject to admission control are encoded as `raftpb.LowPri`,
> # -- regardless of their original priority; to avoid the overhead of
> # -- deserializing the raft admission metadata. Therefore, as the underlying
> # -- test is shared between the v1 and v2 encoding testdata files, the reader
> # -- should interpret any comments referring to regular tokens as referring to
> # -- elastic token.
74,75c81,82
<   kvflowcontrol.tokens.eval.regular.deducted    | 3.0 MiB  
<   kvflowcontrol.tokens.eval.regular.returned    | 3.0 MiB  
---
>   kvflowcontrol.tokens.eval.regular.deducted    | 0 B      
>   kvflowcontrol.tokens.eval.regular.returned    | 0 B      
82,83c89,90
<   kvflowcontrol.tokens.send.regular.deducted    | 3.0 MiB  
<   kvflowcontrol.tokens.send.regular.returned    | 3.0 MiB  
---
>   kvflowcontrol.tokens.send.regular.deducted    | 0 B      
>   kvflowcontrol.tokens.send.regular.returned    | 0 B

blocked_admission

0a1,7
> # -- When using v1 encoding (V2EnabledWhenLeaderV1Encoding), all entries which
> # -- are subject to admission control are encoded as `raftpb.LowPri`,
> # -- regardless of their original priority; to avoid the overhead of
> # -- deserializing the raft admission metadata. Therefore, as the underlying
> # -- test is shared between the v1 and v2 encoding testdata files, the reader
> # -- should interpret any comments referring to regular tokens as referring to
> # -- elastic token.
19,20c26,27
<   kvflowcontrol.tokens.eval.regular.available   | 33 MiB   
<   kvflowcontrol.tokens.eval.regular.deducted    | 15 MiB   
---
>   kvflowcontrol.tokens.eval.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 0 B      
27,28c34,35
<   kvflowcontrol.tokens.send.regular.available   | 33 MiB   
<   kvflowcontrol.tokens.send.regular.deducted    | 15 MiB   
---
>   kvflowcontrol.tokens.send.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 0 B      
48,64c55,71
<   range_id | store_id |  priority  | tokens   
< -----------+----------+------------+----------
<   70       | 1        | normal-pri | 1.0 MiB  
<   70       | 1        | normal-pri | 1.0 MiB  
<   70       | 1        | normal-pri | 1.0 MiB  
<   70       | 1        | normal-pri | 1.0 MiB  
<   70       | 1        | normal-pri | 1.0 MiB  
<   70       | 2        | normal-pri | 1.0 MiB  
<   70       | 2        | normal-pri | 1.0 MiB  
<   70       | 2        | normal-pri | 1.0 MiB  
<   70       | 2        | normal-pri | 1.0 MiB  
<   70       | 2        | normal-pri | 1.0 MiB  
<   70       | 3        | normal-pri | 1.0 MiB  
<   70       | 3        | normal-pri | 1.0 MiB  
<   70       | 3        | normal-pri | 1.0 MiB  
<   70       | 3        | normal-pri | 1.0 MiB  
<   70       | 3        | normal-pri | 1.0 MiB  
---
>   range_id | store_id | priority | tokens   
> -----------+----------+----------+----------
>   70       | 1        | low-pri  | 1.0 MiB  
>   70       | 1        | low-pri  | 1.0 MiB  
>   70       | 1        | low-pri  | 1.0 MiB  
>   70       | 1        | low-pri  | 1.0 MiB  
>   70       | 1        | low-pri  | 1.0 MiB  
>   70       | 2        | low-pri  | 1.0 MiB  
>   70       | 2        | low-pri  | 1.0 MiB  
>   70       | 2        | low-pri  | 1.0 MiB  
>   70       | 2        | low-pri  | 1.0 MiB  
>   70       | 2        | low-pri  | 1.0 MiB  
>   70       | 3        | low-pri  | 1.0 MiB  
>   70       | 3        | low-pri  | 1.0 MiB  
>   70       | 3        | low-pri  | 1.0 MiB  
>   70       | 3        | low-pri  | 1.0 MiB  
>   70       | 3        | low-pri  | 1.0 MiB  
83,84c90,91
<   kvflowcontrol.tokens.eval.regular.deducted    | 15 MiB  
<   kvflowcontrol.tokens.eval.regular.returned    | 15 MiB  
---
>   kvflowcontrol.tokens.eval.regular.deducted    | 0 B     
>   kvflowcontrol.tokens.eval.regular.returned    | 0 B     
91,92c98,99
<   kvflowcontrol.tokens.send.regular.deducted    | 15 MiB  
<   kvflowcontrol.tokens.send.regular.returned    | 15 MiB  
---
>   kvflowcontrol.tokens.send.regular.deducted    | 0 B     
>   kvflowcontrol.tokens.send.regular.returned    | 0 B

class_prioritization

0a1,7
> # -- When using v1 encoding (V2EnabledWhenLeaderV1Encoding), all entries which
> # -- are subject to admission control are encoded as `raftpb.LowPri`,
> # -- regardless of their original priority; to avoid the overhead of
> # -- deserializing the raft admission metadata. Therefore, as the underlying
> # -- test is shared between the v1 and v2 encoding testdata files, the reader
> # -- should interpret any comments referring to regular tokens as referring to
> # -- elastic token.
48,49c55,56
<   kvflowcontrol.tokens.eval.regular.available   | 45 MiB   
<   kvflowcontrol.tokens.eval.regular.deducted    | 3.0 MiB  
---
>   kvflowcontrol.tokens.eval.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 0 B      
56,57c63,64
<   kvflowcontrol.tokens.send.regular.available   | 45 MiB   
<   kvflowcontrol.tokens.send.regular.deducted    | 3.0 MiB  
---
>   kvflowcontrol.tokens.send.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 0 B      
77,78c84,85
<   kvflowcontrol.tokens.eval.regular.deducted    | 3.0 MiB  
<   kvflowcontrol.tokens.eval.regular.returned    | 3.0 MiB  
---
>   kvflowcontrol.tokens.eval.regular.deducted    | 0 B      
>   kvflowcontrol.tokens.eval.regular.returned    | 0 B      
85,86c92,93
<   kvflowcontrol.tokens.send.regular.deducted    | 3.0 MiB  
<   kvflowcontrol.tokens.send.regular.returned    | 3.0 MiB  
---
>   kvflowcontrol.tokens.send.regular.deducted    | 0 B      
>   kvflowcontrol.tokens.send.regular.returned    | 0 B

crashed_node

0a1,7
> # -- When using v1 encoding (V2EnabledWhenLeaderV1Encoding), all entries which
> # -- are subject to admission control are encoded as `raftpb.LowPri`,
> # -- regardless of their original priority; to avoid the overhead of
> # -- deserializing the raft admission metadata. Therefore, as the underlying
> # -- test is shared between the v1 and v2 encoding testdata files, the reader
> # -- should interpret any comments referring to regular tokens as referring to
> # -- elastic token.
19,20c26,27
<   kvflowcontrol.tokens.eval.regular.available   | 22 MiB   
<   kvflowcontrol.tokens.eval.regular.deducted    | 10 MiB   
---
>   kvflowcontrol.tokens.eval.regular.available   | 32 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 0 B      
27,28c34,35
<   kvflowcontrol.tokens.send.regular.available   | 22 MiB   
<   kvflowcontrol.tokens.send.regular.deducted    | 10 MiB   
---
>   kvflowcontrol.tokens.send.regular.available   | 32 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 0 B      
67,69c74,76
<   kvflowcontrol.tokens.eval.regular.available   | 27 MiB   
<   kvflowcontrol.tokens.eval.regular.deducted    | 10 MiB   
<   kvflowcontrol.tokens.eval.regular.returned    | 5.0 MiB  
---
>   kvflowcontrol.tokens.eval.regular.available   | 32 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 0 B      
>   kvflowcontrol.tokens.eval.regular.returned    | 0 B      
75,77c82,84
<   kvflowcontrol.tokens.send.regular.available   | 27 MiB   
<   kvflowcontrol.tokens.send.regular.deducted    | 10 MiB   
<   kvflowcontrol.tokens.send.regular.returned    | 5.0 MiB  
---
>   kvflowcontrol.tokens.send.regular.available   | 32 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 0 B      
>   kvflowcontrol.tokens.send.regular.returned    | 0 B

granter_admit_one_by_one

0a1,7
> # -- When using v1 encoding (V2EnabledWhenLeaderV1Encoding), all entries which
> # -- are subject to admission control are encoded as `raftpb.LowPri`,
> # -- regardless of their original priority; to avoid the overhead of
> # -- deserializing the raft admission metadata. Therefore, as the underlying
> # -- test is shared between the v1 and v2 encoding testdata files, the reader
> # -- should interpret any comments referring to regular tokens as referring to
> # -- elastic token.
19,20c26,27
<   kvflowcontrol.tokens.eval.regular.available   | 45 MiB   
<   kvflowcontrol.tokens.eval.regular.deducted    | 3.0 MiB  
---
>   kvflowcontrol.tokens.eval.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 0 B      
27,28c34,35
<   kvflowcontrol.tokens.send.regular.available   | 45 MiB   
<   kvflowcontrol.tokens.send.regular.deducted    | 3.0 MiB  
---
>   kvflowcontrol.tokens.send.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 0 B      
60,61c67,68
<   kvflowcontrol.tokens.eval.regular.deducted    | 3.0 MiB  
<   kvflowcontrol.tokens.eval.regular.returned    | 3.0 MiB  
---
>   kvflowcontrol.tokens.eval.regular.deducted    | 0 B      
>   kvflowcontrol.tokens.eval.regular.returned    | 0 B      
68,69c75,76
<   kvflowcontrol.tokens.send.regular.deducted    | 3.0 MiB  
<   kvflowcontrol.tokens.send.regular.returned    | 3.0 MiB  
---
>   kvflowcontrol.tokens.send.regular.deducted    | 0 B      
>   kvflowcontrol.tokens.send.regular.returned    | 0 B

leader_not_leaseholder

0a1,7
> # -- When using v1 encoding (V2EnabledWhenLeaderV1Encoding), all entries which
> # -- are subject to admission control are encoded as `raftpb.LowPri`,
> # -- regardless of their original priority; to avoid the overhead of
> # -- deserializing the raft admission metadata. Therefore, as the underlying
> # -- test is shared between the v1 and v2 encoding testdata files, the reader
> # -- should interpret any comments referring to regular tokens as referring to
> # -- elastic token.
19,20c26,27
<   kvflowcontrol.tokens.eval.regular.available   | 45 MiB   
<   kvflowcontrol.tokens.eval.regular.deducted    | 3.0 MiB  
---
>   kvflowcontrol.tokens.eval.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 0 B      
27,28c34,35
<   kvflowcontrol.tokens.send.regular.available   | 45 MiB   
<   kvflowcontrol.tokens.send.regular.deducted    | 3.0 MiB  
---
>   kvflowcontrol.tokens.send.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 0 B      
47,48c54,55
<   kvflowcontrol.tokens.eval.regular.available   | 45 MiB   
<   kvflowcontrol.tokens.eval.regular.deducted    | 3.0 MiB  
---
>   kvflowcontrol.tokens.eval.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 0 B      
55,56c62,63
<   kvflowcontrol.tokens.send.regular.available   | 45 MiB   
<   kvflowcontrol.tokens.send.regular.deducted    | 3.0 MiB  
---
>   kvflowcontrol.tokens.send.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 0 B      
72,73c79,80
<   kvflowcontrol.tokens.eval.regular.deducted    | 3.0 MiB  
<   kvflowcontrol.tokens.eval.regular.returned    | 3.0 MiB  
---
>   kvflowcontrol.tokens.eval.regular.deducted    | 0 B      
>   kvflowcontrol.tokens.eval.regular.returned    | 0 B      
80,81c87,88
<   kvflowcontrol.tokens.send.regular.deducted    | 3.0 MiB  
<   kvflowcontrol.tokens.send.regular.returned    | 3.0 MiB  
---
>   kvflowcontrol.tokens.send.regular.deducted    | 0 B      
>   kvflowcontrol.tokens.send.regular.returned    | 0 B      
101,102c108,109
<   kvflowcontrol.tokens.eval.regular.deducted    | 3.0 MiB  
<   kvflowcontrol.tokens.eval.regular.returned    | 3.0 MiB  
---
>   kvflowcontrol.tokens.eval.regular.deducted    | 0 B      
>   kvflowcontrol.tokens.eval.regular.returned    | 0 B      
109,110c116,117
<   kvflowcontrol.tokens.send.regular.deducted    | 3.0 MiB  
<   kvflowcontrol.tokens.send.regular.returned    | 3.0 MiB  
---
>   kvflowcontrol.tokens.send.regular.deducted    | 0 B      
>   kvflowcontrol.tokens.send.regular.returned    | 0 B

quiesced_range

0a1,7
> # -- When using v1 encoding (V2EnabledWhenLeaderV1Encoding), all entries which
> # -- are subject to admission control are encoded as `raftpb.LowPri`,
> # -- regardless of their original priority; to avoid the overhead of
> # -- deserializing the raft admission metadata. Therefore, as the underlying
> # -- test is shared between the v1 and v2 encoding testdata files, the reader
> # -- should interpret any comments referring to regular tokens as referring to
> # -- elastic token.

raft_membership

0a1,7
> # -- When using v1 encoding (V2EnabledWhenLeaderV1Encoding), all entries which
> # -- are subject to admission control are encoded as `raftpb.LowPri`,
> # -- regardless of their original priority; to avoid the overhead of
> # -- deserializing the raft admission metadata. Therefore, as the underlying
> # -- test is shared between the v1 and v2 encoding testdata files, the reader
> # -- should interpret any comments referring to regular tokens as referring to
> # -- elastic token.
19,20c26,27
<   kvflowcontrol.tokens.eval.regular.available   | 45 MiB   
<   kvflowcontrol.tokens.eval.regular.deducted    | 3.0 MiB  
---
>   kvflowcontrol.tokens.eval.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 0 B      
27,28c34,35
<   kvflowcontrol.tokens.send.regular.available   | 45 MiB   
<   kvflowcontrol.tokens.send.regular.deducted    | 3.0 MiB  
---
>   kvflowcontrol.tokens.send.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 0 B      
118,119c125,126
<   kvflowcontrol.tokens.eval.regular.deducted    | 11 MiB  
<   kvflowcontrol.tokens.eval.regular.returned    | 11 MiB  
---
>   kvflowcontrol.tokens.eval.regular.deducted    | 0 B     
>   kvflowcontrol.tokens.eval.regular.returned    | 0 B     
126,127c133,134
<   kvflowcontrol.tokens.send.regular.deducted    | 11 MiB  
<   kvflowcontrol.tokens.send.regular.returned    | 11 MiB  
---
>   kvflowcontrol.tokens.send.regular.deducted    | 0 B     
>   kvflowcontrol.tokens.send.regular.returned    | 0 B

raft_membership_remove_self

0a1,7
> # -- When using v1 encoding (V2EnabledWhenLeaderV1Encoding), all entries which
> # -- are subject to admission control are encoded as `raftpb.LowPri`,
> # -- regardless of their original priority; to avoid the overhead of
> # -- deserializing the raft admission metadata. Therefore, as the underlying
> # -- test is shared between the v1 and v2 encoding testdata files, the reader
> # -- should interpret any comments referring to regular tokens as referring to
> # -- elastic token.
19,20c26,27
<   kvflowcontrol.tokens.eval.regular.available   | 45 MiB   
<   kvflowcontrol.tokens.eval.regular.deducted    | 3.0 MiB  
---
>   kvflowcontrol.tokens.eval.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 0 B      
27,28c34,35
<   kvflowcontrol.tokens.send.regular.available   | 45 MiB   
<   kvflowcontrol.tokens.send.regular.deducted    | 3.0 MiB  
---
>   kvflowcontrol.tokens.send.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 0 B      
49,50c56,57
<   kvflowcontrol.tokens.eval.regular.deducted    | 3.0 MiB  
<   kvflowcontrol.tokens.eval.regular.returned    | 3.0 MiB  
---
>   kvflowcontrol.tokens.eval.regular.deducted    | 0 B      
>   kvflowcontrol.tokens.eval.regular.returned    | 0 B      
57,58c64,65
<   kvflowcontrol.tokens.send.regular.deducted    | 3.0 MiB  
<   kvflowcontrol.tokens.send.regular.returned    | 3.0 MiB  
---
>   kvflowcontrol.tokens.send.regular.deducted    | 0 B      
>   kvflowcontrol.tokens.send.regular.returned    | 0 B      
89,90c96,97
<   kvflowcontrol.tokens.eval.regular.deducted    | 3.0 MiB  
<   kvflowcontrol.tokens.eval.regular.returned    | 3.0 MiB  
---
>   kvflowcontrol.tokens.eval.regular.deducted    | 0 B      
>   kvflowcontrol.tokens.eval.regular.returned    | 0 B      
97,98c104,105
<   kvflowcontrol.tokens.send.regular.deducted    | 3.0 MiB  
<   kvflowcontrol.tokens.send.regular.returned    | 3.0 MiB  
---
>   kvflowcontrol.tokens.send.regular.deducted    | 0 B      
>   kvflowcontrol.tokens.send.regular.returned    | 0 B

raft_snapshot

0a1,7
> # -- When using v1 encoding (V2EnabledWhenLeaderV1Encoding), all entries which
> # -- are subject to admission control are encoded as `raftpb.LowPri`,
> # -- regardless of their original priority; to avoid the overhead of
> # -- deserializing the raft admission metadata. Therefore, as the underlying
> # -- test is shared between the v1 and v2 encoding testdata files, the reader
> # -- should interpret any comments referring to regular tokens as referring to
> # -- elastic token.
16,17c23,24
<   kvflowcontrol.tokens.eval.regular.available   | 75 MiB   
<   kvflowcontrol.tokens.eval.regular.deducted    | 5.0 MiB  
---
>   kvflowcontrol.tokens.eval.regular.available   | 80 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 0 B      
24,25c31,32
<   kvflowcontrol.tokens.send.regular.available   | 75 MiB   
<   kvflowcontrol.tokens.send.regular.deducted    | 5.0 MiB  
---
>   kvflowcontrol.tokens.send.regular.available   | 80 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 0 B      
77,78c84,85
<   kvflowcontrol.tokens.eval.regular.available   | 70 MiB  
<   kvflowcontrol.tokens.eval.regular.deducted    | 10 MiB  
---
>   kvflowcontrol.tokens.eval.regular.available   | 80 MiB  
>   kvflowcontrol.tokens.eval.regular.deducted    | 0 B     
85,86c92,93
<   kvflowcontrol.tokens.send.regular.available   | 70 MiB  
<   kvflowcontrol.tokens.send.regular.deducted    | 10 MiB  
---
>   kvflowcontrol.tokens.send.regular.available   | 80 MiB  
>   kvflowcontrol.tokens.send.regular.deducted    | 0 B     
124,126c131,133
<   kvflowcontrol.tokens.eval.regular.available   | 74 MiB   
<   kvflowcontrol.tokens.eval.regular.deducted    | 10 MiB   
<   kvflowcontrol.tokens.eval.regular.returned    | 4.0 MiB  
---
>   kvflowcontrol.tokens.eval.regular.available   | 80 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 0 B      
>   kvflowcontrol.tokens.eval.regular.returned    | 0 B      
132,134c139,141
<   kvflowcontrol.tokens.send.regular.available   | 74 MiB   
<   kvflowcontrol.tokens.send.regular.deducted    | 10 MiB   
<   kvflowcontrol.tokens.send.regular.returned    | 4.0 MiB  
---
>   kvflowcontrol.tokens.send.regular.available   | 80 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 0 B      
>   kvflowcontrol.tokens.send.regular.returned    | 0 B      
166,167c173,174
<   kvflowcontrol.tokens.eval.regular.deducted    | 10 MiB  
<   kvflowcontrol.tokens.eval.regular.returned    | 10 MiB  
---
>   kvflowcontrol.tokens.eval.regular.deducted    | 0 B     
>   kvflowcontrol.tokens.eval.regular.returned    | 0 B     
174,175c181,182
<   kvflowcontrol.tokens.send.regular.deducted    | 10 MiB  
<   kvflowcontrol.tokens.send.regular.returned    | 10 MiB  
---
>   kvflowcontrol.tokens.send.regular.deducted    | 0 B     
>   kvflowcontrol.tokens.send.regular.returned    | 0 B

split_merge

0a1,7
> # -- When using v1 encoding (V2EnabledWhenLeaderV1Encoding), all entries which
> # -- are subject to admission control are encoded as `raftpb.LowPri`,
> # -- regardless of their original priority; to avoid the overhead of
> # -- deserializing the raft admission metadata. Therefore, as the underlying
> # -- test is shared between the v1 and v2 encoding testdata files, the reader
> # -- should interpret any comments referring to regular tokens as referring to
> # -- elastic token.
17,18c24,25
<   kvflowcontrol.tokens.eval.regular.deducted    | 3.0 MiB  
<   kvflowcontrol.tokens.eval.regular.returned    | 3.0 MiB  
---
>   kvflowcontrol.tokens.eval.regular.deducted    | 0 B      
>   kvflowcontrol.tokens.eval.regular.returned    | 0 B      
25,26c32,33
<   kvflowcontrol.tokens.send.regular.deducted    | 3.0 MiB  
<   kvflowcontrol.tokens.send.regular.returned    | 3.0 MiB  
---
>   kvflowcontrol.tokens.send.regular.deducted    | 0 B      
>   kvflowcontrol.tokens.send.regular.returned    | 0 B      
47,48c54,55
<   kvflowcontrol.tokens.eval.regular.deducted    | 18 MiB  
<   kvflowcontrol.tokens.eval.regular.returned    | 18 MiB  
---
>   kvflowcontrol.tokens.eval.regular.deducted    | 0 B     
>   kvflowcontrol.tokens.eval.regular.returned    | 0 B     
55,56c62,63
<   kvflowcontrol.tokens.send.regular.deducted    | 18 MiB  
<   kvflowcontrol.tokens.send.regular.returned    | 18 MiB  
---
>   kvflowcontrol.tokens.send.regular.deducted    | 0 B     
>   kvflowcontrol.tokens.send.regular.returned    | 0 B     
88,89c95,96
<   kvflowcontrol.tokens.eval.regular.deducted    | 30 MiB  
<   kvflowcontrol.tokens.eval.regular.returned    | 30 MiB  
---
>   kvflowcontrol.tokens.eval.regular.deducted    | 0 B     
>   kvflowcontrol.tokens.eval.regular.returned    | 0 B     
96,97c103,104
<   kvflowcontrol.tokens.send.regular.deducted    | 30 MiB  
<   kvflowcontrol.tokens.send.regular.returned    | 30 MiB  
---
>   kvflowcontrol.tokens.send.regular.deducted    | 0 B     
>   kvflowcontrol.tokens.send.regular.returned    | 0 B

transfer_lease

0a1,7
> # -- When using v1 encoding (V2EnabledWhenLeaderV1Encoding), all entries which
> # -- are subject to admission control are encoded as `raftpb.LowPri`,
> # -- regardless of their original priority; to avoid the overhead of
> # -- deserializing the raft admission metadata. Therefore, as the underlying
> # -- test is shared between the v1 and v2 encoding testdata files, the reader
> # -- should interpret any comments referring to regular tokens as referring to
> # -- elastic token.
19,20c26,27
<   kvflowcontrol.tokens.eval.regular.available   | 45 MiB   
<   kvflowcontrol.tokens.eval.regular.deducted    | 3.0 MiB  
---
>   kvflowcontrol.tokens.eval.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.eval.regular.deducted    | 0 B      
27,28c34,35
<   kvflowcontrol.tokens.send.regular.available   | 45 MiB   
<   kvflowcontrol.tokens.send.regular.deducted    | 3.0 MiB  
---
>   kvflowcontrol.tokens.send.regular.available   | 48 MiB   
>   kvflowcontrol.tokens.send.regular.deducted    | 0 B      
48,49c55,56
<   kvflowcontrol.tokens.eval.regular.deducted    | 3.0 MiB  
<   kvflowcontrol.tokens.eval.regular.returned    | 3.0 MiB  
---
>   kvflowcontrol.tokens.eval.regular.deducted    | 0 B      
>   kvflowcontrol.tokens.eval.regular.returned    | 0 B      
56,57c63,64
<   kvflowcontrol.tokens.send.regular.deducted    | 3.0 MiB  
<   kvflowcontrol.tokens.send.regular.returned    | 3.0 MiB  
---
>   kvflowcontrol.tokens.send.regular.deducted    | 0 B      
>   kvflowcontrol.tokens.send.regular.returned    | 0 B

unquiesced_range

0a1,7
> # -- When using v1 encoding (V2EnabledWhenLeaderV1Encoding), all entries which
> # -- are subject to admission control are encoded as `raftpb.LowPri`,
> # -- regardless of their original priority; to avoid the overhead of
> # -- deserializing the raft admission metadata. Therefore, as the underlying
> # -- test is shared between the v1 and v2 encoding testdata files, the reader
> # -- should interpret any comments referring to regular tokens as referring to
> # -- elastic token.

pkg/kv/kvserver/testdata/flow_control_integration_v2/v1_to_v2_transition

kvoli · 2024-09-24T14:41:10Z

pkg/kv/kvserver/testdata/flow_control_integration_v2/v1_to_v2_transition

+-- (Transferring range lease to n2 (running v1) and allowing leadership to follow.)
+
+
+-- The v2 flow token metrics from n1 having lost the lease and raft leadership. 


Update to check the early return metric as well (just regen the output).

pkg/kv/kvserver/testdata/flow_control_integration_v2/v1_to_v2_transition

sumeerbhola

nice tests!

Reviewed 1 of 9 files at r36, 1 of 1 files at r41, 3 of 3 files at r43, 1 of 1 files at r44, 29 of 29 files at r45, 2 of 2 files at r46.
Reviewable status: complete! 1 of 0 LGTMs obtained (waiting on @kvoli and @pav-kv)

pkg/kv/kvserver/testdata/flow_control_integration_v2/raft_membership_remove_self line 64 at r45 (raw file):

-- n1 should have no connected streams now after transferring the lease to n2.
-- While, n2 should have 3 connected streams to s2,s3,s4. Query the stream count
-- on n1, then on n2.

This comment does not match the output.

pkg/kv/kvserver/testdata/flow_control_integration_v2/raft_snapshot line 48 at r45 (raw file):


-- Observe the total tracked tokens per-stream on n1. 1MiB is (still) tracked
-- for n1-n5, because they are not in StateSnapshot yet and it have likely been

nit: ... and have likely ...

pkg/kv/kvserver/testdata/flow_control_integration_v2/split_merge line 1 at r45 (raw file):

echo

how is this different from admission_post_split_merge?

pkg/kv/kvserver/flow_control_integration_test.go line 2461 at r44 (raw file):

				rangeID:                    rangeID,
				IncomingRaftMessageHandler: si,
				unreliableRaftHandlerFuncs: unreliableRaftHandlerFuncs{

Is verbose logging all that this is doing? According to the following code comment, it seems RaftHeartbeat and RaftMessageResponse would be dropped.

	// If non-nil, can return false to avoid dropping the msg to
	// unreliableRaftHandler.rangeID. If nil, all messages pertaining to the
	// respective range are dropped.

pkg/kv/kvserver/flow_control_integration_test.go line 3169 at r45 (raw file):

-- n1 should have no connected streams now after transferring the lease to n2.
-- While, n2 should have 3 connected streams to s2,s3,s4. Query the stream count
-- on n1, then on n2.

The following does not query n2.

pkg/kv/kvserver/flow_control_integration_test.go line 3878 at r46 (raw file):

				},
				AdmissionControl: &admission.TestingKnobs{
					DisableWorkQueueFastPath: true,

Does setting this to false change anything in the test output? If not, can we chose this randomly.

kvoli

TYFTR!

Dismissed @sumeerbhola from a discussion.
Reviewable status: complete! 1 of 0 LGTMs obtained (waiting on @pav-kv and @sumeerbhola)

pkg/kv/kvserver/testdata/flow_control_integration_v2/raft_membership_remove_self line 64 at r45 (raw file):