Skip to content

Commit

Permalink
Merge cockroachdb#86141
Browse files Browse the repository at this point in the history 
86141: storage: add mvcc rangedels to BenchmarkMVCCExportToSST r=erikgrinaker a=tbg

```
go test ./pkg/storage -run - -timeout 40m -count 10 -benchtime 1s -bench 'MVCCExportToSST' 2>&1 | tee bench.txt && ./bench-compare-params.sh bench.txt numRangeKeys {0,1,100}
```

```
==== old: numRangeKeys=0 new: numRangeKeys=1 ====
name                                                                                       old time/op  new time/op   delta
MVCCExportToSST/numKeys=64/numRevisions=1/exportAllRevisions=false/numRangeKeys=X-24       92.5µs ± 4%  145.7µs ± 9%  +57.51%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=64/numRevisions=1/exportAllRevisions=true/numRangeKeys=X-24        85.5µs ± 2%  146.2µs ±10%  +71.07%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=64/numRevisions=10/exportAllRevisions=false/numRangeKeys=X-24       212µs ± 2%    289µs ± 4%  +36.17%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=64/numRevisions=10/exportAllRevisions=true/numRangeKeys=X-24        351µs ± 4%    455µs ± 2%  +29.53%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=64/numRevisions=100/exportAllRevisions=false/numRangeKeys=X-24      231µs ± 3%    308µs ± 3%  +33.42%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=64/numRevisions=100/exportAllRevisions=true/numRangeKeys=X-24      1.63ms ± 2%   2.09ms ± 2%  +27.97%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=512/numRevisions=1/exportAllRevisions=false/numRangeKeys=X-24       421µs ± 3%    550µs ± 2%  +30.61%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=512/numRevisions=1/exportAllRevisions=true/numRangeKeys=X-24        387µs ± 2%    517µs ± 2%  +33.74%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=512/numRevisions=10/exportAllRevisions=false/numRangeKeys=X-24     1.33ms ± 1%   1.84ms ± 1%  +37.58%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=512/numRevisions=10/exportAllRevisions=true/numRangeKeys=X-24      2.39ms ± 1%   3.17ms ± 2%  +32.63%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=512/numRevisions=100/exportAllRevisions=false/numRangeKeys=X-24    1.46ms ± 1%   1.97ms ± 1%  +34.70%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=512/numRevisions=100/exportAllRevisions=true/numRangeKeys=X-24     12.6ms ± 1%   15.9ms ± 1%  +26.51%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=1024/numRevisions=1/exportAllRevisions=false/numRangeKeys=X-24      761µs ± 2%   1016µs ± 1%  +33.43%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=1024/numRevisions=1/exportAllRevisions=true/numRangeKeys=X-24       705µs ± 2%    948µs ± 1%  +34.44%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=1024/numRevisions=10/exportAllRevisions=false/numRangeKeys=X-24    2.64ms ± 1%   3.58ms ± 1%  +35.74%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=1024/numRevisions=10/exportAllRevisions=true/numRangeKeys=X-24     4.70ms ± 0%   6.25ms ± 1%  +32.90%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=1024/numRevisions=100/exportAllRevisions=false/numRangeKeys=X-24   2.86ms ± 1%   3.84ms ± 1%  +34.11%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=1024/numRevisions=100/exportAllRevisions=true/numRangeKeys=X-24    25.3ms ± 2%   31.5ms ± 1%  +24.55%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=8192/numRevisions=1/exportAllRevisions=false/numRangeKeys=X-24     5.70ms ± 1%   7.41ms ± 1%  +29.97%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=8192/numRevisions=1/exportAllRevisions=true/numRangeKeys=X-24      5.16ms ± 2%   6.86ms ± 2%  +33.11%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=8192/numRevisions=10/exportAllRevisions=false/numRangeKeys=X-24    21.2ms ± 2%   29.1ms ± 2%  +37.18%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=8192/numRevisions=10/exportAllRevisions=true/numRangeKeys=X-24     38.6ms ± 1%   50.0ms ± 1%  +29.55%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=8192/numRevisions=100/exportAllRevisions=false/numRangeKeys=X-24   24.8ms ± 1%   33.4ms ± 3%  +34.73%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=8192/numRevisions=100/exportAllRevisions=true/numRangeKeys=X-24     210ms ± 3%    264ms ± 2%  +25.93%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=65536/numRevisions=1/exportAllRevisions=false/numRangeKeys=X-24    45.1ms ± 1%   58.6ms ± 1%  +30.02%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=65536/numRevisions=1/exportAllRevisions=true/numRangeKeys=X-24     40.9ms ± 1%   54.1ms ± 1%  +32.40%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=65536/numRevisions=10/exportAllRevisions=false/numRangeKeys=X-24    183ms ± 1%    244ms ± 0%  +33.19%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=65536/numRevisions=10/exportAllRevisions=true/numRangeKeys=X-24     314ms ± 0%    413ms ± 2%  +31.33%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=65536/numRevisions=100/exportAllRevisions=false/numRangeKeys=X-24   260ms ±10%    331ms ±11%  +27.50%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=65536/numRevisions=100/exportAllRevisions=true/numRangeKeys=X-24    1.83s ± 1%    2.27s ± 1%  +24.03%  (p=0.008 n=5+5)
```

```
==== old: numRangeKeys=0 new: numRangeKeys=100 ====
name                                                                                       old time/op  new time/op    delta
MVCCExportToSST/numKeys=64/numRevisions=1/exportAllRevisions=false/numRangeKeys=X-24       92.5µs ± 4%  5081.4µs ± 0%  +5391.36%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=64/numRevisions=1/exportAllRevisions=true/numRangeKeys=X-24        85.5µs ± 2%  5083.3µs ± 1%  +5848.68%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=64/numRevisions=10/exportAllRevisions=false/numRangeKeys=X-24       212µs ± 2%    5283µs ± 1%  +2387.43%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=64/numRevisions=10/exportAllRevisions=true/numRangeKeys=X-24        351µs ± 4%    5385µs ± 1%  +1432.41%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=64/numRevisions=100/exportAllRevisions=false/numRangeKeys=X-24      231µs ± 3%    5318µs ± 1%  +2200.66%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=64/numRevisions=100/exportAllRevisions=true/numRangeKeys=X-24      1.63ms ± 2%    7.06ms ± 1%   +332.92%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=512/numRevisions=1/exportAllRevisions=false/numRangeKeys=X-24       421µs ± 3%   15413µs ± 1%  +3562.62%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=512/numRevisions=1/exportAllRevisions=true/numRangeKeys=X-24        387µs ± 2%   15127µs ± 2%  +3810.76%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=512/numRevisions=10/exportAllRevisions=false/numRangeKeys=X-24     1.33ms ± 1%   17.30ms ± 2%  +1196.69%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=512/numRevisions=10/exportAllRevisions=true/numRangeKeys=X-24      2.39ms ± 1%   18.16ms ± 2%   +659.80%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=512/numRevisions=100/exportAllRevisions=false/numRangeKeys=X-24    1.46ms ± 1%   17.67ms ± 1%  +1109.77%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=512/numRevisions=100/exportAllRevisions=true/numRangeKeys=X-24     12.6ms ± 1%    31.3ms ± 1%   +148.91%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=1024/numRevisions=1/exportAllRevisions=false/numRangeKeys=X-24      761µs ± 2%   18560µs ± 1%  +2338.25%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=1024/numRevisions=1/exportAllRevisions=true/numRangeKeys=X-24       705µs ± 2%   18309µs ± 0%  +2497.43%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=1024/numRevisions=10/exportAllRevisions=false/numRangeKeys=X-24    2.64ms ± 1%   23.30ms ± 1%   +783.25%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=1024/numRevisions=10/exportAllRevisions=true/numRangeKeys=X-24     4.70ms ± 0%   24.54ms ± 1%   +422.09%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=1024/numRevisions=100/exportAllRevisions=false/numRangeKeys=X-24   2.86ms ± 1%   23.76ms ± 1%   +729.73%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=1024/numRevisions=100/exportAllRevisions=true/numRangeKeys=X-24    25.3ms ± 2%    51.2ms ± 1%   +102.28%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=8192/numRevisions=1/exportAllRevisions=false/numRangeKeys=X-24     5.70ms ± 1%   32.34ms ± 3%   +467.60%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=8192/numRevisions=1/exportAllRevisions=true/numRangeKeys=X-24      5.16ms ± 2%   31.52ms ± 2%   +511.28%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=8192/numRevisions=10/exportAllRevisions=false/numRangeKeys=X-24    21.2ms ± 2%    76.5ms ± 2%   +260.02%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=8192/numRevisions=10/exportAllRevisions=true/numRangeKeys=X-24     38.6ms ± 1%    86.6ms ± 1%   +124.44%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=8192/numRevisions=100/exportAllRevisions=false/numRangeKeys=X-24   24.8ms ± 1%    86.1ms ± 1%   +247.75%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=8192/numRevisions=100/exportAllRevisions=true/numRangeKeys=X-24     210ms ± 3%     305ms ± 2%    +45.62%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=65536/numRevisions=1/exportAllRevisions=false/numRangeKeys=X-24    45.1ms ± 1%    77.3ms ± 2%    +71.41%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=65536/numRevisions=1/exportAllRevisions=true/numRangeKeys=X-24     40.9ms ± 1%    73.0ms ± 1%    +78.62%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=65536/numRevisions=10/exportAllRevisions=false/numRangeKeys=X-24    183ms ± 1%     411ms ± 1%   +124.65%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=65536/numRevisions=10/exportAllRevisions=true/numRangeKeys=X-24     314ms ± 0%     520ms ± 1%    +65.46%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=65536/numRevisions=100/exportAllRevisions=false/numRangeKeys=X-24   260ms ±10%     857ms ± 3%   +229.61%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=65536/numRevisions=100/exportAllRevisions=true/numRangeKeys=X-24    1.83s ± 1%     2.63s ± 2%    +44.05%  (p=0.008 n=5+5)
```

Closes cockroachdb#86140.

Release note: None
Release justification: testing-only changes


Co-authored-by: Tobias Grieger <[email protected]>
  • Loading branch information
@tbg
craig[bot] and tbg committed Aug 19, 2022
2 parents 8eabfb7 + ff6d72d commit 2e8c0f6
Show file tree
Hide file tree
Showing 2 changed files with 152 additions and 23 deletions.
153 changes: 135 additions & 18 deletions pkg/storage/bench_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -108,8 +108,16 @@ func BenchmarkMVCCGarbageCollect(b *testing.B) {
}

func BenchmarkMVCCExportToSST(b *testing.B) {
skip.UnderShort(b)
defer log.Scope(b).Close(b)

// To run and compare on range keys:
//
// go test ./pkg/storage -run - -count 5 -bench BenchmarkMVCCExportToSST -timeout 500m 2>&1 | tee bench.txt
// for flavor in numRangeKeys=0 numRangeKeys=1 numRangeKeys=100; do grep -E "${flavor}[^0-9]+" bench.txt | sed -E "s/${flavor}+/X/" > $flavor.txt; done
// benchstat numRangeKeys\={0,1}.txt
// benchstat numRangeKeys\={0,100}.txt

numKeys := []int{64, 512, 1024, 8192, 65536}
numRevisions := []int{1, 10, 100}
exportAllRevisions := []bool{false, true}
Expand All @@ -120,7 +128,17 @@ func BenchmarkMVCCExportToSST(b *testing.B) {
b.Run(fmt.Sprintf("numRevisions=%d", numRevision), func(b *testing.B) {
for _, exportAllRevisionsVal := range exportAllRevisions {
b.Run(fmt.Sprintf("exportAllRevisions=%t", exportAllRevisionsVal), func(b *testing.B) {
runMVCCExportToSST(b, numKey, numRevision, exportAllRevisionsVal)
for _, numRangeKeys := range []int{0, 1, 100} {
b.Run(fmt.Sprintf("numRangeKeys=%d", numRangeKeys), func(b *testing.B) {
opts := mvccExportToSSTOpts{
numKeys: numKey,
numRevisions: numRevision,
exportAllRevisions: exportAllRevisionsVal,
numRangeKeys: numRangeKeys,
}
runMVCCExportToSST(b, opts)
})
}
})
}
})
Expand Down Expand Up @@ -1606,7 +1624,12 @@ func runBatchApplyBatchRepr(
b.StopTimer()
}

func runMVCCExportToSST(b *testing.B, numKeys int, numRevisions int, exportAllRevisions bool) {
type mvccExportToSSTOpts struct {
numKeys, numRevisions, numRangeKeys int
exportAllRevisions bool
}

func runMVCCExportToSST(b *testing.B, opts mvccExportToSSTOpts) {
dir, cleanup := testutils.TempDir(b)
defer cleanup()
engine := setupMVCCPebble(b, dir)
Expand All @@ -1615,14 +1638,50 @@ func runMVCCExportToSST(b *testing.B, numKeys int, numRevisions int, exportAllRe
ctx := context.Background()
st := cluster.MakeTestingClusterSettings()

batch := engine.NewUnindexedBatch(true /* writeOnly */)
for i := 0; i < numKeys; i++ {
key := make([]byte, 16)
key = append(key, 'a', 'a', 'a')
mkKey := func(i int) roachpb.Key {
var key []byte
key = append(key, keys.LocalMax...)
key = append(key, bytes.Repeat([]byte{'a'}, 19)...)
key = encoding.EncodeUint32Ascending(key, uint32(i))
return key
}

for j := 0; j < numRevisions; j++ {
mvccKey := MVCCKey{Key: key, Timestamp: hlc.Timestamp{WallTime: int64(j + 1), Logical: 0}}
mkWall := func(j int) int64 {
wt := int64(j + 2)
return wt
}

// Write range keys.
func() {
rng := rand.New(rand.NewSource(12345))
batch := engine.NewBatch()
defer batch.Close()
for i := 0; i < opts.numRangeKeys; i++ {
// NB: regular keys are written at ts 2+, so this is below any of the
// regular writes and thus won't delete anything.
ts := hlc.Timestamp{WallTime: 1, Logical: int32(i + 1)}
start := rng.Intn(opts.numKeys)
end := start + rng.Intn(opts.numKeys-start) + 1
// As a special case, if we're only writing one range key, write it across
// the entire span.
if opts.numRangeKeys == 1 {
start = 0
end = opts.numKeys + 1
}
startKey := mkKey(start)
endKey := mkKey(end)
require.NoError(b, MVCCDeleteRangeUsingTombstone(
ctx, batch, nil, startKey, endKey, ts, hlc.ClockTimestamp{}, nil, nil, false, 0, nil))
}
require.NoError(b, batch.Commit(false /* sync */))
}()

batch := engine.NewBatch()
for i := 0; i < opts.numKeys; i++ {
key := mkKey(i)

for j := 0; j < opts.numRevisions; j++ {
mvccKey := MVCCKey{Key: key, Timestamp: hlc.Timestamp{WallTime: mkWall(j), Logical: 0}}
mvccValue := MVCCValue{Value: roachpb.MakeValueFromString("foobar")}
err := batch.PutMVCC(mvccKey, mvccValue)
if err != nil {
Expand All @@ -1638,31 +1697,89 @@ func runMVCCExportToSST(b *testing.B, numKeys int, numRevisions int, exportAllRe
b.Fatal(err)
}

startWall := mkWall(opts.numRevisions/2 - 1) // exclusive, at least 1, so never see rangedels
endWall := mkWall(opts.numRevisions + 1) // see latest revision for every key
var expKVsInSST int
if opts.exportAllRevisions {
// First, compute how many revisions are visible for each key.
// Could probably use a closed formula for this but this works.
for i := 0; i < opts.numRevisions; i++ {
wall := mkWall(i)
if wall > startWall && wall <= endWall {
expKVsInSST++
}
}
// Then compute the total.
expKVsInSST *= opts.numKeys
} else {
// See one revision per key.
expKVsInSST = opts.numKeys
}

var buf bytes.Buffer
buf.Grow(1 << 20)
b.ResetTimer()
b.StopTimer()
var assertLen int // buf.Len shouldn't change between runs
for i := 0; i < b.N; i++ {
startTS := hlc.Timestamp{WallTime: int64(numRevisions / 2)}
endTS := hlc.Timestamp{WallTime: int64(numRevisions + 2)}
buf.Reset()
b.StartTimer()
startTS := hlc.Timestamp{WallTime: startWall, Logical: math.MaxInt32} // use 1.infinity to avoid rangedels at 1.<n>
endTS := hlc.Timestamp{WallTime: endWall}
_, _, err := MVCCExportToSST(ctx, st, engine, MVCCExportOptions{
StartKey: MVCCKey{Key: keys.LocalMax},
EndKey: roachpb.KeyMax,
StartTS: startTS,
EndTS: endTS,
ExportAllRevisions: exportAllRevisions,
ExportAllRevisions: opts.exportAllRevisions,
TargetSize: 0,
MaxSize: 0,
StopMidKey: false,
}, noopWriter{})
}, &buf)
if err != nil {
b.Fatal(err)
}
}
b.StopTimer()
}
b.StopTimer()

if i == 0 {
require.NotZero(b, buf.Len())
assertLen = buf.Len()
}

type noopWriter struct{}
require.Equal(b, assertLen, buf.Len())
}

func (noopWriter) Close() error { return nil }
func (noopWriter) Write(p []byte) (int, error) { return len(p), nil }
// Run sanity checks on last produced SST.
it, err := NewPebbleMemSSTIterator(
buf.Bytes(), true /* verify */, IterOptions{
LowerBound: keys.LocalMax,
UpperBound: roachpb.KeyMax,
KeyTypes: IterKeyTypePointsAndRanges,
},
)
it.SeekGE(MakeMVCCMetadataKey(roachpb.LocalMax))
require.NoError(b, err)
var n int // points
var r int // range key stacks
for {
ok, err := it.Valid()
require.NoError(b, err)
if !ok {
break
}
hasPoint, hasRange := it.HasPointAndRange()
if hasPoint {
n++
}
if hasRange && it.RangeKeyChanged() {
r++
}
it.Next()
}
require.Equal(b, expKVsInSST, n)
// Should not see any rangedel stacks.
require.Zero(b, r)
}

func runCheckSSTConflicts(
b *testing.B, numEngineKeys, numVersions, numSstKeys int, overlap, usePrefixSeek bool,
Expand Down
22 changes: 17 additions & 5 deletions pkg/storage/mvcc.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5712,12 +5712,24 @@ func MVCCIsSpanEmpty(

// MVCCExportToSST exports changes to the keyrange [StartKey, EndKey) over the
// interval (StartTS, EndTS] as a Pebble SST. See MVCCExportOptions for options.
// StartTS may be zero.
//
// Tombstones are included if all revisions are requested (all tombstones) or if
// the StartTS is non-zero (latest tombstone), including both MVCC point
// tombstones and MVCC range tombstones. Intents within the time interval will
// return a WriteIntentError, while intents outside the time interval are
// ignored.
// This comes in two principal flavors: all revisions or latest revision only.
// In all-revisions mode, exports everything matching the span and time bounds,
// i.e. extracts contiguous blocks of MVCC history. In latest-revision mode,
// extracts just the changes necessary to transform an MVCC snapshot at StartTS
// into one equivalent to the data at EndTS, but without including all
// intermediate revisions not visible at EndTS. The latter mode is used for
// incremental backups that can only be restored to EndTS, the former allows
// restoring to any intermediate timestamp.
//
// Tombstones (both point and MVCC range tombstones) are treated like revisions.
// That is, if all revisions are requested, all tombstones in (StartTS, EndTS]
// and overlapping [StartKey, EndKey) are returned. If only the latest revision
// is requested, only the most recent matching tombstone is returned.
//
// Intents within the time and span bounds will return a WriteIntentError, while
// intents outside are ignored.
//
// Returns an export summary and a resume key that allows resuming the export if
// it reached a limit. Data is written to dest as it is collected. If an error
Expand Down

0 comments on commit 2e8c0f6

Please sign in to comment.