Skip to content

Commit

Permalink
Add static_bsatn_validator & validate_bsatn (#2029)
Browse files Browse the repository at this point in the history
  • Loading branch information
@Centril
Centril authored Dec 3, 2024
1 parent bc71e25 commit 586929c
Show file tree
Hide file tree
Showing 5 changed files with 435 additions and 4 deletions.
5 changes: 5 additions & 0 deletions crates/sats/src/buffer.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@ pub enum DecodeError {
expected: usize,
given: usize,
},
/// Length did not match the statically expected length.
InvalidLen { expected: usize, given: usize },
/// The tag does not exist for the sum.
InvalidTag { tag: u8, sum_name: Option<String> },
/// Expected data to be UTF-8 but it wasn't.
Expand All @@ -34,6 +36,9 @@ impl fmt::Display for DecodeError {
expected,
given,
} => write!(f, "data too short for {for_type}: Expected {expected}, given {given}"),
DecodeError::InvalidLen { expected, given } => {
write!(f, "unexpected data length: Expected {expected}, given {given}")
}
DecodeError::InvalidTag { tag, sum_name } => {
write!(
f,
Expand Down
7 changes: 7 additions & 0 deletions crates/table/proptest-regressions/static_bsatn_validator.txt
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Seeds for failure cases proptest has generated in the past. It is
# automatically read and these particular cases re-run before any
# novel cases are generated.
#
# It is recommended to check this file in to source control so that
# everyone who runs the test benefits from these saved cases.
cc a6756c136abdcebcb9502aafb473f334ddff07a00500f58527f1d0f9469b3dbf # shrinks to (ty, val) = (ProductType { elements: [ProductTypeElement { name: None, algebraic_type: Sum(SumType { variants: [SumTypeVariant { name: Some("variant_0"), algebraic_type: U8 }] }) }, ProductTypeElement { name: None, algebraic_type: Bool }] }, ProductValue { elements: [Sum(SumValue { tag: 0, value: U8(2) }), Bool(false)] })
1 change: 1 addition & 0 deletions crates/table/src/lib.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ pub mod pointer_map;
pub mod read_column;
pub mod row_hash;
pub mod row_type_visitor;
pub mod static_bsatn_validator;
pub mod table;
pub mod var_len;

Expand Down
6 changes: 2 additions & 4 deletions crates/table/src/row_type_visitor.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -116,10 +116,8 @@ fn sum_type_to_rose_tree(ty: &SumTypeLayout, current_offset: &mut usize) -> VarL

// All variants are stored overlapping at the offset of the sum.
// Don't let them mutate `current_offset`.
// Note that we store sums with data first,
// followed by tag,
// so the variant data goes at `current_offset`,
// not `current_offset + tag + padding`.
// Note that we store sums with tag first,
// followed by data/payload.
//
// `offset_of_variant_data` is defined as 0,
// but included for future-proofing.
Expand Down
Loading

2 comments on commit 586929c

@github-actions
Copy link

@github-actions github-actions bot commented on 586929c Dec 3, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Benchmarking failed. Please check the workflow run for details.

@github-actions
Copy link

@github-actions github-actions bot commented on 586929c Dec 3, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Callgrind benchmark results

Callgrind Benchmark Report

These benchmarks were run using callgrind,
an instruction-level profiler. They allow comparisons between sqlite (sqlite), SpacetimeDB running through a module (stdb_module), and the underlying SpacetimeDB data storage engine (stdb_raw). Callgrind emulates a CPU to collect the below estimates.

Measurement changes larger than five percent are in bold.

In-memory benchmarks

callgrind: empty transaction

db total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw 6396 6396 0.00% 6552 6572 -0.30%
sqlite 5609 5619 -0.18% 5983 6159 -2.86%

callgrind: filter

db schema indices count preload _column data_type total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str no_index 64 128 1 u64 74672 74672 0.00% 75404 75132 0.36%
stdb_raw u32_u64_str no_index 64 128 2 string 116914 117170 -0.22% 117928 118006 -0.07%
stdb_raw u32_u64_str btree_each_column 64 128 2 string 25083 25082 0.00% 25745 25802 -0.22%
stdb_raw u32_u64_str btree_each_column 64 128 1 u64 24048 24048 0.00% 24636 24548 0.36%
sqlite u32_u64_str no_index 64 128 2 string 144415 144416 -0.00% 145825 145804 0.01%
sqlite u32_u64_str no_index 64 128 1 u64 123763 123764 -0.00% 124911 124920 -0.01%
sqlite u32_u64_str btree_each_column 64 128 1 u64 131080 131081 -0.00% 132560 132517 0.03%
sqlite u32_u64_str btree_each_column 64 128 2 string 134222 134223 -0.00% 135848 135867 -0.01%

callgrind: insert bulk

db schema indices count preload total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 64 128 880455 881321 -0.10% 902167 931751 -3.18%
stdb_raw u32_u64_str btree_each_column 64 128 1027640 1032718 -0.49% 1068076 1094738 -2.44%
sqlite u32_u64_str unique_0 64 128 399360 399362 -0.00% 422230 422960 -0.17%
sqlite u32_u64_str btree_each_column 64 128 984611 984613 -0.00% 1023849 1025997 -0.21%

callgrind: iterate

db schema indices count total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 1024 138372 138372 0.00% 138506 138580 -0.05%
stdb_raw u32_u64_str unique_0 64 15797 15797 0.00% 15915 15993 -0.49%
sqlite u32_u64_str unique_0 1024 1042718 1042718 0.00% 1046090 1046124 -0.00%
sqlite u32_u64_str unique_0 64 74728 74704 0.03% 75868 75812 0.07%

callgrind: serialize_product_value

count format total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
64 json 47528 47528 0.00% 50282 50180 0.20%
64 bsatn 25509 25509 0.00% 27719 27719 0.00%
16 bsatn 8200 8200 0.00% 9560 9560 0.00%
16 json 12188 12188 0.00% 14160 14092 0.48%

callgrind: update bulk

db schema indices count preload total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 1024 1024 20570310 20142476 2.12% 21134846 20588676 2.65%
stdb_raw u32_u64_str unique_0 64 128 1288591 1288881 -0.02% 1358951 1353897 0.37%
sqlite u32_u64_str unique_0 1024 1024 1802137 1802079 0.00% 1811497 1811105 0.02%
sqlite u32_u64_str unique_0 64 128 128540 128500 0.03% 131416 131258 0.12%
On-disk benchmarks

callgrind: empty transaction

db total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw 6401 6401 0.00% 6569 6589 -0.30%
sqlite 5651 5651 0.00% 6047 6195 -2.39%

callgrind: filter

db schema indices count preload _column data_type total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str no_index 64 128 1 u64 74677 74677 0.00% 75353 75113 0.32%
stdb_raw u32_u64_str no_index 64 128 2 string 116919 117175 -0.22% 117857 117959 -0.09%
stdb_raw u32_u64_str btree_each_column 64 128 2 string 25087 25086 0.00% 25673 25774 -0.39%
stdb_raw u32_u64_str btree_each_column 64 128 1 u64 24053 24053 0.00% 24593 24513 0.33%
sqlite u32_u64_str no_index 64 128 1 u64 125684 125685 -0.00% 127284 127269 0.01%
sqlite u32_u64_str no_index 64 128 2 string 146336 146337 -0.00% 148126 148193 -0.05%
sqlite u32_u64_str btree_each_column 64 128 2 string 136418 136419 -0.00% 138464 138595 -0.09%
sqlite u32_u64_str btree_each_column 64 128 1 u64 133176 133177 -0.00% 135028 135081 -0.04%

callgrind: insert bulk

db schema indices count preload total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 64 128 830018 831176 -0.14% 882334 880734 0.18%
stdb_raw u32_u64_str btree_each_column 64 128 980186 981989 -0.18% 1051366 1043653 0.74%
sqlite u32_u64_str unique_0 64 128 416914 416916 -0.00% 439324 439896 -0.13%
sqlite u32_u64_str btree_each_column 64 128 1023158 1023160 -0.00% 1061834 1064888 -0.29%

callgrind: iterate

db schema indices count total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 1024 138377 138377 0.00% 138487 138565 -0.06%
stdb_raw u32_u64_str unique_0 64 15802 15802 0.00% 15916 15990 -0.46%
sqlite u32_u64_str unique_0 1024 1045796 1045796 0.00% 1049568 1049590 -0.00%
sqlite u32_u64_str unique_0 64 76486 76486 0.00% 77802 77822 -0.03%

callgrind: serialize_product_value

count format total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
64 json 47528 47528 0.00% 50282 50180 0.20%
64 bsatn 25509 25509 0.00% 27719 27719 0.00%
16 bsatn 8200 8200 0.00% 9560 9560 0.00%
16 json 12188 12188 0.00% 14160 14092 0.48%

callgrind: update bulk

db schema indices count preload total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 1024 1024 19052227 19074169 -0.12% 19664703 19590355 0.38%
stdb_raw u32_u64_str unique_0 64 128 1240898 1242608 -0.14% 1310136 1306720 0.26%
sqlite u32_u64_str unique_0 1024 1024 1809785 1809727 0.00% 1818417 1818253 0.01%
sqlite u32_u64_str unique_0 64 128 132687 132629 0.04% 135691 135583 0.08%

Please sign in to comment.