Filtersets V2

Simplifies filtersets. The primary filter type is now a RelDnf (the ids
of spans matching a Disjunctive Normal Form over a source).
This makes rewrite rules simpler (many -> one (intersection of DNFS)),
and also results in better performance, since we can merge many kinds of
nodes into one DNF, which corresponds to only a single scan on the data.

Author: algorithmiker

Cargo.lock

@@ -1,10 +1,11 @@
-THE FILTERSET CALCULUS
+THE FILTERSET CALCULUS (v2)
 ---
 
 ### TYPES
 
 ```rust
 type FiltersetId = usize;
+type PreidcateId = usize;
 struct Predicate {
     attr: String,
     rel: Ordering,
@@ -14,9 +15,8 @@ struct Predicate {
 enum Filterset {
     Dead,
     Primitive(Roaring),
-    Rel(Predicate, FiltersetId),
-    RelIntersect(Vec<Predicate>, FiltersetId),
-    RelUnion(Vec<Predicate>, FiltersetId),
+    BlackBox(FiltersetId),
+    RelDnf(Vec<Vec<PredicateId>>, FiltersetId),
     And(Vec<FiltersetId>),
     Or(Vec<FiltersetId>),
     Not(FiltersetId),
@@ -30,25 +30,12 @@ enum Filterset {
 And([... And(children) ...]) -> And(flattened)
 Or([... Or(children) ...])  -> Or(flattened)
 Not(Not(X)) -> X
-
-**Eliminating trivial ops (unimplemeneted)**
 And([A]) -> A
 Or([A]) -> A
 
-**REL composition**
-
-I.   Rel(p1, Rel(p2, A)) -> RelIntersect([p1, p2], A)
-II.  Rel(p, RelIntersect(ps, A)) -> RelIntersect([p] + ps, A)
-III. RelIntersect(ps1, RelIntersect(ps2, A)) -> RelIntersect(ps1 + ps2, A)
-IV.  RelIntersect(ps, Rel(p, A)) -> RelIntersect(ps + [p], A)
-V.   RelUnion(ps1, RelUnion(ps2, A)) -> RelUnion(ps1 + ps2, A)
-
+**One Rule To Rule Them All**
+1. RelDnf(clauses, RelDnf(clauses2, A)) -> new clauses: c_1 \times c_2 (if the result won't be too big)
 
-**Flattening on the same level (unimplemeneted)**
-And([RelIntersect(ps, A), RelIntersect(ps2, A), RelIntersect(_, B) ...])
-  -> And([RelIntersect(ps1+ps2, A), RelIntersect(_, B)])
-    - This could be hard to detect, instead there could be just:
-      And([RelIntersect(ps_1, A), .. RelIntersect(ps_n, A)]) -> RelIntersect(ps_1+..+ps_n, A)
-Or([RelUnion(ps, A), RelUnion(ps2, A), RelUnion(_, B) ...])
-    -> Or([RelUnion(ps1+ps2, A), RelIntersect(_, B)])
-    - same applies here.
+**RelDnf in Or/And**
+1. Or([RelDnf(c, A), RelDnf(c2, A), RelDnf(c3, B), RelDnf(c4, B)]) -> Or([RelDnf(c+c2, A), RelDnf(c3+c4, B)])
+2. And([RelDnf(c, A), RelDnf(c2, A), RelDnf(c3, B), RelDnf(c4, B)]) -> And([RelDnf(c x c2, A), RelDnf(c3 x c4, B)]) (if not too big)

docs/filtersets.md

@@ -35,7 +35,8 @@ pub trait LogProvider {
     /// This MUST be cheap as the frontend might call this every frame.
     fn len(&self) -> usize;
 
-    /// The frontent SHOULD call this at the beginning of each painted frame.
+    /// The frontent SHOULD call this at the beginning of each painted frame,
+    /// but there is no guarantee to whether or when it will.
     /// This runs on the main thread.
     /// The [LogProvider] implementation MUST ensure that this terminates quickly,
     /// as it directly affects FPS.

entrace_core/src/log_provider.rs

@@ -10,6 +10,7 @@ anyhow = "1.0.100"
 memchr = "2.7.6"
 thiserror = "2.0.17"
 roaring = "0.11.2"
+itertools = "0.14.0"
 
 [[bin]]
 name="entrace-query-test"