Skip to content

Commit

Permalink
Correct sigma distance in new footnote, moving less topical material to
Browse files Browse the repository at this point in the history
the footnote.
  • Loading branch information
c-blake committed Jul 7, 2023
1 parent 9733726 commit 57f6d8f
Showing 1 changed file with 6 additions and 4 deletions.
10 changes: 6 additions & 4 deletions doc/funnel.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,9 +122,9 @@ First, though methodology here is more careful than average, this is just a one
(machine, OS, source tree, reg-ex, locale) test to demo one wrapper script
possibility for `funnel`. Second, this is a 0 match test. Linus' name is not
misspelled this way in his big project. Third, mean of the best 3/10[^1] are
reproduced run-to-run to within its stderrs[^2]. E.g., the first wall time pair
is 0.72 sigma apart, and all are within 5 sigma.[^3] While other things can (and
maybe should) be done[^4], we use the min of the 2 trials next.
reproduced run-to-run to within its stderrs[^2] or at least within 5 sigma.[^3]
While other things can (and maybe should) be done[^4], we use the min of the 2
trials next.
Interpreting in more detail, here `rg` is 310498/299533 =~ 1.037X faster. Since
I found no way to disable `rg` dir scans/.., it is more fair to subtract an `rg
Expand Down Expand Up @@ -166,14 +166,16 @@ error propagation](https://en.wikipedia.org/wiki/Propagation_of_uncertainty)
holds, `sig(A-B)=(sigA**2+sigB**2)**.5`, for example. "Numbers of sigma" refer
to `|A-B|/sig(A-B)`, an informal two-sided Student T test for zero-hood. In a
Nim setting, https://github.com/SciNim/Measuremancer also has more details.
Sigma distances of the 5 wall time pairs are: 114/30=3.8, 9/28=0.32, 6/5=1.2,
2/10=0.2, and 93/21=4.4.
[^3]: A common workaround for hostile, unknown heavy tails in particle physics
is to use 5 rather than 3 sigma as a "strong" threshold. This actually may be
too optimistic for "computer system timing noise", especially depending on
background activity like web browsers/..where I occasionally see 10+ sigma.
[^4]: E.g. using a global min over all 20 runs, possibly adjusted & with error
estimates.
estimates. [tim](tim.md) docs have some more color on difficulties here.
[^5]: On multi-CPU computers, even shell pipelines are "parallel programming"
with OS kernels doing the heavy lifting. People too readily receive wisdom
Expand Down

0 comments on commit 57f6d8f

Please sign in to comment.