update tests/benchmarks/readme.md

nim-lang · May 25, 2021 · 5eeb09e · 5eeb09e
1 parent 0e00fbf
commit 5eeb09e
Showing 1 changed file with 5 additions and 0 deletions.
diff --git a/tests/benchmarks/readme.md b/tests/benchmarks/readme.md
@@ -3,3 +3,8 @@
 In future work, benchmarks can be added to CI, but for now we provide benchmarks that can be run locally.
 
 See RFC: https://github.com/timotheecour/Nim/issues/425
+
+## guidelines
+* tests should run in CI (so the test keeps working) but should complete fast (so it doesn't slow down CI).
+  they should provide a knob (e.g. via `const numIter {.intdefine.} = 10`), so that users can re-run
+  the benchmark manually with more meaningful parameters (e.g. `-d:numIter:100_000`).