diff --git a/tests/benchmarks/readme.md b/tests/benchmarks/readme.md
index 1e744fc40b1e8..1ee573641611d 100644
--- a/tests/benchmarks/readme.md
+++ b/tests/benchmarks/readme.md
@@ -3,3 +3,8 @@
 In future work, benchmarks can be added to CI, but for now we provide benchmarks that can be run locally.
 
 See RFC: https://github.com/timotheecour/Nim/issues/425
+
+## guidelines
+* tests should run in CI (so the test keeps working) but should complete fast (so it doesn't slow down CI).
+  they should provide a knob (e.g. via `const numIter {.intdefine.} = 10`), so that users can re-run
+  the benchmark manually with more meaningful parameters (e.g. `-d:numIter:100_000`).