lib: add afterEach hook option to each benchmark

[RafaelGSS]

Fixes: #31

[RafaelGSS]

I'm still evaluating how useful would be it since it's an additional instruction that will run after each function call, of course, affecting the benchmark result. I believe this is just replaced by the userland code adding those checks.

[robbiespeed]

afterEach is necessary for benchmarking code that can be negatively impacted by GC pressure which wouldn't normally occur, but do in the context of a tight looping benchmark. This isn't usually an issue for small micro benchmarks, but for larger integration style benchmarks it's quite common.

However as currently implemented it's not useful, because the hook is affecting the timing. Instead each iteration should have individual timing which occurs before the afterEach hook. You could keep the aggregate timing as is for cases without afterEach, but I'm not sure what's gained from that. Individual timings also allow extracting percentiles.

[RafaelGSS]

However as currently implemented it's not useful, because the hook is affecting the timing. Instead each iteration should have individual timing which occurs before the afterEach hook. You could keep the aggregate timing as is for cases without afterEach, but I'm not sure what's gained from that. Individual timings also allow extracting percentiles.

So, your suggestion would be:

• Start timer -> Perform one iteration -> Stop timer -> afterEach hook or
• Start timer -> Perform one cycle/sample of iterations -> Stop timer -> afterEach hook

I assume the first one, right? I'd need to analyse how it would behave since making unique comparisons would be more affected by external code.

[H4ad]

I think we should recommend people to use the managed timer, so they can add before/after hooks without affecting the performance of the benchmark.

[RafaelGSS]

I think we should recommend people to use the managed timer, so they can add before/after hooks without affecting the performance of the benchmark.

Yes, however, deoptimizing the setup/cleanup might not be their intention though https://github.com/RafaelGSS/bench-node?tab=readme-ov-file#setup-and-teardown

[robbiespeed]

@RafaelGSS correct the first one. Having batched samples is pretty easy to do in the user benchmark by wrapping in a loop, or possible with a plugin that does a loop.

[RafaelGSS]

suite.add('my benchmark with gc cleanup', (timer) => {
  timer.start()
  myBenchFunction()
  timer.end(1)
  gc();
})

or

suite.add('my benchmark with gc cleanup', (timer) => {
  for (let i = 0; i < timer.count; i++) {
    timer.start();
    myBenchFunction()
    timer.end(1)
    gc();
  }
})

Wouldn't it solve your problem?