ci: add continuous benchmarking

[CarlWachter]

This will execute the benchmarks in the hermit-rs repo according to the bench.json file. Results will be published to Github Pages.

CI will fail until hermit-os/hermit-rs#708 gets merged.

[jounathaen]

Suggested change

	runs-on: ubuntu-latest
	runs-on: [self-hosted]

[jounathaen]

Ci is unfortunately still failing, despite hermit-os/hermit-rs#708 being merged and also used.

[CarlWachter]

@jounathaen it's passing now and results look good

[jounathaen]

The benchmarks take 27 minutes now, which is 10 mins longer than the tests. Can we change the benchmark setup, so that the CI duration is not dominated by the benchmarks anymore (parallelize it on two runners or reduce the benchmark size)?

[mkroening]

I think the benchmarks taking more time is fine, but I would suggest running the benchmark suite after merging and on PRs only on demand.

[jounathaen]

I think the benchmarks taking more time is fine, but I would suggest running the benchmark suite after merging and on PRs only on demand.

But in that case, we wouldn't know if a PR affects performance before merging it.

[jounathaen]

As @mkroening pointed out, we already have gh-pages installed for this repository (https://hermit-os.github.io/kernel/hermit/). We would need to find a way of publishing the results on a different URL. Martin suggested a separate repository for gh-pages, where this action pushes to. Maybe a combination of both gh-pages into subdirectories is also possible.