Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Historical Benchmark Performance Dashboard #928

Open
lapp0 opened this issue May 29, 2024 · 0 comments
Open

Historical Benchmark Performance Dashboard #928

lapp0 opened this issue May 29, 2024 · 0 comments

Comments

@lapp0
Copy link
Contributor

lapp0 commented May 29, 2024

What behavior of the library made you think about the improvement?

Follow-up to #903 which implements ASV benchmarks comparing main to HEAD of PR.

With the infrastructure introduced in the above PR, we can create useful historical performance dashboards such as https://asv-runner.github.io/asv-collection/pandas/

How would you like it to behave?

  • On merge, ASV benchmarks are run using main, and published to outlines-dev/outlines-benchmarks
  • A github page is setup for outlines-benchmarks
  • To ensure consistency over time, this requires a stable, dedicated machine which must have a guarantee of being idle during benchmark runs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant