Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add benchmark suite #3120

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Add benchmark suite #3120

wants to merge 2 commits into from

Conversation

manno
Copy link
Member

@manno manno commented Dec 4, 2024

  • add benchmarks
  • add test data
  • experiments use group label instead of repo-name label w fleet 0.9.11
  • add toggle for metrics
  • script uses stable benchmark order via --seed
  • check for at least one cluster
  • skip confusing report table printing as we don't use sampling

TODO

@manno manno force-pushed the benchmark-suite branch 5 times, most recently from 095a595 to 674be05 Compare December 11, 2024 15:54
@manno manno marked this pull request as ready for review December 12, 2024 13:43
@manno manno requested a review from a team as a code owner December 12, 2024 13:43
@manno manno force-pushed the benchmark-suite branch 2 times, most recently from b1ab9b0 to 082b115 Compare December 13, 2024 15:08
dev/benchmarks.sh Outdated Show resolved Hide resolved
dev/benchmarks.sh Outdated Show resolved Hide resolved
benchmarks/record/record.go Show resolved Hide resolved
Comment on lines +173 to +177
} else if len(image.Names) > 1 {
name = image.Names[1]
} else {
name = image.Names[0]
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the rationale for this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

    images:
    - names:
      - docker.io/rancher/fleet:dev
      sizeBytes: 419817115
    - names: null
      sizeBytes: 399360934
    - names:
      - ghcr.io/rancherlabs/delve-debugger@sha256:d397fc318f7a5bd06304f15ad0aa07346d7dbcb3690637efd3663b8a5554c844
      - ghcr.io/rancherlabs/delve-debugger:latest
      sizeBytes: 360158186

We need to pick one of the names, on k3d the second name is better.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh I see, thanks for clarifying. A brief comment explaining this would be great :)

benchmarks/record/record.go Show resolved Hide resolved
benchmarks/report/report.go Show resolved Hide resolved
benchmarks/deploy_test.go Outdated Show resolved Hide resolved
benchmarks/deploy_test.go Outdated Show resolved Hide resolved
benchmarks/deploy_test.go Outdated Show resolved Hide resolved
benchmarks/gitrepo_bundle_test.go Outdated Show resolved Hide resolved
* add benchmarks
* add test data
* experiments use group label instead of repo-name label w fleet 0.9.11
* add toggle for metrics
* script uses stable benchmark order via --seed
* check for at least one cluster
* skip confusing report table printing as we don't use sampling
@kkaempf kkaempf added this to the v2.11.0 milestone Dec 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: 👀 In review
Development

Successfully merging this pull request may close these issues.

3 participants