Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: check for benchmark regressions on PRs #12027

Merged
merged 7 commits into from
Jan 23, 2025
+15 −1
Merged

ci: check for benchmark regressions on PRs #12027

Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
abd1614
ci: check for benchmark regressions on PRs
brettlangdon Jan 22, 2025
57a23b8
introduce regression
brettlangdon Jan 22, 2025
dcf2d3b
don't allow failures
brettlangdon Jan 23, 2025
f5b1d8c
chore: Configure ARTIFACTS_DIR to use for regression check
ddyurchenko Jan 23, 2025
f703acc
chore: Run check-big-regressions even when some CI jobs failed
ddyurchenko Jan 23, 2025
9800e8e
chore: Reduce regression size
ddyurchenko Jan 23, 2025
fdfee2c
Apply suggestions from code review
brettlangdon Jan 23, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 15 additions & 1 deletion .gitlab/benchmarks.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,6 @@ variables:
paths:
- reports/
expire_in: 3 months
allow_failure: true # Allow failure, so partial results are uploaded
variables:
UPSTREAM_PROJECT_ID: $CI_PROJECT_ID # The ID of the current project. This ID is unique across all projects on the GitLab instance.
UPSTREAM_PROJECT_NAME: $CI_PROJECT_NAME # "dd-trace-py"
Expand Down Expand Up @@ -78,6 +77,21 @@ benchmarks-pr-comment:
UPSTREAM_COMMIT_SHA: $CI_COMMIT_SHA # The commit revision the project is built for.
KUBERNETES_SERVICE_ACCOUNT_OVERWRITE: dd-trace-py

check-big-regressions:
stage: benchmarks
needs: [ microbenchmarks, benchmark-serverless ]
when: always
tags: ["arch:amd64"]
image: $MICROBENCHMARKS_CI_IMAGE
script:
- export ARTIFACTS_DIR="$(pwd)/reports/"
- git config --global url."https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.ddbuild.io/DataDog/".insteadOf "https://github.com/DataDog/"
- git clone --branch dd-trace-py https://github.com/DataDog/benchmarking-platform /platform && cd /platform
- bp-runner bp-runner.fail-on-regression.yml --debug
variables:
# Gitlab and BP specific env vars. Do not modify.
KUBERNETES_SERVICE_ACCOUNT_OVERWRITE: dd-trace-py

benchmark-serverless:
stage: benchmarks
image: $SLS_CI_IMAGE
Expand Down
Loading