feat: add bench workflow for AWS #1330
Conversation
tschneider-aneo
force-pushed
the
ts/add-bench-aws
branch
from
cdd4d48 to
6e32c34
Compare
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| ttlSecondsAfterFinished: 0 | ||
| template: | ||
| spec: | ||
| containers: |
Check warning
Code scanning / SonarCloud
Service account permissions should be restricted Medium
| template: | ||
| spec: | ||
| containers: | ||
| - name: bench-session |
Check warning
Code scanning / SonarCloud
Memory limits should be enforced Medium
| template: | ||
| spec: | ||
| containers: | ||
| - name: bench-session |
Check warning
Code scanning / SonarCloud
Storage limits should be enforced Medium
tschneider-aneo
force-pushed
the
ts/add-bench-aws
branch
4 times, most recently
from
ee84b2d to
e5e9472
Compare
tschneider-aneo
force-pushed
the
ts/add-bench-aws
branch
2 times, most recently
from
0f1faab to
701ae60
Compare
tschneider-aneo
force-pushed
the
ts/add-bench-aws
branch
from
701ae60 to
e767835
Compare
|
| on: | ||
| push: | ||
| branches: | ||
| - "ts/add-bench-aws" |
There was a problem hiding this comment.
Replace with 'main'.
| if [ "$TRIGGER" = 'push' ]; then | ||
| echo '{"include":[{"type": "localhost", "ntasks":100, "polling-limit": 300}, {"type": "aws", "ntasks":1000, "polling-limit": 600}]}' > matrix.json | ||
| elif [ "$TRIGGER" = 'release' ]; then | ||
| echo '{"include":[{"type": "localhost", "ntasks":100, "polling-limit": 300}]}' > matrix.json | ||
| fi |
There was a problem hiding this comment.
Swap the two condition bodies. and set the number of tasks to a relevant value.
| # test-matrix: | ||
| # name: Test matrix | ||
| # runs-on: ubuntu-latest | ||
| # needs: define-matrix | ||
| # strategy: | ||
| # fail-fast: false | ||
| # matrix: ${{ fromJson(needs.define-matrix.outputs.matrix) }} | ||
| # steps: | ||
| # - id: test | ||
| # name: Test matrix | ||
| # env: | ||
| # TYPE: ${{ matrix.type }} | ||
| # NTASKS: ${{ matrix.ntasks }} | ||
| # POLLING_LIMIT: ${{ matrix.polling-limit }} | ||
| # run: | | ||
| # set -ex | ||
| # echo "Type: $TYPE, Number of tasks: $NTASKS, Polling Limit: $POLLING_LIMIT" |
There was a problem hiding this comment.
Remove this commented job.
| AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }} | ||
| AWS_EC2_METADATA_DISABLED: true | ||
| run: | | ||
| aws s3 cp "$BENCH_RESULTS_PATH" "s3://test-armonik-bench-storage/benchclient_benchmark_${EVENT_NAME}_${TYPE}_${GHRUNID}.json" |
There was a problem hiding this comment.
Maybe you should use an environment variable here instead of the putting directly the bucket uri
| # WARNING FOR ARMONIK CORE TEAM | ||
|
|
||
| # Sometimes infrastructure destruction fails on AWS due to a deadlock | ||
| # between a security group and a subnet that happens unpredictably. | ||
|
|
||
| # When this happens, the destruction must be taken over manually | ||
| # by destroying the security group and the network interface associated with it, | ||
| # and finished with `make` recipes `destroy` and `bootstrap-destroy` with the prefix |
There was a problem hiding this comment.
I'm sure it's very important. But is it the right place?
Motivation
Related to ArmoniK benchmarking automation project, Bench application is the most suited one for testing the ArmoniK's sheer orchestration performance
Description
Use Bench action from Armonik.Action.Deploy to benchmark ArmoniK with Bench application on localhost on each commit on main, and on AWS at each release.
Testing
This workflow was run on every push on the PR's branch.
Impact
This PR implements a milestone for ArmoniK benchmarking capacity, as AK would be benchmarked frequently.
Additional Information
Needs this PR to be merged before being merge
Sometimes infrastructure destruction fails on AWS due to a deadlock between a security group and a subnet that happens unpredictably.
When this happens, the destruction must be taken over manually by destroying the security group and the network interface associated with it, and finished with
makerecipesdestroyandbootstrap-destroywith the prefix used by the GitHub workflow (currentlybenchmark).Checklist