Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: lower time for fuzz/race tests #15141

Merged
merged 13 commits into from
Nov 8, 2024
5 changes: 3 additions & 2 deletions .github/workflows/ci-core.yml
Original file line number Diff line number Diff line change
Expand Up @@ -220,12 +220,13 @@ jobs:
go install ./pkg/chainlink/cmd/chainlink-starknet
popd

- name: Increase Race Timeout
# Increase race timeout for scheduled runs only
- name: Increase Timeouts for Fuzz/Race
# Increase timeouts for scheduled runs only
if: ${{ github.event.schedule != '' && needs.filter.outputs.should-run-ci-core == 'true' }}
run: |
echo "TIMEOUT=10m" >> $GITHUB_ENV
echo "COUNT=50" >> $GITHUB_ENV
echo "FUZZ_TIMEOUT_MINUTES=10">> $GITHUB_ENV

- name: Install gotestloghelper
if: ${{ needs.filter.outputs.should-run-ci-core == 'true' }}
Expand Down
18 changes: 13 additions & 5 deletions tools/bin/go_core_fuzz
erikburt marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,22 @@ set +e

SCRIPT_PATH=`dirname "$0"`; SCRIPT_PATH=`eval "cd \"$SCRIPT_PATH\" && pwd"`
OUTPUT_FILE=${OUTPUT_FILE:-"./output.txt"}
FUZZ_TIMEOUT=${FUZZ_TIMEOUT:-10m}
FUZZ_TIMEOUT_MINUTES=${FUZZ_TIMEOUT_MINUTES:-"6"}

TOTAL_SECONDS=$((FUZZ_TIMEOUT_MINUTES * 60))
if (( TOTAL_SECONDS >= 180 )); then
# Allow for a 120 second buffer between the timeout, and fuzz test runtime
FUZZ_SECONDS=$((TOTAL_SECONDS - 120))
else
echo "Increase FUZZ_TIMEOUT_MINUTES to >=3, received $FUZZ_TIMEOUT_MINUTES"
exit 1
fi

echo "timeout minutes: $FUZZ_TIMEOUT_MINUTES"
echo "fuzz seconds: $FUZZ_SECONDS"
echo "Failed fuzz tests and panics: ---------------------"
echo ""
# the amount of --seconds here is subject to change based on how long the CI job takes in the future
# as we add more fuzz tests, we should take into consideration increasing this timelapse, so we can have enough coverage.
# We are timing out after ~10mins in case the tests hang. (Current CI duration is ~8m, modify if needed)
timeout "${FUZZ_TIMEOUT}" ./fuzz/fuzz_all_native.py --ci --seconds 420 --go_module_root ./ | tee $OUTPUT_FILE
timeout "${FUZZ_TIMEOUT_MINUTES}"m ./fuzz/fuzz_all_native.py --ci --seconds "$FUZZ_SECONDS" --go_module_root ./ | tee $OUTPUT_FILE
EXITCODE=${PIPESTATUS[0]}

# Assert no known sensitive strings present in test logger output
Expand Down
2 changes: 1 addition & 1 deletion tools/bin/go_core_race_tests
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#!/usr/bin/env bash
set -ex
OUTPUT_FILE=${OUTPUT_FILE:-"./output.txt"}
TIMEOUT="${TIMEOUT:-30s}"
TIMEOUT="${TIMEOUT:-20s}"
COUNT="${COUNT:-10}"

echo "Failed tests and panics: ---------------------"
Expand Down
Loading