Skip to content

Commit

Permalink
Migrate "test and compare" test to be done from GH Actions
Browse files Browse the repository at this point in the history
  • Loading branch information
michaliskambi committed Jun 18, 2024
1 parent 99dee39 commit dcce0d6
Show file tree
Hide file tree
Showing 6 changed files with 51 additions and 77 deletions.
46 changes: 46 additions & 0 deletions .github/workflows/test-demo-models.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# ----------------------------------------------------------------------------
# GitHub Actions workflow to test correct output on demo models.
# ----------------------------------------------------------------------------

name: Test Demo Models

on: [push, pull_request]

defaults:
run:
shell: bash

jobs:
test:
name: Test Demo Models
runs-on: ubuntu-latest
container: kambi/castle-engine-cloud-builds-tools:cge-unstable
steps:
- uses: actions/checkout@v4

- name: Checkout demo-models
uses: actions/checkout@v4
with:
repository: castle-engine/demo-models
path: demo-models

- name: Build
run: make

- name: Run Tests
run: test_scripts/run_tests_on_dir_final.sh demo-models/

# We save output before comparing.
# This way we have it even when comparing fails.
# This is useful to later analyze, and to update test_scripts/run_tests_valid_output.txt .
- name: Archive Artifacts (logs)
uses: actions/upload-artifact@v4
with:
name: logs
path: |
run_tests_output.txt
run_tests_output_verbose.txt
if-no-files-found: error

- name: Compare test with last known good output
run: diff -wu run_tests_output.txt test_scripts/run_tests_valid_output.txt
63 changes: 0 additions & 63 deletions Jenkinsfile.test-demo-models

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ set -eu
# Tip: To test manually how it works on a single file, you can use this:
# $ cd castle-model-viewer/
# $ rm -f output-short.txt output-verbose.txt
# $ ./jenkins_scripts/run_test_on_model.sh output-short.txt output-verbose.txt ../demo-models/texturing_advanced/warnings/tex3d_composed_warnings.x3dv
# $ ./test_scripts/run_test_on_model.sh output-short.txt output-verbose.txt ../demo-models/texturing_advanced/warnings/tex3d_composed_warnings.x3dv
# ... and consult output-short.txt output-verbose.txt output
#
# This bash script uses settings similar to bash strict mode, for similar reasons,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ set -eu
# Change LANG to something hardcoded,
# because `sort' results are affected by the current locale (see man sort).
# And we want them to be predictable, the same on every system,
# to make comparing two outputs (like run_tests_on_dir_and_compare.sh does) reliable.
# to make comparing two outputs (like run_tests_on_dir_final.sh does) reliable.
export LANG=C
export LC_COLLATE=C

Expand All @@ -73,7 +73,7 @@ set +e
# The "find" output is run through sort and then xargs.
# Otherwise, find prints files in an unpredictable order (from readdir),
# which would make comparing two outputs from two different systems (like
# run_tests_on_dir_and_compare.sh does) impossible.
# run_tests_on_dir_final.sh does) impossible.
# Fortunately, find + sort go very fast (compared to actual
# run_test_on_model.sh time).

Expand All @@ -97,6 +97,6 @@ find "$@" \
-iname '*.starling-xml' ')' \
-print0 ')' | \
sort --zero-terminated | \
xargs -0 --max-args=1 jenkins_scripts/run_test_on_model.sh "${OUTPUT_SHORT}" "${OUTPUT_VERBOSE}"
xargs -0 --max-args=1 test_scripts/run_test_on_model.sh "${OUTPUT_SHORT}" "${OUTPUT_VERBOSE}"

set -e
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ set -eu
# ----------------------------------------------------------------------------

# run tests
jenkins_scripts/run_tests_on_dir.sh run_tests_output.txt run_tests_output_verbose.txt "$@"
test_scripts/run_tests_on_dir.sh run_tests_output.txt run_tests_output_verbose.txt "$@"

# remove OpenAL trash from outpt
sed --in-place=.bak \
Expand All @@ -29,12 +29,3 @@ REPOSITORY_DIR="`pwd`"
sed --in-place=.bak2 -e "s|${REPOSITORY_DIR}|DIR|g" run_tests_output.txt

cat run_tests_output.txt

# compare with last correct output
#
# TODO: -w added only temporarily, since "Possible reasons:" was for some time
# with extra space in CGE repo, but in "run_tests_valid_output.txt" has
# trimmed space. Once CGE updates, it should be everywhere without space,
# and -w can be removed below.

diff -wu run_tests_output.txt jenkins_scripts/run_tests_valid_output.txt
File renamed without changes.

0 comments on commit dcce0d6

Please sign in to comment.