Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding Jenkins #150

Merged
merged 16 commits into from
Sep 27, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 0 additions & 8 deletions .github/scripts/build_legacy.sh

This file was deleted.

14 changes: 0 additions & 14 deletions .github/scripts/build_tests_xcore.sh

This file was deleted.

137 changes: 0 additions & 137 deletions .github/workflows/ci.yml

This file was deleted.

59 changes: 0 additions & 59 deletions .github/workflows/docs.yml

This file was deleted.

1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,4 @@ lib_xcore_math/lib/

# test cruft
test/deps
*.csv
2 changes: 2 additions & 0 deletions CHANGELOG.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ lib_xcore_math change log

* Fixes bug (issue #147) in `s16_to_s32()`.
* Fixes bug (issue #146) in `bfp_s32_macc()` and `bfp_s32_nmacc()`.
* Fixes bug with the `vect_s32_prepare_api` not appearing in the documentation.
* Fixes bug in `bfp_s32_mean()` and `bfp_s16_mean()` when hitting a corner case scenario.

2.1.2
-----
Expand Down
3 changes: 0 additions & 3 deletions CODEOWNERS

This file was deleted.

135 changes: 135 additions & 0 deletions Jenkinsfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
@Library('[email protected]')

def runningOn(machine) {
println "Stage running on:"
println machine
}

getApproval()
pipeline {
agent none

parameters {
string(
name: 'TOOLS_VERSION',
defaultValue: '15.2.1',
description: 'The XTC tools version'
)
booleanParam(
name: 'XMATH_SMOKE_TEST',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how long the tests last if XMATH_SMOKE_TEST is set to False? I wonder we should use it by default if they last less than 30 min. If we decide to run the tests in smoke test mode, when do we run the full tests?

defaultValue: true,
description: 'Enable smoke run'
)
} // parameters
options {
skipDefaultCheckout()
timestamps()
buildDiscarder(xmosDiscardBuildSettings())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I should note here that this will modify build deletion settings going forward.
details: https://github.com/xmos/xmos_jenkins_shared_library/pull/331

} // options

stages {
stage('Bullds and tests') {
parallel {
stage('Linux builds and tests') {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why don't we parallelize the xs3 tests with the x86 ones? We will reduce the duration of the Jenkins run by a considerable amount.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The way it's designed right now is that the smoke run takes 4 min, and the non-smoke run takes 30 min. This whole stage is executed on a single agent. It's not gonna bring much benefit for a smoke run. and will reduce a non-smoke run by 4 min as xs3 takes 27 min and x86 takes 4 min.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the explanation and the details

agent {
label 'xcore.ai'
}
stages {
stage('Build') {
steps {
runningOn(env.NODE_NAME)
dir('lib_xcore_math') {
checkout scm
// fetch submodules
sh 'git submodule update --init --recursive --jobs 4'
withTools(params.TOOLS_VERSION) {
// xs3a build
sh "cmake -B build_xs3a -DXMATH_SMOKE_TEST=${params.XMATH_SMOKE_TEST} --toolchain=etc/xmos_cmake_toolchain/xs3a.cmake"
sh 'make -C build_xs3a -j4'
// x86 build
sh "cmake -B build_x86 -DXMATH_SMOKE_TEST=${params.XMATH_SMOKE_TEST}"
sh 'make -C build_x86 -j4'
// xmake build
dir('test/legacy_build') {
sh 'xmake -j4'
sh 'xrun --io --id 0 bin/legacy_build.xe'
}
}
}
}
} // Build

stage('Unit tests xs3a') {
steps {
dir('lib_xcore_math/build_xs3a/test') {
withTools(params.TOOLS_VERSION) {
sh 'xrun --xscope --id 0 --args bfp_tests/bfp_tests.xe -v'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what does "-v" option do? Doesn't it just print the version? If you want to increase the verbosity, you need to use "--verbose", but I never used it myself,

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-v here is for the test itself, not xrun, hence it has --args. It means verbose

sh 'xrun --xscope --id 0 --args dct_tests/dct_tests.xe -v'
sh 'xrun --xscope --id 0 --args fft_tests/fft_tests.xe -v'
sh 'xrun --xscope --id 0 --args filter_tests/filter_tests.xe -v'
sh 'xrun --xscope --id 0 --args scalar_tests/scalar_tests.xe -v'
sh 'xrun --xscope --id 0 --args vect_tests/vect_tests.xe -v'
sh 'xrun --xscope --id 0 --args xs3_tests/xs3_tests.xe -v'
}
}
}
} // Unit tests xs3a

stage('Unit tests x86') {
steps {
dir('lib_xcore_math/build_x86/test') {
sh './bfp_tests/bfp_tests -v'
sh './dct_tests/dct_tests -v'
sh './fft_tests/fft_tests -v'
sh './filter_tests/filter_tests -v'
sh './scalar_tests/scalar_tests -v'
sh './vect_tests/vect_tests -v'
sh './xs3_tests/xs3_tests -v'
}
}
} // Unit tests x86
} // stages
post {
cleanup {
xcoreCleanSandbox()
}
}
} // Linux builds and tests

stage ('Build Documentation') {
agent {
label 'docker'
}
environment {
XMOSDOC_VERSION = "v3.0.0"
}
stages {
stage('Build Docs') {
steps {
runningOn(env.NODE_NAME)
checkout scm
sh "docker pull ghcr.io/xmos/doc_builder:${XMOSDOC_VERSION}"
sh """docker run --user "\$(id -u):\$(id -g)" \
--rm \
-v ${WORKSPACE}:/build \
-e EXCLUDE_PATTERNS="/build/doc/doc_excludes.txt" \
-e DOXYGEN=1 -e DOXYGEN_INCLUDE=/build/doc/Doxyfile.inc \
-e PDF=1 \
ghcr.io/xmos/doc_builder:${XMOSDOC_VERSION} \
|| echo "PDF build is badly broken, ignoring for now till it's fixed." """

archiveArtifacts artifacts: "doc/_build/**", allowEmptyArchive: true
} // steps
} // Build Docs
} // stages
post {
cleanup {
xcoreCleanSandbox()
}
}
} // Build Documentation

} // parallel
} // Bullds and tests
} // stages
} // pipeline
2 changes: 1 addition & 1 deletion etc/build_options.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ include(CMakeDependentOption)
set(DEV_LIB_XCORE_MATH OFF CACHE BOOL "Include unit tests and examples as CMake Targets." )

## If enabled, the unit tests will be run fewer repetitions of each test to speed up execution time
option(XCORE_MATH_SMOKE_TEST "Build unit tests in 'smoke test' mode. This mostly just reduces the number of repetitions to reduce simulation time." OFF)
option(XMATH_SMOKE_TEST "Build unit tests in 'smoke test' mode. This mostly just reduces the number of repetitions to reduce simulation time." OFF)

## If enabled, the FFT look-up tables will be auto-generated at build time (requires Python 3 and numpy)
set( XCORE_MATH_GEN_FFT_LUT OFF CACHE BOOL "Auto-generate FFT look-up tables." )
Expand Down
Loading