Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Continuous Integration (CI) using GitHub actions #203

Merged
merged 8 commits into from
Apr 12, 2024

Conversation

niketagrawal
Copy link

@niketagrawal niketagrawal commented Mar 19, 2024

Summary of code changes

  • This GitHub actions workflow automates the process of building and testing new code changes to the tudat repository. It is triggered upon push and pull request to master and develop.
  • Closes Setup Continuous Integration (CI): Automated testing for pushes and pull requests  #199
  • ccache is used to cache previous compilation results to speed up the build process across subsequent runs.
  • Three tests were found to be failing for MacOS. They will be reported as separate issues upon merging this pull request. The test failure is not related to the CI workflow.
  • The CI workflow runs in parallel for the three operating systems defined in the matrix. The time taken for the workflow to finish execution is determined by the OS that takes the most time.

Results

  • The CI workflow takes a total of 2 hours to run for the first time. The compiler cache is stored at the end of the first run.
  • After a compiler cache hit, the CI workflow takes approximately 20 minutes to finish its execution across all the three operating systems. Execution time per operating system is the following:
    • Linux: 10 minutes
    • Windows: 20 minutes
    • MacOS: 15 minutes
  • Note: There is a cache limit of 10GB for the repository. If the workflow fails to store cache, it could be due to the lack of storage space for the cache. Compiler cache entries are deleted automatically by GitHub after 1 week, but, they may need to be deleted manually if the storage fills up before a week. You can monitor and manage the cache storage for the repository here

niketagrawal and others added 5 commits March 6, 2024 11:04
…ull request.

Cache the conda environment to speed up the build across subsequent runs.
Data files in tudat-resources are not cached. They are downloaded
separately in an event of a cache hit.
Failing tests in MacOS are temporarily ignored to let the workflow pass
and the cache feature to be tested.
ccache works out of the box for the GNU compilers in Ubuntu and the
Clang compiler in MacOS. The MSVC compilers require an extra
configuration step which was added to the CMakeLists file.
Rename the ccache step to describe what it does.
Fix typo in referencing the matrix variable build type.
@niketagrawal niketagrawal changed the title Add Continuous Integration (CI) Add Continuous Integration (CI) using GitHub actions Mar 19, 2024
@niketagrawal niketagrawal added the enhancement New feature or request label Mar 19, 2024
@niketagrawal niketagrawal marked this pull request as draft March 19, 2024 12:30
…se proceeds with the build step. Caching of compilation results of the current run happens post completion of the build step.
…o a feature branch that is also a source in a pull request
@niketagrawal niketagrawal marked this pull request as ready for review March 21, 2024 11:07
@niketagrawal niketagrawal mentioned this pull request Mar 27, 2024
- name: Build
# Build your program with the given configuration. Note that --config is needed because the default Windows generator is a multi-config generator (Visual Studio generator). Use multiple cpu cores available in the runner.
shell: bash -l {0}
run: cmake --build "${{ github.workspace }}/build" --config "${{ matrix.build_type }}" -j4

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure if this would cause any problems but according to this: https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories macos-latest only has 3 cores available if I understand correctly.

Copy link

@yiquintero yiquintero Apr 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the catch @LeonardoPohl :) ! We had a look and it appears the Apple M1s have 8 cores each. The macos-latest runner has 3 M1 processors, thus giving us a total of 24 cores.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah ok great, good to know thanks :)

@DominicDirkx
Copy link
Member

Thanks! This looks awesome :) I'm merging this now, and will test it out soon!

@DominicDirkx DominicDirkx merged commit 7284c6b into develop Apr 12, 2024
4 of 6 checks passed
@niketagrawal niketagrawal deleted the feature/199-add-continuous-integration branch June 24, 2024 12:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Setup Continuous Integration (CI): Automated testing for pushes and pull requests
4 participants