Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rearrange CI tests #341

Merged
merged 5 commits into from
Jan 17, 2024
Merged

Rearrange CI tests #341

merged 5 commits into from
Jan 17, 2024

Conversation

teutoburg
Copy link
Contributor

@teutoburg teutoburg commented Jan 15, 2024

  • Mark whole test class if all tests in that class were marked.
  • Integrate (cloned) notebook tests as a step in the overall test suite, as they were triggered by the same events anyway and it's übersichtlicher this way.
  • Use separate webtests and also "updated-dependencies" test from DevOps.

@teutoburg teutoburg added the tests Related to unit or integration tests label Jan 15, 2024
@teutoburg teutoburg self-assigned this Jan 15, 2024
Copy link

codecov bot commented Jan 15, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (3d174ef) 75.15% compared to head (c4d45c4) 77.04%.

Additional details and impacted files
@@              Coverage Diff               @@
##           dev_master     #341      +/-   ##
==============================================
+ Coverage       75.15%   77.04%   +1.88%     
==============================================
  Files              57       57              
  Lines            7693     7693              
==============================================
+ Hits             5782     5927     +145     
+ Misses           1911     1766     -145     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@teutoburg teutoburg changed the title Fh/no webtest Rearrange CI tests Jan 16, 2024
@teutoburg teutoburg marked this pull request as ready for review January 16, 2024 14:14
Copy link
Collaborator

@hugobuddel hugobuddel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I personally might prefer to have the notebook tests in a separate workflow from the regular tests, because we might decide to run them in different ways later.

In particular, I was hoping we could get the coverage of the regular tests up, so it would not be necessary to run the notebook tests as often. E.g. run the regular tests for each PR, but the notebooks only nightly. And maybe we should not run the notebooks on github at all, but on zeus in Vienna.

Those considerations apply mostly to the IRDB, and less so to ScopeSIM. As in, I suppose we do want to test the ScopeSim notebooks more often. So separating the tests here would be mainly to keep the repository structures similar.

Nevertheless/therefore, it is fine to make this change. It is indeed overzichtelijker this way.

@teutoburg teutoburg merged commit 44f5926 into dev_master Jan 17, 2024
24 checks passed
@teutoburg teutoburg deleted the fh/no-webtest branch January 17, 2024 13:24
@teutoburg
Copy link
Contributor Author

I personally might prefer to have the notebook tests in a separate workflow from the regular tests, because we might decide to run them in different ways later.

In particular, I was hoping we could get the coverage of the regular tests up, so it would not be necessary to run the notebook tests as often. E.g. run the regular tests for each PR, but the notebooks only nightly. And maybe we should not run the notebooks on github at all, but on zeus in Vienna.

Those considerations apply mostly to the IRDB, and less so to ScopeSIM. As in, I suppose we do want to test the ScopeSim notebooks more often. So separating the tests here would be mainly to keep the repository structures similar.

Nevertheless/therefore, it is fine to make this change. It is indeed overzichtelijker this way.

Yeah, the IRDB is different in several ways, might come around to that in the near future...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tests Related to unit or integration tests
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

2 participants