Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add numerical test to CI #19260

Merged
merged 5 commits into from
Mar 8, 2024
Merged

add numerical test to CI #19260

merged 5 commits into from
Mar 8, 2024

Conversation

haifeng-jin
Copy link
Contributor

Resolves #19216.
Add numerical test to the CI to catch more bugs.

@codecov-commenter
Copy link

codecov-commenter commented Mar 7, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 54.02%. Comparing base (c8700f4) to head (0855748).
Report is 79 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff             @@
##           master   #19260       +/-   ##
===========================================
- Coverage   80.14%   54.02%   -26.13%     
===========================================
  Files         341      363       +22     
  Lines       36163    39422     +3259     
  Branches     7116     7630      +514     
===========================================
- Hits        28982    21296     -7686     
- Misses       5578    16575    +10997     
+ Partials     1603     1551       -52     
Flag Coverage Δ
keras 54.02% <ø> (-25.97%) ⬇️
keras-jax ?
keras-numpy 54.02% <ø> (-3.07%) ⬇️
keras-tensorflow ?
keras-torch ?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@fchollet
Copy link
Collaborator

fchollet commented Mar 7, 2024

Thanks for the PR!

ModuleNotFoundError: No module named 'tf_keras'

We can add this step to the GitHub action (no need to add it to requirements.txt)

@@ -41,7 +44,8 @@ def build_keras_model(keras_module, num_classes):
keras_module.layers.Conv2D(
64, kernel_size=(3, 3), activation="relu"
),
keras_module.layers.BatchNormalization(scale=False, center=True),
# TODO: Renable the following line.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the issue? Numerical difference across backends?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, if I add this layer, TF and JAX tests would fail due to numerical differences.

@google-ml-butler google-ml-butler bot added kokoro:force-run ready to pull Ready to be merged into the codebase labels Mar 8, 2024
@fchollet fchollet merged commit 618a674 into keras-team:master Mar 8, 2024
9 checks passed
@google-ml-butler google-ml-butler bot removed awaiting review ready to pull Ready to be merged into the codebase kokoro:force-run labels Mar 8, 2024
@haifeng-jin haifeng-jin deleted the numerical branch March 8, 2024 18:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: Assigned Reviewer
Development

Successfully merging this pull request may close these issues.

Enable integration_test/numerical_test.py for CI
4 participants