Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use sqrtf to avoid cast to double #252

Merged
merged 2 commits into from
Mar 21, 2024
Merged

Conversation

braingram
Copy link
Collaborator

@braingram braingram commented Mar 21, 2024

The use of sqrt from libc results in a cast to/from double in the ols_cas22 ramp fitting (with jump detection). This can result in small numerical differences that lead to romancal regtests that passed on jenkins but failed locally (due to a single extra jump in one pixel).

This PR switches sqrt for sqrtf to keep the intermediate values as float which leads to (at the moment) the same failure locally and on jenkins (where test_rampfit_step[spec_full] fails with a single pixel difference):

    {'arrays_differ': {"root['roman']['data']": {'abs_diff': <Quantity 1.9270487 DN / s>,
                                                 'n_diffs': 1,
                                                 'worst_abs_diff': {'index': (2798,
                                                                              2758),
                                                                    'value': <Quantity 1.9270487 DN / s>},

Regtest run here:
https://plwishmaster.stsci.edu:8081/job/RT/job/Roman-Developers-Pull-Requests/681/
The test_resample failure is unrelated and also occurs on main.

Checklist

  • added entry in CHANGES.rst (either in Bug Fixes or Changes to API)
  • updated relevant tests
  • updated relevant documentation
  • updated relevant milestone(s)
  • added relevant label(s)

@braingram braingram marked this pull request as ready for review March 21, 2024 14:57
@braingram braingram requested a review from a team as a code owner March 21, 2024 14:57
@braingram braingram marked this pull request as draft March 21, 2024 14:57
Copy link
Collaborator

@schlafly schlafly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, thank you!

@braingram braingram marked this pull request as ready for review March 21, 2024 15:23
@braingram
Copy link
Collaborator Author

@zacharyburnett can we squeeze this into 1.6.2? If not I'll need to update the changelog entry for this PR.

Copy link

codecov bot commented Mar 21, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 85.18%. Comparing base (41259f3) to head (af5aefb).
Report is 11 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #252      +/-   ##
==========================================
- Coverage   85.90%   85.18%   -0.73%     
==========================================
  Files          35       35              
  Lines        6557     6797     +240     
==========================================
+ Hits         5633     5790     +157     
- Misses        924     1007      +83     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Collaborator

@zacharyburnett zacharyburnett left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wow what a simple fix! I'll run these through the romancal regtests on github actions too when it's merged

@braingram
Copy link
Collaborator Author

Thanks! I'd hold off re-running them until @schlafly can update the truth files. The change here only breaks jenkins in the same way as the local tests were failing (so this PR will break 2 regtests in romancal until the truth files can be updated).

@braingram braingram merged commit 4b5ea2f into spacetelescope:main Mar 21, 2024
24 of 25 checks passed
@braingram braingram deleted the sqrtf branch March 21, 2024 15:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants