Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[enhancement] add sklearnex version of validate_data, _check_sample_weight #2177

Open
wants to merge 135 commits into
base: main
Choose a base branch
from

Conversation

icfaust
Copy link
Contributor

@icfaust icfaust commented Nov 20, 2024

Description

This is another interim PR towards introducing the new onedal finiteness checker into the sklearnex estimator workflows. This is not yet introduced into any of the estimators, and so performance benchmarks are not necessary. This PR focuses on making sure that input and outputs of validate_data and _check_sample_weight are respected for sycl_usm_ndarray types and that the new finite checker is properly called and yields results in a range of scenarios. This is also done to minimize the review burden, as changing all the estimators is a large change.

The new process for all estimators will be as follows:

  • All estimators will call validate_data and _check_sample_weight once in sklearnex in _onedal_* methods called by device_offload's dispatch
  • All estimators will call assert_all_finite no where else but in validate_data or _check_sample_weight unless an operation before the oneDAL backend can yield a inf/NaN (this is a strict condition, and is expected to be extremely uncommon/ hard to allow)
  • Calls to check_array anywhere in the onedal or sklearnex folders must have assert_all_finite checks turned off.

A follow up PR will create a design test for this, and will introduce the new validate_data in one estimator. Other estimators will occur in individual PRs due to the depth of the changes.


PR should start as a draft, then move to ready for review state after CI is passed and all applicable checkboxes are closed.
This approach ensures that reviewers don't spend extra time asking for regular requirements.

You can remove a checkbox as not applicable only if it doesn't relate to this PR in any way.
For example, PR with docs update doesn't require checkboxes for performance while PR with any change in actual code should have checkboxes and justify how this code change is expected to affect performance (or justification should be self-evident).

Checklist to comply with before moving PR from draft:

PR completeness and readability

  • I have reviewed my changes thoroughly before submitting this pull request.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have updated the documentation to reflect the changes or created a separate PR with update and provided its number in the description, if necessary.
  • Git commit message contains an appropriate signed-off-by string (see CONTRIBUTING.md for details).
  • I have added a respective label(s) to PR if I have a permission for that.
  • I have resolved any merge conflicts that might occur with the base branch.

Testing

  • I have run it locally and tested the changes extensively.
  • All CI jobs are green or I have provided justification why they aren't.
  • I have extended testing suite if new functionality was introduced in this PR.

Performance

  • I have measured performance for affected algorithms using scikit-learn_bench and provided at least summary table with measured data, if performance change is expected.
  • I have provided justification why performance has changed or why changes are not expected.
  • I have provided justification why quality metrics have changed or why changes are not expected.
  • I have extended benchmarking suite and provided corresponding scikit-learn_bench PR if new measurable functionality was introduced in this PR.

@icfaust
Copy link
Contributor Author

icfaust commented Nov 25, 2024

/intelci: run

@icfaust
Copy link
Contributor Author

icfaust commented Nov 25, 2024

/intelci: run

@icfaust
Copy link
Contributor Author

icfaust commented Nov 26, 2024

Please manually check private CI, failure is a 503 http error (unrelated to the PR).

@icfaust icfaust marked this pull request as ready for review November 26, 2024 21:57
@@ -438,3 +443,31 @@ def _is_csr(x):
return isinstance(x, sp.csr_matrix) or (
hasattr(sp, "csr_array") and isinstance(x, sp.csr_array)
)


def _assert_all_finite(X, allow_nan=False, input_name=""):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is a version check required here that sets _assert_all_finite() to the daal4py version for older oneDAL releases?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes made for circumstance (won't come up in any of the CIs though)

sklearnex/utils/tests/test_validation.py Outdated Show resolved Hide resolved
sklearnex/utils/tests/test_validation.py Outdated Show resolved Hide resolved
@ahuber21
Copy link
Contributor

Is there any additional need to check if "old" and "new" finiteness checks produce the same results. I'd say no, because they're both tested independently, and neither is a gold standard. But let me know if you think otherwise.

@icfaust
Copy link
Contributor Author

icfaust commented Nov 27, 2024

/intelci: run

@icfaust
Copy link
Contributor Author

icfaust commented Nov 29, 2024

/intelci: run

@icfaust
Copy link
Contributor Author

icfaust commented Dec 2, 2024

/intelci: run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants