Rework the precision metric. #9222
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Rework the precision metric for both CPU and GPU. - Mention it in document. - Cleanup old support code for GPU ranking metric. - Add proper support for binary classification.
hcho3
requested changes
Jun 1, 2023
| @@ -372,12 +372,37 @@ bool IsBinaryRel(linalg::VectorView<float const> label, AllOf all_of) { | |||
| * both CPU and GPU. | |||
| */ | |||
| template <typename AllOf> | |||
| void CheckMapLabels(linalg::VectorView<float const> label, AllOf all_of) { | |||
| void CheckPreLabels(StringView name, linalg::VectorView<float const> label, AllOf all_of) { | |||
There was a problem hiding this comment.
Update the docstring description: \brief Validate label for the Precision metric.
Comment on lines
-4
to
-22
| // When device ordinal is present, we would want to build the metrics on the GPU. It is *not* | ||
| // possible for a valid device ordinal to be present for non GPU builds. However, it is possible | ||
| // for an invalid device ordinal to be specified in GPU builds - to train/predict and/or compute | ||
| // the metrics on CPU. To accommodate these scenarios, the following is done for the metrics | ||
| // accelerated on the GPU. | ||
| // - An internal GPU registry holds all the GPU metric types (defined in the .cu file) | ||
| // - An instance of the appropriate GPU metric type is created when a device ordinal is present | ||
| // - If the creation is successful, the metric computation is done on the device | ||
| // - else, it falls back on the CPU | ||
| // - The GPU metric types are *only* registered when xgboost is built for GPUs | ||
| // | ||
| // This is done for 2 reasons: | ||
| // - Clear separation of CPU and GPU logic | ||
| // - Sorting datasets containing large number of rows is (much) faster when parallel sort | ||
| // semantics is used on the CPU. The __gnu_parallel/concurrency primitives needed to perform | ||
| // this cannot be used when the translation unit is compiled using the 'nvcc' compiler (as the | ||
| // corresponding headers that brings in those function declaration can't be included with CUDA). | ||
| // This precludes the CPU and GPU logic to coexist inside a .cu file | ||
|
|
There was a problem hiding this comment.
This comment is no longer valid?
There was a problem hiding this comment.
The GPU registry is removed in this PR, as the precision metric is the last one being rewritten.
src/metric/rank_metric.cc
Outdated
Comment on lines
321
to
323
| << "Invalid size of weight. For a binary classification task, it's size should be equal " | ||
| "to the number of samples. For a learning to rank task, it's size should be equal to " | ||
| "the number of query groups."; |
There was a problem hiding this comment.
I thought the Precision metric is only used for the learning-to-rank task? If so, we should not mention the binary classification task from this error message.
There was a problem hiding this comment.
Removed. This comment didn't get cleaned up. I tried to support binary classification at one point but gave up as the existing implementations are feature rich.
hcho3
approved these changes
Jun 1, 2023
hcho3
approved these changes
Jun 1, 2023
1 task
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I just realized this was not a documented feature before the PR.
The new implementation supports only learning to rank. Please consider one of the existing library implementations like sklearn for classification tasks. The previous implementation claims support for binary classification in a comment, which is false.
It is extracted from #8822.