-
Notifications
You must be signed in to change notification settings - Fork 96
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Inverted Uplift scores of revert label (CLassTransformation) ? #199
Comments
@CoteDave Hello! Thanks a lot for providing information. Could you provide the code and data to reproduce this bug? |
Hi Maksim, here is the code (As you can see, pretty simple out of the box and exactly the same data is used for SoloModel and ClassTransformation but the predicted uplift scores for ClassTransformation seems distributed weirdly). The other 2 models are SLearner and XLearner from the Econml library. Unfortunately, I can't share the data (enterprise). Some facts about the set up:
And that's it! |
Thanks a lot! We will try to reproduce the bug, and then we will return with the results 🙌 |
Hi, I found the problem. The Class Transformation is only made for balanced T0, T1 datasets. My dataset is highly skewed (fewer T1). Soo I can't use the class transformation algorithms as the first assumption is that the T1 and T0 are balanced. |
🐛 Bug
scikit-uplift==0.5.1
Hi I just compared the uplift scores of the ClassTransformation with other uplift strategies (SoloModel(slearner), xlearner) and the uplift scores of the class transformation seems very off !
As the ClassTransformation transform the target before the training, should it transform back the predicted uplift scores in the output ? How to interpret the scores compared to the other strategies ? Is this a bug ?
And if we draw the qini curves, we clearly see that the uplift scores of the revert label (ClassTransformation) seem inverted:
![image](https://user-images.githubusercontent.com/49559881/195647502-8e2d8fa5-b6bd-4f10-aff3-c52734900e70.png)
Nothing special in the .fit(), X, Y, T, Estimator are the same for the SoloModel() and the ClassTransformation()
The text was updated successfully, but these errors were encountered: