Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added asym_rmse and asym_mae accuracy measures #156

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

martincousi
Copy link

Hi, I have created two new accuracy measures for a project I am working on. These measures are asymmetric and can put more emphasis on lower or higher ratings. They are useful when the rating distribution is asymmetric. See [1] for more details.

I don't know how to include these measures in GridSearchCV since they require a weight parameter.

P.S. This is my first time contributing to a library so I am open to suggestions on how to improve.

[1] R. Zhu, D. Niu, L. Kong, and Z. Li, “Expectile Matrix Factorization for Skewed Data Analysis,” in Proceedings of the 31th Conference on Artificial Intelligence (AAAI 2017), 2017, pp. 259–265.

@NicolasHug
Copy link
Owner

NicolasHug commented Mar 29, 2018

Thanks for the PR and for the details :)

Can you please fix the pep8 issue?

I don't know how to include these measures in GridSearchCV since they require a weight parameter.

Yes that's a good point. I guess we'll have to implement a more sophisticated way to pass the measures parameter in GridSearchCV and in cross_validate. I haven't gone through the details but I think the way to go with scikit-learn is to use make_scorer, so we could make something like that.

@martincousi
Copy link
Author

I think that I solved all pep8 issues now.

@NicolasHug
Copy link
Owner

NicolasHug commented Apr 2, 2018

Thanks,

I'll leave the PR open and merge it when we find a way to address the previous point. I'd rather not merge it now as it's not fully compatible with GridSearchCV and cross_validate.

@martincousi
Copy link
Author

Ok, if I will see if I get the time to adjust the way a measure is passed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants