Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Metadata comparison for retrieval metrics #54

Open
yisz opened this issue Apr 1, 2024 · 1 comment
Open

Metadata comparison for retrieval metrics #54

yisz opened this issue Apr 1, 2024 · 1 comment
Labels
enhancement New feature or request

Comments

@yisz
Copy link
Contributor

yisz commented Apr 1, 2024

No description provided.

@yisz yisz added the enhancement New feature or request label Apr 1, 2024
@yisz
Copy link
Contributor Author

yisz commented Jun 14, 2024

@kelvinchanwh I actually had this issue open a while ago. I will find some time to implement it. I plan to do the below. Let me know if you have any suggestions.

TODO: Create a new metric PrecisionRecallF1_Metadata that compares the metadata of retrieved_context vs. metadata of ground_truth_context. metadata should be a json containing information such as can be document id / page number / etc.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@yisz