Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Save scores per prompt #21

Merged
merged 4 commits into from
Feb 13, 2024
Merged

Save scores per prompt #21

merged 4 commits into from
Feb 13, 2024

Conversation

natolambert
Copy link
Collaborator

@natolambert natolambert commented Feb 12, 2024 •
edited
Loading

WIP, but:

  • Saves scores per prompt along side bucketed results
  • Cleans up model loading logic / run_rm.py
  • Adds util for saving
  • Cleaning & clearer variable names

Close #4 , Example data here https://huggingface.co/datasets/ai2-adapt-dev/herm-debug/tree/main

jacob-morrison
jacob-morrison requested changes Feb 12, 2024
View reviewed changes
README.md Show resolved Hide resolved
scripts/run_rm.py Show resolved Hide resolved
@natolambert natolambert merged commit f441f9c into main Feb 13, 2024
3 checks passed
@ljvmiranda921 ljvmiranda921 deleted the save_scores branch March 16, 2024 21:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jacob-morrison jacob-morrison jacob-morrison approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Save reward scores for each prompt
2 participants
@natolambert @jacob-morrison