Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(dependencies): Include evaluate metrics in lm-evaluation-harness #13

Merged
merged 1 commit into from
Dec 16, 2024
Merged

feat(dependencies): Include evaluate metrics in lm-evaluation-harness #13

merged 1 commit into from
Dec 16, 2024

Conversation

ruivieira
Copy link
Member

Include evaluate metrics in lm-evaluation-harness for offline usage.
Metrics taken from https://github.com/opendatahub-io/hf-evaluate/tree/odh-2.22.
Refer to RHOAIENG-16451.

Description

How Has This Been Tested?

Merge criteria:

  • The commits are squashed in a cohesive manner and have meaningful messages.
  • Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
  • The developer has manually tested the changes and verified that the changes work

@ruivieira ruivieira added the dependencies Pull requests that update a dependency file label Dec 13, 2024
@ruivieira ruivieira self-assigned this Dec 13, 2024
danielezonca
danielezonca approved these changes Dec 13, 2024
View reviewed changes
Copy link

@danielezonca danielezonca left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

ruivieira added a commit to ruivieira/trustyai-service-operator that referenced this pull request Dec 13, 2024
@ruivieira
fix(lmeval): Remove unneeded metrics fetch
cbff4ba 
Metrics are now included in the lm-evaluation-harness repository with opendatahub-io/lm-evaluation-harness#13
@ruivieira ruivieira mentioned this pull request Dec 13, 2024
Merged
@ruivieira ruivieira merged commit 6ed16f5 into opendatahub-io:release-0.4.5 Dec 16, 2024
0 of 3 checks passed
ruivieira added a commit to trustyai-explainability/trustyai-service-operator that referenced this pull request Dec 16, 2024
@ruivieira
fix(lmeval): Remove unneeded metrics fetch (#382)
b986a1a 
Metrics are now included in the lm-evaluation-harness repository with opendatahub-io/lm-evaluation-harness#13
@ruivieira ruivieira mentioned this pull request Dec 16, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@RobGeada RobGeada RobGeada approved these changes

@danielezonca danielezonca danielezonca approved these changes

Assignees

@ruivieira ruivieira

Labels
dependencies Pull requests that update a dependency file
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@ruivieira @danielezonca @RobGeada