feat(weave): Add xtra Scorers to Weave #3006
Open
+5,452
−127
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Preview this PR with FeatureBee: https://beta.wandb.ai/?betaVersion=857eb4902e28b98edc92742da425775a864becdc |
tcapelle
changed the title
tcapelle
changed the title
* add robustness scorer * add tests * update init * add eval test * lint * handle if ground truth is present+str/bool dtypes * fix scorer - slight logic falacy * cover more ground with tests * lint * update RobustnessScorer to support continuous values using cohen d * chore(ui): lighten trace tree lines (#3037) * feat(ui): object comparison (#2942) * feat(weave): simple prompt classes (#3032) * fix(weave): Fix issue where Evaluation can raise unexpectedly due to bad scorer (#3039) * support cohen d for continuous values * Remove highlight-next-line to fix indentation issue (#3021) * chore(weave): Pull str limit magic numbers into const #3042 * chore(weave): Cleanup save object basic (#3015) * chore(weave): Remove signature from Op (#2989) * chore(weave): Better names for internal methods (#2982) * fix(weave): Bump display name limit to 1024 chars (#3043) * Release version: 0.51.22-dev0 → 0.51.22 * Release version: 0.51.22 → 0.51.23-dev0 * chore(weave): Rethrow useExpandedNode error for proper serialization (#3017) * chore(ui): change dataset link (#3040) * fix(weave): Elide display names that exceed the char limit (#3046) * chore(weave): hide annotations button temporarily #3048 * add docstring with details * lint * update scorer-tests deps * make robustness dataset * add tests for dataset builder --------- Co-authored-by: Jamie Rasmussen <[email protected]> Co-authored-by: Andrew Truong <[email protected]> Co-authored-by: J2-D2-3PO <[email protected]> Co-authored-by: Weave Build Bot <[email protected]> Co-authored-by: domphan-wandb <[email protected]> Co-authored-by: Tim Sweeney <[email protected]>
…rait. remove TrustScore prints
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds