Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Correct typo #213

Merged
merged 1 commit into from
Aug 7, 2024
Merged

Correct typo #213

merged 1 commit into from
Aug 7, 2024

Conversation

dragonstyle
Copy link
Collaborator

This PR contains:

  • New features
  • Changes to dev-tools e.g. CI config / github tooling
  • Docs
  • Bug fixes
  • Code refactor

What is the current behavior? (You can also link to an open issue here)

What is the new behavior?

Does this PR introduce a breaking change? (What changes might users need to make in their application due to this PR?)

Other information:

╭─ gpqa (20 x 5 samples): openai/gpt-4o ───────────────────────────────────────────────────────────╮
│ epochs_reducer: ['pass_at_1', 'pass_at_2', 'pass_at_5'], max_connections:     dataset: gpqa_main │
│ 10                                                                                scorer: choice │
│                                                                                                  │
│ total time:              0:00:05                                                                 │
│ openai/gpt-4o            24,947 tokens [24,647 + 300]                                            │
│                                                                                                  │
│ accuracy[pass_at_1]: 0.48  stderr[pass_at_1]: 0.0851  accuracy[pass_at_2]:                       │
│ 0.62  stderr[pass_at_2]: 0.0866  accuracy[pass_at_5]: 0.8                                        │
│ stderr[pass_at_5]: 0.0918                                                                        │
│                                                                                                  │
│ Log: logs/2024-08-07T18-26-36-04-00_gpqa_VhMP7qnJ3BASYABnogXM2M.json                             │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯

@dragonstyle dragonstyle requested a review from jjallaire August 7, 2024 22:28
@jjallaire jjallaire merged commit cfea94b into main Aug 7, 2024
9 checks passed
@jjallaire jjallaire deleted the bugfix/pass branch August 7, 2024 22:34
lauritowal pushed a commit to lauritowal/inspect_ai that referenced this pull request Aug 19, 2024
max-kaufmann pushed a commit that referenced this pull request Sep 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants