Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add gpt-4o-mini to smoke test github workflow and make smoke test judge more reliable #873

Merged
merged 2 commits into from
Oct 14, 2024
Merged

Add gpt-4o-mini to smoke test github workflow and make smoke test judge more reliable #873

merged 2 commits into from
Oct 14, 2024
+2,613 −12,339

Commits on Oct 14, 2024

  1. test: refine smoke judge comparison rules and output 

    - Focus comparison on matching event types to reduce false negatives
- Drop "ignore callProgress" rule (we're eliding them from the event
  stream before sending them to the judge now)

Signed-off-by: Nick Hale <[email protected]>
    @njhale
    njhale committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    89a9398 View commit details
    Browse the repository at this point in the history

  2. test: add gpt-4o-mini to smoke test github workflow 

    - Add gpt-4o-mini to smoke test github workflow
- Add gpt-4o-mini smoke test golden files
- Remove outdated gpt-4o model and 4-turbo smoke test config and golden
files
- Add golden files for gpt-4o-2024-08-06
- Regenerate golden files for existing models to drop callProgress
  events (we weren't comparing these anyway)

Signed-off-by: Nick Hale <[email protected]>
    @njhale
    njhale committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    ef6c6fc View commit details
    Browse the repository at this point in the history