Skip to content

Conversation

finbarrtimbers
Copy link
Collaborator

@finbarrtimbers finbarrtimbers commented Aug 8, 2025

Here, I add two tests:

  1. There was a pre-existing test for ToolUseLLM in open_instruct/tool_utils/tool_vllm.py. Here, I pull this out into test_tool_vllm.py. I also add a mocked vllm instance so that we will automatically run the tool tests as part of our CI. The tests pass on both GPU and CPU.
  2. I have updated the beaker integration test so that it runs with 2 vllm engines and uses a smaller Qwen model (0.6B). This test would have caught the timeout issue (example repro). This also succeeds otherwise (example run).

@finbarrtimbers finbarrtimbers marked this pull request as ready for review August 8, 2025 19:31
@finbarrtimbers finbarrtimbers changed the title Adds a test for ToolUseLLM. Adds more tests! Aug 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant