Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use AsyncLLM for CodeActAgent and other components #6726

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

tofarr
Copy link
Collaborator

@tofarr tofarr commented Feb 14, 2025

This PR updates the CodeActAgent class to use the async variant of LLM completion, which should improve performance by avoiding blocking operations. The changes have been made in a way that maintains backward compatibility with existing code while enabling async operations.

Changes:

  • Update CodeActAgent to use AsyncLLM and async step method
  • Update base Agent class to make step method async
  • Update all other agent implementations to use async step:
    • VisualBrowsingAgent
    • BrowsingAgent
    • DummyAgent
    • DelegatorAgent
    • MicroAgent
  • Update LLM-based condensers to use AsyncLLM and async_completion
  • Update InvariantAnalyzer to use AsyncLLM and async_completion
  • Update FileEditRuntimeMixin to use AsyncLLM and async_completion
  • Update tests to handle async operations

The main changes were:

  1. Replacing LLM with AsyncLLM in imports and type hints
  2. Making methods async that use LLM completion
  3. Replacing completion() calls with async_completion()
  4. Updating tests to handle async operations

To run this PR locally, use the following command:

docker run -it --rm   -p 3000:3000   -v /var/run/docker.sock:/var/run/docker.sock   --add-host host.docker.internal:host-gateway   -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:74fff3f-nikolaik   --name openhands-app-74fff3f   docker.all-hands.dev/all-hands-ai/openhands:74fff3f

* Update CodeActAgent to use AsyncLLM and async step method
* Update base Agent class to make step method async
* Update all other agent implementations to use async step
* Update LLM-based condensers to use AsyncLLM
* Update InvariantAnalyzer to use AsyncLLM
* Update FileEditRuntimeMixin to use AsyncLLM
* Update tests to handle async operations
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants