feat(tools): add use_computer tool to Strands tools repository #114
Conversation
| @@ -66,7 +67,7 @@ pip install strands-agents-tools | |||
| To install the dependencies for optional tools: | |||
|
|
|||
| ```bash | |||
| pip install strands-agents-tools[mem0_memory, use_browser] | |||
| pip install strands-agents-tools[mem0_memory, use_browser, use_computer] | |||
There was a problem hiding this comment.
pip install strands-agents-tools[use_computer] isn't working for me
There was a problem hiding this comment.
Do you need to do something like the following? pip install "strands-agents-tools[use_computer]@file:///Volumes/workplace/tools/" Until the pyproject.toml file is published I think we have to download them that way. I tested in a new environment and there are a few more dependencies I need to add under use_computer in the pyproject.toml file
There was a problem hiding this comment.
I did tested deeply, here's the agent code and conversation history: https://gist.github.com/cagataycali/c1af4f368b4159bc3a2474372c884c03
Here're some screenshots captured by use_computer:
jimbrub
force-pushed
the
use_computer
branch
3 times, most recently
from
40f4ca5 to
2dfe4e6
Compare
| image_path = screenshot_path | ||
| should_delete = False | ||
| else: | ||
| image_path = create_screenshot(region) |
There was a problem hiding this comment.
Are we returning the screenshot parameters to LLM back, seems like it's not returned back to the model ^^
…he model for context
Description
Key Features
This tool was tested on macOS, so some features may not work on Windows/Linux
Basic Computer Control:
Screen Analysis:
Cross-Platform Support:
Technical Implementation
pyautoguifor computer controlDependencies
Download Tesseract OCR on mac:
brew install tesseractRelated Issues
[Link to related issues using #issue-number format]
Documentation PR
TBD
Type of Change
Testing
[How have you tested the change?]
hatch fmt --linterhatch fmt --formatterhatch test --allChecklist
I have read the CONTRIBUTING document
I have added tests that prove my fix is effective or my feature works
I have updated the documentation accordingly
I have added an appropriate example to the documentation to outline the feature
My changes generate no new warnings
Any dependent changes have been merged and published
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.