add vision client template #526
Conversation
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
There was a problem hiding this comment.
PR Summary
Added a new vision client template that enables both image and text embedding functionality through a unified API interface.
- Added
InfinityVisionAPIclass in/libs/client_infinity/template/vision_client.pywith concurrent request handling via ThreadPoolExecutor - Implemented robust error handling with request retries and HTTP session management using HTTPAdapter
- Added base64 encoding/decoding support for image processing with PIL integration
- Added semaphore-based request throttling with a limit of 64 concurrent requests
- Added automatic hidden dimension detection during client initialization for proper embedding shape handling
💡 (5/5) You can turn off certain types of comments like style here!
1 file(s) reviewed, 5 comment(s)
Edit PR Review Bot Settings | Greptile
| self.tp = ThreadPoolExecutor() | ||
| self.tp.__enter__() |
There was a problem hiding this comment.
logic: ThreadPoolExecutor is entered but never exited. Need a del or close method to properly clean up resources.
| self.tp = ThreadPoolExecutor() | |
| self.tp.__enter__() | |
| self.tp = ThreadPoolExecutor() | |
| self.tp.__enter__() | |
| # Add destructor to ensure cleanup | |
| def __del__(self): | |
| self.tp.__exit__(None, None, None) |
| import numpy as np # Import numpy | ||
| from typing import Union | ||
| HAS_IMPORTS = True | ||
| try: | ||
| from PIL import Image | ||
| import numpy as np |
There was a problem hiding this comment.
style: numpy is imported twice unnecessarily
| import numpy as np # Import numpy | |
| from typing import Union | |
| HAS_IMPORTS = True | |
| try: | |
| from PIL import Image | |
| import numpy as np | |
| import numpy as np # Import numpy | |
| from typing import Union | |
| HAS_IMPORTS = True | |
| try: | |
| from PIL import Image |
| def embed(self, model: str, sentences: list[str]) -> Future[list]: | ||
| self.health() | ||
| with self.sem: |
There was a problem hiding this comment.
logic: health check failure should prevent the embedding request rather than just raising separately
| req.raise_for_status() | ||
| return req.status_code == 200 | ||
|
|
||
| def _request(self, model: str, images_or_text: list[Union["Image.Image", str]]) -> dict: |
There was a problem hiding this comment.
syntax: return type hint should be tuple[list, int] instead of dict
| def _request(self, model: str, images_or_text: list[Union["Image.Image", str]]) -> dict: | |
| def _request(self, model: str, images_or_text: list[Union["Image.Image", str]]) -> tuple[list, int]: |
| def test_colpali(): | ||
| colpali = InfinityVisionAPI() | ||
| future = colpali.embed("michaelfeil/colqwen2-v0.1", ["test"]) | ||
| embeddings, total_tokens = future.result() | ||
| print(embeddings, total_tokens) |
There was a problem hiding this comment.
style: test function should use assertions instead of print statements
|
Codecov ReportAll modified and coverable lines are covered by tests ✅
❗ Your organization needs to install the Codecov GitHub app to enable full functionality. Additional details and impacted files@@ Coverage Diff @@
## main #526 +/- ##
==========================================
+ Coverage 79.74% 79.86% +0.11%
==========================================
Files 43 43
Lines 3486 3486
==========================================
+ Hits 2780 2784 +4
+ Misses 706 702 -4 ☔ View full report in Codecov by Sentry. |
Related Issue
Checklist
Additional Notes
Add any other context about the PR here.