Support specifying a similarity threshold #81
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Related to wagtail#21 Add support for specifying a similarity threshold in vector search methods. * **VectorIndex Class**: Add `similarity_threshold` parameter to `query`, `find_similar`, and `search` methods in `src/wagtail_vector_index/storage/base.py`. * **PgvectorIndexMixin Class**: Update `get_similar_documents` and `aget_similar_documents` methods to accept `similarity_threshold` parameter and filter results based on it in `src/wagtail_vector_index/storage/pgvector/provider.py`. * **QdrantIndexMixin Class**: Update `get_similar_documents` method to accept `similarity_threshold` parameter and filter results based on it in `src/wagtail_vector_index/storage/qdrant/provider.py`. * **WeaviateIndexMixin Class**: Update `get_similar_documents` method to accept `similarity_threshold` parameter and filter results based on it in `src/wagtail_vector_index/storage/weaviate/provider.py`. * **NumpyIndexMixin Class**: Update `get_similar_documents` method to accept `similarity_threshold` parameter and filter results based on it in `src/wagtail_vector_index/storage/numpy/provider.py`. * **Documentation**: Update `docs/vector-indexes.md` to include information on the new `similarity_threshold` parameter and provide examples of its usage. * **Tests**: Add tests for the new `similarity_threshold` parameter in `query`, `find_similar`, and `search` methods in `tests/test_index.py`. --- For more details, open the [Copilot Workspace session](https://copilot-workspace.githubnext.com/wagtail/wagtail-vector-index/issues/21?shareId=XXXX-XXXX-XXXX-XXXX).
|
I've marked this as a draft to indicate that it is work-in-progress, mainly due to failing tests. Early feedback is more than welcome :-) |
|
This is looking great thanks @brylie! I'll do a more thorough review this week and we get this in. |
tomusher
reviewed
Aug 8, 2024
|
Thanks @brylie! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Related to #21
Add support for specifying a similarity threshold in vector search methods.
similarity_thresholdparameter toquery,find_similar, andsearchmethods insrc/wagtail_vector_index/storage/base.py.get_similar_documentsandaget_similar_documentsmethods to acceptsimilarity_thresholdparameter and filter results based on it insrc/wagtail_vector_index/storage/pgvector/provider.py.get_similar_documentsmethod to acceptsimilarity_thresholdparameter and filter results based on it insrc/wagtail_vector_index/storage/qdrant/provider.py.get_similar_documentsmethod to acceptsimilarity_thresholdparameter and filter results based on it insrc/wagtail_vector_index/storage/weaviate/provider.py.get_similar_documentsmethod to acceptsimilarity_thresholdparameter and filter results based on it insrc/wagtail_vector_index/storage/numpy/provider.py.docs/vector-indexes.mdto include information on the newsimilarity_thresholdparameter and provide examples of its usage.similarity_thresholdparameter inquery,find_similar, andsearchmethods intests/test_index.py.For more details, open the Copilot Workspace session.