An enterprise-grade AI retriever designed to streamline AI integration into your applications, ensuring cutting-edge accuracy.
Denser Retriever combines multiple search technologies into a single platform. It utilizes gradient boosting ( xgboost) machine learning technique to combine:
- Keyword-based searches that focus on fetching precisely what the query mentions.
- Vector databases that are great for finding a wide range of potentially relevant answers.
- Machine Learning rerankers that fine-tune the results to ensure the most relevant answers top the list.
- Our experiments on MTEB datasets show that the combination of keyword search, vector search and a reranker via a xgboost model (denoted as ES+VS+RR_n) can significantly improve the vector search (VS) baseline.
- Check out Denser Retriever experiments using the Anthropic Contextual Retrieval dataset at here.
The initial release of Denser Retriever provides the following features.
- Supporting heterogeneous retrievers such as keyword search, vector search, and ML model reranking
- Leveraging xgboost ML technique to effectively combine heterogeneous retrievers
- State-of-the-art accuracy on MTEB Retrieval benchmarking
- Demonstrating how to use Denser retriever to power an end-to-end applications such as chatbot and semantic search
We recommend installing Python via Anaconda, as we have received feedback about issues with Numpy installation when using the installer from https://www.python.org/downloads/. We are working on providing a solution to this problem. To install Denser Retriever, you can run:
pip install git+https://github.com/denser-org/denser-retriever.git#main
poetry add git+https://github.com/denser-org/denser-retriever.git#main
The official documentation is hosted on retriever.denser.ai. Click here to get started.
You can start developing Denser Retriever on your local machine.
See DEVELOPMENT.md for more details.
This project is licensed under the terms of the MIT
license.
See LICENSE for more details.
@misc{denser-retriever,
author = {denser-org},
title = {An enterprise-grade AI retriever designed to streamline AI integration into your applications, ensuring cutting-edge accuracy.},
year = {2024},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {\url{https://github.com/denser-org/denser-retriever}}
}