Update Rockset details under rockset.json #151
Conversation
ankit1khare
changed the title
There was a problem hiding this comment.
Hi @ankit1khare - thank you for your PR!
Please see my comments next to your changes.
We combine ANN (vector search) with metadata filtering which is hybrid search in essence. Updated the links. |
| "support": "", | ||
| "source_url": "", | ||
| "comment": "" | ||
| "support": "full", |
There was a problem hiding this comment.
Please change this to none as Hybrid Search has a specific meaning in this tab le, which is different from what you explained.
There was a problem hiding this comment.
Hi @ankit1khare, I'm guessing you mean to point to this section https://github.com/sofia099/search_autocomplete#step-3-create-3-query-lambdas in the new repo you've linked. It seems to combine a set of exact matches with approximate lexical matches.
Hybrid Search in this table refers to "ability to merge between ranked lists of keyword search and vector search results within the DB".
Is it possible to do something like Reciprocal Rank Fusion or take a linear combination of scores from vector search and keyword search in a similar way?
There was a problem hiding this comment.
Hey @dhruv-anand-aintech, after all it's SQL we are talking about where we have built our own engine from scratch such that all standard SQL operations can be done on different data modalities. One can build their own weighting mechanism using SQL and get all of the analytical capabilities built into the query language (like linear combination, reciprocal rank fusion, etc.).
There was a problem hiding this comment.
Thanks for the explanation. It'll still be good to have an example notebook that does these types of fusion in Rockset, so that users are easily able to migrate to it from other DBs that do it under the hood.
| "support": "full", | ||
| "source_url": "https://docs.rockset.com/documentation/reference/user-defined-functions", | ||
| "comment": "" | ||
| }, | ||
| "embeddings_image": { | ||
| "support": "", | ||
| "source_url": "", | ||
| "support": "full", | ||
| "source_url": "https://docs.rockset.com/documentation/reference/user-defined-functions", |
There was a problem hiding this comment.
The ability to run user-defined functions counts as "partial" support for these 2 columns. Full support would be for ready to use embedding modules (see examples in the table).
I'll switch these and merge the PR. You can comment here if there is further clarification.
Feel free to reach out to me if something is unclear :)