Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support estimating distance between two compressed datasets #9

Open
thomasahle opened this issue Apr 12, 2023 · 2 comments
Open

Support estimating distance between two compressed datasets #9

thomasahle opened this issue Apr 12, 2023 · 2 comments
Labels
enhancement New feature or request good first issue Good for newcomers

Comments

@thomasahle
Copy link
Owner

thomasahle commented Apr 12, 2023

Often we use PQ to estimate the distance from a full precision vector to a bunch of compressed points.
However, we can also try to compute the distance between all pairs of points in two compressed datasets (even possibly with distinct FastPQ instances).

This is relevant, for example, when inserting a batch of points into the data structure, when we quickly want to compute all the relevant close cluster centers.
Currently we compute this using full precision distance computations.

Edit: Maybe #13 is more relevant for speeding up building the index. However, supporting estimating distances between compressed datasets is still interesting and worthwhile.

@thomasahle thomasahle added enhancement New feature or request good first issue Good for newcomers labels Apr 12, 2023
@tsabbir96
Copy link

@thomasahle Hi, I am new to opensource contribution and would like to take up on this issue, if thats okay... Could you tell me more about this issue?

@thomasahle
Copy link
Owner Author

Awesome! Do you have a rough idea about how the library works already?
In particular the distance_table method in the FastPQ class?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

2 participants