In memory vector search information retrieval metrics

[kuraisle]

Is this the right issue type?

• Yes, I'm planning work for this project team.

Summary

For quick comparisons of embeddings models for vector search, it would be useful to have an option that doesn't require the faffing about needed to load embeddings into a database, then querying that. A faster version that simply loads the embeddings for some subset of the OMOP vocabulary into memory and computes similarity there would be useful for rapid prototyping.

I've kept embeddings in parquet files, so it should be able to read those.

Acceptance Criteria

• Embeddings can be loaded from parquet files
• A subset of embeddings can be taken from the parquet
• Cosine similarity and dot product can be calculated
• Metrics exist for vector search
  • top k precision, recall, f-score
  • relative position of correct answer in the vocabulary

Tasks

• Embeddings loading code
• Similarity calculation code
• metrics
• Unit tests.
• Documentation.

Confirm creation

• This issue is ready