DPR create_embeddings/update_embeddings FAISS is so slow!

[shahad2099]

Hello everyone,

I hope you're all doing well. I am using the Haystack framework to build a retriever. After training DPR, I wanted to use FAISS as my vector database. However, updating or creating embeddings is so slow! I have 3 million short documents (100 words/document), and for only 17% of them, it takes almost 5 hours and 38 minutes. This is incredibly frustrating for me.

What should I do? Are there any optimizations that I should implement in the code?

I also have some simple questions:

1-If I want to add more data, should I store all the documents and create embeddings from scratch? Or can I add and create embeddings for the new ones only?
2-In your opinion, what is the most efficient vector database for a retriever?
3-Does the update_embeddings function utilize GPUs or multi-threading?

Many thanks to all of you!

[julian-risch]

Hi @shahad2099 happy to answer your questions! 🙂

1. You can add more data later and then create embeddings only for newly added data. For that purpose, the update_embeddings method has a update_existing_embeddings=False parameter.
2. When it comes to picking the best document store for your use case I recommend our overview table on this documentation page: https://docs.haystack.deepset.ai/docs/document_store#choosing-the-right-document-store
3. Could you please check whether you installed faiss-cpu or faiss-gpu? You can run pip freeze | grep faiss in the command line to check that. If the speed is slow you're most likely not utilizing your GPU.

[shahad2099]

I am using faiss-gpu==1.7.2, but it's so slow—almost 34 hours for 3 million documents of length 100 words.

I am also wondering if the results differ based on the vector DB? I mean, if I used the same model, the results should always be the same, no matter what vector DB I am using ?.

[shahad2099]

Can someone answer why the embedding takes all this time ?
Thanks

[ZanSara]

Hey @shahad2099 are you really sure you are using the GPU? There might be a misconfiguration somewhere. And in any case, 3 million documents is a lot! Unless you have a powerful GPU it can take quite some time. For such large usecases we normally recommend a more production-ready document store like Milvus, Weaviate, Pinecone, ...

[shahad2099]

Hello,

Yes, I'm sure that I am using GPUs. However, the utilization of the GPUs differs: one is at 70%, and the other is at 30%. I want both GPUs to be running at 100%.

Thank you.