Few questions

[cloudcompute]

Hi

I am very much new to vector databases and ML. So havesome questions in this regard.

a. Do we need to use Kafka as a middleware that sits between our application and a vector database like Qdrant in case of real-time streaming data like capturing click streams.

Or, can we directly insert events data from our application to the db without effecting the app's performance or any data loss.. I mean to ask can we completely ignore heavy-weight Kafka while working with vector databases.

Does Qdrant support backpressure? In our application, I think we need to make sure of the backpressure / retry strategy if we write to Qdrant directly.. just like we need to in case of Kafka.

b. Does it make sense to store the entire application data in a vector database. It could be for example accounts data (debit, credit, invoice) on which we do not want to apply any neural search. How about the operational costs if we decide to keep such data in Qdrant?

c. How much minimum memory and CPU does Qdrant require?

d. Approximately how much minimum memory and CPU/GPU and Memory is required in production and development for running an entire e-commerce application which has features like Personalization, Recommendations. If we need to use a neural framework like Jina and/or Transformers (Hugging Face), kindly include that too. Great if you can give a rough idea.

Thanks

[azayarni]

Hi, a lot of specific questions. :) Let us maybe have a short call? Please drop me a line if interested. Alternatively, you can join our Discord community (https://qdrant.to/discord) and post your questions there, so our engineers can reply directly. Sorry, we are not really using GitHub discussions.

[cloudcompute]

Okay, I'd join discord.

Thanks.