-
Notifications
You must be signed in to change notification settings - Fork 512
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
embeddings persistence #83
Comments
@sbradford006 you should have redis dump in storage service |
Thanks for your input itmilos. Do you mean the storage service for the documents? Currently documents are stored in Azure blob storage but there is a local (containerizes) redis for embeddings. I don't see anything other than explicitly uploaded documents in the Azure storage unfortunately. Have i misunderstood your comment? |
Thanks again for your help - I did have that share on the storage account and have removed it. Currently assuming it was a hangover from a previous iteration of the project where the entire service was built in Azure (before i realized how expensive Redis was going to be!) as that file share hasn't been recreated. Unfortunately the local Redis instance still appears to lose all embeddings after a container restart and unfortunately simply using the batch operation "convert all files and embeddings" doesn't reproduce them. Any chance you know where the embeddings might be held on the API container, if it isn't /data? |
I'm attempting to have the embeddings in the redis (api) container persist a restart.
having mounted /data to a dir on localhost I only ever see two dirs (/data/redis and /data/redisinsight). Neither of these seem to contain any data...
i've played around with adding --save config in docker-compose but i am no docker wizard and it looks like any config passed at compose time nukes the default config configured in the container.
very possible i am misunderstanding how this should all hang together... but any advice would be welcome!
The text was updated successfully, but these errors were encountered: