Backup storage infos

[arita37]

Just a small sharing post on backup best practices,
could not find extensive docs here, it can be useful to anyboyd looking for backup best pratices
https://qdrant.tech/documentation/storage/

1. Is "safe" to copy /qdrant_storage as backup... ?
   Suppose there is the hot writing issue...

2. What about the data in RAM, how to force the flush on disk (?)

3. A way to shut down gracefully qdrant (ie writes finish or cut the network In/ to prevent further writes). ?

For info, Cassandra/ ScyllaDB in the KVS has extra CLI to manage the server part.

https://docs.datastax.com/en/cassandra-oss/3.x/cassandra/tools/toolsNodetool.html

[generall]

Is "safe" to copy /qdrant_storage as backup... ?
Suppose there is the hot writing issue...

There is a Write-Ahead-Log, which is flushed once the update is acknowledged. If some changes are not flushed in actual storage - they will be restored from WAL during the recovery.

What about the data in RAM, how to force the flush on disk (?)

It is flushed with flush_interval_sec intervals, but they are not critical for recovery

It worth mentioning, that Qdrant is not intended to be a source-of-truth in your system. It is essential to be able to re-create qdrant collections from some other real database.

For info, Cassandra/ ScyllaDB in the KVS has extra CLI to manage the server part.

We don't currently have any special tools for deployment management, but we are looking into it.

[arita37]

Sure, Makes sense to focus on a re-insert process. thx

…

On Jan 4, 2022, at 22:27, Andrey Vasnetsov ***@***.***> wrote:  Is "safe" to copy /qdrant_storage as backup... ? Suppose there is the hot writing issue... There is a Write-Ahead-Log, which is flushed once the update is acknowledged. If some changes are not flushed in actual storage - they will be restored from WAL during the recovery. What about the data in RAM, how to force the flush on disk (?) It is flushed with flush_interval_sec intervals, but they are not critical for recovery It worth mentioning, that Qdrant is not intended to be a source-of-truth in your system. It is essential to be able to re-create qdrant collections from some other real database. For info, Cassandra/ ScyllaDB in the KVS has extra CLI to manage the server part. We don't currently have any special tools for deployment management, but we are looking into it. — Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you authored the thread.