You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In order to solve different issue and old blocked behavior, implementing a server-to-server communication can solve many issues. First one would be the possibility to sync db without external tools.
The text was updated successfully, but these errors were encountered:
Workflow to sync a database in an efficient way (commands will needs to be executed as admin):
First lock a namespace (in read-only) mode, with a special command (LOCKSYNC) to prepare to sync
1.1. This command will reply current data id and current data offset
1.2. Because the namespace is in read-only, no changes will occurs during the lock
Note: we need to keep the namespace locked the shortest time possible
Let's transfert the whole index, file by file
3.1. Because we know data id from first command, we know how many index files to request
3.2. Request index file, file-by-file INDEXFILE [id] [offset]
3.3. Return value will be an array with: the payload, optional offset for chunk
3.4. Because payload are limited to 8 MB, it's possible that payload needs to be chunked, let's repeat until we have the full file
Now that we have the index fully sync locally, we can unlock the namespace (UNLOCKSYNC)
4.1. Because data are immuable and we have the offset of the current last datafile, we can use local index to query the current database at this time snapshot, now we have two options:
4.2. Or we use external trigger to request datafiles on runtime when needed
4.3. Or we start downloading all datafiles "slowly" with a new command DATAFILE [id] [offset] which works the same way as the index
4.4. Because the namespace is unlocked, we don't block anyone else and can take time to fetch the whole data
That way, we can sync a namespace without any external tools because we will talk zdb-to-zdb directly.
In order to solve different issue and old blocked behavior, implementing a server-to-server communication can solve many issues. First one would be the possibility to sync db without external tools.
The text was updated successfully, but these errors were encountered: