(feat) Snapshot file to interrupt concurrent transmission(#503) #632
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Hi @lilei0527, welcome to SOFAStack community, Please sign Contributor License Agreement! After you signed CLA, we will automatically sync the status of this pull request in 3 minutes. |
|
@killme2008 大佬，看看我的PR |
|
@lilei0527 ok，我看下 |
|
这个是否重复开发了？ 这个任务已经被其他人认领过了, #596 |
不好意思，我看之前这个issue迟迟没有动静，所以自己也尝试了一下。 |
|
感谢贡献，因为有一个重复 PR 并且进行了 review，因此这个关闭了。 |
|
@killme2008 之前你们review的那个pr怎么没有下文了，是否可以看看我的pr |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation:
快照在网络传输过程中可能由于各种原因导致传输中断的问题，此改动增加了大文件并行传输和断点续传的功能
Modification:
设计思路：
1.首先leader在InstallSnapshot流程中的loadMetaTable的时候计算快照文件的分片数目，将快照文件的分片数据放入LocalFileMeta
传输给follower。
2.follower获取到LocalFileMeta，取其中的文件名，然后查看本机是否有已接收的快照分片段，将各个快照分片段的文件大小并行
传输给leader。
3.leader接收到follower的GetFileRequest，从指定的文件断点位置传输数据。
4.所有的快照分片传输完毕，将分片合并，合并完成后删除掉分片文件。
Result:
Fixes #503