-
Notifications
You must be signed in to change notification settings - Fork 87
BlockChain Data Synchronization
We divided the block synchronization into several segments.
Provided the configuration:
sync: zone: 100000 cpu: 4
We get:
[Zone(from=1, to=81732), Zone(from=81733, to=163464), Zone(from=163465, to=245196), Zone(from=245197, to=326928)]
and for each segment, we start a thread to synchronize paralleled.
- Thread 45 starting populate
- Thread 48 starting populate
- Thread 47 starting populate
- Thread 46 starting populate
In each segment, we keep the thread retry for fail connection or unstable rpc access. And each thread would never affect others.
Meantime, we subscribe to the block link to append the toppest block.
You would find the data as below:
+--------+ | height | +--------+ | 1 | | 2 | | 3 | | 4 | | 5 | | 6 | | 7 | | 8 | | 9 | | 10 | | 11 | | 12 | | 13 | | 14 | | 15 | | 16 | | 17 | | 18 | | 19 | | 20 | | 21 | | 22 | | 23 | | 24 | | 25 | | 26 | | 27 | | 28 | | 29 | | 30 | | 31 | | 32 | | 33 | | 34 | | 35 | | 36 | | 37 | | 38 | | 39 | | 40 | | 41 | | 42 | | 43 | | 44 | | 45 | | 46 | | 47 | | 48 | | 49 | | 50 | | 51 | | 52 | | 53 | | 54 | | 55 | | 56 | | 57 | | 58 | | 59 | | 60 | | 61 | | 62 | | 63 | | 64 | | 65 | | 66 | | 67 | | 68 | | 69 | | 70 | | 71 | | 72 | | 73 | | 74 | | 75 | | 76 | | 77 | | 78 | | 79 | | 80 | | 81 | | 82 | | 83 | | 84 | | 85 | | 86 | | 87 | | 88 | | 89 | | 90 | | 91 | | 92 | | 93 | | 94 | | 95 | | 96 | | 97 | | 98 | | 99 | | 100 | | 101 | | 102 | | 103 | | 104 | | 105 | | 106 | | 107 | | 108 | | 81824 | | 81825 | | 81826 | | 81827 | | 81828 | | 81829 | | 81830 | | 81831 | | 81832 | | 81833 | | 81834 | | 81835 | | 81836 | | 81837 | | 81838 | | 81839 | | 81840 | | 81841 | | 81842 | | 81843 | | 81844 | | 81845 | | 81846 | | 81847 | | 81848 | | 81849 | | 81850 | | 81851 | | 81852 | | 81853 | | 81854 | | 81855 | | 81856 | | 81857 | | 81858 | | 81859 | | 81860 | | 81861 | | 81862 | | 81863 | | 81864 | | 81865 | | 81866 | | 81867 | | 81868 | | 81869 | | 81870 | | 81871 | | 81872 | | 81873 | | 81874 | | 81875 | | 81876 | | 81877 | | 81878 | | 81879 | | 81880 | | 81881 | | 81882 | | 81883 | | 81884 | | 81885 | | 81886 | | 81887 | | 81888 | | 81889 | | 81890 | | 81891 | | 81892 | | 81893 | | 81894 | | 81895 | | 81896 | | 81897 | | 81898 | | 81899 | | 81900 | | 81901 | | 81902 | | 81903 | | 81904 | | 81905 | | 81906 | | 81907 | | 81908 | | 81909 | | 81910 | | 81911 | | 81912 | | 81913 | | 81914 | | 81915 | | 81916 | | 81917 | | 81918 | | 81919 | | 81920 | | 81921 | | 81922 | | 81923 | | 81924 | | 81925 | | 81926 | | 81927 | | 81928 | | 163647 | | 163648 | | 163649 | | 163650 | | 163651 | | 163652 | | 163653 | | 163654 | | 163655 | | 163656 | | 163657 | | 163658 | | 163659 | | 163660 | | 163661 | | 163662 | | 163663 | | 163664 | | 163665 | | 163666 | | 163667 | | 163668 | | 163669 | | 163670 | | 163671 | | 163672 | | 163673 | | 163674 | | 163675 | | 163676 | | 163677 | | 163678 | | 163679 | | 163680 | | 163681 | | 163682 | | 163683 | | 163684 | | 163685 | | 163686 | | 163687 | | 163688 | | 163689 | | 163690 | | 163691 | | 163692 | | 163693 | | 163694 | | 163695 | | 163696 | | 163697 | | 163698 | | 163699 | | 163700 | | 163701 | | 163702 | | 163703 | | 163704 | | 163705 | | 163706 | | 163707 | | 163708 | | 163709 | | 163710 | | 163711 | | 163712 | | 163713 | | 163714 | | 163715 | | 163716 | | 163717 | | 163718 | | 163719 | | 163720 | | 163721 | | 163722 | | 163723 | | 163724 | | 163725 | | 163726 | | 163727 | | 163728 | | 163729 | | 163730 | | 163731 | | 163732 | | 163733 | | 163734 | | 163735 | | 163736 | | 163737 | | 163738 | | 163739 | | 163740 | | 163741 | | 163742 | | 163743 | | 163744 | | 163745 | | 163746 | | 163747 | | 163748 | | 163749 | | 163750 | | 163751 | | 163752 | | 163753 | | 245470 | | 245471 | | 245472 | | 245473 | | 245474 | | 245475 | | 245476 | | 245477 | | 245478 | | 245479 | | 245480 | | 245481 | | 245482 | | 245483 | | 245484 | | 245485 | | 245486 | | 245487 | | 245488 | | 245489 | | 245490 | | 245491 | | 245492 | | 245493 | | 245494 | | 245495 | | 245496 | | 245497 | | 245498 | | 245499 | | 245500 | | 245501 | | 245502 | | 245503 | | 245504 | | 245505 | | 245506 | | 245507 | | 245508 | | 245509 | | 245510 | | 245511 | | 245512 | | 245513 | | 245514 | | 245515 | | 245516 | | 245517 | | 245518 | | 245519 | | 245520 | | 245521 | | 245522 | | 245523 | | 245524 | | 245525 | | 245526 | | 245527 | | 245528 | | 245529 | | 245530 | | 245531 | | 245532 | | 245533 | | 245534 | | 245535 | | 245536 | | 245537 | | 245538 | | 245539 | | 245540 | | 245541 | | 245542 | | 245543 | | 245544 | | 245545 | | 245546 | | 245547 | | 245548 | | 245549 | | 245550 | | 245551 | | 245552 | | 245553 | | 245554 | | 245555 | | 245556 | | 245557 | | 245558 | | 245559 | | 245560 | | 245561 | | 245562 | | 245563 | | 245564 | | 245565 | | 245566 | | 245567 | | 245568 | | 245569 | | 245570 | | 245571 | | 245572 | | 245573 | | 245574 | | 245575 | | 245576 | | 327295 | +--------+