Skip to content

BlockChain Data Synchronization

Bill edited this page Jan 27, 2018 · 3 revisions

We divided the block synchronization into several segments.

Provided the configuration:

sync: zone: 100000 cpu: 4

We get:

[Zone(from=1, to=81732), Zone(from=81733, to=163464), Zone(from=163465, to=245196), Zone(from=245197, to=326928)]

and for each segment, we start a thread to synchronize paralleled.

  • Thread 45 starting populate
  • Thread 48 starting populate
  • Thread 47 starting populate
  • Thread 46 starting populate

In each segment, we keep the thread retry for fail connection or unstable rpc access. And each thread would never affect others.

Meantime, we subscribe to the block link to append the toppest block.

You would find the data as below:

+--------+ | height | +--------+ | 1 | | 2 | | 3 | | 4 | | 5 | | 6 | | 7 | | 8 | | 9 | | 10 | | 11 | | 12 | | 13 | | 14 | | 15 | | 16 | | 17 | | 18 | | 19 | | 20 | | 21 | | 22 | | 23 | | 24 | | 25 | | 26 | | 27 | | 28 | | 29 | | 30 | | 31 | | 32 | | 33 | | 34 | | 35 | | 36 | | 37 | | 38 | | 39 | | 40 | | 41 | | 42 | | 43 | | 44 | | 45 | | 46 | | 47 | | 48 | | 49 | | 50 | | 51 | | 52 | | 53 | | 54 | | 55 | | 56 | | 57 | | 58 | | 59 | | 60 | | 61 | | 62 | | 63 | | 64 | | 65 | | 66 | | 67 | | 68 | | 69 | | 70 | | 71 | | 72 | | 73 | | 74 | | 75 | | 76 | | 77 | | 78 | | 79 | | 80 | | 81 | | 82 | | 83 | | 84 | | 85 | | 86 | | 87 | | 88 | | 89 | | 90 | | 91 | | 92 | | 93 | | 94 | | 95 | | 96 | | 97 | | 98 | | 99 | | 100 | | 101 | | 102 | | 103 | | 104 | | 105 | | 106 | | 107 | | 108 | | 81824 | | 81825 | | 81826 | | 81827 | | 81828 | | 81829 | | 81830 | | 81831 | | 81832 | | 81833 | | 81834 | | 81835 | | 81836 | | 81837 | | 81838 | | 81839 | | 81840 | | 81841 | | 81842 | | 81843 | | 81844 | | 81845 | | 81846 | | 81847 | | 81848 | | 81849 | | 81850 | | 81851 | | 81852 | | 81853 | | 81854 | | 81855 | | 81856 | | 81857 | | 81858 | | 81859 | | 81860 | | 81861 | | 81862 | | 81863 | | 81864 | | 81865 | | 81866 | | 81867 | | 81868 | | 81869 | | 81870 | | 81871 | | 81872 | | 81873 | | 81874 | | 81875 | | 81876 | | 81877 | | 81878 | | 81879 | | 81880 | | 81881 | | 81882 | | 81883 | | 81884 | | 81885 | | 81886 | | 81887 | | 81888 | | 81889 | | 81890 | | 81891 | | 81892 | | 81893 | | 81894 | | 81895 | | 81896 | | 81897 | | 81898 | | 81899 | | 81900 | | 81901 | | 81902 | | 81903 | | 81904 | | 81905 | | 81906 | | 81907 | | 81908 | | 81909 | | 81910 | | 81911 | | 81912 | | 81913 | | 81914 | | 81915 | | 81916 | | 81917 | | 81918 | | 81919 | | 81920 | | 81921 | | 81922 | | 81923 | | 81924 | | 81925 | | 81926 | | 81927 | | 81928 | | 163647 | | 163648 | | 163649 | | 163650 | | 163651 | | 163652 | | 163653 | | 163654 | | 163655 | | 163656 | | 163657 | | 163658 | | 163659 | | 163660 | | 163661 | | 163662 | | 163663 | | 163664 | | 163665 | | 163666 | | 163667 | | 163668 | | 163669 | | 163670 | | 163671 | | 163672 | | 163673 | | 163674 | | 163675 | | 163676 | | 163677 | | 163678 | | 163679 | | 163680 | | 163681 | | 163682 | | 163683 | | 163684 | | 163685 | | 163686 | | 163687 | | 163688 | | 163689 | | 163690 | | 163691 | | 163692 | | 163693 | | 163694 | | 163695 | | 163696 | | 163697 | | 163698 | | 163699 | | 163700 | | 163701 | | 163702 | | 163703 | | 163704 | | 163705 | | 163706 | | 163707 | | 163708 | | 163709 | | 163710 | | 163711 | | 163712 | | 163713 | | 163714 | | 163715 | | 163716 | | 163717 | | 163718 | | 163719 | | 163720 | | 163721 | | 163722 | | 163723 | | 163724 | | 163725 | | 163726 | | 163727 | | 163728 | | 163729 | | 163730 | | 163731 | | 163732 | | 163733 | | 163734 | | 163735 | | 163736 | | 163737 | | 163738 | | 163739 | | 163740 | | 163741 | | 163742 | | 163743 | | 163744 | | 163745 | | 163746 | | 163747 | | 163748 | | 163749 | | 163750 | | 163751 | | 163752 | | 163753 | | 245470 | | 245471 | | 245472 | | 245473 | | 245474 | | 245475 | | 245476 | | 245477 | | 245478 | | 245479 | | 245480 | | 245481 | | 245482 | | 245483 | | 245484 | | 245485 | | 245486 | | 245487 | | 245488 | | 245489 | | 245490 | | 245491 | | 245492 | | 245493 | | 245494 | | 245495 | | 245496 | | 245497 | | 245498 | | 245499 | | 245500 | | 245501 | | 245502 | | 245503 | | 245504 | | 245505 | | 245506 | | 245507 | | 245508 | | 245509 | | 245510 | | 245511 | | 245512 | | 245513 | | 245514 | | 245515 | | 245516 | | 245517 | | 245518 | | 245519 | | 245520 | | 245521 | | 245522 | | 245523 | | 245524 | | 245525 | | 245526 | | 245527 | | 245528 | | 245529 | | 245530 | | 245531 | | 245532 | | 245533 | | 245534 | | 245535 | | 245536 | | 245537 | | 245538 | | 245539 | | 245540 | | 245541 | | 245542 | | 245543 | | 245544 | | 245545 | | 245546 | | 245547 | | 245548 | | 245549 | | 245550 | | 245551 | | 245552 | | 245553 | | 245554 | | 245555 | | 245556 | | 245557 | | 245558 | | 245559 | | 245560 | | 245561 | | 245562 | | 245563 | | 245564 | | 245565 | | 245566 | | 245567 | | 245568 | | 245569 | | 245570 | | 245571 | | 245572 | | 245573 | | 245574 | | 245575 | | 245576 | | 327295 | +--------+

Clone this wiki locally