Code

Feat/hyperblock#18

Code

Open

AdoAdoAdo wants to merge 248 commits intomainmultiversx/mx-chain-ws-connector-firehose-go:mainfrom feat/hyperblockmultiversx/mx-chain-ws-connector-firehose-go:feat/hyperblockCopy head branch name to clipboard

+19,267-554

Commits

Commits on Mar 14, 2024

• added base components structure

  ssd04

  committed
  61a89af

• switch default ws connector mode to server

  ssd04

  committed
  5188b70

• added testdata and ws connector tool

  ssd04

  committed
  3e09445

• added hyper outport block proto struct

  ssd04

  committed
  8811cf2

• update data aggregator

  ssd04

  committed
  8990a7c

• refactor hyper outport block propagation

  ssd04

  committed
  eca2bba

• fix notarized refence + updated test data

  ssd04

  committed
  5859a2a

• fix storage cacher reference

  ssd04

  committed
  08fd7d4

Commits on Mar 15, 2024

• added blocks caches

  ssd04

  committed
  590d499

• blocks pool with rounds map

  ssd04

  committed
  0878889

Commits on Mar 18, 2024

• added persistent storage for events

  ssd04

  committed
  bdca33f

• added pruning for persister

  ssd04

  committed
  7ee4a1d

Commits on Mar 19, 2024

• update ws connector tool

  ssd04

  committed
  cd715f5

• added missing comments

  ssd04

  committed
  58540fc

• added unit tests

  ssd04

  committed
  9a136dd

• remove ws connector tool

  ssd04

  committed
  c485bd7

• remove test data

  ssd04

  committed
  d48bb17

• fix linter issue

  ssd04

  committed
  0b011cf

• fix merge conflicts

  ssd04

  committed
  e7362e7

• fix merge conflicts

  ssd04

  committed
  934e2a3

• Merge branch 'hyperblock-processor' into implement-blocks-pool

  ssd04

  committed
  fb80947

• rm test data and test tools

  ssd04

  committed
  a4f5236

Commits on Mar 20, 2024

• updated config + refactoring factory

  ssd04

  committed
  4f38d98

• use camel case format for websocket config

  ssd04

  committed
  b6915da

• renamings + unit test fix

  ssd04

  committed
  0fd8ca9

• added pruning storer

  ssd04

  committed
  15aa4b9

Commits on Mar 21, 2024

• pruning storer unit tests

  ssd04

  committed
  615f939

• added cacher mock and stub

  ssd04

  committed
  abff3c0

• cleanup old persisters

  ssd04

  committed
  0afb4cc

• use pruning storer in data pool

  ssd04

  committed
  5c9b702

• add test make command

  ssd04

  committed
  bb4c05f

Commits on Mar 22, 2024

• added unit test blocks pool

  ssd04

  committed
  205614c

• added import db mode

  ssd04

  committed
  4967f79

• fix linter issue

  ssd04

  committed
  fefbcab

• create dir of not existing

  ssd04

  committed
  d526525

Commits on Mar 25, 2024

• Added Dockerfile.

  cristure

  committed
  bb0a6ba

Commits on Mar 26, 2024

• dump cache data to persister

  ssd04

  committed
  cf7b8b6

• pruning storer close

  ssd04

  committed
  a9f1a5c

• refactor to use connector runner component

  ssd04

  committed
  c289b13

• fix tests - update storer stub

  ssd04

  committed
  a722602

• added config flag and Dockerfile.

  cristure

  committed
  a9d4926

Commits on Mar 27, 2024

• updated default value for config file.

  cristure

  committed
  e34b630

• fixes after review - update storage version + small checks

  ssd04

  committed
  5bdb010

Commits on Mar 28, 2024

• cosmetic changes.

  cristure

  committed
  c2c1da4

Commits on Mar 29, 2024

• Merge pull request #4 from multiversx/hyperblock-processor

  Show description for d480d5a
  ssd04

  authored
  d480d5a

Commits on Apr 1, 2024

• Merge branch 'feat/hyperblock' into implement-blocks-pool

  ssd04

  committed
  500d14c

Commits on Apr 3, 2024

• set checkpoint

  ssd04

  committed
  f5707e5

Commits on Apr 11, 2024

• added converter from gogo protobuf to google protobuf.

  cristure

  committed
  1754bcf

• fix after review.

  cristure

  committed
  18f44a9

• added proto marshalizer and tests.

  cristure

  committed
  2e28e61

• integrated protoMarshalizer in converter.

  cristure

  committed
  1b67dfc

• fix linter and cosmetic changes.

  cristure

  committed
  5dc249f

• add t.Parallel()

  cristure

  committed
  06c9374

• fix linter.

  cristure

  committed
  820a0f7

Commits on Apr 12, 2024

• update test stubs

  ssd04

  committed
  0f96e5f

• added separate outport and hyper outport blocks pool

  ssd04

  committed
  e2e8481

• refactor base blocks pool to use marshalled data

  ssd04

  committed
  9472b98

• added disabled blocks pool components

  ssd04

  committed
  f665e4d

• added grpc publisher

  ssd04

  committed
  dbccd44

• added publisher factory

  ssd04

  committed
  79a6731

Commits on Apr 15, 2024

• remove disabled components

  ssd04

  committed
  ba9aae3

• fix blocks pool get test

  ssd04

  committed
  af89d47

• refactor data components factory

  ssd04

  committed
  b792ea1

• added disabled hyper blocks pool

  ssd04

  committed
  de57511

Commits on Apr 16, 2024

• added failed to put block error

  ssd04

  committed
  e419464

• added persister data into prunning storer

  ssd04

  committed
  5744bf8

• added db modes

  ssd04

  committed
  af7c083

• added storer destroy + fix mutex deadlock

  ssd04

  committed
  9dff85c

• update storer stub

  ssd04

  committed
  2658d64

• pruning storer dump unit test

  ssd04

  committed
  276bffc

• data pool unit tests

  ssd04

  committed
  e16d307

• blocks pool integration tests

  ssd04

  committed
  bcfa48c

• fix integration test

  ssd04

  committed
  5578a9c

Commits on Apr 17, 2024

• added grpc blocks handler

  ssd04

  committed
  ba1ce92

• remove separated outport blocks pools

  ssd04

  committed
  72f84e1

Commits on Apr 18, 2024

• fixes after review.

  cristure

  committed
  6216b8b

• cosmetic changes.

  cristure

  committed
  c93f2a3

Commits on Apr 19, 2024

• moved checkpoint functionality to blocks pool

  ssd04

  committed
  a91eb76

• removed number of shards field

  ssd04

  committed
  d44c435

• Merge branch 'implement-blocks-pool' into blocks-pool-improvements

  ssd04

  committed
  dcd19b8

• removed not necessary tests

  ssd04

  committed
  9ff2ebf

• Merge pull request #11 from multiversx/pruning-storer-tests

  Show description for 90afa46
  ssd04

  authored
  90afa46

• Merge pull request #5 from multiversx/implement-blocks-pool

  Show description for e6e6512
  ssd04

  authored
  e6e6512

• Merge pull request #10 from multiversx/converterV1

  Show description for 6527f62
  ssd04

  authored
  6527f62

• remove init from start

  ssd04

  committed
  7e0d337

• added gRPC server and slight modifications.

  cristure

  committed
  d34b43a

• moved marshaler from gogo to standard proto.

  cristure

  committed
  7b3cac1

• bump go version in github actions.

  cristure

  committed
  a05329b

• modify linter job.

  cristure

  committed
  0aac083

• fix linter.yml

  cristure

  committed
  bfe6aa4

• replace deprecated functions.

  cristure

  committed
  25a6483

Commits on Apr 21, 2024

• added first commitable block config option

  ssd04

  committed
  5b5b126

Commits on Apr 22, 2024

• reverted config file override.

  cristure

  committed
  2dbaf6e

• fixes after review.

  cristure

  committed
  cabb4a7

• add previous block succesive check

  ssd04

  committed
  abd010d

• improved gRPC server and factory.

  cristure

  committed
  ac64822

• Merge branch 'feat/hyperblock' into dockerfile

  Show description for f43b5a9
  cristure

  committed
  f43b5a9

• cosmetic changes.

  cristure

  committed
  b7e3bed

• fixes after review.

  cristure

  committed
  b26c4b8

• Merge pull request #6 from multiversx/dockerfile

  Show description for 4ddee09
  AdoAdoAdo

  authored
  4ddee09

• fix tests.

  cristure

  committed
  0df8a40

• fixed blocksPool expression.

  cristure

  committed
  0ab621d

• Merge remote-tracking branch 'refs/remotes/origin/feat/hyperblock' into grpcServer

  Show description for 017c0d8
  cristure

  committed
  017c0d8

Commits on Apr 23, 2024

• Update service/hyperOutportBlock/blockService.go

  Show description for 46b0f95
  

  cristure

  and

  ssd04

  authored
  46b0f95

• fix imports.

  cristure

  committed
  f9d87c4

• added missing comments.

  cristure

  committed
  ab37eb9

• improve slice allocation by specifying the capacity.

  cristure

  committed
  9e78fa1

• add integration test for optimized persister mode

  ssd04

  committed
  27188c4

• more slice allocation fixes.

  cristure

  committed
  143e3c0

• avoid setting meta index twice

  ssd04

  committed
  cdb69fe

• reverted disabling data processor tests.

  cristure

  committed
  6fa0e69

• cosmetic changes.

  cristure

  committed
  3ee852c

• added tmp dir file path

  ssd04

  committed
  38ae825

• added stream endpoint.

  cristure

  committed
  99447a3

• remove client.

  cristure

  committed
  f32cfbc

• fixes after review.

  cristure

  committed
  a408d37

• fixes after review.

  cristure

  committed
  36bdd0d

• remove breaking test.

  cristure

  committed
  b7b93fc

Commits on Apr 24, 2024

• added common package

  ssd04

  committed
  a990c00

• added db modes comments

  ssd04

  committed
  e24e931

• Merge branch 'grpcServer' into grpc-streaming

  cristure

  authored
  baf50c0

• Merge pull request #15 from multiversx/grpcServer

  Show description for 99c597a
  ssd04

  authored
  99c597a

• Merge branch 'feat/hyperblock' into blocks-pool-improvements

  ssd04

  committed
  7f36b93

• fix blocks pool tests - use gogo proto marshaller

  ssd04

  committed
  f1da267

• added hyper blocks pool

  ssd04

  committed
  692f909

• change block checkpoint to use standard proto

  ssd04

  committed
  0078764

• added error check for updating meta state

  ssd04

  committed
  d38a478

• removed queue and added buffered channel.

  cristure

  committed
  eacfdf5

• fix linter.

  cristure

  committed
  a58a31d

• fix unit tests

  ssd04

  committed
  105c3d9

• fix integration tests

  ssd04

  committed
  f914bb9

Commits on Apr 25, 2024

• fix linter issue - added err check in test

  ssd04

  committed
  8932d2d

• rename go module to match repo name

  ssd04

  committed
  f3dfb83

• rename also proto struct

  ssd04

  committed
  e61cb62

• update firehose message type

  ssd04

  committed
  f619fee

• Merge remote-tracking branch 'refs/remotes/origin/feat/hyperblock' into improve-slice-allocation

  Show description for 621dd65
  cristure

  committed
  621dd65

• fix conflicts.

  cristure

  committed
  e29adbc

• improved the graceful shutdown mechanism.

  cristure

  committed
  899d714

• fix broken tests.

  cristure

  committed
  0dd37ec

• added db mode type

  ssd04

  committed
  88b0fb4

Commits on Apr 26, 2024

• update error formats

  ssd04

  committed
  11de24b

• Merge pull request #19 from multiversx/rename-module-name

  Show description for 95d497d
  AdoAdoAdo

  authored
  95d497d

• revamped the streaming mechanism.

  cristure

  committed
  2a7a1a0

• comsetic changes.

  cristure

  committed
  e26404d

• comsetic changes.

  cristure

  committed
  5e6d1fa

• fix tests and linter.

  cristure

  committed
  4752172

• Merge branch 'refs/heads/feat/hyperblock' into grpc-streaming

  cristure

  committed
  c642304

• Merge remote-tracking branch 'refs/remotes/origin/feat/hyperblock' into grpc-streaming

  Show description for 4c0fdfa
  cristure

  committed
  4c0fdfa

• fix after review

  ssd04

  committed
  0424845

• Merge branch 'feat/hyperblock' into blocks-pool-improvements

  ssd04

  committed
  15c28dd

• fix broken imports.

  cristure

  committed
  6fac50e

• Update server/grpcServer.go

  Show description for 72e5040
  

  cristure

  and

  ssd04

  authored
  72e5040

• added TODOS and cosmetic changes.

  cristure

  committed
  7ad43a4

• move succesive check to hyper blocks pool

  ssd04

  committed
  648a401

Commits on Apr 29, 2024

• added common publisher

  ssd04

  committed
  254357a

• adapt to use common publisher

  ssd04

  committed
  af2df4b

• handle revert event

  ssd04

  committed
  10bdaa2

Commits on Apr 30, 2024

• fix data processor unit tests

  ssd04

  committed
  b193fc4

• added publisher handler unit tests

  ssd04

  committed
  2066e6a

• added varibles to config

  ssd04

  committed
  2c71761

• more unit tests for first commitable block check

  ssd04

  committed
  5477209

• fix linter checks

  ssd04

  committed
  41e287d

• fix concurrency tests issue

  ssd04

  committed
  146ea69

• Merge pull request #17 from multiversx/grpc-streaming

  Show description for 9ea7570
  AdoAdoAdo

  authored
  9ea7570

• Merge branch 'feat/hyperblock' into blocks-pool-improvements

  ssd04

  committed
  39a0e00

• fix after merge

  ssd04

  committed
  0dcc0a6

• update proto to time duration

  ssd04

  committed
  79e3df8

• add protogen command

  ssd04

  committed
  3b59b8a

• added separate grpc proto file

  ssd04

  committed
  f79a750

• update gprc service names

  ssd04

  committed
  e63cfa8

• fix after review

  ssd04

  committed
  879c289

Commits on May 2, 2024

• added polling unit tests

  ssd04

  committed
  09ec534

• added grpc server wrapper

  ssd04

  committed
  6e06b53

Commits on May 7, 2024

• fix renaming

  ssd04

  committed
  afe37fd

• fix linter issue

  ssd04

  committed
  fdc1a8d

• added unit test for stream requests

  ssd04

  committed
  eb2111b

• fix linter issue in unit test

  ssd04

  committed
  94d4d8f

• Update process/blocksPool.go

  Show description for 47128b9
  

  ssd04

  and

  sstanculeanu

  authored
  47128b9

• fixes after review

  ssd04

  committed
  851e289

• moved custom outport block handling to separate files

  ssd04

  committed
  b374c06

• Merge pull request #20 from multiversx/update-firehose-message-type

  Show description for da30cd6
  ssd04

  authored
  da30cd6

• fixes after review: rename mock files

  ssd04

  committed
  2f65057

Commits on May 8, 2024

• updated to use nonce instead of round for index reference

  ssd04

  committed
  09d54b7

• rename hyper blocks pool

  ssd04

  committed
  8abaaca

• rename blocks pool files

  ssd04

  committed
  46f4e74

• fixes after review

  ssd04

  committed
  32810dc

• update changes in connector

  ssd04

  committed
  042f242

Commits on May 9, 2024

• added first commitable blocks per shard

  ssd04

  committed
  b369d17

• aupdate first commitable blocks per shard config

  ssd04

  committed
  5edf104

• fixes after review

  ssd04

  committed
  4acca74

• Merge pull request #13 from multiversx/blocks-pool-improvements

  Show description for fa73294
  AdoAdoAdo

  authored
  fa73294

Commits on May 10, 2024

• Merge branch 'feat/hyperblock' into add-meta-blocks-optimistically

  ssd04

  committed
  c668fed

Commits on May 13, 2024

• fixes after review

  ssd04

  committed
  2fcf6be

• fixes after review

  ssd04

  committed
  f0be16a

Commits on May 14, 2024

• fix after review - renaming

  ssd04

  committed
  e89a44d

• added publisher checkpoint

  ssd04

  committed
  4b6d7dd

• added nil check

  ssd04

  committed
  42ff2a5

• change publish and update state order

  ssd04

  committed
  e944cb7

• handle checkpoint from init on goroutine

  ssd04

  committed
  4b56518

• fix unit test

  ssd04

  committed
  d5e6f0b

Commits on May 15, 2024

• added concurrency test

  ssd04

  committed
  32b889a

Commits on May 17, 2024

• fix variable typo

  ssd04

  committed
  282e663

Commits on May 20, 2024

• Merge pull request #21 from multiversx/add-meta-blocks-optimistically

  Show description for 925c0e8
  ssd04

  authored
  925c0e8

• Merge branch 'feat/hyperblock' into grpc-streaming-updates

  ssd04

  committed
  5ab5c15

• rename mock file

  ssd04

  committed
  6f04819

• update grpc server mock

  ssd04

  committed
  1ef30bc

• fix concurrency test issue

  ssd04

  committed
  78dec20

Commits on May 24, 2024

• Merge pull request #22 from multiversx/grpc-streaming-updates

  Show description for 1544e6c
  AdoAdoAdo

  authored
  1544e6c

Commits on Jun 11, 2024

• save last soft checkpoint

  ssd04

  committed
  e7293a6

Commits on Jun 12, 2024

• moved first commitable blocks check to data processor

  ssd04

  committed
  8d7930c

Commits on Jun 13, 2024

• fix unit tests

  ssd04

  committed
  da37707

• added unit test for data processor first commitable blocks

  ssd04

  committed
  8f78bfe

Commits on Jul 12, 2024

• changed proto definition along with converting system.

  cristure

  committed
  cbdc4f0

Commits on Jul 15, 2024

• added v2 for shard conversion.

  cristure

  committed
  774931a

Commits on Jul 16, 2024

• added first commitable blocks check to publisher handler

  ssd04

  committed
  83b04d6

• fix header hash reference

  ssd04

  committed
  7cc6f22

Commits on Jul 18, 2024

• refactored proto definitions.

  cristure

  committed
  46d8ea8

• added reset checkpoints flag

  ssd04

  committed
  141d57e

Commits on Jul 19, 2024

• added interfaces for v2.

  cristure

  committed
  23ee1d5

Commits on Jul 25, 2024

• improved copying mechanism.

  cristure

  committed
  b523e87

• fixed nil pointer.

  cristure

  committed
  81456a2

• fix linter.

  cristure

  committed
  88c11dc

Commits on Jul 26, 2024

• remove debug part.

  cristure

  committed
  3379e9a

• update config

  ssd04

  committed
  afd89ba

• restore config

  ssd04

  committed
  c8539ed

Commits on Aug 5, 2024

• remove revent events handling

  ssd04

  committed
  865512d

Commits on Aug 6, 2024

• get last checkpoint fix in publisher handler

  ssd04

  committed
  7fbca4c

• fixes after review.

  cristure

  committed
  8169fac

• revert to v1.

  cristure

  committed
  2126233

Commits on Aug 7, 2024

• cosmetic changes.

  cristure

  committed
  a8bd56e

• fixes after review

  ssd04

  committed
  d6de3c3

• use deep copy for map

  ssd04

  committed
  40e9bfc

• cosmetic changes.

  cristure

  committed
  527f494

• Merge pull request #23 from multiversx/data-pool-fixes

  Show description for 7564475
  ssd04

  authored
  7564475

• Merge pull request #24 from multiversx/proto-v2

  Show description for 8f76917
  cristure

  authored
  8f76917

Commits on Aug 26, 2024

• improve slice allocation by specifying the capacity.

  cristure

  committed
  73fcc71

• more slice allocation fixes.

  cristure

  committed
  0601975

• fixes after review.

  cristure

  committed
  4a6335e

• fix conflicts.

  cristure

  committed
  d41e16b

• fix after rebase.

  cristure

  committed
  22f5a7c

• Merge remote-tracking branch 'origin/improve-slice-allocation' into improve-slice-allocation

  Show description for e146985
  cristure

  committed
  e146985

• Merge remote-tracking branch 'refs/remotes/origin/feat/hyperblock' into improve-slice-allocation

  cristure

  committed
  5978baf

• Merge remote-tracking branch 'origin/improve-slice-allocation' into improve-slice-allocation

  cristure

  committed
  1f29ce0

• Merge pull request #16 from multiversx/improve-slice-allocation

  Show description for ab42226
  AdoAdoAdo

  authored
  ab42226

Commits on Sep 23, 2024

• added state changes to convertor.

  cristure

  committed
  c1e7b86

Commits on Sep 25, 2024

• fixes after review.

  cristure

  committed
  d7bc540

Commits on Sep 26, 2024

• cut down content from test file to decrease size.

  cristure

  committed
  93c93a9

Commits on Oct 23, 2024

• Merge pull request #25 from multiversx/convert_state_changes

  Show description for ffe985a
  cristure

  authored
  ffe985a