Sync branches

* check mode on invalid action_send_msg

* Fix random seed generation

* Explicitly skip invalid actions

* Count skipped valid messages, rename cfg option to message_skip_enabled

* Allow unfreeze via external messages

* Detect and handle bounce_on_fail mode for invalid messages

* Fix codestyle

* Adjust doc

* add github action for macOS 14 (arm64, M1)

* add github action (portable) for macOS 14 (arm64, M1)

* rename macOS arm64 output artifact

* Update libsodium on windows

* Compile libsodium

* Update build-windows.bat

* use upgraded libsodium 1.0.20; use compiled static libsodium for Windows instead of precompiled;

* revert libsodium 1.0.20; use compiled static libsodium for Windows instead of precompiled;

* use upgraded libsodium 1.0.20; use compiled static libsodium for Windows instead of precompiled;

* fix libsodium version 1.0.19; use compiled static libsodium for Windows instead of precompiled;

* try 1.0.20 libsodium precompiled on github

* try 1.0.18 libsodium precompiled on github

* try windows build on win server 2019

* and use PlatformToolset=v142

* use cmake -G "Visual Studio 16 2019"

* fix path to msvc 2019 on github

* separate github windows build on win server 2019 and build on win server 2022

* Update assembly/native/build-windows-2019.bat

add retry mechanism

Co-authored-by: Dr. Awesome Doge <[email protected]>

---------

Co-authored-by: neodiX <[email protected]>
Co-authored-by: Dr. Awesome Doge <[email protected]>

* nullify incorrect actions directly in skipped_action_list

* Correct documentation link

Update moved URL in comments

* Update all doc links

Signed-off-by: snoppy <[email protected]>

* Added nullptr check for emulation_result

* Typo

* added cast to EmulationExternalNotAccepted and diagnostic message

* formatting

* formatting

* removed unnecessary nesting

---------

Co-authored-by: dbaranov34 <[email protected]>

* Deferred messages and msg metadata

* Store out msg queue size in state

* Add checks for queue processing

1. Collator must process at least one message from AccountDispatchQueue (unless block is full)
2. The first message from a transaction is not counted, it cannot be deferred (unless AccountDispatchQueue is not empty)

* Return msg metadata from LS in listBlockTransactions[Ext]

* Enable new features by capabilities

* Changes in deferred messages

* Process deferred messages via new_msgs in collator
* Rework setting deferred_lt, bring back check_message_processing_order, check order of deferred_lt in validator

* Use have_unprocessed_account_dispatch_queue_ in collator

* Fix setting transaction lt for deferred messages

* Fix lite-client compilation error

* Changes in process_dispatch_queue, rename deferred_lt -> emitted_lt

* Fix compilation error

* Use uint64 for msg queue size

* Add liteServer.getBlockOutMsgQueueSize

* Fix compilation error

* Fix typos in comments

---------

Co-authored-by: SpyCheese <[email protected]>

…eueSizes (#1027)

* add method smc.getRawFullAccountState

* Add method blocks.getOutMsgQueueSizes

* Set collator options from validator console

* Fix compilation error in manager-disk

* Defer all messages if out msg queue is too big

* Fix checking queue size in collator

---------

Co-authored-by: SpyCheese <[email protected]>

* add github action for macOS 14 (arm64, M1)

* add github action (portable) for macOS 14 (arm64, M1)

* rename macOS arm64 output artifact

* Update libsodium on windows

* Compile libsodium

* Update build-windows.bat

* use upgraded libsodium 1.0.20; use compiled static libsodium for Windows instead of precompiled;

* revert libsodium 1.0.20; use compiled static libsodium for Windows instead of precompiled;

* use upgraded libsodium 1.0.20; use compiled static libsodium for Windows instead of precompiled;

* fix libsodium version 1.0.19; use compiled static libsodium for Windows instead of precompiled;

* try 1.0.20 libsodium precompiled on github

* try 1.0.18 libsodium precompiled on github

* try windows build on win server 2019

* and use PlatformToolset=v142

* use cmake -G "Visual Studio 16 2019"

* fix path to msvc 2019 on github

* separate github windows build on win server 2019 and build on win server 2022

* Update assembly/native/build-windows-2019.bat

add retry mechanism

Co-authored-by: Dr. Awesome Doge <[email protected]>

* rework docker image; provide installation, configuration and troubleshooting guidelines; add nc, ifconfig, netstat and iptraf-ng utilities for troubleshooting;

* put back control.template

* add tcpdump and curl to the docker image;
update default validator ports;
add kubernetes deployment guidelines with network=host;
test metalLB load balancer

* tested metalLB load balancer

* tested aws deployment

* tested gcp deployment

* todo ali cloud and storage mount points, currently only the networking was tested

* add storage/pv/pvc; repair broken links, adjust docu

* change to dynamic storage provisioning without node affinity (statefulSet+headless service)
WIP

* modify gcp deployment WIP

* modify aws deployment WIP

* add resource requests/limits

* some docu changes

* some docu changes; aws tested

* support $DUMP_URL parameter as well as $ZFS_POOL_NAME;
add pv and plzip to docker image for dump extraction;
use mainnet dump by default in k8s deployments;

* support $DUMP_URL parameter as well as $ZFS_POOL_NAME;
add pv and plzip to docker image for dump extraction;
use mainnet dump by default in k8s deployments;
add AliCloud support

* minor remarks, final tests

* remove ZFS_POOL_NAME parameter

* improve docker github action - run test and add release tag, compile against arm64

* set docker test timeout

* test if validator-engine inside the docker image is valid

* test if validator-engine inside the docker image is valid

* test if validator-engine inside the docker image is valid

* adjust recommended node values for ali cloud deployment

---------

Co-authored-by: neodiX <[email protected]>
Co-authored-by: Dr. Awesome Doge <[email protected]>

* add github action for macOS 14 (arm64, M1)

* add github action (portable) for macOS 14 (arm64, M1)

* rename macOS arm64 output artifact

* Update libsodium on windows

* Compile libsodium

* Update build-windows.bat

* use upgraded libsodium 1.0.20; use compiled static libsodium for Windows instead of precompiled;

* revert libsodium 1.0.20; use compiled static libsodium for Windows instead of precompiled;

* use upgraded libsodium 1.0.20; use compiled static libsodium for Windows instead of precompiled;

* fix libsodium version 1.0.19; use compiled static libsodium for Windows instead of precompiled;

* try 1.0.20 libsodium precompiled on github

* try 1.0.18 libsodium precompiled on github

* try windows build on win server 2019

* and use PlatformToolset=v142

* use cmake -G "Visual Studio 16 2019"

* fix path to msvc 2019 on github

* separate github windows build on win server 2019 and build on win server 2022

* Update assembly/native/build-windows-2019.bat

add retry mechanism

Co-authored-by: Dr. Awesome Doge <[email protected]>

* add test-emulator; disable test groovy pipeline

* trigger all gh actions

* fix win build

* call test-emulator

* fix test-emulator

---------

Co-authored-by: neodiX <[email protected]>
Co-authored-by: Dr. Awesome Doge <[email protected]>
Co-authored-by: ms <[email protected]>

Co-authored-by: SpyCheese <[email protected]>

* "getcollatoroptionsjson" command in validator console

* Improve state serializer

Use previous persistent state to speed up reading

Co-authored-by: hey-researcher <[email protected]>

…1070)

* More verbose VM logs

* Fix parsing slice literals in fift

* add github action for macOS 14 (arm64, M1)

* add github action (portable) for macOS 14 (arm64, M1)

* rename macOS arm64 output artifact

* Update libsodium on windows

* Compile libsodium

* Update build-windows.bat

* use upgraded libsodium 1.0.20; use compiled static libsodium for Windows instead of precompiled;

* revert libsodium 1.0.20; use compiled static libsodium for Windows instead of precompiled;

* use upgraded libsodium 1.0.20; use compiled static libsodium for Windows instead of precompiled;

* fix libsodium version 1.0.19; use compiled static libsodium for Windows instead of precompiled;

* try 1.0.20 libsodium precompiled on github

* try 1.0.18 libsodium precompiled on github

* try windows build on win server 2019

* and use PlatformToolset=v142

* use cmake -G "Visual Studio 16 2019"

* fix path to msvc 2019 on github

* separate github windows build on win server 2019 and build on win server 2022

* Update assembly/native/build-windows-2019.bat

add retry mechanism

Co-authored-by: Dr. Awesome Doge <[email protected]>

* add test-emulator; disable test groovy pipeline

* trigger all gh actions

* fix win build

* call test-emulator

* remove usage of rocksdb in fift-lib

* put back some code for test-db

* fix test-emulator

* remove usage of db-path parameter in fift

* some func adjustments

* fix checkout of openssl in fift-func-wasm-build-ubuntu.sh

* typo

* improve wasm build script for quicker turn around

* remove sENVIRONMENT=web,worker for funcfiftlib. will be added later.

* remove sENVIRONMENT=web,worker for funcfiftlib. will be added later.

* remove sENVIRONMENT=web,worker for funcfiftlib. will be added later.

* minor adjustments

* remove -d option in fift; optimize fift-lib usage

* reduce tondb usage

---------

Co-authored-by: neodiX <[email protected]>
Co-authored-by: Dr. Awesome Doge <[email protected]>
Co-authored-by: ms <[email protected]>

Co-authored-by: SpyCheese <[email protected]>

…hannel with Nop (#1096)

* Add --fast-state-serializer and tools for jemalloc

* Disable fast state serializer by default unless RAM is >= 90GB
* Print jemalloc stats once a minute
* Dump jemalloc profile on request

* Respond to AdnlMessageCreateChannel with Nop

---------

Co-authored-by: SpyCheese <[email protected]>

* add github action for macOS 14 (arm64, M1)

* add github action (portable) for macOS 14 (arm64, M1)

* rename macOS arm64 output artifact

* Update libsodium on windows

* Compile libsodium

* Update build-windows.bat

* use upgraded libsodium 1.0.20; use compiled static libsodium for Windows instead of precompiled;

* revert libsodium 1.0.20; use compiled static libsodium for Windows instead of precompiled;

* use upgraded libsodium 1.0.20; use compiled static libsodium for Windows instead of precompiled;

* fix libsodium version 1.0.19; use compiled static libsodium for Windows instead of precompiled;

* try 1.0.20 libsodium precompiled on github

* try 1.0.18 libsodium precompiled on github

* try windows build on win server 2019

* and use PlatformToolset=v142

* use cmake -G "Visual Studio 16 2019"

* fix path to msvc 2019 on github

* separate github windows build on win server 2019 and build on win server 2022

* Update assembly/native/build-windows-2019.bat

add retry mechanism

Co-authored-by: Dr. Awesome Doge <[email protected]>

* add test-emulator; disable test groovy pipeline

* trigger all gh actions

* fix win build

* call test-emulator

* fix test-emulator

* test ubuntu 24.04 native build

* ubuntu 24.04 has clang-16 by default

---------

Co-authored-by: neodiX <[email protected]>
Co-authored-by: Dr. Awesome Doge <[email protected]>
Co-authored-by: ms <[email protected]>