Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Txindex #401

Draft
wants to merge 76 commits into
base: master
Choose a base branch
from
Draft

Txindex #401

wants to merge 76 commits into from

Conversation

D-Stacks
Copy link
Collaborator

@D-Stacks D-Stacks commented Jan 22, 2024

Merge Request for TxIndex

Summary

This PR introduces the TxIndex, a new index that enables querying of transaction data.

Changes:

(None exhaustive)

Consensus Codebase

  • Expands the get_virtual_chain_from_block API to allow chunked calls and walking the chain to a specific hash.
  • Adds public APIs for retrieving block transactions, history root, highest common chain block ancestor, chain block status, and compact header.
  • Optimizes SinkBlueScoreChanged, VirtualDaaScoreChanged, and VirtualChainChanged notifications to only emit when relevant data changes.
  • Introduces a ChainAcceptanceDataPruned notification emitted by the pruning processor.
  • Modifies VirtualChainChanged to include removed acceptance data.
  • Moves CachePolicyBuilder to the kaspa-database crate and makes some functions public.
  • Converts Testconsensus::with_db to use a dummy sender.
  • Refactors test_helpers to testutils and adds generators for acceptance data.
  • Moves CompactHeader to the core crate.
  • Renames AcceptedTxEntry to TxEntry to accommodate for future support for unaccepted transactions.

Notify Codebase

  • Add relevant parts for ChainAcceptanceDataPruned

Index Codebase

  • Moves all core TxIndex and UTXOIndex structures to the core crate.
  • Configures the index processor to register for ChainAcceptanceDataPruned and VirtualChainChanged notifications if the TxIndex is activated.
  • Updates the UTXOIndex to also receive a notification in its update method for consistency in the processor.
  • Integrates new notifications into the TxIndex in the processor.
  • Adds the TxIndex itself, including relevant stores, reindexer, API, minimal configuration with memory-aware caches and resyncing.

Kaspad Codebase

  • Enables indexes to initialize and resync in parallel using threads.
  • Adds relevant database directories and setup for the TxIndex.
  • Adds flags and arguments for configuring the TxIndex.

RPC Codebase

  • Integrates the TxIndex into the RPC framework.
  • Add new TransactionData structure for querying transactions.
  • Adds code for a getTransactionData RPC call in both gRPC and wRPC and into ./kaspa-cli

Integration Tests Codebase

  • Adds TxIndex-related tests to the RPC calls tests.
  • Adds TxIndex-related tests to the JSON tests.

Known Issues

1. There is a race condition or similar issue when pruning is interrupted, which causes the TxIndex history root to lose sync with consensus. - Solved
2. There is a lack of testing for pruning and resyncing due to limitations in the testing framework. A separate PR may be needed to expand Simpa to support testing with indexes and enable controlled testing of reorg scenarios.

@D-Stacks D-Stacks marked this pull request as ready for review January 23, 2024 01:47
D-Stacks and others added 20 commits January 23, 2024 12:35
* bump to 0.13.4

* bump to 0.13.4

* fix incorrect version due to a toolchain issue
* replace NetworkType with NetworkId in the tx generator and related APIs

* fix url path rendering in format_url

* Improve mass related errors

* refactor fee handling

* renaming fee variants

* mass combination strategy

* wip - storage mass

* additional unit tests and limit checks

* Introduce per-network configurable additional compound transaction compute mass (currently set to 100 on tn10 and tn11)

* expand sanity checks to handle 0 value outputs

* documentation updates

* code formatting

* Set new input sequence to 0

* change aggregate input value check to use saturating_sub() to prevent potential overflows (re github feedback)

* change storage mass fee calc to return u64::MAX if the resulting mass is None (re GitHub feedback)
* increase semaphore yield duration to 30 micros + tighten the test (so that w/o this fix it would fail certianly)

* allow the test 2 read capture attempts + add macos experiment results
wanshuangcheng and others added 12 commits July 10, 2024 19:12
* init the pruning point depth store, for the rare case where it is the first sink during initial ibd

* perform pruning proof revalidation vs the current consensus and not the staging one

* minor
* Write panic errors to logs (kaspanet#445)

* Remove the comment as it doesn't need to be addressed

Reason: Waiting here will block the thread, and as the thread is locked
and we wait for the logger system to be initiated that would wait for
the thread to be released itself, making the waiting pointless

* Add column to the log message for the panic error

* Move panic setup after the logger init in all places

* Add the thread name to the panic error

* default hook invoke position + minor style changes
…et#459)

* using IConnectOptions struct instead of Jsvalue for socket connection

* Cargo.lock

* WIP - initial Wallet API typescript bindings

* WIP - Wallet API typescript bindings

* typescript setup

* TS related issue

* relocate TypeScript StringArray to kaspa_consensus_wasm

* Update deploy GitHub Action to use folders inside WASM archives

* WIP: TSDoc setup

* WIP: TSdoc

* build-docs updated for typedoc uses

* XPublicKey toString/fromString support for JS

* WASM + Wallet API refactoring for TypeScript

* rearrange nodejs JS/TS examples

* moved ts files to src dir

* rpc frame size

* utxo context listener tests

* WIP - public node wRPC beacons

* fix TS merge

* refactor/restructure past typedoc merge

* Add a warning on build-node-dev

* cleanup

* utxo events issue

* rename README

* rename README

* WIP - typescript bindings and related refactoring

* code formatting

* wasm32-sdk feature gate

* wasm32-sdk feature gating TS/JS-side APIs and types; typedoc handling in macros; relocate wasm RpcClient API into kaspa-wrpc-wasm;

* documentation parsing

* refactor typescript types & JS/TS examples

* simplify HexString (typescript type)

* doc processing in macros

* wRPC Beacon

* #!/bin/bash headers for bash scripts

* wRPC Beacon Cargo.toml update

* WASM wRPC Beacon

* INetworkId type

* toString for XPrv

* renaming SerdeJson to Json

* fix wasm32-sdk propagation

* fix wasm32-sdk propagation

* WIP JS examples

* code formatting

* fix beacon config loading

* updating examples init script

* SerdeJson renaming

* WIP TS: rename XPxxKey to PxxKeyGenerator

* module rename

* finish merging with master

* build*-dev warning text

* WIP: Wallet API updates

* wasm32-sdk infra

* fix IndexMap "deprecated" warning in orphan processing

* kaspa-beacon CORS handling

* Prefix : ktrv, ktub

* key generator (xprv,xpub, signer, keypair) moved to wallet/core/wasm

* XPrv creation from string (xprvxxxx), examples

* change wallet xpub example

* restructure docs location

* hex serialization for RPC components

* WASM32 SDK examples (WIP)

* code formatting

* set /test and /check bash scripts to use -e for immediate exit on failure

* rename from_str to from_xprv_str (clippy)

* wasm32-sdk changelog

* CI updates for wasm32-sdk

* WASM32 SDK CHANGELOG updates

* WIP RPC & Wallet event types

* code formatting

* docs

* Segregating wallet core from keys crates (WASM32 WIP)

* WIP WASM32 release

* WIP WASM32 release

* WIP WASM32 structure

* WASM32 rename packages

* docs

* fix build scripts

* script updates

* fix gitignore

* update kaspa-beacon; fix misc issues, improve election algorithm;

* display error in case of metrics failure

* rename client-side Beacon to Resolver; impl related WASM32 bindings and handling for RpcClient and Resolver

* WASM32 wRPC addEventListener() handling & misc bug fixes

* mix fixes, docs, update WASM32 examples to use updated APIs

* update WASM32 examples

* replace EventDispatcher with localized event handlers

* Refactor wasm event listeners (RPC, UtxoProcessor, Wallet) + update LICENSE

* Support contexts in event listeners

* additional TypeScript interfaces and docs

* WASM32 changelog updates

* WASM32 release scripts

* Kaspa Beacon - tentative access lists

* Kaspa Beacon ACL

* Kaspa Beacon ACLs

* Fix parsing issues with typedoc

* Fix TypeDoc parsing issues

* WASM32 updates to UtxoProcessor (ctors, UTXO access)

* misc UtxoContext updates

* fix typo in cli wizard

* WASM32 - refactor key APIs; fix GetBalancesByAddresses conversion; rename IXxx types to XxxT (convention).

* WASM32 / client - additional key management APIs

* WASM32 Wallet API

* CryptoBox wrapper (+ WASM32 interface)

* WASM32 - PublicKeyGenerator functions, hash functions to support plain text hashing, CryptoBox API, fix RPC shutdown/disconnect mishandling in certain cases.

* Fix broken typedoc link

* fix createAddresses() TS type handling

* WASM32 improve UtxoProcessor restart messaging

* WASM - Fix Generator priorityFee type binding

* WIP - WASM CastFromJS+TryCastFromJS derive macros and traits. Refactoring JsValue casting to Rust objects. secp256k1 update to latest (pending warning resolution)

* WASM - TryCastFromJs updates

* Update PrvKeyDataCreateArgs to use Secret for mnemonic + related CLI changes

* Fix IGeneratorSettingsObject "entries" ingest order

* logging for generator testing

* Generator logs

* browser extension scaffolding

* cleanup logging after debugging

* update WASM changelog

* gate wasm binding by feature

* put comments back

* add and rename crate

* wallet transport

* Change decrypt error to be more generic

* impl and isolate WASM Header bindings from consensus Header

* Fix From<HexString> for JsValue recursion (fixes WASM message signing)

* lowercase hex

* Change decryption failure error message to be more generic

* Update attribute cleaner deps

* Cleanup

* add --weak-refs to wasm-pack

* implement Wallet API accounts_ensure_default() helper

* Update WASM examples/init script to the latest API changes

* Fix typescript mixed array types

* remove sync monitor logs

* Fix mixed typescript array declarations

* Add Debug to WASM event sinks

* Fix grammar

* re-order NetworkTypeT to NetworkType conversion due to failing native WBG enum ABI resolution during type mix (try_from_js_value)

* WASM: fix multi-listener handling for RPC-subscribed events

* Replace from_slice with from_digest_slice to handle warnings

* cleanup logs

* Fix incorrect call on wRPC client disconnect

* WIP - transaction serialization and input signing

* Fix txscript collision due to multiple implementations

* Fix incorrect opcode enum variant name

* WIP transaction serialization

* PendingTransaction -> serializeXXX impl

* Adapter examples (kaspanet#30)

* wip: wallet extension connection
* dark theme
* Update Cargo.toml

* docs + TransactionInput.utxo()

* Fix handling of passing arrays to Generator

* WIP - signing

* code formatting

* WASM RPC: accept extraData as hex or vec in GetBlockTemplateRequest

* WASM RPC: GetBlockTemplateRequest - accept extraData as plain text instead of a hex string

* WASM: remove ? from header hash interface

* RpcClient: Refactor notification handling in wRPC client + add various guards around connect/disconnect and start/stop API calls

* unify WASM logs with workflow-rs logger

* Split connect/disconnect guards to allow concurrent disconnect while connect is pending...

* cleanup logs

* Stop RPC client services on connection error in fallback mode

* Fix misc issues post-merge with master

* wRPC add prelude to the kaspa-wrpc-client crate

* add kaspa-wrpc-example-subscriber example

* cleanup

* add TransactionRecord.has_address() helper to check if an address is used in the transaction record

* Update WASM TypeScript example for event casing and ITransactionRecord access

* addEventListener event union (kaspanet#31)

* Cargo.lock

* rename kaspa-beacon app to kaspa-resolver

* WASM wasm_bindgen bindings for TransactionRecord and a dedicated path on

* Wallet Metrics event;
retain_context(), change_network_id() and account_select() api calls;
get_status_call() updates;

* Cleanup

* TS wallet events (kaspanet#32)

* Apply map-based event typing on wallet events (following UtxoProcessor type design)
* WASM: implement event type maps on RpcClient event notifications

* WASM TS events - remove "All" variants;
Wallet::set_network_id() -> wrpc_client.set_network_id();
set_network_id() updates;

* Cargo.lock

* Cleanup

* add Wallet::retain_context() to WASM Wallet Class

* cleanup

* Migrate WASM functions that use SignableTransaction to Transaction (kaspanet#33)

* Migrate WASM functions that use SignableTransaction to Transaction

* invalid data handling via returning error

* Cargo.lock

* WASM: declare events that receive undefined data

* typo (kaspanet#34)

* transaction.addresses getter added, SignableTransaction struct removed (kaspanet#35)

* transaction.addresses getter

* SignableTransaction struct removed

* docs

* docs

* WASM changelog

* master merge

* docs

* todo (kaspanet#36)

* Todo (kaspanet#37)

* Todo (kaspanet#38)

* transaction.addresses getter

* SignableTransaction struct removed

* docs

* Address.isValid(string)  : a static method for address validation

* typescript docs: Optional NetworkId

* Address.isValid -> Address.validate

* tx.addresses getter to tx.addresses() method

* transaction.addresses() updated for address creation from SPK

* using NetworkTypeT instead of NetworkType

* minor doc updates

* Update to published workflow-rs 0.11.1

* Fix unit tests and WASM SDK build process

* cleanup

* Fix symbols missed during the publicKey() to toPublicKey() rename.

* Update README info for building WASM SDK

* Update README info for building WASM SDK

* Rename RPC open/close events to connect/disconnect

* Add try getters for rpc in UtxoProcessor, Wallet, Wallet API

* Update WASM SDK README

* DIsable Account class (but keep the scaffolding for potential future re-integration)

* Update WASM SDK README with instructions on how to run NodeJs examples

* WalletEvent structure, wallet api example update,  (kaspanet#40)

* transaction.addresses getter

* SignableTransaction struct removed

* docs

* Address.isValid(string)  : a static method for address validation

* typescript docs: Optional NetworkId

* Address.isValid -> Address.validate

* tx.addresses getter to tx.addresses() method

* transaction.addresses() updated for address creation from SPK

* using NetworkTypeT instead of NetworkType

* batch mode issue

* Delete mod copy.rs

* AccountKind constructor, WalletApiObjectExtension helper method

* TS IAccountsCreateRequest type updated using union
* IWalletEvent "event" property renamed to "type"
* examples/wallet.js WIP

* TransactionRecord  notification structure

* TransactionRecord notification structure

* wallet example update

* ecdsa address creation issue

* create_address : ecdsa param use case

* Update CHANGELOG.md

* changelogs, deps version update

* Update wallet.js

* Update package.json

* Update CLI to use a public node Resolver (kaspanet#39)

* Move Tracker memory configuration message from info!() to debug!()

* cleanup

* cleanup

* support public resolver in CLI (default server now is 'public')

* lints

* Update connect command to print a notice when connecting to a public node.
Rename the local url variable.

* cleanup

* cleanup

* Remove optional from ISerializableTransactionInput::utxo field and cleanup previousOutpoint comment

* Cleanup and comments

* UtxoEntry and UtxoEntryReference: toString override for js (kaspanet#41)

* Fix resolver docs, improve resolver URL argument handling.

* Remove IUtxosByAddressesEntry as in WASM we optimize by converting GetUtxosByAddress entries directly to UtxoEntryReference

* bump MSRV to 1.78.0

* CI: update arduino/setup-protoc to v3

---------

Co-authored-by: Surinder Singh Matoo <[email protected]>
Co-authored-by: max143672 <[email protected]>
Co-authored-by: coderofstuff <[email protected]>
* Update README and remove mainnet mining restriction

* Update testnet11.md

* add link to TN11 doc

* add tn11 subtitle

* hint when utxoindex is needed
…spanet#439)

* Add support for IP only for --rpclisten-borsh/json

* Fix cehck complaints
* initial support of pskt:
    supported roles: creator, constructor, updater, signer roles

* add builder

* handle combine errors

* finalize

* extractor

* chore: typo

* style: fmt

* expose txid to global

* chore: change version

* feat: serde for optional bytes

* feat: impl (de)serialization

* style: fmt

* add example, fixes

* style: fmt

* style: clippy

* rollback unrelated changes

* psbt -> pskt

* refactor: avoid copy-paste by using recursion

* docs: add description of roles
@D-Stacks D-Stacks marked this pull request as draft July 10, 2024 17:43
D-Stacks and others added 17 commits July 12, 2024 08:36
* bump to 0.13.4

* bump to 0.13.4

* fix incorrect version due to a toolchain issue
* replace NetworkType with NetworkId in the tx generator and related APIs

* fix url path rendering in format_url

* Improve mass related errors

* refactor fee handling

* renaming fee variants

* mass combination strategy

* wip - storage mass

* additional unit tests and limit checks

* Introduce per-network configurable additional compound transaction compute mass (currently set to 100 on tn10 and tn11)

* expand sanity checks to handle 0 value outputs

* documentation updates

* code formatting

* Set new input sequence to 0

* change aggregate input value check to use saturating_sub() to prevent potential overflows (re github feedback)

* change storage mass fee calc to return u64::MAX if the resulting mass is None (re GitHub feedback)
* allow using a toml config file
* linting fix, serde lowercase WrpcNetAddress
* Fix handling of defaults, strict deserialize of config file + error message
* Update README
* Revert debug default, README fixes
…panet#427)

* Add an integration test covering UTXOs propagation

* Refactor daemon_utxos_propagation_test()

* Add heap profiling feature

* Cover VirtualDaaScoreChanged notifications in test

* Merge branch 'master' into rpc-memory-benchmark

* Assert all changed UTXOs

* Use UtxosChangedScope ctor

* Make active non-blanket UtxosChanged subscription unique

* Refactor broadcaster subscription unregistering

* Add Display to Connection trait

* Save creation of UtxosChanged address vec on listener unregistration

* Reduce UtxosChangedSubscription memory footprint on mutation

* Add gRPC client timeout to every request & remove tpc keep alive

* Reconnect gRPC client on broken pipe

* Disable gRPC server http2 keepalive

* Add a test benchmarking UtxosChanged notifications memory footprint

* Fix log

* Merge branch 'master' into rpc-memory-benchmark

* Remove tokio dependency in crate notify

* Add a UtxosChanged mutation policy to the notification system

* Share subscriptions between listeners and broadcasters

* Replace UtxoAddress with Address

* Refactor Single::mutate

* Refactor `UtxosChangedSubscription` internals

* Make `UtxosChangedSubscription::hash()` cheaper to compute

* mimalloc

* mimalloc disable purge_decommits

* Add to simpa and integration tests

* remove redundant unix and replace with win/linux/mac

* Add comment

* Add comment

* Sort out features

* Remove request timeout from `heap` feature

* Enhance `Broadcaster` memory release and logs

* Track global `UtxosChangedSubscription` count

* Fix heap profile

* Let the number of broadcasters in gRPC server be configurable

* Identify an active and filtering UtxosChangedSubscription with a ListenerId

* Give address.rs its folder

* Address tracker, indexing and counting registrations

* Add a sharable subscription context to the notification system

* Use a subscription context in `Notification` trait and in `single::UtxosChangedSubscription`

* Add an index counter and use short names

* Use a subscription context in `compounded::UtxosChangedSubscription` and in `Subscription` trait

* Rely on hash sets & maps instead of sorted vectors in single and compounded UtxosChanged subscriptions

* fix lint

* Add an optional maximum capacity to the address tracker

* Introduce a mutation outcome

* Remove unneeded CompoundedClone::clone_arc

* Provide inner mutability to `Indexes` and `Counters`

* Restore the filtering of UtxosChanged notifications based on compounded subscriptions in RPC core,  gRPC server and wRPC server

* Measure memory footprint of CounterMap

* Extend `UtxosChangedSubscription` inner mutability to its state

* Group all wildcard `UtxosChangedSubscription` in broadcaster plan

* Have kaspad use a single `SubscriptionContext`

* Add event_type() to Scope

* Reduce the number of mutation clones

* Log some memory stats in a file

* Consume the address vector of UtxosChangedScope

* Retain the original address vector of a UtxosChanged mutation along the full chain of notifiers up to the IndexProcessor

* Enhance the termination of all gRPC server connections

* Put `UtxosChangedSubscription` state and indexes under the same lock

* Some Tracker lock fairness enhancements

* Preallocate static UtxosChanged subscriptions

* Address new lint rules

* Move memory_monitor

* Silent on simnet

* Add a shutdown listener to `Core`

* Add a `Task` trait and implement some tasks

* New daemon memory benchmark running in its own child process

* Refactor `ClientPool` with a start fn returning a vector of `JoinHandle`

* Add start and shutdown signaling to `ClientPool`

* Add full miner, tx sender, subscriber tasks and all their sub-tasks

* Use the tasks in fn utxos_changed_subscriptions_client

* Cleaning

* Fix a rare case potentially preventing subscriber tasks to exit

* Fill the mempool up to the target

* Run actual memory benchmarks

* Add a main task to `TasksRunner`

* Move tasks

* Move tasks (2)

* Rename full to group

* Rename full to group (2)

* Fix cargo syntax error

* Add a stopper status to some tasks

* Let the main task run before adding sub tasks that need it alive

* Mempool benchmark based on tasks

* Small adjustments on the utxos changed subscribe benchmark

* Prevent a race condition

* Refactor

* Move the core shutdown request signaling into `RpcCoreService`

* Add a signal indicating the gRPC server has started

* Recycle emptied tracker entries

* Add `max-tracked-addresses` argument

* Rename `UtxosChangedMutationPolicy` `AllOrNothing` to `Wildcard`

* Cleaning: remove R&D code

* Merge branch 'master' into address-tracker-subscriptions

* Some comments and documentation

* Use a preset listener id in direct mode

* Add lower and upper bounds to the tracker max address count & change the default value

* For each event type the notifier can have at most one subscriber

* Add and document `GrpcClient::connect_with_args`

* Some doc

* Complete `UtxosChangedMutationPolicy` description

* Validate --max-tracked-addresses argument

* remove unused AddressesHash

* fix minor warnings under `devnet-prealloc` feature code
…t#449)

* add a strict assertion which should catch the pruning bug before actual data is pruned

* possible fix: add `block_at_depth_2m` as an additional traversal root

* rollback: rollback the previous fix since it's not the root cause

* add additional dbg info to assertion

* bug fix: write level relations for trusted blocks (blocks in the pruning point anticone of a newly synced node)

* enable mainnet mining by default

* simplify kip 9 beta condition + more mass tests

* set default tracked addresses to 1M

* fix tracker prealloc property + adds compile time assertion for upper bound
…et#459)

* using IConnectOptions struct instead of Jsvalue for socket connection

* Cargo.lock

* WIP - initial Wallet API typescript bindings

* WIP - Wallet API typescript bindings

* typescript setup

* TS related issue

* relocate TypeScript StringArray to kaspa_consensus_wasm

* Update deploy GitHub Action to use folders inside WASM archives

* WIP: TSDoc setup

* WIP: TSdoc

* build-docs updated for typedoc uses

* XPublicKey toString/fromString support for JS

* WASM + Wallet API refactoring for TypeScript

* rearrange nodejs JS/TS examples

* moved ts files to src dir

* rpc frame size

* utxo context listener tests

* WIP - public node wRPC beacons

* fix TS merge

* refactor/restructure past typedoc merge

* Add a warning on build-node-dev

* cleanup

* utxo events issue

* rename README

* rename README

* WIP - typescript bindings and related refactoring

* code formatting

* wasm32-sdk feature gate

* wasm32-sdk feature gating TS/JS-side APIs and types; typedoc handling in macros; relocate wasm RpcClient API into kaspa-wrpc-wasm;

* documentation parsing

* refactor typescript types & JS/TS examples

* simplify HexString (typescript type)

* doc processing in macros

* wRPC Beacon

* #!/bin/bash headers for bash scripts

* wRPC Beacon Cargo.toml update

* WASM wRPC Beacon

* INetworkId type

* toString for XPrv

* renaming SerdeJson to Json

* fix wasm32-sdk propagation

* fix wasm32-sdk propagation

* WIP JS examples

* code formatting

* fix beacon config loading

* updating examples init script

* SerdeJson renaming

* WIP TS: rename XPxxKey to PxxKeyGenerator

* module rename

* finish merging with master

* build*-dev warning text

* WIP: Wallet API updates

* wasm32-sdk infra

* fix IndexMap "deprecated" warning in orphan processing

* kaspa-beacon CORS handling

* Prefix : ktrv, ktub

* key generator (xprv,xpub, signer, keypair) moved to wallet/core/wasm

* XPrv creation from string (xprvxxxx), examples

* change wallet xpub example

* restructure docs location

* hex serialization for RPC components

* WASM32 SDK examples (WIP)

* code formatting

* set /test and /check bash scripts to use -e for immediate exit on failure

* rename from_str to from_xprv_str (clippy)

* wasm32-sdk changelog

* CI updates for wasm32-sdk

* WASM32 SDK CHANGELOG updates

* WIP RPC & Wallet event types

* code formatting

* docs

* Segregating wallet core from keys crates (WASM32 WIP)

* WIP WASM32 release

* WIP WASM32 release

* WIP WASM32 structure

* WASM32 rename packages

* docs

* fix build scripts

* script updates

* fix gitignore

* update kaspa-beacon; fix misc issues, improve election algorithm;

* display error in case of metrics failure

* rename client-side Beacon to Resolver; impl related WASM32 bindings and handling for RpcClient and Resolver

* WASM32 wRPC addEventListener() handling & misc bug fixes

* mix fixes, docs, update WASM32 examples to use updated APIs

* update WASM32 examples

* replace EventDispatcher with localized event handlers

* Refactor wasm event listeners (RPC, UtxoProcessor, Wallet) + update LICENSE

* Support contexts in event listeners

* additional TypeScript interfaces and docs

* WASM32 changelog updates

* WASM32 release scripts

* Kaspa Beacon - tentative access lists

* Kaspa Beacon ACL

* Kaspa Beacon ACLs

* Fix parsing issues with typedoc

* Fix TypeDoc parsing issues

* WASM32 updates to UtxoProcessor (ctors, UTXO access)

* misc UtxoContext updates

* fix typo in cli wizard

* WASM32 - refactor key APIs; fix GetBalancesByAddresses conversion; rename IXxx types to XxxT (convention).

* WASM32 / client - additional key management APIs

* WASM32 Wallet API

* CryptoBox wrapper (+ WASM32 interface)

* WASM32 - PublicKeyGenerator functions, hash functions to support plain text hashing, CryptoBox API, fix RPC shutdown/disconnect mishandling in certain cases.

* Fix broken typedoc link

* fix createAddresses() TS type handling

* WASM32 improve UtxoProcessor restart messaging

* WASM - Fix Generator priorityFee type binding

* WIP - WASM CastFromJS+TryCastFromJS derive macros and traits. Refactoring JsValue casting to Rust objects. secp256k1 update to latest (pending warning resolution)

* WASM - TryCastFromJs updates

* Update PrvKeyDataCreateArgs to use Secret for mnemonic + related CLI changes

* Fix IGeneratorSettingsObject "entries" ingest order

* logging for generator testing

* Generator logs

* browser extension scaffolding

* cleanup logging after debugging

* update WASM changelog

* gate wasm binding by feature

* put comments back

* add and rename crate

* wallet transport

* Change decrypt error to be more generic

* impl and isolate WASM Header bindings from consensus Header

* Fix From<HexString> for JsValue recursion (fixes WASM message signing)

* lowercase hex

* Change decryption failure error message to be more generic

* Update attribute cleaner deps

* Cleanup

* add --weak-refs to wasm-pack

* implement Wallet API accounts_ensure_default() helper

* Update WASM examples/init script to the latest API changes

* Fix typescript mixed array types

* remove sync monitor logs

* Fix mixed typescript array declarations

* Add Debug to WASM event sinks

* Fix grammar

* re-order NetworkTypeT to NetworkType conversion due to failing native WBG enum ABI resolution during type mix (try_from_js_value)

* WASM: fix multi-listener handling for RPC-subscribed events

* Replace from_slice with from_digest_slice to handle warnings

* cleanup logs

* Fix incorrect call on wRPC client disconnect

* WIP - transaction serialization and input signing

* Fix txscript collision due to multiple implementations

* Fix incorrect opcode enum variant name

* WIP transaction serialization

* PendingTransaction -> serializeXXX impl

* Adapter examples (kaspanet#30)

* wip: wallet extension connection
* dark theme
* Update Cargo.toml

* docs + TransactionInput.utxo()

* Fix handling of passing arrays to Generator

* WIP - signing

* code formatting

* WASM RPC: accept extraData as hex or vec in GetBlockTemplateRequest

* WASM RPC: GetBlockTemplateRequest - accept extraData as plain text instead of a hex string

* WASM: remove ? from header hash interface

* RpcClient: Refactor notification handling in wRPC client + add various guards around connect/disconnect and start/stop API calls

* unify WASM logs with workflow-rs logger

* Split connect/disconnect guards to allow concurrent disconnect while connect is pending...

* cleanup logs

* Stop RPC client services on connection error in fallback mode

* Fix misc issues post-merge with master

* wRPC add prelude to the kaspa-wrpc-client crate

* add kaspa-wrpc-example-subscriber example

* cleanup

* add TransactionRecord.has_address() helper to check if an address is used in the transaction record

* Update WASM TypeScript example for event casing and ITransactionRecord access

* addEventListener event union (kaspanet#31)

* Cargo.lock

* rename kaspa-beacon app to kaspa-resolver

* WASM wasm_bindgen bindings for TransactionRecord and a dedicated path on

* Wallet Metrics event;
retain_context(), change_network_id() and account_select() api calls;
get_status_call() updates;

* Cleanup

* TS wallet events (kaspanet#32)

* Apply map-based event typing on wallet events (following UtxoProcessor type design)
* WASM: implement event type maps on RpcClient event notifications

* WASM TS events - remove "All" variants;
Wallet::set_network_id() -> wrpc_client.set_network_id();
set_network_id() updates;

* Cargo.lock

* Cleanup

* add Wallet::retain_context() to WASM Wallet Class

* cleanup

* Migrate WASM functions that use SignableTransaction to Transaction (kaspanet#33)

* Migrate WASM functions that use SignableTransaction to Transaction

* invalid data handling via returning error

* Cargo.lock

* WASM: declare events that receive undefined data

* typo (kaspanet#34)

* transaction.addresses getter added, SignableTransaction struct removed (kaspanet#35)

* transaction.addresses getter

* SignableTransaction struct removed

* docs

* docs

* WASM changelog

* master merge

* docs

* todo (kaspanet#36)

* Todo (kaspanet#37)

* Todo (kaspanet#38)

* transaction.addresses getter

* SignableTransaction struct removed

* docs

* Address.isValid(string)  : a static method for address validation

* typescript docs: Optional NetworkId

* Address.isValid -> Address.validate

* tx.addresses getter to tx.addresses() method

* transaction.addresses() updated for address creation from SPK

* using NetworkTypeT instead of NetworkType

* minor doc updates

* Update to published workflow-rs 0.11.1

* Fix unit tests and WASM SDK build process

* cleanup

* Fix symbols missed during the publicKey() to toPublicKey() rename.

* Update README info for building WASM SDK

* Update README info for building WASM SDK

* Rename RPC open/close events to connect/disconnect

* Add try getters for rpc in UtxoProcessor, Wallet, Wallet API

* Update WASM SDK README

* DIsable Account class (but keep the scaffolding for potential future re-integration)

* Update WASM SDK README with instructions on how to run NodeJs examples

* WalletEvent structure, wallet api example update,  (kaspanet#40)

* transaction.addresses getter

* SignableTransaction struct removed

* docs

* Address.isValid(string)  : a static method for address validation

* typescript docs: Optional NetworkId

* Address.isValid -> Address.validate

* tx.addresses getter to tx.addresses() method

* transaction.addresses() updated for address creation from SPK

* using NetworkTypeT instead of NetworkType

* batch mode issue

* Delete mod copy.rs

* AccountKind constructor, WalletApiObjectExtension helper method

* TS IAccountsCreateRequest type updated using union
* IWalletEvent "event" property renamed to "type"
* examples/wallet.js WIP

* TransactionRecord  notification structure

* TransactionRecord notification structure

* wallet example update

* ecdsa address creation issue

* create_address : ecdsa param use case

* Update CHANGELOG.md

* changelogs, deps version update

* Update wallet.js

* Update package.json

* Update CLI to use a public node Resolver (kaspanet#39)

* Move Tracker memory configuration message from info!() to debug!()

* cleanup

* cleanup

* support public resolver in CLI (default server now is 'public')

* lints

* Update connect command to print a notice when connecting to a public node.
Rename the local url variable.

* cleanup

* cleanup

* Remove optional from ISerializableTransactionInput::utxo field and cleanup previousOutpoint comment

* Cleanup and comments

* UtxoEntry and UtxoEntryReference: toString override for js (kaspanet#41)

* Fix resolver docs, improve resolver URL argument handling.

* Remove IUtxosByAddressesEntry as in WASM we optimize by converting GetUtxosByAddress entries directly to UtxoEntryReference

* bump MSRV to 1.78.0

* CI: update arduino/setup-protoc to v3

---------

Co-authored-by: Surinder Singh Matoo <[email protected]>
Co-authored-by: max143672 <[email protected]>
Co-authored-by: coderofstuff <[email protected]>
* initial support of pskt:
    supported roles: creator, constructor, updater, signer roles

* add builder

* handle combine errors

* finalize

* extractor

* chore: typo

* style: fmt

* expose txid to global

* chore: change version

* feat: serde for optional bytes

* feat: impl (de)serialization

* style: fmt

* add example, fixes

* style: fmt

* style: clippy

* rollback unrelated changes

* psbt -> pskt

* refactor: avoid copy-paste by using recursion

* docs: add description of roles
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants