Skip to content

v0.9.0
Compare
Choose a tag to compare
@brennanjl brennanjl released this 01 Oct 23:30
· 143 commits to main since this release
v0.9.0
ef61535

Kwil v0.9

This is a major release of Kwil DB with several new features.

The highlights are:

  • Zero downtime migrations
  • Data privacy modes: node peer filtering, and private RPC mode with authenticated calls
  • Support builds with customizable branding
  • Genesis and end-block hooks for extension authors
  • Kuneiform updates: many new functions and syntax improvements
  • A framework for testing Kuneiform schemas before deployment
  • Improved node monitoring
  • Removal of the previously-deprecated gRPC server and the legacy HTTP gateway

Contents

Important Upgrading Notes

Upgrading to this release requires a network migration. See the network migration documents for instructions on how to perform a network migration. Only upgrading from v0.8 is supported.

The application's protocol version is bumped from 0 to 1. This prevents nodes that are running different versions of Kwil from operating on the same network. The genesis file no longer includes a "version" section since this is determined by the application logic.

In certain cases, deployed schemas may not be compatible with v0.9. See the Kuneiform Language section for details.

Build Requirements and Dependencies

  • The minimum required Go version is now 1.22. (724fe4f)
  • Builds and tests use Go 1.23. (3aecf89, d50159e)
  • Update the consensus engine (CometBFT) to v0.38.12. (89bad53)

Security Policy

The supported versions are now as follows:

Version Supported
Latest beta or release candidate ✓
v0.9.x ✓
v0.8.x ✓
< v0.8 ❌

See SECURITY.md for details.

Notable Features and Changes

Zero Downtime Migrations

This release adds a new type of migration called a Zero Downtime (ZDT) migration. Unlike the existing offline process, ZDT migrations launch the new network prior to terminating the existing network. There is a migration period during which changes on the existing network are replicated on the new network. See the docs for more information.

Note that the existing network must also support ZDT migrations. Since this is the first release of Kwil that supports such migrations, it cannot be used to go from a v0.8 network to a new v0.9 network. Future major releases will support ZDT migrations allowing this.

These changes also introduced two new transaction types: create resolution and approve resolution. These support generalized resolution processing, migration proposals being one such type.

Relevant code changes: f290d6c, 2f24aae, 0706798, ba6e3e, d175b7f

To support changeset creation from the logical replication stream from PostgreSQL, the type system in the pg package was reworked. See the following relevant changes to the DB internals and exported types in core/types: c983b22, dfa1d8a, 1a16488, 4ece583

NOTE: To support creation of changesets that enable detection and resolution of conflicts between the networks, all tables are now created with "full" replica identity". This changes "apphash" computation, so Kwil v0.9 cannot be installed in place over a v0.8 deployment's data.

Data Privacy Modes

To support use cases where a network needs to maintain data privacy, this release adds two main access control functionalities at both the P2P layer and in the RPC service.

Node Peer Filtering Capability

To run a private network of nodes that only permits inbound P2P connections from certain peers, there is now a private mode that uses peer filtering to only allow certain nodes to connect. The filtering is based on Node ID, which cryptographically ensures the identity of the peer.

A node's whitelist is composed of the current validator set, nodes with active join requests with an approval from the operator, configured seed nodes and persistent peers, and any manually whitelisted peers. This latter case is used to support the addition of authorized sentry (non-validator) nodes to support other functions such as RPC providers, indexers, and seeders.

It is enabled in config.toml with the chain.p2p.private_mode setting. Whitelisted peers may be set in config.toml with chain.p2p.whitelist_peers, or managed at runtime with the kwil-admin whitelist subcommands.
This functionality was added in 559b027.

Private RPCs and Authenticated Calls

A "private" RPC mode has been added with the following features:

  • All call requests are authenticated. The Kwil clients handle this automatically when communicating with an RPC provider in private mode. A private key must be configured with the client in order to sign the call requests.
  • Ad hoc SQL queries are prohibited.
  • Transaction status query responses do not include the full transaction, which could potentially contain data that the user is not authorized to access.

For Go SDK users, note that this adds a signature field to the core/client.CallMessage type.

Related commits: 6537114, 559b027, 177891a

Ability to Rebrand Customized Binary Builds

Projects now have the ability to build custom-branded binaries that group in kwild, kwil-cli, and kwil-admin. These binaries can be given customized default configurations.

The global cmd.BinaryConfig structure is added to allow changing the project name and other details. Projects should set their configuration in an instance of the cmd/custom.CommonCmdConfig and then create their application's root command with the NewCustomCmd function.

This was added in 2aee07c and efc3af4.

Extension Hooks for Genesis Initialization and End-Block Actions

There are now hooks for extension authors to perform arbitrary actions:

  • GenesisHook is a function that is run exactly once, at network genesis. It can be used to create initial state or perform other setup tasks.
    Any state changed or error returned should be deterministic, as all nodes will run the same genesis hooks in the same order.
    Named genesis hooks are registered with the extensions/hooks.RegisterGenesisHook function.
  • EndBlockHook is a function that is run at the end of each block, after all of the transactions in the block have been processed, but before the any state has been committed.
    It is meant to be used to alter state, send data to external services, or perform cleanup tasks for other extensions.
    All state changes and errors should be deterministic, as all nodes will run the same end-block hooks in the same order.
    Extensions register named end-block hooks with the extensions/hooks.RegisterEndBlockHook function.

These changes were made in 79a1e9d.

Kuneiform Language

There are several additions and changes to the Kuneiform language and SQL statement parsing rules. Review this section carefully and use the kwil-cli utils parse to identify changes in the validity of an existing Kuneiform schema prior to starting a network migration. Some schemas that were previously invalid may now be valid, and vice versa.

Kuneiform Additions

  • Support array variable assignment to an existing array like $arr[2] := 5;. (4ea9379)
  • Add the @foreign_caller contextual variable, which carries the name of the schema that called into another schema's procedure. This is empty for the outermost (direct) call. (714a1b9)
  • Add the parse_unix_timestamp and format_unix_timestamp functions, whose formatting rules match PostgreSQL to_timestamp formatting. (5a5f36d)
  • Add the notice function, which can be used to emit transaction logs. NOTE: In addition to being a Kuneiform change, this affects consensus as all nodes must handle and record these logged messages faithfully. (8e8e09c)
  • Add the @block_timestamp variable, which contains the UNIX epoch time stamp of the block that is processing the transaction. It is 0 if not in a transaction. (f684fd2)
  • Add the @authenticator variable, which prints the name of the type of authenticator used to sign the transaction. For example, "secp256k1_ep". (0f7de25)
  • Add the array_remove function, and array slicing. For example, $my_arr[1:3], $my_arr[1:], $my_arr[3]. c08b6c3
  • Add the array_agg function. (55caa0e)

Kuneiform Changes

In addition to the newly added Kuneiform functionality, there are several notable changes to Kuneiform parsing and SQL rules that have the potential to change how procedures from previous release are executed with this version. Some of these are merely fixes, but may change execution regardless:

  • A reserved delimiter, $kwil_reserved_delim$, is now prohibited in procedures. (48d8f65)
  • Equality and assignment operators take a higher precedence than arithmetic operators. (2e1507d)
  • A bare JOIN now defaults to INNER JOIN. Previously it was required to specify the join type. (ce85f23).
  • The common table expression (CTE) syntax no longer requires users to specify their return columns. For example, the following is now acceptable: WITH cte AS (SELECT id FROM users) SELECT * FROM cte; (22bdf85)

New Kuneiform Testing Framework

A new testing framework for validating a Kuneiform schema prior to validation was added in 3893313. The use case is for Kwil users who want to set up tests for their schemas that can run locally, avoiding the need to wait for consensus or create unnecessary network utilization.

There are two ways to use it:

  1. testing package: Users can import the testing package to write their own tests in Go. This gives a lot of flexibility, as they can code any sort of function they want against the engine.
  2. kwil-cli utils test command: Users can use kwil-cli to run tests. Tests can be defined in JSON, where users can specify the schemas to deploy, seed data, and execute actions/procedures and check the results.

The tests can be configured to talk to any Postgres connection, or users can tell the test to setup and teardown a test container.

See the docs for more information.

Related git commits: 3893313, a2d2fd8, 3062a33, 7ff2536

Health Check Endpoint and JSON-RPC Methods

This release adds new health checks for kwild. These are added to support common generic health checks services that have limited configuration i.e. only the http status code is considered.

  • GET /api/v1/health/{svc}

    This endpoint returns the health status of the service in {svc}. For example, "/api/v1/health/user".
    The response code is 200 if healthy, otherwise 503 (unavailable).
    The response body is JSON serialization of a HealthResponse type in the corresponding core/rpc/{svc} Go package.

  • GET /api/v1/health

    This RESTish endpoint returns a core/rpc/json.HealthResponse.
    The code is 200 if all registered services report healthy and
    503 if any of the services are not healthy.
    This response body includes a fingerprint boolean ("kwil_alive")
    and the aggregate health ("healthy"), which corresponds to the response code.
    There is a "services" field that is an object keyed by service
    names. The individual service health status objects are those
    returned by the service-specific "/api/v1/health/{svc}" endpoint described above.

  • rpc.health JSON-RPC method. This method in the reserved rpc namespace corresponds to the aggregate health endpoint.

  • user.health and admin.health JSON-RPC methods. These correspond to the /api/v1/health/user and /api/v1/health/admin endpoints.

Reminder: The HTTP server on port 8484, which is used for user facing RPCs, has only the "user" and "function" services registered. Thus, a request for "admin" health on this server will result in a 404 or method not found. The admin server, which is on a unix socket by default or TCP port 8485, will have all of "user", "function", and "admin" health statuses reported. While it is less likely to have a health check from a cloud provider for this secure server, it is available for secure monitoring setups on the host machine.

Log Rotation and Archival

Rotation and archival of log file segments now happens automatically. Any configured log file, such as the default kwild.log, will be compressed into a sequenced gzip file, and a new empty log file will begin.

The new setting --log.file-roll-size is a number in KB at which the current log file will be compressed, named the next gz in the sequence, and a new uncompressed log file started.

For example, when an uncompressed kwild.log file reaches the configured size threshold, it will create "kwild.log.1.gz", and kwild.log be cleared. When it reaches the threshold again, it will create "kwild.log.2.gz" and so on.

-rw-r--r--   1 usr grp   18433 Sep 12 11:43 kwild.log
-rw-r--r--   1 usr grp    3296 Sep 11 17:50 kwild.log.1.gz
-rw-r--r--   1 usr grp    3596 Sep 11 17:52 kwild.log.2.gz

The default threshold is 100MB.

This was added in b9e424a.

Expose CometBFT Prometheus Metrics

A prometheus metrics server may now be enabled with kwild by setting instrumentation.prometheus = true in the new [instrumentation] setting of the config file.

The default listen address is "0.0.0.0:26660", but may be configured with instrumentation.prometheus_listen_addr.

Presently this only includes metrics with the "cometbft" and "go" namespaces. In future releases, this will be expanded with additional metrics in the "kwild" namespace for application related data.

Extension Interfaces

There are several changes and improvements to extension interface.

The context structures accepted by several functions have been reworked. (c55b5f4)

The node configuration packages have been reorganized and partially exposed to extension authors. (efc3af4)

See the corresponding section on breaking changes for guidance on upgrading.

Breaking Changes

Removal of the gRPC service and its HTTP API gateway

The gRPC service and the HTTP (REST) gateway server that were deprecated in v0.8 in favor of a JSON-RPC service are now removed with c5d0127 and 5e91c02.

  • removed the proto git submodule
  • removed the generated types and service code from core/rpc/protobuf
  • removed the gRPC "txsvc" implementation and server from internal/services/grpc{,_server}
  • removed the gRPC gateway from internal/services/grpc_gateway
  • removed the generated client for the http gateway from core/rpc/http
  • removed the wrapper client for the http gateway from core/rpc/client
  • cleaned out the Taskfile.yml and remove Taskfile-pb.yml
  • updated CI and other scripts
  • updated Dockerfiles and compose definitions
  • updated cmd apps and their config
  • go mod tidy all modules (task tidy)
  • updated integration and acceptance test framework

Breaking Changes and Deprecations in CLI Applications

  • The kwil-admin peer command is deprecated, which, like the init command, initializes the configuration for a new node. The equivalent functionality of the peer command is now achieved with the init command by using the --genesis flag to provide a genesis.json file to use. In addition, the init command now accepts all of the node's flags to specify settings to use in the generated config.toml. (d1cf754

  • With kwil-cli, the --action flag used with the database execute and database call commands is now deprecated. Since this is a required input, it is now a positional argument (just omit --action). (852df6a)

  • With kwild, the rpc_req_limit setting has been renamed to rpc_max_req_size to reflect that it is a size limit rather than a rate limit. rpc_req_limit is now deprecated. (56a765d)

  • The reset and reset-state subcommands of kwil-admin setup are reworked to also reset the PostgreSQL database. The reset-state command only resets PostgreSQL, while reset also deletes the data folders in the application's root directory. (2f6b18e, e876a07)

  • The kwil-admin migrate genesis-state command's --root-dir flag is renamed to --out-dir, and the default is now ~/.genesis-state so as not to accidentally overwrite an existing genesis file if node is running locally. (5dc5f27)

Breaking Changes to Kuneiform and SQL Rules

Please note the changes to Kuneiform and the parser in Kuneiform Changes. Existing schemas that are migrated from an older version of kwild may have functional changes.

Breaking Changes to Transaction Execution

  • If a validator remove transaction targets a public key that is not in the validator set, the transaction fails (non-zero result code) rather than silently doing nothing. Integration tests were already designed with this behavior in mind, but did not query status of the removal transaction on a node where it was executed. This is a breaking change to applications the expected the previous outcome. (b8a52a3)

Breaking Changes to Extension Interfaces

  • Removed the transaction-specific information from common.ExecutionData to avoid duplication with common.TxContext. Changed the common/Engine.CreateDataset() and common/Engine.DeleteDataset() methods to take a transaction context as the first parameter, replacing the context.Context and removing the fourth parameter common.TransactionData which previously contained the transaction data. Changed the common/Engine.Procedure() and common/Engine.Execute() methods to take a transaction context as the first parameter, replacing the context.Context. Extensions that previously passed transaction information to any of the above methods using either common.TransactionData or common.ExecutionData should now pass the information to each methods' first parameter common.TxContext. common.Route.PreTx() and common.Route.InTx() methods now use common.TxContext as the first parameter, instead of context.Context.
    (c55b5f4).

Minor Features and Changes

Minor New Features

CLI Apps

  • Add the kwil-cli utils dbid command to generate a DBID for a given schema name and deployer. (2214f45)

  • Enable setting and providing a password for the admin RPC service with kwil-admin. This must be used with a secure transport layer (either TLS, a secure UNIX or loopback TCP socket). If the service is listening on a non-loopback TCP address, TLS is automatically enabled unless the admin_notls setting is provided to override that behavior. (898430e)

  • Add a configurable request size limit to kwild: app.rpc_req_limit. The default is 4,200,000 bytes. If the network's genesis config sets max transaction size larger
    than this limit, kwild warns on startup. (a0ac88c)

  • Add the kwil-cli utils generate-key command, which generates a new secp256k1 private key and displays the corresponding public key and identifier. (c059786)

  • Add the --genesis-state flag to kwil-admin init to load a snapshot file into the root directory and set genesis_state in the generated config file to the snapshot file. (8974481)

Go SDK

  • Add the ability for the Go client in core/client to skip the initial remote chain ID verification step. This can be used to reduce repeated chain_info requests with KGW. (8c2e545)

Parser

  • Add the parse.ParseSQLWithoutValidation function to allow parsing raw SQL without validation against schema-context. (b494977)

Node Internals

  • Give the ABCI application the ability to remember various consensus variables that are specific to the Kwil blockchain application. This also improves the ability to pass important contextual information to the internal execution engine as well as registered extensions. (TODO: Cross reference the final state of context structs elsewhere in this doc.) (dd5719a), (9997f24), (2ee5b21)

  • The internal pg package has new functions and methods to allow pre-instantiating variables for scan targets. There are several corresponding updates to types in core/types to support scanning and valuing with SQL queries. (3aecf89)

  • The foundation for SQL cost estimation has been added, although it does not affect users of this Kwil release. See e7ac91f and 559b027.

Minor Changes

  • The Kwil version is now written to the configured log file in addition to standard out on node startup. (4c84ae8)
  • Update SQL railroad diagrams (6a93517, 8e061a7)
  • Change the default mempool size limit to 50,000 KB, and the default mempool cache size to 60,000 KB. (c32cf80)
  • Silence the logging of errors in user dataset queries, which are not application logic errors. (23a1832)
  • The help output from kwild -h no longer reports default durations as are actually not known when displaying usage for CLI flags. (3613af5)
  • Add a 10 sec timeout during statesync when requesting a header from the configured RPC provider. This prevents an possible infinite hang if the provider is misconfigured. (af55d9c)
  • Suggest a 4 character tab width in .editorconfig. (4702a91)
  • Go Extension authors executing procedures no longer need to prefix the variable names with a dollar sign ($). For example, when inserting an entry into the arguments map, you may do params["id"] = "me" in addition to params["$id"] = "me". (51cecdc)
  • If snapshot creation is enabled, a snapshot is now created immediately on startup in any of the following cases: none exists, it is height 1, or blocksync is done. Subsequent snapshots are still created when the height is a multiple of the configured app.snapshots.recurring_height setting. b4da094
  • An ActionCall is no longer enumerated as a valid transaction payload since they should never appear in a blockchain transaction, only in a CallMessage. 9f4cf81
  • With kwil-cli database call when using --output json, if the call execution has an error, the empty "result" field will be set to null instead of "". This is done to resolve cryptic errors in the testing framework, but it is also more correct as the result type is not universally a string. 355b77
  • Discrepancies between configured resolutions and compiled-in resolution support is now detected. This prevents situations where an existing deployment contains resolutions that may not exist in a different Kwil version. bcf9592
  • A sanity check is added at the beginning of statesync to ensure the local database is fresh before restoring a snapshot. The presence and versions of the psql and pg_dump tools are also checked. (24addc7)
  • The default validator join expiry (a genesis parameter) is changed to 1 week (about 100800 blocks for 6 second blocks). This default is written by the kwil-admin setup commands. (8271fae)
  • The client example application in core/client/example is updated to work with networks with or without gas enabled. A faucet URL may also be set to automatically request funds if needed. Finally, if a private key is not specified, it will generate one. These changes are meant to make example application easier to run, although they are not realistic situations on most production networks. (bd3be25)

Fixes

  • Repeated validator misbehavior in consecutive blocks is handled more gracefully, avoiding a consensus failure. (0538e2)
  • Snapshots no longer incorrectly remove certain statements from procedure bodies. (51cfe56)
  • Improve errors returned by parser to improve error messages on the CLI and web IDE. (0fd8e09)
  • Numerous other parser error fixes. (fcde95c, 09d647c) @brennanjl: call out any of these in kuneiform breaking changes?
  • The decimal.(*Decimal).Cmp method no longer modifies the receiver. (a96d108)
  • Properly fail to start up if the configured trusted snapshot providers are invalid. This avoids a confusing trusted_height is required error message on startup. (096043f)
  • Allow the generate_dbid function to be used in procedures with the assignment operator. For example, $dbid := generate_dbid('aa', decode('B7E2d6DABaf3B0038cFAaf09688Fa104f4409697', 'hex'));. (8553f23)
  • Remove inaccurate documentation for a non-existent --peer flag with the setup peer command. The --chain.p2p.persistent-peers flag should be used. (a499074)
  • The kwil-admin setup testnet command no longer incorrectly overrides defaults for validator join and vote expiries. cc87c18

Backported to v0.8

  • Prevent creating a decimal type where the exponent is greater than precision. (9ecbbcd)
  • Fix parsing of SQL expressions with LIMIT and ORDER. This was a regression in the v0.8 release that was fixed in a patch release. (412791f)
  • Prevent execution errors in resolution functions from halting the node. ((412791f), (d259784))
  • Various fixes to statesync, most notably, preventing node halt in the case of failure to apply a snapshot chunk. (a02ee22)
  • Allow configuring extensions via flags. (87d45cc)
  • In the internal voting store, fix a unique index erroneously including resolution body, which need not be unique. (60d8e6d)
  • Pass a cancellable context into an internal function in the PostgreSQL logical replication system. (2dbb725)
  • Properly set various configurable file paths to be relative to the root directory if not absolute paths. (3785397)
  • Remove a duplicate log output path for "kwild.log" from the config generated by kwil-admin. (f6c471b)
  • Fix a parser panic found when parsing a procedure that expects a return but does not return anything. (2d85b4c)
  • Fix a parser bug in generation of unary expressions that lacked a space. (5ef667b)
  • Remove incorrect handling of unsupported UUID types in the parser. (99ea751)
  • Give the parser the ability to gracefully handle and report unknown columns in indexes and foreign keys (e2f6708).
  • Allow the parser to work with parameterized SQL (containing $var placeholders). (fd23ff7)
  • Graceful handling of failed snapshot creation. (77b08dd)
  • Fix an inadequate internal buffer in both snapshot creation and statesync. (c674ee9)
  • Fix incorrect SQL generation for actions with inline statements with typecasts for ambiguous types. (90f8b05)
  • Fix whitespace handling (c067834)
  • Prevent adding a stray semicolon at the end of statements that end in spaces. (c067834)
  • During statesync, query for the latest snapshot height instead of the chain height when determining a trusted height to use. (d155a3f)

Testing

  • Include procedures in the stress tool. (6502e30)
  • More strict type assertions in acceptance tests. (65d268c)
  • Integration tests now include scenarios including node restarts. (8974481)
  • The network migration tests now expose the user and admin RPC services to the host machine on unique TCP ports. (65f74c8)
  • The Docker networks generated by integration tests now specify a subnet that is believed to not overlap with subnets already in used by the Github Actions runner. (bd49a7b)
  • The release CI workflow is updated for v0.9. (e39483c and ae1129b)

Diff summary

v0.8.1...v0.9.0

519 changed files with 33,853 additions and 24,856 deletions

Contributors

brennanjl
Assets 7
Loading