- Seath framework and Milestone 3 demo
This documents describes simple stateful protocol that can be run on-chain and the way how to integrate and run it with Seath framework.
The demo protocol is called "Additional Protocol" and is quite simple:
- The state of the protocol is represented by a single UTXO with a single number in the Datum.
- Users of the protocol perform an "addition action" to increment the current state by a desired amount. That's it. Users build transactions to spend the current script UTXO and produce a new one where the number in the datum is incremented by the desired amount. The validator script checks that the amount to be added is greater than or equal to zero and that the datum was properly updated.
Although this protocol is really simple, it suffers from the UTXO contention problem. If two or more users build and submit transactions at the same time, they will use the same single UTXO with the current state from the validator script address. But only the first transaction accepted by the blockchain will succeed. It will spend the current state UTXO and produce a new one. Other transactions will fail because the input state UTXO will have already been spent.
You can find more details and explanations of the UTXO contention problem in the introductory sections of the following demo recordings:
Seath aims to help developers mitigate issues by enabling transaction chaining. With Seath, it is possible to run a network of nodes where one node acts as the leader and listen users' requests. Users send their requests to the leader, stating the actions they want to perform on the protocol. The leader then builds a chain of non-conflicting transactions and submits it to the blockchain. Transactions are chained by Seath in a way that allows users of the protocol to perform their step without the fear of encountering UTXO contention problems.
The on-chain part of the protocol is represented by a validator script written in PlutusTx, which can be found in the on-chain
directory of the repository inside the AdditionValidator.hs Haskell module.
The repository is divided into two main parts, represented by the on-chain
and off-chain
directories. To facilitate a fast and easy setup, the repository provides a flake.nix
file. It is recommended to use Nix with flake feature to run the code in this repository, and all following examples will use Nix capabilities.
As mentioned previously, the on-chain
directory contains a validation script written in PlutusTx. Additionally, the directory contains tools written in Haskell to compile and serialize the script into CBOR format. The serialized script can then be integrated into the off-chain
part to build transactions.
To change the validator script, you can start a Nix environment with the Haskell dependencies required for PlutusTx. From the root of the repository, run:
nix develop .#on-chain
# when Nix shell is up and ready
cd on-chain
# now you have environment with cabal and HLS
After changes are made and ready to go, run this command from the root of the repo:
nix run .#script-export
The script will be compiled, serialized to CBOR, and inserted into the correct location within the off-chain
directory. From there, the Seath framework will be able to pick it up and use it in transactions.
The off-chain part contains the Seath framework logic, which is built on top of the Cardano Transaction Library (CTL) written in PureScript. It has all the required capabilities for the Cardano blockchain.
To access the development shell with spago
, navigate to the root of the repository and run:
nix develop .#off-chain
# when Nix shell is up and ready
cd off-chain
From here you can run tests with
spago run -m Seath.Test.Main
or run automated end-to-end test for addition protocol with disposable local cluster with (see details on testing with Plutip tool)
spago run -m Seath.Test.Main -b addition-e2e-plutip
Or run demo on pre-production testnet.
To communicate with the blockchain, Seath (which is implemented using CTL) requires additional runtime components, namely Kupo and Ogmios (more details can be found in the CTL documentation). The flake.nix
file in the current repository provides a command to start all necessary services. The only requirement is that Docker is installed and running. To start the runtime, navigate to the root of the repository and run the following command (to stop the runtime, press Ctrl/Cmd+C
in the same terminal):
nix run .#preprod-ctl-runtime
Next, wait until the node has synced to 100%. You can check the current sync progress using the docker
command:
# find id or name of the node container with command
docker ps
# then use id or name to execute command in the node container
docker exec -ti [id_or_name] sh -c "CARDANO_NODE_SOCKET_PATH=/ipc/node.socket cardano-cli query tip --testnet-magic=1"
Once the node is synced, you can launch Seath tests and components designed for execution on the pre-production network:
-> an automated end-to-end test with a single leader node and four users submitting actions simultaneously
spago run -m Seath.Test.Main -b preprod -b auto-e2e-test
-> start full Seath node that can accept user requests via IPv4 network
spago run -m Seath.Test.Main -b preprod -b start-leader
-> start scenario in which 4 users send their action simultaneously (currently users are set to send requests to the node started with start-leader
option shown above)
spago run -m Seath.Test.Main -b preprod -b start-users
The primary unit of the Seath framework is the Seath node
. A Seath node
can function as both a leader and a user.
Under the hood, a Seath node
runs the following components (note: actions
will be further explained):
LeaderNode
: the process responsible for running the leader logic. It acceptsactions
from users, translates them into a chain of transactions, handles the signing process, and submits the chain of transactions to the blockchain. TheLeaderNode
uses theCore
functionality of the Seath framework, which needs to be extended by framework users according to the specifics of the particular protocol (more on that below).UserNode
: the process responsible for running the user logic. It createsactions
and sends them to the leader, monitors the current action status, and reacts to status changes by inspecting and signing transactions when necessary (or refusing to sign). It also detects if a transaction was submitted successfully or has failed.- Web server: provides a REST API for the
LeaderNode
to accept user requests. TheUserNode
uses this known REST API of the leader to enable communication betweenSeath nodes
over the network. Currently, there are no requests from the leader to users. Instead, the leader changes the status of theaction
submitted by the user, and by monitoringaction
status, the user sends the required information to the leader or queries the leader.
In terms of architecture, Seath framework divided into 3 main parts (see src dir):
Core
-core functionality
of SeathNetwork
- logic ofLeaderNode
andUserNode
HTTP
- logic of web-server part
Dependency graph: Core
<- Network
<- HTTP
All of these parts will be explained in more detail below, along with an explanation of the demo setup.
Seath chains transactions in "rounds":
- First, it accumulates a number of incoming requests to perform an
action
from users. - It then takes a batch of requests and:
- Translates the
actions
into a chain of transactions using theCore
functionality. - Marks the chained transactions as ready to be signed by the users.
- Waits for users to sign the transactions. If some users don't respond or explicitly refuse to sign the transaction, the leader figures out where the chain is broken and which transactions can still be submitted. Not-signed transactions are discarded from further processing. Transactions that were signed but cannot be submitted due to chain breakage are put into a special priority queue and will be processed first during the next "round". Such transactions will be rebuilt from
actions
from scratch. Users of such transactions will see in the status response that their transaction is in a priority queue for the next "round". - Submits the chain of signed transactions. If any transaction fails, further submission is aborted. Already submitted transactions are marked as processed successfully. Failed transactions are discarded, and the user receives a notification that the transaction failed. The remaining transactions in the chain can't be submitted anymore, so they will be put into a special priority queue and will be processed first during the next "round". Such transactions will be rebuilt from
actions
from scratch. Users of such transactions will see in the status response that their transaction is in a priority queue for the next "round".
- Translates the
- After the batch is processed, the leader waits until the submitted chain is confirmed on the blockchain.
- After the chain is confirmed, the leader starts the next "round", repeating the whole process.
The modules required to run the demo are located in the Demo directory of Addition example.
FullLeaderNode.purs
module contains all the necessary setup to start a full Seath node and handle the Addition protocol. Let us start from here and go layer by layer: first, the core functionality, then user and leader logic, and finally the web server.
The Core functionality
is the central part of the Seath framework. Users of the Seath framework need to integrate their own off-chain logic into Seath core to enable transaction chaining.
The code required to integrate the Addition protocol into Seath core can be found in the Addition directory inside tests. Most important modules are Actions.purs
and Types.purs
To create a Seath node in FullLeaderNode.purs
, we first obtain the CoreConfiguration
by running the CTL Contract.:
coreConfig <- runContractInEnv env $ withKeyWallet leader $
Addition.buildAdditionCoreConfig
(for more details on running Contracts in CTL see related docs, and there will be also some explanations below)
The result of this call will provide us with CoreConfiguration
data from the Seath.Core.Types module. It contains the necessary "parts" that a user of the Seath framework will need to define in order to integrate a particular protocol into Seath. Let us take a closer look at what is done to integrate the Addition protocol.
To produce CoreConfiguration
, you will need a total of 4 pieces:
- The public key hash of the leader -
leader
field. - The hash of the validator script that will hold state UTXOs of the protocol -
stateValidatorHash
field. - A function that can interpret the user's
actions
into lookups and constraints -actionHandler
. This function enables transaction chaining. - A CTL Contract that can query the state of the protocol from the blockchain -
queryBlockchainState
. To build a new chain, Seath need to determine the current state of the protocol. Once it has that information, it can pass changes to the initial state from one transaction in the chain to another. When the chain is submitted, Seath will use this function again to start building a new chain with the current state acquired from the blockchain.
Action
refers to a step in the protocol that the user wishes to perform, expressed via a certain type. In the Addition protocol, users want to increment the current state, represented by a number, by a certain amount. To accomplish this, the following type from the Types.purs
module of the Addition example is used.
newtype AdditionAction = AddAmount BigInt
So there is our action
- add some amount. This type need to be JSON serializable, so nodes can send data over the network.
Now we need to define how to apply this action
to the current protocol state. Usually, we will create a contract that builds a transaction which gets the current state, modifies it, and creates a new output with the updated state. In the case of Seath, we do not need to build a full transaction. Instead, we need to define how the current action
should change the state on-chain, make corresponding transaction constraints and lookups (CTL contracts were inspired by plutus-apps
contracts), and describe the new state.
In the case of the Addition protocol, the state is described by the AdditionState
type from Types.purs
and is simply a number that reflects the state representation on-chain.
type AdditionState = BigInt
For actionHandler
we need to provide a function with the following type:
handleAction
:: UserAction AdditionAction
-> AdditionState
-> Contract UtxoMap
-> Contract
( Seath.Core.Types.StateReturn AdditionValidator AdditionDatum
AdditionRedeemer
AdditionState
)
handleAction userAction lockedValue getScriptUtxos = ...
userAction
is AdditionAction
wrapped in UserAction
. UserAction
is part of the internal Seath machinery and does not need to be constructed by the user of the framework.
lockedValue
is a representation of the on-chain state - AdditionState
type. It is passed by the State framework under the hood during chaining.
getScriptUtxos
is a function provided (injected) by the Seath framework. It gives access to the current or predicted outputs at the validator script address, depending on whether the action
is first in the chain or not.
You can find the full code in Seath.Test.Examples.Addition.Actions.handleAction. Here we extract the addition amount from the AddAmount
constructor of the action
. We then get the validator script hash and current (or predicted during chaining) UTXOs from the validator script that holds the protocol state. Finally, we build the required parts for a future transaction: Datum, Redeemer, constraints, and lookups. If you are familiar with state machines from Plutus Pioneer Program, you may see a closely related idea here. We describe how to perform a single step on some state depending on input without building the transaction explicitly by hand.
With handleAction
done, we have one pice for CoreConfiguration
ready.
The next required piece of code is also located in Seath.Test.Examples.Addition.Actions
and is named queryBlockchainState
. In the case of the Addition protocol, it is a simple CTL Contract that can query the script validator of the protocol. The script validator was compiled and serialized from the on-chain
part and injected as CBOR into Seath.Test.Examples.Addition.Validator. Using several helper functions queryBlockchainState
can use CBOR from Seath.Test.Examples.Addition.Validator
to build the CTL Contract that can query UTXOs from the script and extract the current Datum, which represents the protocol state.
The last two pieces needed are much simpler - the hash of the validator script and the hash of the leader public key. You can check out how to obtain them in the case of the Addition example in Seath.Test.Examples.Addition.ContractUtils.buildAdditionCoreConfig. Getting the script hash is straightforward - we just use helper functions to deserialize CBOR and obtain the script's hash. For the leader key, we get it by running another CTL Contract using the KeyWallet
feature provided by CTL. Depending on the environment, it can return either the hash of the key generated for pre-production testnet or the hash of the key generated by the Plutip tool when Seath runs on a private local cluster. In the current demo, we run Seath on pre-production testnet, and the required keys are located in the keys dir.
That is it for the Core
part. With CoreConfiguration
, the Seath node will be able to build transactions and chain them to run the Addition protocol.
If we further examine FullLeaderNode.purs
we will see:
mkRunner :: KeyWallet -> RunContract
mkRunner kw = RunContract (\c -> runContractInEnv env $ withKeyWallet kw c)
leaderNodeConfig =
Seath.mkLeaderConfig
3000 -- timeout for building chain
4 -- number of pending actions in queue
3000 -- timeout for signatures awaiting
coreConfig
(mkRunner leader)
userNodeConfig = Seath.mkUserConfig leaderUrl (mkRunner leader)
(pure <<< Right)
Let us start from the top.
The first thing to note is the creation of the RunContract
, which is then used to create leader and user configs. Both the leader and user logic use CTL contracts to interact with the blockchain, so they need a way to run Contracts. CTL can execute contracts in several environments, including using cardano-cli-generated keys or integrating with light wallets. In this case, we are using cardano-cli-generated keys and the KeyWallet
CTL feature to run Contracts using them.
We construct a function that can use KeyWallet
and the current CTL environment (see docs for CTL's withContractEnv) to build a function that can execute a contract using the provided key inside the Seath node
.
The Seath.mkLeaderConfig
function builds the configuration for the internal LeaderNode
that the Seath node
will run. It is strongly recommended to use this framework-provided function to build the config. Node configs are quite involved, but this function simplifies their creation by making some required wiring under the hood in a way that is 100% correct.
The current configurable options are:
- Timeout before the leader starts processing user actions - in milliseconds. No matter how many
actions
currently in the leader's "mailbox" queue (see next option), after this period of time, the leader will start processing them anyway. - Number of user
actions
in the leader's "mailbox" that will trigger chain building and submission. When the leader receives a request from the user to submit theiraction
to the chain, the action is added to the leader's queue first. When the number ofactions
reaches this threshold, the leader pulls the actions from the queue and starts processing them. - Time that the leader waits for signatures - in milliseconds. After the leader starts processing user actions, it builds a chain of transactions from them according to the provided
CoreConfiguration
. Once the chain is ready, the leader waits for users to sign their transactions. After this timeout, the leader checks the signed transactions it received from users and submits them. If a transaction was not signed on time or the user rejected signing it, it will be discarded from the processing pipeline.
In addition to options above, we need to provide already explained CoreConfiguration
and contract runner to get the config for the LeaderNode
.
To create config for UserNode
, use the Seath.mkUserConfig
function. It is strongly advised to use this framework-provided function instead of manually building the config using a constructor.
The UserNode
config accepts the following parameters:
- Leader URL:
mkUserConfig
creates the necessary handlers to enable network communication required by user logic under the hood. - Contract runner function: We use the leader key again, as we are creating a full
Seath node
that can act as both leader and user. However, it will play the role of leader for the demo. - Function to check transactions: This function should be defined by the user of the Seath framework. When the leader builds a chain of transactions, it marks them as ready to be signed. The user pulls their transaction from the leader, but before signing it, the user can examine the entire transaction. If something looks incorrect, the user can refuse to sign it by returning an error (
Left
) from this function. The node will notify the leader that the user refused to sign the transaction, and it will be excluded from further processing.
Further in the FullLeaderNode.purs
we see the last required part:
serverConf :: SeathServerConfig
serverConf = { port: leaderPort }
To start the web server, you only need to specify the port number. The server will serve a REST API that allows the node acting as leader to receive requests from user nodes.
With all that prepared we can start Seath node
:
seathNode <- SeathNode.start serverConf leaderNodeConfig userNodeConfig
To stop the node:
SeathNode.stop seathNode
In the demo we leave leader node to run forever until interruption.
As a reminder, to start leader node from terminal (CTL runtime should be up and running):
nix develop .#off-chain
# when Nix shell is up and ready
cd off-chain
spago run -m Seath.Test.Main -b preprod -b start-leader
Test.Examples.Addition.Demo.SeathUsers
In contrast to the leader setup, nodes that will act as users in the demo are not started as full Seath nodes
. Instead, only "internal" UserNodes
are started. This approach has several benefits:
- Simplified demo scenario setup, as there is no need to configure a full
Seath node
. - Only logs from user logic are printed to the terminal. Leader logic generates a lot of logs while running, and there is no logging configuration in Seath yet.
- Easier building of automated scenarios for demo purposes, which can be fine-tuned this way.
We use the framework function mkUserConfig
to configure the nodes in the same way as we did in the leader setup. mkUserConfig
will install all required network handlers.
mkUserConfig
is wrapped in the helper function mkNumeratedUserNodes
, which accepts the refuser
flag.
let
refuser = Just 2
numeratedNodes <- mkNumeratedUserNodes leaderUrl mkRunner refuser setup
The current demo setup for the pre-production network involves actions from four users. By setting refuser = Just 2
, we force user 2
to refuse to sign the transaction, causing a chain break and recovery. To make all users sign their transactions, set the flag to Nothing
.
mkNumeratedUserNodes
will start four UserNode
s that are still capable of sending actions to the leader and tracking the status of the transaction.
mkNumeratedUserNodes
returns an Array
of already running UserNodes
together with their indexes. From here, we can submit an action
directly to the node, and the node will process the action
, send a request to the leader, start monitoring the status of the transaction and sign it when needed (or refuse signing).
for_ numeratedNodes $ \(ix /\ node) -> do
log $ ixName ix <> ": preform include action request"
node `Users.performAction` (AddAmount $ BigInt.fromInt ix)
After this, the process will continue running until it is interrupted. After interruption, the scenario will output the results for each user's submitted actions. This output can include information about successfully submitted transactions or errors encountered during the process:
liftEffect $ onSignal SIGINT $ launchAff_ do
for_ numeratedNodes $ \(ix /\ node) -> do
log $ ixName ix <> " results"
readResults node >>= log <<< show
As a reminder, to start leader node from terminal (CTL runtime and leader node should be up and running):
nix develop .#off-chain
# when Nix shell is up and ready
cd off-chain
spago run -m Seath.Test.Main -b preprod -b start-users
There are some limitations in the current version, but they are not fundamental or unsolvable and can be eliminated during further development. These limitations include:
- Users cannot send another
action
until the previous action is still in progress. When a user submits an action, the data that is sent to the leader includes the user's public key hash. This way, the leader can detect that there is already anaction
in process from this user. The reason for this is that when a user submits an action, they also submit inputs from their address to balance the transaction. At the moment, the algorithm just grabs all UTXOs from the user's address. Sending the same UTXOs twice will cause the 2nd transaction from the same user in the chain to fail. So the problem can be solved by a smarter way of picking input UTXOs on the user side. - The leader must be known upfront. With the current state, we need to know the leader URL and public key upfront before starting the node. This makes it impossible to change the leader dynamically without a node restart. However, this may be improved during further development, and the current architecture does not introduce any blockers for this.
- When a user refuses to sign a transaction, the rest of the transactions in the chain are moved to a priority queue for the next "round", which can potentially slow down the submission of
action
s. During further development, we could try to continue the chain from the breakage point by re-building and re-signing such transactions right away, without putting them in the queue for the next "round".