Skip to content

Files

Latest commit

a416c30 · Jan 21, 2025

History

History

e2e

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
Jan 8, 2025
Nov 18, 2024
Sep 3, 2024
Mar 26, 2024
Jun 26, 2024
Dec 11, 2023
Dec 11, 2023
Jun 14, 2022
Mar 1, 2021
Apr 7, 2023
Oct 31, 2022
Jun 11, 2024
Jul 3, 2024
Jan 9, 2025
Jan 21, 2025
Jan 10, 2024
Jan 9, 2025
Dec 1, 2023
Jul 1, 2024
Jul 1, 2024
Jun 18, 2024
Mar 5, 2021
Jun 28, 2024

E2E testing

E2E functional tests of cardano-wallet are running nightly on cardano preprod testnet. Running tests against public testnet allows to exercise cardano-wallet on environment close to production (mainnet) utilizing and integrating maximally all components of the Cardano ecosystem like Stake pools, SMASH, metadata token server etc.

Running tests

In order to run tests one needs to have ruby (at least 3.1.2).

Configuring test project

  1. Get necessary gems.
cd test/e2e
bundle install
  1. Decrypt secret files using $TESTS_E2E_FIXTURES secret:
  • fixture_wallets.json.gpg containing mnemonics of testnet fixture wallets
export TESTS_E2E_FIXTURES=*******
rake secrets_decode

ℹ️ TESTS_E2E_FIXTURES secret Note that this step is also executed on very first test run. :information_source: TESTS_E2E_FIXTURES is also kept in the team's 1Password.

Fixture wallets

The command rake secrets_decode can be used to decode the file fixture_wallets.json.gpg into a plain JSON file named fixture_wallets.json. This file contains mnemonic sentences of the wallets that are utilized in tests. The test framework will retrieve these wallets from the file. It is important to note that there are separate sets of wallets for different operating systems, namely Linux, MacOS, and Windows. This allows tests to be run in parallel on different operating systems.

Fixture wallets update

If there is a need to update the mnemonics to new ones, it is certainly possible. After making the updates, remember to use the rake secrets_encode command to encode the updated plain JSON file into a GPG-encoded file. The updated GPG file then needs to be pushed to the repository so that it can be used by the continuous integration (CI) system.

ℹ️ There is a utility command rake fixture_wallets_template which generates the file fixture_wallets.json from scratch with new mnemonic sentences.

Please note that the wallets used in tests must have ADA and specific assets on their balance. Test ADA can be obtained from the Faucet. The assets required on the wallet's balance are:

Both assets have metadata in the Testnet Metadata Registry, and there are tests in place to ensure that the wallet reads data from there correctly. Both assets can be minted outside of the wallet using tools such as cardano-cli or token-minter, and then sent to the balances of the fixture wallets. Policy scripts and keys required for minting are availeble in tests/e2e/fixtuers/wallet_assets.

Further test setup

There may be certain tests that require additional actions to be performed before they can be successfully executed. In such cases, the test will fail and display an appropriate message with instructions. An example of such a test is AlwaysFails.plutus with collateral return to the wallet, which requires a specific UTXO to be set up on the testnet where the test will be executed. This setup is a one-time operation and is only necessary when running tests on a new testnet or when the existing testnet has been reset. The scenario is accompanied by detailed comments and step-by-step instructions on how to proceed. If the required actions have not been taken, the test will fail and display the appropriate message.

Running all tests

In order to run all e2e tests one can simply run single rake task:

$ rake run_on[preprod]

This master task is performing also all the necessary configuration steps (i.e. getting latest testnet configs and wallet/node binaries from Cardano Book, starting everything up). All steps can also be executed as separate tasks , i.e.:

$ rake secrets_decode
$ rake get_latest_bins
$ rake get_latest_configs[preprod]
$ rake start_node_and_wallet[preprod]
$ rake wait_until_node_synced
$ rake spec
$ rake stop_node_and_wallet[preprod]

ℹ️ Linux / MacOS cardano-node and cardano-wallet are started as separate screen sessions. One can attach to the respective session using:

$ screen -r NODE_preprod
$ screen -r WALLET_preprod

ℹ️ Windows cardano-node and cardano-wallet are started as separate Windows services using nssm tool. One can examine services using Windows service manager like services.msc.

ℹ️ Docker One can also start tests against cardano-wallet docker. There is docker-compose-test.yml provided that includes cardano-node and cardano-wallet. To start it several env variables need to be set to feed docker-compose:

NETWORK=preprod \
TESTS_E2E_TOKEN_METADATA=https://metadata.world.dev.cardano.org/ \
WALLET=dev-master \
NODE=10.1.4 \
NODE_CONFIG_PATH=`pwd`/state/configs/$NETWORK \
DATA=`pwd`/state/node_db/$NETWORK
docker-compose -f docker-compose-test.yml up

Then running tests against docker is just:

$ rake wait_until_node_synced
$ rake spec

Running tests against local wallet

One can also run tests against cardano-wallet and cardano-node which are specified on machine's $PATH:

$ TESTS_E2E_BINDIR="" rake run_on[testnet]

Running tests as such skips downloading latest wallet and node binaries.

Test artifacts

By default following locations are used for different artifacts used by the tests:

  • ./bins - location for wallet and node binaries (will be downloaded here from Buildkite)
  • ./state - wallet/node databases, logs and configs

Locations are relative to test/e2e directory. Default values can be changed by providing environment variables, for instance:

TESTS_E2E_STATEDIR=~/state \
TESTS_E2E_BINDIR=~/bins \
rake run_on[preprod]

Full list of environment variables is available in e2e/env.rb.

Running specific tests

There are two types of tests within the suite:

  • e2e - tests that require node and wallet to be synced with the network (i.e. rake wait_until_node_synced step needs to pass before running them)
  • non-e2e - tests do not require node to be fully synced

One can run specific tests using by providing SPEC or SPEC_OPTS arguments to the rake task. For example:

  • run all tests (by default it'll download binaries from Hydra and wait for node to be synced before starting tests)
$ rake run_on[preprod]
  • run only e2e tests
$ rake run_on[preprod] SPEC_OPTS="-t e2e"
  • run only non-e2e tests on downloaded binaries and don't wait for node to be synced
$ rake run_on[preprod,no-sync] SPEC_OPTS="-t ~e2e"
  • run only tests matching specific string
$ rake run_on[preprod] SPEC_OPTS="-e 'CardanoWallet::Shelley::Wallets'"
  • run only specific test identified by line of test code against node and wallet from the $PATH (skips downloading from CI)
$ TESTS_E2E_BINDIR="" rake run_on[preprod] SPEC=spec/shelley_spec.rb:9

ℹ️ Tests can be also executed using rspec.

Skipping / making test pending

When test is failing due to a bug it can be marked as pending. This mark expects test to fail. When such test actually passes the report will indicate such test as failure indicating that it can be turned on again.

it "Wallet can make multi-address transaction" do
  pending "ADP-777 - Failures on multi-address transactions"
  ...
end

Test can be also skipped, so it is not executed at all.

it "Wallet can show utxo distribution" do
  skip "This functionality works intermittently - to be investigated"
  ...
end

Running tests from CI

There are several steps in the Buildkite pipeline for testing against different platforms (Linux, MacOS, Windows), which can be triggered manually if desired.

Node DB cache

For speeding up execution in wallet's full mode we use cardano-node DB from cache. Thanks to this we don't have to wait for hours on each execution until cardano-node is synced with the chain.

Documentation

Cardano-wallet-rb repository: https://github.com/piotr-iohk/cardano-wallet-rb.

Ruby doc: https://piotr-iohk.github.io/cardano-wallet-rb/master/.