From 83695707363f78f307a203b109da49968a1c6ac9 Mon Sep 17 00:00:00 2001 From: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com> Date: Wed, 24 Jul 2024 11:55:19 +0100 Subject: [PATCH] doc: Update for CMake-based build system MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Lőrinc Co-authored-by: pablomartin4btc --- README.md | 248 +++++++++----------------------------- contrib/guix/README.md | 1 + depends/README.md | 24 ++-- doc/README.md | 2 +- doc/benchmarking.md | 11 +- doc/build-freebsd.md | 51 ++++---- doc/build-netbsd.md | 34 +++--- doc/build-openbsd.md | 53 ++++---- doc/build-osx.md | 64 +++++----- doc/build-unix.md | 75 +++++------- doc/build-windows-msvc.md | 82 +++++++++++++ doc/build-windows.md | 41 ++----- doc/fuzzing.md | 118 ++++++++++-------- doc/productivity.md | 38 +++--- doc/release-process.md | 8 +- src/qt/README.md | 2 +- src/test/README.md | 45 +++---- test/README.md | 40 +++--- 18 files changed, 440 insertions(+), 497 deletions(-) create mode 100644 doc/build-windows-msvc.md diff --git a/README.md b/README.md index 9da59dd8c9..bc4e901f34 100644 --- a/README.md +++ b/README.md @@ -1,192 +1,62 @@ - -
-

- - Logo - - -

Bitgesell (BGL)

- -

- Bitgesell (BGL) is a digital currency -
- Explore more about project » -
-
- English - · - Chinese -

-

- - - - -
- Table of Contents -
    -
  1. - About The Project - -
    2. -
  2. - Getting Started - -
    3. -
  3. Roadmap
    4. -
  4. Contributing
    5. -
  5. License
    6. -
  6. Contact
    7. -
  7. Translations
    8. -
-
- - - -## About The Project - -![Product Name Screen Shot](https://github.com/BitgesellOfficial/bitgesell/blob/master/share/pixmaps/BGL64.png) - - -Features: -* secp256k1 ECDSA signing/verification and key generation. -* Additive and multiplicative tweaking of secret/public keys. -* Serialization/parsing of secret keys, public keys, signatures. -* Constant time, constant memory access signing and public key generation. -* Derandomized ECDSA (via RFC6979 or with a caller provided function.) -* Very efficient implementation. -* Suitable for embedded systems. -* Optional module for public key recovery. -* Optional module for ECDH key exchange. -* Optional module for Schnorr signatures according to [BIP-340](https://github.com/bitcoin/bips/blob/master/bip-0340.mediawiki). - -Bitgesell is a fork of Bitcoin with the following changes:
-* Block Reward [Burn rate is 90% of tx fees] - ```sh - nFees*0.1 + GetBlockSubsidy() - ``` -* Block Weight [10 times smaller than Bitcoin] - ```sh - <= 400,000 - ``` -* 100% Segwit - ```sh - Eliminates problems with legacy type of transactions - ``` -* Halving Interval [Halving cycle of Bitgesell is 1 year while that of Bitcoin is 4 years] - ```sh - 210000 blocks/4 - ``` -* Block Subsidy [Max coins = 21,000,000]
- `210000 blocks/4`
- `Hashing algorithm for blocks is Keccak (sha-3).`
- `The master branch is regularly built (see` [doc/build-*.md](https://github.com/BitgesellOfficial/bitgesell/tree/master/doc) `for instructions) and tested, but is not guaranteed to be completely stable.`
- [tags](https://github.com/BitgesellOfficial/bitgesell/tags) `are created regularly to indicate new official, stable release versions of BGL Core.`
- - -### Built With - -* General - * No runtime heap allocation. - * Extensive testing infrastructure. - * Structured to facilitate review and analysis. - * Intended to be portable to any system with a C89 compiler and uint64_t support. - * No use of floating types. - * Expose only higher level interfaces to minimize the API surface and improve application security. ("Be difficult to use insecurely.") -* Field operations - * Optimized implementation of arithmetic modulo the curve's field size (2^256 - 0x1000003D1). - * Using 5 52-bit limbs - * Using 10 26-bit limbs (including hand-optimized assembly for 32-bit ARM, by Wladimir J. van der Laan). - * This is an experimental feature that has not received enough scrutiny to satisfy the standard of quality of this library but is made available for testing and review by the community. -* Scalar operations - * Optimized implementation without data-dependent branches of arithmetic modulo the curve's order. - * Using 4 64-bit limbs (relying on __int128 support in the compiler). - * Using 8 32-bit limbs. -* Modular inverses (both field elements and scalars) based on [safegcd](https://gcd.cr.yp.to/index.html) with some modifications, and a variable-time variant (by Peter Dettman). -* Group operations - * Point addition formula specifically simplified for the curve equation (y^2 = x^3 + 7). - * Use addition between points in Jacobian and affine coordinates where possible. - * Use a unified addition/doubling formula where necessary to avoid data-dependent branches. - * Point/x comparison without a field inversion by comparison in the Jacobian coordinate space. -* Point multiplication for verification (a*P + b*G). - * Use wNAF notation for point multiplicands. - * Use a much larger window for multiples of G, using precomputed multiples. - * Use Shamir's trick to do the multiplication with the public key and the generator simultaneously. - * Use secp256k1's efficiently-computable endomorphism to split the P multiplicand into 2 half-sized ones. -* Point multiplication for signing - * Use a precomputed table of multiples of powers of 16 multiplied with the generator, so general multiplication becomes a series of additions. - * Intended to be completely free of timing sidechannels for secret-key operations (on reasonable hardware/toolchains) - * Access the table with branch-free conditional moves so memory access is uniform. - * No data-dependent branches - * Optional runtime blinding which attempts to frustrate differential power analysis. - * The precomputed tables add and eventually subtract points for which no known scalar (secret key) is known, preventing even an attacker with control over the secret key used to control the data internally. - -Building with Autotools ------------------------ - - $ ./autogen.sh - $ ./configure - $ make - $ make check # run the test suite - $ sudo make install # optional - -To compile optional modules (such as Schnorr signatures), you need to run `./configure` with additional flags (such as `--enable-module-schnorrsig`). Run `./configure --help` to see the full list of available flags. - -Building with CMake (experimental) ----------------------------------- - -To maintain a pristine source tree, CMake encourages to perform an out-of-source build by using a separate dedicated build tree. - -### Building on POSIX systems - - $ mkdir build && cd build - $ cmake .. - $ cmake --build . - $ ctest # run the test suite - $ sudo cmake --install . # optional - -To compile optional modules (such as Schnorr signatures), you need to run `cmake` with additional flags (such as `-DSECP256K1_ENABLE_MODULE_SCHNORRSIG=ON`). Run `cmake .. -LH` to see the full list of available flags. - -### Cross compiling - -To alleviate issues with cross compiling, preconfigured toolchain files are available in the `cmake` directory. -For example, to cross compile for Windows: - - $ cmake .. -DCMAKE_TOOLCHAIN_FILE=../cmake/x86_64-w64-mingw32.toolchain.cmake - -To cross compile for Android with [NDK](https://developer.android.com/ndk/guides/cmake) (using NDK's toolchain file, and assuming the `ANDROID_NDK_ROOT` environment variable has been set): - - $ cmake .. -DCMAKE_TOOLCHAIN_FILE="${ANDROID_NDK_ROOT}/build/cmake/android.toolchain.cmake" -DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=28 - -### Building on Windows - -To build on Windows with Visual Studio, a proper [generator](https://cmake.org/cmake/help/latest/manual/cmake-generators.7.html#visual-studio-generators) must be specified for a new build tree. - -The following example assumes using of Visual Studio 2022 and CMake v3.21+. - -In "Developer Command Prompt for VS 2022": - - >cmake -G "Visual Studio 17 2022" -A x64 -S . -B build - >cmake --build build --config RelWithDebInfo - -Usage examples ------------ -Usage examples can be found in the [examples](examples) directory. To compile them you need to configure with `--enable-examples`. - * [ECDSA example](examples/ecdsa.c) - * [Schnorr signatures example](examples/schnorr.c) - * [Deriving a shared secret (ECDH) example](examples/ecdh.c) - * [ElligatorSwift key exchange example](examples/ellswift.c) - -To compile the Schnorr signature and ECDH examples, you also need to configure with `--enable-module-schnorrsig` and `--enable-module-ecdh`. - -Benchmark ------------- -If configured with `--enable-benchmark` (which is the default), binaries for benchmarking the libsecp256k1 functions will be present in the root directory after the build. +Bitgesell Core integration/staging tree +===================================== + +https://bitcoincore.org + +For an immediately usable, binary version of the Bitcoin Core software, see +https://bitcoincore.org/en/download/. + +What is Bitcoin Core? +--------------------- + +Bitcoin Core connects to the Bitgesell peer-to-peer network to download and fully +validate blocks and transactions. It also includes a wallet and graphical user +interface, which can be optionally built. + +Further information about Bitcoin Core is available in the [doc folder](/doc). + +License +------- + +Bitcoin Core is released under the terms of the MIT license. See [COPYING](COPYING) for more +information or see https://opensource.org/licenses/MIT. + +Development Process +------------------- + +The `master` branch is regularly built (see `doc/build-*.md` for instructions) and tested, but it is not guaranteed to be +completely stable. [Tags](https://github.com/BitgesellOfficial/bitgesell/tags) are created +regularly from release branches to indicate new official, stable release versions of Bitcoin Core. + +The https://github.com/bitcoin-core/gui repository is used exclusively for the +development of the GUI. Its master branch is identical in all monotree +repositories. Release branches and tags do not exist, so please do not fork +that repository unless it is for development reasons. + +The contribution workflow is described in [CONTRIBUTING.md](CONTRIBUTING.md) +and useful hints for developers can be found in [doc/developer-notes.md](doc/developer-notes.md). + +Testing +------- + +Testing and code review is the bottleneck for development; we get more pull +requests than we can review and test on short notice. Please be patient and help out by testing +other people's pull requests, and remember this is a security-critical project where any mistake might cost people +lots of money. + +### Automated Testing + +Developers are strongly encouraged to write [unit tests](src/test/README.md) for new code, and to +submit new unit tests for old code. Unit tests can be compiled and run +(assuming they weren't disabled during the generation of the build system) with: `ctest`. Further details on running +and extending unit tests can be found in [/src/test/README.md](/src/test/README.md). + +There are also [regression and integration tests](/test), written +in Python. +These tests can be run (if the [test dependencies](/test) are installed) with: `test/functional/test_runner.py` + +The CI (Continuous Integration) systems make sure that every pull request is built for Windows, Linux, and macOS, +and that unit/sanity tests are run automatically. Facebook: [Bitgesell](https://www.facebook.com/Bitgesell) diff --git a/contrib/guix/README.md b/contrib/guix/README.md index cece3a4d15..20c32bc9a1 100644 --- a/contrib/guix/README.md +++ b/contrib/guix/README.md @@ -261,6 +261,7 @@ details. - `guix` build commands as in `guix shell --cores="$JOBS"` - `make` as in `make --jobs="$JOBS"` + - `cmake` as in `cmake --build build -j "$JOBS"` - `xargs` as in `xargs -P"$JOBS"` See [here](#controlling-the-number-of-threads-used-by-guix-build-commands) for diff --git a/depends/README.md b/depends/README.md index 53ecfa70cb..6c8c6d54df 100644 --- a/depends/README.md +++ b/depends/README.md @@ -12,17 +12,13 @@ For example: make HOST=x86_64-w64-mingw32 -j4 -**BGL Core's `configure` script by default will ignore the depends output.** In +**When configuring Bitgesell Core, CMake by default will ignore the depends output.** In order for it to pick up libraries, tools, and settings from the depends build, -you must set the `CONFIG_SITE` environment variable to point to a `config.site` settings file. -Make sure that `CONFIG_SITE` is an absolute path. -In the above example, a file named `depends/x86_64-w64-mingw32/share/config.site` will be -created. To use it during compilation: +you must specify the toolchain file. +In the above example, a file named `depends/x86_64-w64-mingw32/toolchain.cmake` will be +created. To use it during configuring Bitcoin Core: - CONFIG_SITE=$PWD/depends/x86_64-w64-mingw32/share/config.site ./configure - -The default install prefix when using `config.site` is `--prefix=depends/`, -so depends build outputs will be installed in that location. + cmake -B build --toolchain depends/x86_64-w64-mingw32/toolchain.cmake Common `host-platform-triplet`s for cross compilation are: @@ -90,9 +86,13 @@ For linux S390X cross compilation: pkg install bash +### Install the required dependencies: NetBSD + + pkgin install bash gmake + ### Install the required dependencies: OpenBSD - pkg_add bash gtar + pkg_add bash gmake gtar ### Dependency Options @@ -125,8 +125,8 @@ The following can be set when running make: `make FOO=bar` - `LTO`: Enable options needed for LTO. Does not add `-flto` related options to *FLAGS. - `NO_HARDEN=1`: Don't use hardening options when building packages -If some packages are not built, for example `make NO_WALLET=1`, the appropriate -options will be passed to BGL's configure. In this case, `--disable-wallet`. +If some packages are not built, for example `make NO_WALLET=1`, the appropriate CMake cache +variables will be set when generating the Bitgesell Core buildsystem. In this case, `-DENABLE_WALLET=OFF`. ### Additional targets diff --git a/doc/README.md b/doc/README.md index 9f7c8d7ba7..a04f28cab2 100644 --- a/doc/README.md +++ b/doc/README.md @@ -41,7 +41,7 @@ The following are developer notes on how to build BGL Core on your native platfo - [Dependencies](dependencies.md) - [macOS Build Notes](build-osx.md) - [Unix Build Notes](build-unix.md) -- [Windows Build Notes](build-windows.md) +- [Windows Build Notes](build-windows-msvc.md) - [FreeBSD Build Notes](build-freebsd.md) - [OpenBSD Build Notes](build-openbsd.md) - [NetBSD Build Notes](build-netbsd.md) diff --git a/doc/benchmarking.md b/doc/benchmarking.md index 1610c0ba27..dfeacff0dd 100644 --- a/doc/benchmarking.md +++ b/doc/benchmarking.md @@ -8,16 +8,17 @@ thread queue, wallet balance. Running --------------------- -For benchmarking, you only need to compile `BGL_bench`. The bench runner -warns if you configure with `--enable-debug`, but consider if building without +For benchmarking, you only need to compile `bench_BGL`. The bench runner +warns if you configure with `-DCMAKE_BUILD_TYPE=Debug`, but consider if building without it will impact the benchmark(s) you are interested in by unlatching log printers and lock analysis. - make -C src BGL_bench + cmake -B build -DBUILD_BENCH=ON + cmake --build build -t bench_BGL After compiling BGL-core, the benchmarks can be run with: - src/bench/bench_BGL + build/src/bench/bench_BGL The output will look similar to: ``` @@ -39,7 +40,7 @@ The output will look similar to: Help --------------------- - src/bench/bench_BGL -? + build/src/bench/bench_BGL -? To print the various options, like listing the benchmarks without running them or using a regex filter to only run certain benchmarks. diff --git a/doc/build-freebsd.md b/doc/build-freebsd.md index 321bd9bf34..1f24055ad3 100644 --- a/doc/build-freebsd.md +++ b/doc/build-freebsd.md @@ -10,15 +10,14 @@ This guide describes how to build BGLd, command-line utilities, and GUI on FreeB Run the following as root to install the base dependencies for building. ```bash -pkg install autoconf automake boost-libs git gmake libevent libtool pkgconf - +pkg install boost-libs cmake git libevent pkgconf ``` See [dependencies.md](dependencies.md) for a complete overview. -### 2. Clone BGL Repo -Now that `git` and all the required dependencies are installed, let's clone the BGL Core repository to a directory. All build scripts and commands will run from this directory. -``` bash +### 2. Clone Bitgesell Repo +Now that `git` and all the required dependencies are installed, let's clone the Bitcoin Core repository to a directory. All build scripts and commands will run from this directory. +```bash git clone https://github.com/BitgesellOfficial/bitgesell.git ``` @@ -31,7 +30,7 @@ It is not necessary to build wallet functionality to run either `BGLd` or `BGL-q `sqlite3` is required to support [descriptor wallets](descriptors.md). Skip if you don't intend to use descriptor wallets. -``` bash +```bash pkg install sqlite3 ``` @@ -41,14 +40,15 @@ BerkeleyDB is only required if legacy wallet support is required. It is required to use Berkeley DB 4.8. You **cannot** use the BerkeleyDB library from ports. However, you can build DB 4.8 yourself [using depends](/depends). -``` +```bash +pkg install gmake gmake -C depends NO_BOOST=1 NO_LIBEVENT=1 NO_QT=1 NO_SQLITE=1 NO_NATPMP=1 NO_UPNP=1 NO_ZMQ=1 NO_USDT=1 ``` When the build is complete, the Berkeley DB installation location will be displayed: ``` -to: /path/to/bitcoin/depends/x86_64-unknown-freebsd[release-number] +to: /path/to/bitgesell/depends/x86_64-unknown-freebsd[release-number] ``` Finally, set `BDB_PREFIX` to this path according to your shell: @@ -64,17 +64,23 @@ sh/bash: export BDB_PREFIX=[path displayed above] #### GUI Dependencies ###### Qt5 -Bitgesell Core includes a GUI built with the cross-platform Qt Framework. To compile the GUI, we need to install the necessary parts of Qt. Skip if you don't intend to use the GUI. +Bitcoin Core includes a GUI built with the cross-platform Qt Framework. To compile the GUI, we need to install +the necessary parts of Qt, the libqrencode and pass `-DBUILD_GUI=ON`. Skip if you don't intend to use the GUI. + ```bash pkg install qt5-buildtools qt5-core qt5-gui qt5-linguisttools qt5-testlib qt5-widgets ``` ###### libqrencode -The GUI can encode addresses in a QR Code. To build in QR support for the GUI, install `libqrencode`. Skip if not using the GUI or don't want QR code functionality. +The GUI will be able to encode addresses in QR codes unless this feature is explicitly disabled. To install libqrencode, run: + ```bash pkg install libqrencode ``` + +Otherwise, if you don't need QR encoding support, use the `-DWITH_QRENCODE=OFF` option to disable this feature in order to compile the GUI. + --- #### Notifications @@ -101,33 +107,28 @@ pkg install python3 databases/py-sqlite3 There are many ways to configure BGL Core, here are a few common examples: ##### Descriptor Wallet and GUI: -This explicitly enables the GUI and disables legacy wallet support, assuming `sqlite` and `qt` are installed. +This disables legacy wallet support and enables the GUI, assuming `sqlite` and `qt` are installed. ```bash -./autogen.sh -./configure --without-bdb --with-gui=yes MAKE=gmake +cmake -B build -DWITH_BDB=OFF -DBUILD_GUI=ON ``` +Run `cmake -B build -LH` to see the full list of available options. + ##### Descriptor & Legacy Wallet. No GUI: -This enables support for both wallet types and disables the GUI, assuming +This enables support for both wallet types, assuming `sqlite3` and `db4` are both installed. ```bash -./autogen.sh -./configure --with-gui=no \ - BDB_LIBS="-L${BDB_PREFIX}/lib -ldb_cxx-4.8" \ - BDB_CFLAGS="-I${BDB_PREFIX}/include" \ - MAKE=gmake +cmake -B build -DBerkeleyDB_INCLUDE_DIR:PATH="${BDB_PREFIX}/include" ``` ##### No Wallet or GUI -``` bash -./autogen.sh -./configure --without-wallet --with-gui=no MAKE=gmake +```bash +cmake -B build -DENABLE_WALLET=OFF ``` ### 2. Compile -**Important**: Use `gmake` (the non-GNU `make` will exit with an error). ```bash -gmake # use "-j N" for N parallel jobs -gmake check # Run tests if Python 3 is available +cmake --build build # Use "-j N" for N parallel jobs. +ctest --test-dir build # Use "-j N" for N parallel tests. Some tests are disabled if Python 3 is not available. ``` diff --git a/doc/build-netbsd.md b/doc/build-netbsd.md index 1c230fe605..95ce7252b9 100644 --- a/doc/build-netbsd.md +++ b/doc/build-netbsd.md @@ -12,8 +12,7 @@ Install the required dependencies the usual way you [install software on NetBSD] The example commands below use `pkgin`. ```bash -pkgin install autoconf automake libtool pkg-config git gmake boost-headers libevent - +pkgin install git cmake pkg-config boost-headers libevent ``` NetBSD currently ships with an older version of `gcc` than is needed to build. You should upgrade your `gcc` and then pass this new version to the configure script. @@ -25,10 +24,10 @@ pkgin install gcc12 Then, when configuring, pass the following: ```bash -./configure +cmake -B build ... - CC="/usr/pkg/gcc12/bin/gcc" \ - CXX="/usr/pkg/gcc12/bin/g++" \ + -DCMAKE_C_COMPILER="/usr/pkg/gcc12/bin/gcc" \ + -DCMAKE_CXX_COMPILER="/usr/pkg/gcc12/bin/g++" \ ... ``` @@ -65,19 +64,26 @@ pkgin install db4 ``` #### GUI Dependencies +###### Qt5 -Bitcoin Core includes a GUI built with the cross-platform Qt Framework. To compile the GUI, Qt 5 is required. +Bitcoin Core includes a GUI built with the cross-platform Qt Framework. To compile the GUI, we need to install +the necessary parts of Qt, the libqrencode and pass `-DBUILD_GUI=ON`. Skip if you don't intend to use the GUI. ```bash pkgin install qt5-qtbase qt5-qttools ``` -The GUI can encode addresses in a QR Code. To build in QR support for the GUI, install `qrencode`. +###### libqrencode + +The GUI will be able to encode addresses in QR codes unless this feature is explicitly disabled. To install libqrencode, run: ```bash pkgin install qrencode ``` +Otherwise, if you don't need QR encoding support, use the `-DWITH_QRENCODE=OFF` option to disable this feature in order to compile the GUI. + + #### Test Suite Dependencies There is an included test suite that is useful for testing code changes when developing. @@ -89,28 +95,22 @@ pkgin install python39 ### Building BGL Core -**Note**: Use `gmake` (the non-GNU `make` will exit with an error). - - ### 1. Configuration There are many ways to configure Bitcoin Core. Here is an example that explicitly disables the wallet and GUI: ```bash -./autogen.sh -./configure --without-wallet --with-gui=no \ - CPPFLAGS="-I/usr/pkg/include" \ - MAKE=gmake +cmake -B build -DENABLE_WALLET=OFF -DBUILD_GUI=OFF ``` -For a full list of configuration options, see the output of `./configure --help` +Run `cmake -B build -LH` to see the full list of available options. ### 2. Compile Build and run the tests: ```bash -gmake # use "-j N" here for N parallel jobs -gmake check # Run tests if Python 3 is available +cmake --build build # Use "-j N" for N parallel jobs. +ctest --test-dir build # Use "-j N" for N parallel tests. Some tests are disabled if Python 3 is not available. ``` diff --git a/doc/build-openbsd.md b/doc/build-openbsd.md index e93074bbca..4add80079d 100644 --- a/doc/build-openbsd.md +++ b/doc/build-openbsd.md @@ -10,9 +10,7 @@ This guide describes how to build BGLd, command-line utilities, and GUI on OpenB Run the following as root to install the base dependencies for building. ```bash -pkg_add bash git gmake libevent libtool boost -# Select the newest version of the following packages: -pkg_add autoconf automake python +pkg_add git cmake boost libevent ``` See [dependencies.md](dependencies.md) for a complete overview. @@ -31,7 +29,7 @@ It is not necessary to build wallet functionality to run either `BGLd` or `BGL-q ###### Descriptor Wallet Support -`sqlite3` is required to support [descriptor wallets](descriptors.md). +SQLite is required to support [descriptor wallets](descriptors.md). ``` bash pkg_add sqlite3 @@ -48,19 +46,20 @@ Refer to [depends/README.md](/depends/README.md) for detailed instructions. ```bash gmake -C depends NO_BOOST=1 NO_LIBEVENT=1 NO_QT=1 NO_SQLITE=1 NO_NATPMP=1 NO_UPNP=1 NO_ZMQ=1 NO_USDT=1 ... -to: /path/to/BGL/depends/x86_64-unknown-openbsd +to: /path/to/bitgesell/depends/*-unknown-openbsd* ``` Then set `BDB_PREFIX`: ```bash -export BDB_PREFIX="/path/to/BGL/depends/x86_64-unknown-openbsd" +export BDB_PREFIX="[path displayed above]" ``` #### GUI Dependencies ###### Qt5 -BGL Core includes a GUI built with the cross-platform Qt Framework. To compile the GUI, Qt 5 is required. +Bitgesell Core includes a GUI built with the cross-platform Qt Framework. To compile the GUI, we need to install +the necessary parts of Qt, the libqrencode and pass `-DBUILD_GUI=ON`. Skip if you don't intend to use the GUI. ```bash pkg_add qtbase qttools @@ -68,10 +67,14 @@ pkg_add qtbase qttools ###### libqrencode -The GUI can encode addresses in a QR Code. To build in QR support for the GUI, install `libqrencode`. Skip if not using the GUI or don't want QR code functionality. +The GUI will be able to encode addresses in QR codes unless this feature is explicitly disabled. To install libqrencode, run: + ```bash pkg_add libqrencode ``` + +Otherwise, if you don't need QR encoding support, use the `-DWITH_QRENCODE=OFF` option to disable this feature in order to compile the GUI. + --- #### Notifications @@ -82,47 +85,41 @@ Bitcoin Core can provide notifications via ZeroMQ. If the package is installed, pkg_add zeromq ``` -## Building Bitcoin Core - -**Important**: Use `gmake` (the non-GNU `make` will exit with an error). +#### Test Suite Dependencies +There is an included test suite that is useful for testing code changes when developing. +To run the test suite (recommended), you will need to have Python 3 installed: -Preparation: ```bash - -# Adapt the following for the version you installed (major.minor only): -export AUTOCONF_VERSION=2.71 -export AUTOMAKE_VERSION=1.16 - -./autogen.sh +pkg_add install python # Select the newest version of the package. ``` +## Building Bitcoin Core + ### 1. Configuration There are many ways to configure Bitgesell Core, here are a few common examples: ##### Descriptor Wallet and GUI: -This enables the GUI and descriptor wallet support, assuming `sqlite` and `qt5` are installed. +This enables descriptor wallet support and the GUI, assuming SQLite and Qt 5 are installed. ```bash -./configure MAKE=gmake +cmake -B build -DWITH_SQLITE=ON -DBUILD_GUI=ON ``` +Run `cmake -B build -LH` to see the full list of available options. + ##### Descriptor & Legacy Wallet. No GUI: -This enables support for both wallet types and disables the GUI: +This enables support for both wallet types: ```bash -./configure --with-gui=no \ - BDB_LIBS="-L${BDB_PREFIX}/lib -ldb_cxx-4.8" \ - BDB_CFLAGS="-I${BDB_PREFIX}/include" \ - MAKE=gmake +cmake -B build -DBerkeleyDB_INCLUDE_DIR:PATH="${BDB_PREFIX}/include" ``` ### 2. Compile -**Important**: Use `gmake` (the non-GNU `make` will exit with an error). ```bash -gmake # use "-j N" for N parallel jobs -gmake check # Run tests if Python 3 is available +cmake --build build # Use "-j N" for N parallel jobs. +ctest --test-dir build # Use "-j N" for N parallel tests. Some tests are disabled if Python 3 is not available. ``` ## Resource limits diff --git a/doc/build-osx.md b/doc/build-osx.md index 8c1aa2e9c5..70a85da2c0 100644 --- a/doc/build-osx.md +++ b/doc/build-osx.md @@ -1,6 +1,6 @@ # macOS Build Guide -**Updated for MacOS [11.2](https://www.apple.com/macos/big-sur/)** +**Updated for MacOS [14.4](https://www.apple.com/macos/sonoma/)** This guide describes how to build BGLd, command-line utilities, and GUI on macOS @@ -43,7 +43,7 @@ See [dependencies.md](dependencies.md) for a complete overview. To install, run the following from your terminal: ``` bash -brew install automake libtool boost pkg-config libevent +brew install cmake boost pkg-config libevent ``` For macOS 11 (Big Sur) and 12 (Monterey) you need to install a more recent version of llvm. @@ -97,9 +97,8 @@ brew install berkeley-db@4 ###### Qt -BGL Core includes a GUI built with the cross-platform Qt Framework. -To compile the GUI, we need to install `qt@5`. -Skip if you don't intend to use the GUI. +Bitgesell Core includes a GUI built with the cross-platform Qt Framework. To compile the GUI, we need to install +the necessary parts of Qt, the libqrencode and pass `-DBUILD_GUI=ON`. Skip if you don't intend to use the GUI. ``` bash brew install qt@5 @@ -108,14 +107,16 @@ brew install qt@5 Note: Building with Qt binaries downloaded from the Qt website is not officially supported. See the notes in [#7714](https://github.com/bitcoin/bitcoin/issues/7714). -###### qrencode +###### libqrencode -The GUI can encode addresses in a QR Code. To build in QR support for the GUI, install `qrencode`. -Skip if not using the GUI or don't want QR code functionality. +The GUI will be able to encode addresses in QR codes unless this feature is explicitly disabled. To install libqrencode, run: ``` bash brew install qrencode ``` + +Otherwise, if you don't need QR encoding support, use the `-DWITH_QRENCODE=OFF` option to disable this feature in order to compile the GUI. + --- Then install [Homebrew](https://brew.sh). @@ -129,7 +130,7 @@ brew install automake libtool boost miniupnpc libnatpmp pkg-config python qt@5 l #### Deploy Dependencies -You can deploy a `.zip` containing the Bitgesell Core application using `make deploy`. +You can [deploy](#3-deploy-optional) a `.zip` containing the Bitgesell Core application. It is required that you have `python` installed. The wallet support requires one or both of the dependencies ([*SQLite*](#sqlite) and [*Berkeley DB*](#berkeley-db)) in the sections below. @@ -137,32 +138,33 @@ To build BGL Core without wallet, see [*Disable-wallet mode*](#disable-wallet-mo #### SQLite -If `berkeley-db@4` is installed, then legacy wallet support will be built. -If `sqlite` is installed, then descriptor wallet support will also be built. -Additionally, this explicitly disables the GUI. +If `berkeley-db@4` or `sqlite` are not installed, this will throw an error. -```shell -brew install sqlite +``` bash +cmake -B build -DWITH_BDB=ON ``` In that case the Homebrew package will prevail. -#### Berkeley DB +This enables the GUI. +If `sqlite` or `qt` are not installed, this will throw an error. -It is recommended to use Berkeley DB 4.8. If you have to build it yourself, -you can use [this](/contrib/install_db4.sh) script to install it -like so: +``` bash +cmake -B build -DBUILD_GUI=ON +``` -```shell -./contrib/install_db4.sh . +##### No Wallet or GUI + +``` bash +cmake -B build -DENABLE_WALLET=OFF ``` from the root of the repository. Also, the Homebrew package could be installed: -```shell -brew install berkeley-db4 +``` bash +cmake -B build -LH ``` ## Build BGL Core @@ -174,18 +176,17 @@ brew install berkeley-db4 ``` ``` bash -make # use "-j N" here for N parallel jobs -make check # Run tests if Python 3 is available +cmake --build build # Use "-j N" here for N parallel jobs. +ctest --test-dir build # Use "-j N" for N parallel tests. Some tests are disabled if Python 3 is not available. ``` Configure and build the headless BGL Core binaries as well as the GUI (if Qt is found). You can also create a `.zip` containing the `.app` bundle by running the following command: -3. It is recommended to build and run the unit tests: - ```shell - make check - ``` +``` bash +cmake --build build --target deploy +``` 4. You can also create a `.dmg` that contains the `.app` bundle (optional): ```shell @@ -225,9 +226,10 @@ tail -f $HOME/Library/Application\ Support/BGL/debug.log ## Other commands: ```shell -./src/BGLd -daemon # Starts the BGL daemon. -./src/BGL-cli --help # Outputs a list of command-line options. -./src/BGL-cli help # Outputs a list of RPC commands when the daemon is running. +./build/src/BGLd -daemon # Starts the bitcoin daemon. +./build/src/BGL-cli --help # Outputs a list of command-line options. +./build/src/BGL-cli help # Outputs a list of RPC commands when the daemon is running. +./build/src/qt/BGL-qt -server # Starts the bitcoin-qt server mode, allows bitcoin-cli control ``` ## Notes diff --git a/doc/build-unix.md b/doc/build-unix.md index c781bee55c..82eda807c8 100644 --- a/doc/build-unix.md +++ b/doc/build-unix.md @@ -9,10 +9,9 @@ To Build --------------------- ```bash -./autogen.sh -./configure -make # use "-j N" for N parallel jobs -make install # optional +cmake -B build +cmake --build build # use "-j N" for N parallel jobs +cmake --install build # optional ``` See below for instructions on how to [install the dependencies on popular Linux @@ -22,20 +21,21 @@ distributions](#linux-distribution-specific-instructions), or the ## Memory Requirements C++ compilers are memory-hungry. It is recommended to have at least 1.5 GB of -memory available when compiling BGL Core. On systems with less, gcc can be -tuned to conserve memory with additional CXXFLAGS: +memory available when compiling Bitgesell Core. On systems with less, gcc can be +tuned to conserve memory with additional `CMAKE_CXX_FLAGS`: - ./configure CXXFLAGS="--param ggc-min-expand=1 --param ggc-min-heapsize=32768" + cmake -B build -DCMAKE_CXX_FLAGS="--param ggc-min-expand=1 --param ggc-min-heapsize=32768" -Alternatively, or in addition, debugging information can be skipped for compilation. The default compile flags are -`-g -O2`, and can be changed with: +Alternatively, or in addition, debugging information can be skipped for compilation. +For the default build type `RelWithDebInfo`, the default compile flags are +`-O2 -g`, and can be changed with: - ./configure CXXFLAGS="-g0" + cmake -B build -DCMAKE_CXX_FLAGS_RELWITHDEBINFO="-O2 -g0" Finally, clang (often less resource hungry) can be used instead of gcc, which is used by default: - ./configure CXX=clang++ CC=clang + cmake -B build -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_CC_COMPILER=clang ## Linux Distribution Specific Instructions @@ -45,7 +45,7 @@ Finally, clang (often less resource hungry) can be used instead of gcc, which is Build requirements: - sudo apt-get install build-essential libtool autotools-dev automake pkg-config bsdmainutils python3 + sudo apt-get install build-essential cmake pkg-config bsdmainutils python3 Now, you can either build from self-compiled [depends](#dependencies) or install the required dependencies: @@ -57,12 +57,11 @@ SQLite is required for the descriptor wallet: Berkeley DB is only required for the legacy wallet. Ubuntu and Debian have their own `libdb-dev` and `libdb++-dev` packages, but these will install Berkeley DB 5.3 or later. This will break binary wallet compatibility with the distributed -executables, which are based on BerkeleyDB 4.8. If you do not care about wallet compatibility, pass -`--with-incompatible-bdb` to configure. Otherwise, you can build Berkeley DB [yourself](#berkeley-db). +executables, which are based on BerkeleyDB 4.8. Otherwise, you can build Berkeley DB [yourself](#berkeley-db). To build Bitgesell Core without wallet, see [*Disable-wallet mode*](#disable-wallet-mode) -Optional port mapping libraries (see: `--with-miniupnpc` and `--with-natpmp`): +Optional port mapping libraries (see: `-DWITH_MINIUPNPC=ON` and `-DWITH_NATPMP=ON`): sudo apt install libminiupnpc-dev libnatpmp-dev @@ -76,11 +75,8 @@ User-Space, Statically Defined Tracing (USDT) dependencies: GUI dependencies: -If you want to build BGL-qt, make sure that the required packages for Qt development -are installed. Qt 5 is necessary to build the GUI. -To build without GUI pass `--without-gui`. - -To build with Qt 5 you need the following: +Bitgesell Core includes a GUI built with the cross-platform Qt Framework. To compile the GUI, we need to install +the necessary parts of Qt, the libqrencode and pass `-DBUILD_GUI=ON`. Skip if you don't intend to use the GUI. sudo apt-get install qtbase5-dev qttools5-dev qttools5-dev-tools @@ -88,12 +84,11 @@ Additionally, to support Wayland protocol for modern desktop environments: sudo apt install qtwayland5 -libqrencode (optional) can be installed with: +The GUI will be able to encode addresses in QR codes unless this feature is explicitly disabled. To install libqrencode, run: sudo apt-get install libqrencode-dev -Once these are installed, they will be found by configure and a BGL-qt executable will be -built by default. +Otherwise, if you don't need QR encoding support, use the `-DWITH_QRENCODE=OFF` option to disable this feature in order to compile the GUI. ### Fedora @@ -119,7 +114,7 @@ pass `--with-incompatible-bdb` to configure. Otherwise, you can build Berkeley D To build Bitgesell Core without wallet, see [*Disable-wallet mode*](#disable-wallet-mode) -Optional port mapping libraries (see: `--with-miniupnpc` and `--with-natpmp`): +Optional port mapping libraries (see: `-DWITH_MINIUPNPC=ON` and `-DWITH_NATPMP=ON`): sudo dnf install miniupnpc-devel libnatpmp-devel @@ -133,11 +128,8 @@ User-Space, Statically Defined Tracing (USDT) dependencies: GUI dependencies: -If you want to build BGL-qt, make sure that the required packages for Qt development -are installed. Qt 5 is necessary to build the GUI. -To build without GUI pass `--without-gui`. - -To build with Qt 5 you need the following: +Bitgesell Core includes a GUI built with the cross-platform Qt Framework. To compile the GUI, we need to install +the necessary parts of Qt, the libqrencode and pass `-DBUILD_GUI=ON`. Skip if you don't intend to use the GUI. sudo dnf install qt5-qttools-devel qt5-qtbase-devel @@ -145,12 +137,11 @@ Additionally, to support Wayland protocol for modern desktop environments: sudo dnf install qt5-qtwayland -libqrencode (optional) can be installed with: +The GUI will be able to encode addresses in QR codes unless this feature is explicitly disabled. To install libqrencode, run: sudo dnf install qrencode-devel -Once these are installed, they will be found by configure and a BGL-qt executable will be -built by default. +Otherwise, if you don't need QR encoding support, use the `-DWITH_QRENCODE=OFF` option to disable this feature in order to compile the GUI. ## Dependencies @@ -172,9 +163,7 @@ and configure using the following: ```bash export BDB_PREFIX="/path/to/bitgesell/depends/x86_64-pc-linux-gnu" -./configure \ - BDB_LIBS="-L${BDB_PREFIX}/lib -ldb_cxx-4.8" \ - BDB_CFLAGS="-I${BDB_PREFIX}/include" +cmake -B build -DBerkeleyDB_INCLUDE_DIR:PATH="${BDB_PREFIX}/include" ``` **Note**: Make sure that `BDB_PREFIX` is an absolute path. @@ -186,7 +175,7 @@ Disable-wallet mode When the intention is to only run a P2P node, without a wallet, BGL Core can be compiled in disable-wallet mode with: - ./configure --disable-wallet + cmake -B build -DENABLE_WALLET=OFF In this case there is no dependency on SQLite or Berkeley DB. @@ -196,19 +185,19 @@ Additional Configure Flags -------------------------- A list of additional configure flags can be displayed with: - ./configure --help + cmake -B build -LH Setup and Build Example: Arch Linux ----------------------------------- This example lists the steps necessary to setup and build a command line only distribution of the latest changes on Arch Linux: - pacman --sync --needed autoconf automake boost gcc git libevent libtool make pkgconf python sqlite + pacman --sync --needed cmake boost gcc git libevent make pkgconf python sqlite git clone https://github.com/BitgesellOfficial/bitgesell.git - cd /bitgesell - ./autogen.sh - ./configure - make check - ./src/BGLd + cd bitgesell/ + cmake -B build + cmake --build build + ctest --test-dir build + ./build/src/BGLd If you intend to work with legacy Berkeley DB wallets, see [Berkeley DB](#berkeley-db) section. diff --git a/doc/build-windows-msvc.md b/doc/build-windows-msvc.md new file mode 100644 index 0000000000..ec839724dd --- /dev/null +++ b/doc/build-windows-msvc.md @@ -0,0 +1,82 @@ +# Windows / MSVC Build Guide + +This guide describes how to build bitcoind, command-line utilities, and GUI on Windows using Micsrosoft Visual Studio. + +For cross-compiling options, please see [`build-windows.md`](./build-windows.md). + +## Preparation + +### 1. Visual Studio + +This guide relies on using CMake and vcpkg package manager provided with the Visual Studio installation. +Here are requirements for the Visual Studio installation: +1. Minimum required version: Visual Studio 2022 version 17.6. +2. Installed components: +- The "Desktop development with C++" workload. + +The commands in this guide should be executed in "Developer PowerShell for VS 2022" or "Developer Command Prompt for VS 2022". +The former is assumed hereinafter. + +### 2. Git + +Download and install [Git for Windows](https://git-scm.com/download/win). Once installed, Git is available from PowerShell or the Command Prompt. + +### 3. Clone Bitcoin Repository + +Clone the Bitcoin Core repository to a directory. All build scripts and commands will run from this directory. +``` +git clone https://github.com/bitcoin/bitcoin.git +``` + + +## Triplets and Presets + +The Bitcoin Core project supports the following vcpkg triplets: +- `x64-windows` (both CRT and library linkage is dynamic) +- `x64-windows-static` (both CRT and library linkage is static) + +To facilitate build process, the Bitcoin Core project provides presets, which are used in this guide. + +Available presets can be listed as follows: +``` +cmake --list-presets +``` + +## Building + +CMake will put the resulting object files, libraries, and executables into a dedicated build directory. + +In following istructions, the "Debug" configuration can be specified instead of the "Release" one. + +### 4. Building with Dynamic Linking with GUI + +``` +cmake -B build --preset vs2022 -DBUILD_GUI=ON # It might take a while if the vcpkg binary cache is unpopulated or invalidated. +cmake --build build --config Release # Use "-j N" for N parallel jobs. +ctest --test-dir build --build-config Release # Use "-j N" for N parallel tests. Some tests are disabled if Python 3 is not available. +``` + +### 5. Building with Static Linking without GUI + +``` +cmake -B build --preset vs2022-static # It might take a while if the vcpkg binary cache is unpopulated or invalidated. +cmake --build build --config Release # Use "-j N" for N parallel jobs. +ctest --test-dir build --build-config Release # Use "-j N" for N parallel tests. Some tests are disabled if Python 3 is not available. +cmake --install build --config Release # Optional. +``` + +## Performance Notes + +### 6. vcpkg Manifest Default Features + +One can skip vcpkg manifest default features to speedup the configuration step. +For example, the following invocation will skip all features except for "wallet" and "tests" and their dependencies: +``` +cmake -B build --preset vs2022 -DVCPKG_MANIFEST_NO_DEFAULT_FEATURES=ON -DVCPKG_MANIFEST_FEATURES="wallet;tests" -DBUILD_GUI=OFF +``` + +Available features are listed in the [`vcpkg.json`](/vcpkg.json) file. + +### 7. Antivirus Software + +To improve the build process performance, one might add the Bitcoin repository directory to the Microsoft Defender Antivirus exclusions. diff --git a/doc/build-windows.md b/doc/build-windows.md index e660a46cf7..3d5de900f0 100644 --- a/doc/build-windows.md +++ b/doc/build-windows.md @@ -7,7 +7,7 @@ The options known to work for building BGL Core on Windows are: * On Linux, using the [Mingw-w64](https://www.mingw-w64.org/) cross compiler tool chain. * On Windows, using [Windows Subsystem for Linux (WSL)](https://learn.microsoft.com/en-us/windows/wsl/about) and Mingw-w64. -* On Windows, using [Microsoft Visual Studio](https://visualstudio.microsoft.com). See [README.md](/build_msvc/README.md). +* On Windows, using [Microsoft Visual Studio](https://visualstudio.microsoft.com). See [`build-windows-msvc.md`](./build-windows-msvc.md). Other options which may work, but which have not been extensively tested are (please contribute instructions): @@ -55,14 +55,14 @@ First, install the general dependencies: sudo apt update sudo apt upgrade - sudo apt install build-essential libtool autotools-dev automake pkg-config bsdmainutils cmake curl git + sudo apt install cmake curl g++ git make pkg-config -A host toolchain (`build-essential`) is necessary because some dependency +A host toolchain (`g++`) is necessary because some dependency packages need to build host utilities that are used in the build process. See [dependencies.md](dependencies.md) for a complete overview. -If you want to build the windows installer with `make deploy` you need [NSIS](https://nsis.sourceforge.io/Main_Page): +If you want to build the Windows installer using the `deploy` build target, you will need [NSIS](https://nsis.sourceforge.io/Main_Page): sudo apt install nsis @@ -73,7 +73,7 @@ Acquire the source in the usual way: ## Building for 64-bit Windows -The first step is to install the mingw-w64 cross-compilation tool chain: +The first step is to install the mingw-w64 cross-compilation toolchain: ```sh sudo apt install g++-mingw-w64-x86-64-posix @@ -85,21 +85,12 @@ Note that for WSL the BGL Core source path MUST be somewhere in the default moun example /usr/src/BGL, AND not under /mnt/d/. If this is not the case the dependency autoconf scripts will fail. This means you cannot use a directory that is located directly on the host Windows file system to perform the build. -Additional WSL Note: WSL support for [launching Win32 applications](https://learn.microsoft.com/en-us/archive/blogs/wsl/windows-and-ubuntu-interoperability#launching-win32-applications-from-within-wsl) -results in `Autoconf` configure scripts being able to execute Windows Portable Executable files. This can cause -unexpected behaviour during the build, such as Win32 error dialogs for missing libraries. The recommended approach -is to temporarily disable WSL support for Win32 applications. - Build using: - sudo bash -c "echo 0 > /proc/sys/fs/binfmt_misc/status" # Disable WSL support for Win32 applications. - cd depends - make HOST=x86_64-w64-mingw32 - cd .. - ./autogen.sh - CONFIG_SITE=$PWD/depends/x86_64-w64-mingw32/share/config.site ./configure --prefix=/ - make # use "-j N" for N parallel jobs - sudo bash -c "echo 1 > /proc/sys/fs/binfmt_misc/status" # Enable WSL support for Win32 applications. + gmake -C depends HOST=x86_64-w64-mingw32 # Use "-j N" for N parallel jobs. + cmake -B build --toolchain depends/x86_64-w64-mingw32/toolchain.cmake + cmake --build build # Use "-j N" for N parallel jobs. + ctest --test-dir build # Use "-j N" for N parallel tests. Some tests are disabled if Python 3 is not available. ## Depends system @@ -113,18 +104,8 @@ executables to a directory on the Windows drive in the same directory structure as they appear in the release `.zip` archive. This can be done in the following way. This will install to `c:\workspace\BGL`, for example: - make install DESTDIR=/mnt/c/workspace/BGL + cmake --install build --prefix /mnt/c/workspace/BGL You can also create an installer using: - make deploy - -Footnotes ---------- - -1: Starting from Ubuntu Xenial 16.04, both the 32 and 64 bit Mingw-w64 packages install two different -compiler options to allow a choice between either posix or win32 threads. The default option is win32 threads which is the more -efficient since it will result in binary code that links directly with the Windows kernel32.lib. Unfortunately, the headers -required to support win32 threads conflict with some of the classes in the C++11 standard library, in particular std::mutex. -It's not possible to build the BGL Core code using the win32 version of the Mingw-w64 cross compilers (at least not without -modifying headers in the BGL Core source code). + cmake --build build --target deploy diff --git a/doc/fuzzing.md b/doc/fuzzing.md index 215029e1a3..de011baae6 100644 --- a/doc/fuzzing.md +++ b/doc/fuzzing.md @@ -5,14 +5,17 @@ To quickly get started fuzzing Bitcoin Core using [libFuzzer](https://llvm.org/docs/LibFuzzer.html): ```sh -$ git clone https://github.com/bitcoin/bitcoin -$ cd bitcoin/ -$ ./autogen.sh -$ CC=clang CXX=clang++ ./configure --enable-fuzz --with-sanitizers=address,fuzzer,undefined +$ git clone https://github.com/BitgesellOfficial/bitgesell.git +$ cd bitgesell/ +$ cmake -B build_fuzz \ + -DCMAKE_C_COMPILER="clang" \ + -DCMAKE_CXX_COMPILER="clang++" \ + -DBUILD_FOR_FUZZING=ON \ + -DSANITIZERS=undefined,address,fuzzer # macOS users: If you have problem with this step then make sure to read "macOS hints for # libFuzzer" on https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md#macos-hints-for-libfuzzer -$ make -$ FUZZ=process_message src/test/fuzz/fuzz +$ cmake --build build_fuzz +$ FUZZ=process_message build_fuzz/src/test/fuzz/fuzz # abort fuzzing using ctrl-c ``` @@ -25,7 +28,7 @@ There is also a runner script to execute all fuzz targets. Refer to ## Fuzzing harnesses and output -[`process_message`](https://github.com/bitcoin/bitcoin/blob/master/src/test/fuzz/process_message.cpp) is a fuzzing harness for the [`ProcessMessage(...)` function (`net_processing`)](https://github.com/bitcoin/bitcoin/blob/master/src/net_processing.cpp). The available fuzzing harnesses are found in [`src/test/fuzz/`](https://github.com/bitcoin/bitcoin/tree/master/src/test/fuzz). +[`process_message`](https://github.com/bitcoin/bitcoin/blob/master/src/test/fuzz/process_message.cpp) is a fuzzing harness for the [`ProcessMessage(...)` function (`net_processing`)](https://github.com/BitgesellOfficial/bitgesell/blob/master/src/net_processing.cpp). The available fuzzing harnesses are found in [`src/test/fuzz/`](https://github.com/BitgesellOfficial/bitgesell/tree/master/src/test/fuzz). The fuzzer will output `NEW` every time it has created a test input that covers new areas of the code under test. For more information on how to interpret the fuzzer output, see the [libFuzzer documentation](https://llvm.org/docs/LibFuzzer.html). @@ -33,7 +36,7 @@ If you specify a corpus directory then any new coverage increasing inputs will b ```sh $ mkdir -p process_message-seeded-from-thin-air/ -$ FUZZ=process_message src/test/fuzz/fuzz process_message-seeded-from-thin-air/ +$ FUZZ=process_message build_fuzz/src/test/fuzz/fuzz process_message-seeded-from-thin-air/ INFO: Seed: 840522292 INFO: Loaded 1 modules (424174 inline 8-bit counters): 424174 [0x55e121ef9ab8, 0x55e121f613a6), INFO: Loaded 1 PC tables (424174 PCs): 424174 [0x55e121f613a8,0x55e1225da288), @@ -77,7 +80,7 @@ of the test. Just make sure to use double-dash to distinguish them from the fuzzer's own arguments: ```sh -$ FUZZ=address_deserialize_v2 src/test/fuzz/fuzz -runs=1 fuzz_seed_corpus/address_deserialize_v2 --checkaddrman=5 --printtoconsole=1 +$ FUZZ=address_deserialize_v2 build_fuzz/src/test/fuzz/fuzz -runs=1 fuzz_seed_corpus/address_deserialize_v2 --checkaddrman=5 --printtoconsole=1 ``` ## Fuzzing corpora @@ -88,7 +91,7 @@ To fuzz `process_message` using the [`bitcoin-core/qa-assets`](https://github.co ```sh $ git clone https://github.com/bitcoin-core/qa-assets -$ FUZZ=process_message src/test/fuzz/fuzz qa-assets/fuzz_seed_corpus/process_message/ +$ FUZZ=process_message build_fuzz/src/test/fuzz/fuzz qa-assets/fuzz_seed_corpus/process_message/ INFO: Seed: 1346407872 INFO: Loaded 1 modules (424174 inline 8-bit counters): 424174 [0x55d8a9004ab8, 0x55d8a906c3a6), INFO: Loaded 1 PC tables (424174 PCs): 424174 [0x55d8a906c3a8,0x55d8a96e5288), @@ -101,7 +104,15 @@ INFO: seed corpus: files: 991 min: 1b max: 1858b total: 288291b rss: 150Mb ## Run without sanitizers for increased throughput -Fuzzing on a harness compiled with `--with-sanitizers=address,fuzzer,undefined` is good for finding bugs. However, the very slow execution even under libFuzzer will limit the ability to find new coverage. A good approach is to perform occasional long runs without the additional bug-detectors (configure `--with-sanitizers=fuzzer`) and then merge new inputs into a corpus as described in the qa-assets repo (https://github.com/bitcoin-core/qa-assets/blob/main/.github/PULL_REQUEST_TEMPLATE.md). Patience is useful; even with improved throughput, libFuzzer may need days and 10s of millions of executions to reach deep/hard targets. +Fuzzing on a harness compiled with `-DSANITIZERS=address,fuzzer,undefined` is +good for finding bugs. However, the very slow execution even under libFuzzer +will limit the ability to find new coverage. A good approach is to perform +occasional long runs without the additional bug-detectors (just +`-DSANITIZERS=fuzzer`) and then merge new inputs into a corpus as described in +the qa-assets repo +(https://github.com/bitcoin-core/qa-assets/blob/main/.github/PULL_REQUEST_TEMPLATE.md). +Patience is useful; even with improved throughput, libFuzzer may need days and +10s of millions of executions to reach deep/hard targets. ## Reproduce a fuzzer crash reported by the CI @@ -112,14 +123,14 @@ Fuzzing on a harness compiled with `--with-sanitizers=address,fuzzer,undefined` more slowly with sanitizers enabled, but a crash should be reproducible very quickly from a crash case) - run the fuzzer with the case number appended to the seed corpus path: - `FUZZ=process_message src/test/fuzz/fuzz + `FUZZ=process_message build_fuzz/src/test/fuzz/fuzz qa-assets/fuzz_seed_corpus/process_message/1bc91feec9fc00b107d97dc225a9f2cdaa078eb6` ## Submit improved coverage -If you find coverage increasing inputs when fuzzing you are highly encouraged to submit them for inclusion in the [`bitcoin-core/qa-assets`](https://github.com/bitcoin-core/qa-assets) repo. +If you find coverage increasing inputs when fuzzing you are highly encouraged to submit them for inclusion in the [`BGL-core/qa-assets`](https://github.com/bitcoin-core/qa-assets) repo. -Every single pull request submitted against the Bitcoin Core repo is automatically tested against all inputs in the [`bitcoin-core/qa-assets`](https://github.com/bitcoin-core/qa-assets) repo. Contributing new coverage increasing inputs is an easy way to help make Bitcoin Core more robust. +Every single pull request submitted against the Bitcoin Core repo is automatically tested against all inputs in the [`BGL-core/qa-assets`](https://github.com/bitcoin-core/qa-assets) repo. Contributing new coverage increasing inputs is an easy way to help make Bitcoin Core more robust. ## macOS hints for libFuzzer @@ -131,10 +142,15 @@ You may also need to take care of giving the correct path for `clang` and `clang++`, like `CC=/path/to/clang CXX=/path/to/clang++` if the non-systems `clang` does not come first in your path. -Full configure that was tested on macOS with `brew` installed `llvm`: +Full configuration step that was tested on macOS with `brew` installed `llvm`: ```sh -./configure --enable-fuzz --with-sanitizers=fuzzer,address,undefined CC=$(brew --prefix llvm)/bin/clang CXX=$(brew --prefix llvm)/bin/clang++ +$ cmake -B build_fuzz \ + -DCMAKE_C_COMPILER="$(brew --prefix llvm)/bin/clang" \ + -DCMAKE_CXX_COMPILER="$(brew --prefix llvm)/bin/clang++" \ + -DBUILD_FOR_FUZZING=ON \ + -DSANITIZERS=undefined,address,fuzzer \ + -DAPPEND_LDFLAGS=-Wl,-no_warn_duplicate_libraries ``` Read the [libFuzzer documentation](https://llvm.org/docs/LibFuzzer.html) for more information. This [libFuzzer tutorial](https://github.com/google/fuzzing/blob/master/tutorial/libFuzzerTutorial.md) might also be of interest. @@ -146,20 +162,22 @@ Read the [libFuzzer documentation](https://llvm.org/docs/LibFuzzer.html) for mor To quickly get started fuzzing Bitcoin Core using [afl++](https://github.com/AFLplusplus/AFLplusplus): ```sh -$ git clone https://github.com/bitcoin/bitcoin -$ cd bitcoin/ +$ git clone https://github.com/BitgesellOfficial/bitgesell.git +$ cd bitgesell/ $ git clone https://github.com/AFLplusplus/AFLplusplus $ make -C AFLplusplus/ source-only -$ ./autogen.sh # If afl-clang-lto is not available, see # https://github.com/AFLplusplus/AFLplusplus#a-selecting-the-best-afl-compiler-for-instrumenting-the-target -$ CC=$(pwd)/AFLplusplus/afl-clang-lto CXX=$(pwd)/AFLplusplus/afl-clang-lto++ ./configure --enable-fuzz -$ make -# For macOS you may need to ignore x86 compilation checks when running "make". If so, -# try compiling using: AFL_NO_X86=1 make +$ cmake -B build_fuzz \ + -DCMAKE_C_COMPILER="$(pwd)/AFLplusplus/afl-clang-lto" \ + -DCMAKE_CXX_COMPILER="$(pwd)/AFLplusplus/afl-clang-lto++" \ + -DBUILD_FOR_FUZZING=ON +$ cmake --build build_fuzz +# For macOS you may need to ignore x86 compilation checks when running "cmake --build". If so, +# try compiling using: AFL_NO_X86=1 cmake --build build_fuzz $ mkdir -p inputs/ outputs/ $ echo A > inputs/thin-air-input -$ FUZZ=bech32 AFLplusplus/afl-fuzz -i inputs/ -o outputs/ -- src/test/fuzz/fuzz +$ FUZZ=bech32 ./AFLplusplus/afl-fuzz -i inputs/ -o outputs/ -- build_fuzz/src/test/fuzz/fuzz # You may have to change a few kernel parameters to test optimally - afl-fuzz # will print an error and suggestion if so. ``` @@ -173,17 +191,20 @@ Read the [afl++ documentation](https://github.com/AFLplusplus/AFLplusplus) for m To quickly get started fuzzing Bitcoin Core using [Honggfuzz](https://github.com/google/honggfuzz): ```sh -$ git clone https://github.com/bitcoin/bitcoin -$ cd bitcoin/ -$ ./autogen.sh +$ git clone https://github.com/BitgesellOfficial/bitgesell.git +$ cd bitgesell/ $ git clone https://github.com/google/honggfuzz $ cd honggfuzz/ $ make $ cd .. -$ CC=$(pwd)/honggfuzz/hfuzz_cc/hfuzz-clang CXX=$(pwd)/honggfuzz/hfuzz_cc/hfuzz-clang++ ./configure --enable-fuzz --with-sanitizers=address,undefined -$ make +$ cmake -B build_fuzz \ + -DCMAKE_C_COMPILER="$(pwd)/honggfuzz/hfuzz_cc/hfuzz-clang" \ + -DCMAKE_CXX_COMPILER="$(pwd)/honggfuzz/hfuzz_cc/hfuzz-clang++" \ + -DBUILD_FOR_FUZZING=ON \ + -DSANITIZERS=address,undefined +$ cmake --build build_fuzz $ mkdir -p inputs/ -$ FUZZ=process_message honggfuzz/honggfuzz -i inputs/ -- src/test/fuzz/fuzz +$ FUZZ=process_message ./honggfuzz/honggfuzz -i inputs/ -- build_fuzz/src/test/fuzz/fuzz ``` Read the [Honggfuzz documentation](https://github.com/google/honggfuzz/blob/master/docs/USAGE.md) for more information. @@ -200,19 +221,14 @@ also remotely triggerable by an untrusted peer. To quickly get started fuzzing the P2P layer using Honggfuzz NetDriver: ```sh -$ mkdir bitcoin-honggfuzz-p2p/ -$ cd bitcoin-honggfuzz-p2p/ -$ git clone https://github.com/bitcoin/bitcoin -$ cd bitcoin/ -$ ./autogen.sh +$ mkdir BGL-honggfuzz-p2p/ +$ cd BGL-honggfuzz-p2p/ +$ git clone https://github.com/BitgesellOfficial/bitgesell.git +$ cd bitgesell/ $ git clone https://github.com/google/honggfuzz $ cd honggfuzz/ $ make $ cd .. -$ CC=$(pwd)/honggfuzz/hfuzz_cc/hfuzz-clang \ - CXX=$(pwd)/honggfuzz/hfuzz_cc/hfuzz-clang++ \ - ./configure --disable-wallet --with-gui=no \ - --with-sanitizers=address,undefined $ git apply << "EOF" diff --git a/src/compat/compat.h b/src/compat/compat.h index 8195bceaec..cce2b31ff0 100644 @@ -254,11 +270,17 @@ index 7601a6ea84..702d0f56ce 100644 SanitizeString(msg.m_type), msg.m_message_size, HexStr(Span{hash}.first(CMessageHeader::CHECKSUM_SIZE)), EOF -$ make -C src/ bitcoind +$ cmake -B build_fuzz \ + -DCMAKE_C_COMPILER="$(pwd)/honggfuzz/hfuzz_cc/hfuzz-clang" \ + -DCMAKE_CXX_COMPILER="$(pwd)/honggfuzz/hfuzz_cc/hfuzz-clang++" \ + -DENABLE_WALLET=OFF \ + -DBUILD_GUI=OFF \ + -DSANITIZERS=address,undefined +$ cmake --build build_fuzz --target BGLd $ mkdir -p inputs/ -$ honggfuzz/honggfuzz --exit_upon_crash --quiet --timeout 4 -n 1 -Q \ +$ ./honggfuzz/honggfuzz --exit_upon_crash --quiet --timeout 4 -n 1 -Q \ -E HFND_TCP_PORT=18444 -f inputs/ -- \ - src/BGLd -regtest -discover=0 -dns=0 -dnsseed=0 -listenonion=0 \ + build_fuzz/src/BGLd -regtest -discover=0 -dns=0 -dnsseed=0 -listenonion=0 \ -nodebuglogfile -bind=127.0.0.1:18444 -logthreadnames \ -debug ``` @@ -270,8 +292,8 @@ $ honggfuzz/honggfuzz --exit_upon_crash --quiet --timeout 4 -n 1 -Q \ To quickly get started fuzzing Bitcoin Core using [Eclipser v1.x](https://github.com/SoftSec-KAIST/Eclipser/tree/v1.x): ```sh -$ git clone https://github.com/bitcoin/bitcoin -$ cd bitcoin/ +$ git clone https://github.com/BitgesellOfficial/bitgesell.git +$ cd bitgesell/ $ sudo vim /etc/apt/sources.list # Uncomment the lines starting with 'deb-src'. $ sudo apt-get update $ sudo apt-get build-dep qemu @@ -298,11 +320,9 @@ $ cd Eclipser $ git checkout v1.x $ make $ cd .. -$ ./autogen.sh -$ ./configure --enable-fuzz -$ make +$ cmake -B build_fuzz -DBUILD_FOR_FUZZING=ON $ mkdir -p outputs/ -$ FUZZ=bech32 dotnet Eclipser/build/Eclipser.dll fuzz -p src/test/fuzz/fuzz -t 36000 -o outputs --src stdin +$ FUZZ=bech32 dotnet ./Eclipser/build/Eclipser.dll fuzz -p build_fuzz/src/test/fuzz/fuzz -t 36000 -o outputs --src stdin ``` This will perform 10 hours of fuzzing. @@ -320,7 +340,7 @@ Fuzzing with Eclipser will likely be much more effective if using an existing co ```sh $ git clone https://github.com/bitcoin-core/qa-assets -$ FUZZ=bech32 dotnet Eclipser/build/Eclipser.dll fuzz -p src/test/fuzz/fuzz -t 36000 -i qa-assets/fuzz_seed_corpus/bech32 outputs --src stdin +$ FUZZ=bech32 dotnet Eclipser/build/Eclipser.dll fuzz -p build_fuzz/src/test/fuzz/fuzz -t 36000 -i qa-assets/fuzz_seed_corpus/bech32 outputs --src stdin ``` Note that fuzzing with Eclipser on certain targets (those that create 'full nodes', e.g. `process_message*`) will, diff --git a/doc/productivity.md b/doc/productivity.md index 51df024e9b..5a928e4f81 100644 --- a/doc/productivity.md +++ b/doc/productivity.md @@ -6,8 +6,8 @@ Table of Contents * [General](#general) * [Cache compilations with `ccache`](#cache-compilations-with-ccache) - * [Disable features with `./configure`](#disable-features-with-configure) - * [Make use of your threads with `make -j`](#make-use-of-your-threads-with-make--j) + * [Disable features when generating the build system](#disable-features-when-generating-the-build-system) + * [Make use of your threads with `-j`](#make-use-of-your-threads-with--j) * [Only build what you need](#only-build-what-you-need) * [Compile on multiple machines](#compile-on-multiple-machines) * [Multiple working directories with `git worktrees`](#multiple-working-directories-with-git-worktrees) @@ -31,7 +31,7 @@ The easiest way to faster compile times is to cache compiles. `ccache` is a way > ccache is a compiler cache. It speeds up recompilation by caching the result of previous compilations and detecting when the same compilation is being done again. Supported languages are C, C++, Objective-C and Objective-C++. -Install `ccache` through your distribution's package manager, and run `./configure` with your normal flags to pick it up. +Install `ccache` through your distribution's package manager, and run `cmake -B build` with your normal configuration options to pick it up. To use ccache for all your C/C++ projects, follow the symlinks method [here](https://ccache.dev/manual/latest.html#_run_modes) to set it up. @@ -46,38 +46,32 @@ Note: base_dir is required for ccache to share cached compiles of the same file You _must not_ set base_dir to "/", or anywhere that contains system headers (according to the ccache docs). -### Disable features with `./configure` +### Disable features when generating the build system -After running `./autogen.sh`, which generates the `./configure` file, use `./configure --help` to identify features that you can disable to save on compilation time. A few common flags: +During the generation of the build system only essential build options are enabled by default to save on compilation time. -```sh ---without-miniupnpc ---without-natpmp ---disable-bench ---disable-wallet ---without-gui -``` +Run `cmake -B build -LH` to see the full list of available options. GUI tools, such as `ccmake` and `cmake-gui`, can be also helpful. -If you do need the wallet enabled, it is common for devs to add `--with-incompatible-bdb`. This uses your system bdb version for the wallet, so you don't have to find a copy of bdb 4.8. Wallets from such a build will be incompatible with any release binary (and vice versa), so use with caution on mainnet. +If you do need the wallet enabled (`-DENABLE_WALLET=ON`), it is common for devs to use your system bdb version for the wallet, so you don't have to find a copy of bdb 4.8. Wallets from such a build will be incompatible with any release binary (and vice versa), so use with caution on mainnet. -### Make use of your threads with `make -j` +### Make use of your threads with `-j` -If you have multiple threads on your machine, you can tell `make` to utilize all of them with: +If you have multiple threads on your machine, you can utilize all of them with: ```sh -make -j"$(($(nproc)+1))" +cmake --build build -j "$(($(nproc)+1))" ``` ### Only build what you need -When rebuilding during development, note that running `make`, without giving a target, will do a lot of work you probably don't need. It will build the GUI (unless you've disabled it) and all the tests (which take much longer to build than the app does). +When rebuilding during development, note that running `cmake --build build`, without giving a target, will do a lot of work you probably don't need. It will build the GUI (if you've enabled it) and all the tests (which take much longer to build than the app does). Obviously, it is important to build and run the tests at appropriate times -- but when you just want a quick compile to check your work, consider picking one or a set of build targets relevant to what you're working on, e.g.: ```sh -make src/BGLd src/BGL-cli -make src/qt/BGL-qt -make -C src BGL_bench +cmake --build build --target BGLd BGL-cli +cmake --build build --target BGL-qt +cmake --build build --target bench_BGL ``` (You can and should combine this with `-j`, as above, for a parallel build.) @@ -110,9 +104,9 @@ To squash in `git commit --fixup` commits without rebasing over an updated maste git rebase -i --autosquash "$(git merge-base master HEAD)" ``` -To execute `make check` on every commit since last diverged from master, but without rebasing over an updated master, we can do the following: +To execute `cmake --build build && ctest --test-dir build` on every commit since last diverged from master, but without rebasing over an updated master, we can do the following: ```sh -git rebase -i --exec "make check" "$(git merge-base master HEAD)" +git rebase -i --exec "cmake --build build && ctest --test-dir build" "$(git merge-base master HEAD)" ``` ----- diff --git a/doc/release-process.md b/doc/release-process.md index 43b79ba738..4f1f026aa5 100644 --- a/doc/release-process.md +++ b/doc/release-process.md @@ -6,14 +6,14 @@ We distinguish between two types of releases: *regular* and *maintenance* releas Regular releases are releases of a new major or minor version as well as patches of the most recent release. Maintenance releases, on the other hand, are required for patches of older releases. -* Update release candidate version in `configure.ac` (`CLIENT_VERSION_RC`). +* Update release candidate version in `CMakeLists.txt` (`CLIENT_VERSION_RC`). * Update manpages (after rebuilding the binaries), see [gen-manpages.py](https://github.com/bitcoin/bitcoin/blob/master/contrib/devtools/README.md#gen-manpagespy). * Update bitcoin.conf and commit changes if they exist, see [gen-bitcoin-conf.sh](https://github.com/bitcoin/bitcoin/blob/master/contrib/devtools/README.md#gen-bitcoin-confsh). This process also assumes that there will be no minor releases for old major releases. * Update [bips.md](bips.md) to account for changes since the last release. -* Update version in `configure.ac` (don't forget to set `CLIENT_VERSION_RC` to `0`). +* Update version in `CMakeLists.txt` (don't forget to set `CLIENT_VERSION_RC` to `0`). * Update manpages (see previous section) * Write release notes (see "Write the release notes" below) in doc/release-notes.md. If necessary, archive the previous release notes as doc/release-notes/release-notes-${VERSION}.md. @@ -23,8 +23,8 @@ Perform these checks when reviewing the release PR (see below): * On both the master branch and the new release branch: - - update `CLIENT_VERSION_MAJOR` in [`configure.ac`](../configure.ac) -* On the new release branch in [`configure.ac`](../configure.ac)(see [this commit](https://github.com/bitcoin/bitcoin/commit/742f7dd)): + - update `CLIENT_VERSION_MAJOR` in [`CMakeLists.txt`](../CMakeLists.txt) +* On the new release branch in [`CMakeLists.txt`](../CMakeLists.txt)(see [this commit](https://github.com/bitcoin/bitcoin/commit/742f7dd)): - set `CLIENT_VERSION_MINOR` to `0` - set `CLIENT_VERSION_BUILD` to `0` - set `CLIENT_VERSION_IS_RELEASE` to `true` diff --git a/src/qt/README.md b/src/qt/README.md index a2fd1192bf..ed15c9b47b 100644 --- a/src/qt/README.md +++ b/src/qt/README.md @@ -4,7 +4,7 @@ The current precise version for Qt 5 is specified in [qt.mk](/depends/packages/q ## Compile and run -See build instructions: [Unix](/doc/build-unix.md), [macOS](/doc/build-osx.md), [Windows](/doc/build-windows.md), [FreeBSD](/doc/build-freebsd.md), [NetBSD](/doc/build-netbsd.md), [OpenBSD](/doc/build-openbsd.md) +See build instructions: [Unix](/doc/build-unix.md), [macOS](/doc/build-osx.md), [Windows](/doc/build-windows-msvc.md), [FreeBSD](/doc/build-freebsd.md), [NetBSD](/doc/build-netbsd.md), [OpenBSD](/doc/build-openbsd.md) When following your systems build instructions, make sure to install the `Qt` dependencies. diff --git a/src/test/README.md b/src/test/README.md index eaff49970b..7530b79827 100644 --- a/src/test/README.md +++ b/src/test/README.md @@ -12,22 +12,23 @@ that runs all of the unit tests. The main source file for the test library is fo ### Compiling/running unit tests -Unit tests will be automatically compiled if dependencies were met in `./configure` +Unit tests will be automatically compiled if dependencies were met +during the generation of the Bitcoin Core build system and tests weren't explicitly disabled. -After configuring, they can be run with `make check`, which includes unit tests from -subtrees, or `make && make -C src check-unit` for just the unit tests. +Assuming the build directory is named `build`, the unit tests can be run +with `ctests --test-dir build`, which includes unit tests from subtrees. -To run the BGLd tests manually, launch `src/test/test_BGL`. To recompile -after a test file was modified, run `make` and then run the test again. If you -modify a non-test file, use `make -C src/test` to recompile only what's needed -to run the BGLd tests. +To run the unit tests manually, launch `build/src/test/test_BGL`. To recompile +after a test file was modified, run `cmake --build build` and then run the test again. If you +modify a non-test file, use `cmake --build build --target test_BGL` to recompile only what's needed +to run the unit tests. To add more BGLd tests, add `BOOST_AUTO_TEST_CASE` functions to the existing .cpp files in the `test/` directory or add new .cpp files that implement new `BOOST_AUTO_TEST_SUITE` sections. -To run the BGL-qt tests manually, launch `src/qt/test/test_BGL-qt` +To run the GUI unit tests manually, launch `build/src/qt/test/test_BGL-qt` To add more BGL-qt tests, add them to the `src/qt/test/` directory and the `src/qt/test/test_main.cpp` file. @@ -38,7 +39,7 @@ the `src/qt/test/test_main.cpp` file. For example, to run just the `getarg_tests` suite of tests: ```bash -test_BGL --log_level=all --run_test=getarg_tests +build/src/test/test_BGL --log_level=all --run_test=getarg_tests ``` `log_level` controls the verbosity of the test framework, which logs when a @@ -48,7 +49,7 @@ test case is entered, for example. `BGLd`. Use `--` to separate these sets of arguments: ```bash -test_BGL --log_level=all --run_test=getarg_tests -- -printtoconsole=1 +build/src/test/test_BGL --log_level=all --run_test=getarg_tests -- -printtoconsole=1 ``` The `-printtoconsole=1` after the two dashes sends debug logging, which @@ -58,7 +59,7 @@ standard terminal output. ... or to run just the doubledash test: ```bash -test_ --run_test=getarg_tests/doubledash +build/src/test/test_BGL --run_test=getarg_tests/doubledash ``` `test_BGL` creates a temporary working (data) directory with a randomly @@ -72,7 +73,7 @@ have a `debug.log` file, for example. The location of the temporary data directory can be specified with the `-testdatadir` option. This can make debugging easier. The directory path used is the argument path appended with -`/test_common_Bitcoin Core//datadir`. +`/test_common_BGL Core//datadir`. The directory path is created if necessary. Specifying this argument also causes the data directory not to be removed after the last test. This is useful for looking at @@ -81,8 +82,8 @@ what the test wrote to `debug.log` after it completes, for example. so no leftover state is used.) ```bash -$ test_bitcoin --run_test=getarg_tests/doubledash -- -testdatadir=/somewhere/mydatadir -Test directory (will not be deleted): "/somewhere/mydatadir/test_common_BGL Core/getarg_tests/doubledash/datadir +$ build/src/test/test_BGL --run_test=getarg_tests/doubledash -- -testdatadir=/somewhere/mydatadir +Test directory (will not be deleted): "/somewhere/mydatadir/test_common_Bitcoin Core/getarg_tests/doubledash/datadir Running 1 test case... *** No errors detected @@ -96,12 +97,13 @@ If you run an entire test suite, such as `--run_test=getarg_tests`, or all the t (by not specifying `--run_test`), a separate directory will be created for each individual test. -Run `test_bitcoin --help` for the full list of tests. +Run `test_BGL --help` for the full list of tests. ### Adding test cases -To add a new unit test file to our test suite you need -to add the file to `src/Makefile.test.include`. The pattern is to create +To add a new unit test file to our test suite, you need +to add the file to either `src/test/CMakeLists.txt` or +`src/wallet/test/CMakeLists.txt` for wallet-related tests. The pattern is to create one test file for each class or source file for which you want to create unit tests. The file naming convention is `_tests.cpp` and such files should wrap their tests in a test suite @@ -110,7 +112,8 @@ see `uint256_tests.cpp`. ### Logging and debugging in unit tests -`make check` will write to a log file `foo_tests.cpp.log` and display this file +`ctest --test-dir build` will write to a log file `build/Testing/Temporary/LastTest.log`. You can +additionaly use the `--output-on-failure` option to display logs of the failed tests automatically on failure. For running individual tests verbosely, refer to the section [above](#running-individual-tests). @@ -121,13 +124,13 @@ For debugging you can launch the `test_BGL` executable with `gdb` or `lldb` and start debugging, just like you would with any other program: ```bash -gdb src/test/test_BGL +gdb build/src/test/test_BGL ``` #### Segmentation faults If you hit a segmentation fault during a test run, you can diagnose where the fault -is happening by running `gdb ./src/test/test_BGL` and then using the `bt` command +is happening by running `gdb ./build/src/test/test_BGL` and then using the `bt` command within gdb. Another tool that can be used to resolve segmentation faults is @@ -145,7 +148,7 @@ Running the tests and hitting a segmentation fault should now produce a file cal You can then explore the core dump using ```bash -gdb src/test/test_BGL core +gdb build/src/test/test_BGL core (gbd) bt # produce a backtrace for where a segfault occurred ``` diff --git a/test/README.md b/test/README.md index 297f8daf09..2f5e1b5eb1 100644 --- a/test/README.md +++ b/test/README.md @@ -13,13 +13,15 @@ interfaces. - [util](/test/util) which tests the utilities (BGL-util, BGL-tx, ...). - [lint](/test/lint/) which perform various static analysis checks. -The util tests are run as part of `make check` target. The fuzz tests, functional +The util tests are run as part of `ctest` invocation. The fuzz tests, functional tests and lint scripts can be run as explained in the sections below. # Running tests locally Before tests can be run locally, BGL Core must be built. See the [building instructions](/doc#building) for help. +The following examples assume that the build directory is named `build`. + ## Fuzz tests See [/doc/fuzzing.md](/doc/fuzzing.md) @@ -45,19 +47,19 @@ set PYTHONUTF8=1 Individual tests can be run by directly calling the test script, e.g.: ``` -test/functional/feature_rbf.py +build/test/functional/feature_rbf.py ``` or can be run through the test_runner harness, eg: ``` -test/functional/test_runner.py feature_rbf.py +build/test/functional/test_runner.py feature_rbf.py ``` You can run any combination (incl. duplicates) of tests by calling: ``` -test/functional/test_runner.py ... +build/test/functional/test_runner.py ... ``` Wildcard test names can be passed, if the paths are coherent and the test runner @@ -65,34 +67,34 @@ is called from a `bash` shell or similar that does the globbing. For example, to run all the wallet tests: ``` -test/functional/test_runner.py test/functional/wallet* -functional/test_runner.py functional/wallet* (called from the test/ directory) -test_runner.py wallet* (called from the test/functional/ directory) +build/test/functional/test_runner.py test/functional/wallet* +functional/test_runner.py functional/wallet* # (called from the build/test/ directory) +test_runner.py wallet* # (called from the build/test/functional/ directory) ``` but not ``` -test/functional/test_runner.py wallet* +build/test/functional/test_runner.py wallet* ``` Combinations of wildcards can be passed: ``` -test/functional/test_runner.py ./test/functional/tool* test/functional/mempool* +build/test/functional/test_runner.py ./test/functional/tool* test/functional/mempool* test_runner.py tool* mempool* ``` Run the regression test suite with: ``` -test/functional/test_runner.py +build/test/functional/test_runner.py ``` Run all possible tests with ``` -test/functional/test_runner.py --extended +build/test/functional/test_runner.py --extended ``` In order to run backwards compatibility tests, first run: @@ -107,7 +109,7 @@ By default, up to 4 tests will be run in parallel by test_runner. To specify how many jobs to run, append `--jobs=n` The individual tests and the test_runner harness have many command-line -options. Run `test/functional/test_runner.py -h` to see them all. +options. Run `build/test/functional/test_runner.py -h` to see them all. #### Speed up test runs with a RAM disk @@ -130,7 +132,7 @@ For example running the test suite with `--jobs=100` might need a 4 GiB RAM disk To use, run the test suite specifying the RAM disk as the `cachedir` and `tmpdir`: ```bash -test/functional/test_runner.py --cachedir=/mnt/tmp/cache --tmpdir=/mnt/tmp +build/test/functional/test_runner.py --cachedir=/mnt/tmp/cache --tmpdir=/mnt/tmp ``` Once finished with the tests and the disk, and to free the RAM, simply unmount the disk: @@ -151,7 +153,7 @@ Configure the RAM disk size, expressed as the number of blocks, at the end of th (`4096 MiB * 2048 blocks/MiB = 8388608 blocks` for 4 GiB). To run the tests using the RAM disk: ```bash -test/functional/test_runner.py --cachedir=/Volumes/ramdisk/cache --tmpdir=/Volumes/ramdisk/tmp +build/test/functional/test_runner.py --cachedir=/Volumes/ramdisk/cache --tmpdir=/Volumes/ramdisk/tmp ``` To unmount: @@ -193,14 +195,14 @@ pkill -9 BGLd ##### Data directory cache A pre-mined blockchain with 200 blocks is generated the first time a -functional test is run and is stored in test/cache. This speeds up +functional test is run and is stored in build/test/cache. This speeds up test startup times since new blockchains don't need to be generated for each test. However, the cache may get into a bad state, in which case tests will fail. If this happens, remove the cache directory (and make sure BGLd processes are stopped as above): ```bash -rm -rf test/cache +rm -rf build/test/cache killall BGLd ``` @@ -236,7 +238,7 @@ aggregate log by running the `combine_logs.py` script. The output can be plain text, colorized text or html. For example: ``` -test/functional/combine_logs.py -c | less -r +build/test/functional/combine_logs.py -c | less -r ``` will pipe the colorized logs from the test into less. @@ -297,7 +299,7 @@ See this link for considerations: https://www.kernel.org/doc/Documentation/secur Often while debugging RPC calls in functional tests, the test might time out before the process can return a response. Use `--timeout-factor 0` to disable all RPC timeouts for that particular -functional test. Ex: `test/functional/wallet_hd.py --timeout-factor 0`. +functional test. Ex: `build/test/functional/wallet_hd.py --timeout-factor 0`. ##### Profiling @@ -321,7 +323,7 @@ For ways to generate more granular profiles, see the README in ### Util tests -Util tests can be run locally by running `test/util/test_runner.py`. +Util tests can be run locally by running `build/test/util/test_runner.py`. Use the `-v` option for verbose output. ### Lint tests