diff --git a/README.md b/README.md index 5f4289dcdd05..37b3260758ec 100644 --- a/README.md +++ b/README.md @@ -5,7 +5,7 @@ [![codecov](https://codecov.io/gh/hashgraph/hedera-services/graph/badge.svg?token=ZPMV8C93DV)](https://codecov.io/gh/hashgraph/hedera-services) [![Latest Version](https://img.shields.io/github/v/tag/hashgraph/hedera-services?sort=semver&label=version)](README.md) [![Made With](https://img.shields.io/badge/made_with-java-blue)](https://github.com/hashgraph/hedera-services/) -[![Development Branch](https://img.shields.io/badge/docs-quickstart-green.svg)](hedera-node/docs/gradle-quickstart.md) +[![Development Branch](https://img.shields.io/badge/docs-quickstart-green.svg)](docs/gradle-quickstart.md) [![License](https://img.shields.io/badge/license-apache2-blue.svg)](LICENSE) # Hedera Services @@ -14,27 +14,16 @@ Implementation of the Platform and the [services offered](https://github.com/has nodes in the [Hedera public network](https://hedera.com). ## Overview of child modules -* _platform-sdk/_ - the basic Platform. -* _hedera-node/_ - implementation of Hedera services on the Platform. +* _platform-sdk/_ - the basic Platform – [documentation](platform-sdk/docs/platformWiki.md) +* _hedera-node/_ - implementation of Hedera services on the Platform – [documentation](hedera-node/docs/) -## JVM -An [Eclipse Adoptium](https://adoptium.net/) build of the Java 21 JDK is required. If an Adoptium JDK is not installed, -the Gradle build will download an appropriate Adoptium JDK. The JDK version used to execute Gradle must be Java 21+ in -order for the `checkAllModuleInfo` task to succeed. -## Solidity -Hedera Contracts support `pragma solidity <=0.8.9`. - -## Docker Compose quickstart +## Getting Started -The [Docker quickstart](hedera-node/docs/docker-quickstart.md) covers how to -start a local network of Hedera Services nodes using Docker Compose. +Refer to the [Quickstart Guide](docs/README.md) for how to work with this project. -## Developer IntelliJ quickstart - -The [IntelliJ quickstart](hedera-node/docs/intellij-quickstart.md) covers how to -start a local network of Services nodes from IntelliJ for testing and -development. +## Solidity +Hedera Contracts support `pragma solidity <=0.8.9`. ## Support diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 000000000000..193e4b925a77 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,13 @@ +# Quickstart Guide + +## Preliminaries: JDK + +An [Eclipse Adoptium](https://adoptium.net/) build of the Java **21.0.1** JDK is required. + +## Building, Running and Working with the project + +- [Building, Testing and Running with Gradle](gradle-quickstart.md) +- [Developing with IntelliJ](intellij-quickstart.md) +- [Branch Naming Conventions](branch-naming-conventions.md) +- [Java Style Guide](hedera-java-style-guide.md) +- [Maintainers Guide](maintainers-guide.md) diff --git a/hedera-node/docs/assets/gitflow-branching-model.png b/docs/assets/gitflow-branching-model.png similarity index 100% rename from hedera-node/docs/assets/gitflow-branching-model.png rename to docs/assets/gitflow-branching-model.png diff --git a/docs/assets/gradle-jdk.png b/docs/assets/gradle-jdk.png new file mode 100644 index 000000000000..d15b49f7e4fa Binary files /dev/null and b/docs/assets/gradle-jdk.png differ diff --git a/docs/assets/gradle-reload.png b/docs/assets/gradle-reload.png new file mode 100644 index 000000000000..5424312ade10 Binary files /dev/null and b/docs/assets/gradle-reload.png differ diff --git a/docs/assets/gradle-tasks.png b/docs/assets/gradle-tasks.png new file mode 100644 index 000000000000..1a605f9c0ead Binary files /dev/null and b/docs/assets/gradle-tasks.png differ diff --git a/hedera-node/docs/assets/labels-on-issue.png b/docs/assets/labels-on-issue.png similarity index 100% rename from hedera-node/docs/assets/labels-on-issue.png rename to docs/assets/labels-on-issue.png diff --git a/hedera-node/docs/branch-naming-conventions.md b/docs/branch-naming-conventions.md similarity index 100% rename from hedera-node/docs/branch-naming-conventions.md rename to docs/branch-naming-conventions.md diff --git a/docs/gradle-quickstart.md b/docs/gradle-quickstart.md new file mode 100644 index 000000000000..d77a89b8ad89 --- /dev/null +++ b/docs/gradle-quickstart.md @@ -0,0 +1,174 @@ +# Gradle quickstart + +## Installation + +The repo contains a `gradlew` (or `gradlew.bat` on windows) script. This script will automatically download +the right version of Gradle for this project, scoped to this project. This means you never need to have Gradle installed +manually on your computer. It also means, as the project updates to newer versions of Gradle, you will pick them up +automatically. And, the version of Gradle is checked in, meaning you will always have the right version for whatever +commit you are building from. + +The only requirement to run Gradle is having a recent JDK installed. In order to ensure reproducible builds, this +project is configured to check the JDK you are currently using and fail if it does not correspond to the JDK you are +currently using. If you get an error, please download the indicated JDK and make sure the `java` command on your `PATH` +is of that JDK or that your `JAVA_HOME` is pointing at that JDK. + +## Building the project + +This documents explains how to use Gradle directly from the command line via the `./gradlew ` command. +All Gradle tasks can also be invoked from the Gradle view in [IntelliJ IDEA](intellij-quickstart.md). + +There are several Gradle tasks you can use. Most notably: + +- `./gradlew assemble` compile all code and create all Jar files +- `./gradlew qualityGate` in addition to the above, run all quality checks and auto-fix formatting where possible +- `./gradlew ::` run all tests in one module of the given [test type](#Testing). + +You may run `./gradlew` (without arguments) for a detailed overview + +## Running a services instance or example apps + +- `./gradlew :app:modrun` runs a services instance +- `./gradlew :test-clients:runTestClient -PtestClient=com.hedera.services.bdd.suites.crypto.HelloWorldSpec` +- `./gradlew :swirlds-platform-base-example:run` runs Platform-base Example App + +## Using Gradle during Development + +### Changing or adding Modules of Hedera (aka Gradle Subprojects) + +All modules are listed in [settings.gradle.kts](../settings.gradle.kts) using +`include(":", "")`. The `module-folder-path` should be a folder in a subdirectory like +[platform-sdk](../platform-sdk) or [hedera-node](../hedera-node). In the folder, the following files are expected: + +- `build.gradle.kts` specifies to which group of modules the module belongs, e.g. + `id("com.hedera.gradle.services")` or `id("com.hedera.gradle.platform")` and may contain + [dependency definitions](#changing-or-adding-dependencies) for tests. +- `src/main/java/module-info.java` is the Java Module specification that is also used to determine the + [dependencies of the module](#changing-or-adding-dependencies) by Gradle. Note that the last segment of the module name defined in + the `module-info.java` file needs to correspond to the name of the module defined in + [settings.gradle.kts](../settings.gradle.kts). + +### Changing or Adding Dependencies + +This project use of the _Java Module System (JPMS)_. With this, dependencies between modules are defined in the +`src/main/java/module-info.java` files that each module contains. Other modules are identified by their _Module Name_ +there. For example, a dependency to the `swirlds-logging` module is expressed by `requires com.swirlds.logging`. A +dependency to the 3rd party library `com.fasterxml.jackson.core` is expressed by `requires com.fasterxml.jackson.core`. +Note: This project utilizes the +[org.gradlex.java-module-dependencies](https://github.com/gradlex-org/java-module-dependencies) +plugin to achieve this integration between Gradle and the Java Module System. + +Each dependency definition contains a scope – e.g. `requires` or `requires transitive`. If you are unsure about a +scope, use `requires` when adding a dependency. Then execute `./gradlew qualityGate` which runs a dependency scope +check that analysis the code to determine which Java types are visible (and should be visible) to which modules. If +the check fails, it will advise you how to change the scope. + +### Adding or Changing the Version of a 3rd party dependency + +If you use a 3rd party module lke `com.fasterxml.jackson.core`, a version for that module needs to be selected. +For this, the [hedera-dependency-versions/build.gradle.kts](../hedera-dependency-versions/build.gradle.kts) defines a so-called _Gradle platform_ (also called BOM) +that contains the versions of all 3rd party modules used. If you want to upgrade the version of a module, do this here. +Remember to run `./gradlew qualityGate` after the change. If you need to use a new 3rd party module in a +`src/main/java/module-info.java` file, you need to add the version here. (If the new module is not completely Java +Module System compatible, you may also need to add [patching rules](#patching-3rd-party-modules)). + +### Patching 3rd Party Modules + +Some 3rd party libraries we use are not yet fully Java Module System compatible. And some modules pull in other +dependencies that we can neglect. Situations like this are treated as wrong/incomplete metadata in our Gradle +setup and the file +[com.hedera.gradle.jpms-modules.gradle.kts](../gradle/plugins/src/main/kotlin/com.hedera.gradle.jpms-modules.gradle.kts) +contains the rules to adjust or extend the metadata of 3rd party libraries to address such problems. + +Note: This project utilizes the +[org.gradlex.extra-java-module-info](https://github.com/gradlex-org/extra-java-module-info) +and +[org.gradlex.jvm-dependency-conflict-resolution](https://gradlex.org/jvm-dependency-conflict-resolution/#resolution-plugin) +plugins to ease the definition of patching rules. + +### Incrementing the Version of Hedera itself + +Our Gradle build has a single version number for all modules. It is defined in [version.txt](../version.txt). +Changing this version number will automatically apply to every module. + +### Testing + +We have different types of tests, defined in different folders – so-called _source sets_ – in each module. + +#### Unit Tests + +- located in `src/main/test` of the corresponding module +- run with `./gradlew ::test` + +Unit tests will **always** be executed in PR builds and must pass before merging. The vast majority of +our tests should be unit tests (measured in the 10's of thousands). These tests are never flaky and should avoid +arbitrary waits and timeouts at all costs. The full body of unit tests should execute in roughly 5 minutes. + +#### Integration Tests + +- located in `src/main/itest` of the corresponding module +- run with `./gradlew ::itest` + +We define integration tests as those that involve several components, but not an entire working instance. These use +JUnit. Integration tests take longer to execute than unit tests. These should be the second most plentiful type of test. +They are designed to ensure two or more components work together. We recommend +using [Testcontainers](https://www.testcontainers.org/) for databases, mirror nodes, explorers, or other components that +live in different repos. These tests should be written carefully to avoid flakiness. If a test fails, it should +**always** mean that there is a real problem. Per module or subproject, integration tests should take no more than 10 +minutes to execute. Across the entire repo, there should be thousands of integration tests. + +Integration tests must **all pass** before merging to **main**, so they must be fast and reliable. + +#### Hammer Tests + +- located in `src/main/hammer` of the corresponding module +- run with `./gradlew ::hammer` + +A hammer test is a unit test that "hammers" the code. A more common and less visceral name for this type of test is a +"fuzzing" test. These usually take the form of pseudo-random tests that run for an extended period of time and attempt +to use a component in as many ways as possible. + +Hammer tests by their nature take longer to execute. These are run on a nightly basis. They have concrete pass/fail +behavior. If any hammer test fails, this should mean there is **definitely** a bug that needs to be triaged. + +#### Micro-benchmarks + +- located in `src/main/jmh` of the corresponding module +- run with `./gradlew ::jmh` + +Micro-benchmarks are like the unit-tests of performance testing. They should be used liberally for establishing +metric-driven decisions about different designs. The specific numbers produced by a microbenchmark are not themselves +very useful because different hardware under different conditions can give different numbers. But they are useful when +comparing A/B implementations on the same hardware. These tests also take a significant amount of time to execute, +and are not very good at giving pass/fail criteria after execution. + +Rather, micro-benchmarks exist to help developers verify the impact of their changes in a particular part of the system. +Appropriate benchmarks should be run prior to creation of a PR. These are run nightly, and we record the results, so +we can do trend analysis over time. + +We use the [Java Micro-benchmarking Harness](https://github.com/openjdk/jmh), or JMH, for writing and executing our +micro-benchmarks. + +#### End-to-End Tests + +- located in `src/main/eet` of the `test-clients` module +- run with `./gradlew :test-clients:eet` + +End-to-end tests need a running instance. You can start one in one terminal with `./gradlew modrun` and then execute +the tests from a second terminal with `./gradlew eet`. Or you can use JRS to start an instance, or use some existing +environment. + +### Cleaning + +Gradle projects put all build artifacts into `build` directories. To clean your workspace of all these build artifacts, +use `./gradlew clean`. Note: cleaning is not necessary to get correct built results. You only need to do it if you want +to free disc space. + +## Changing details in the Gradle setup + +Generally, Gradle is configured through so-called Gradle _convention plugins_. A convention plugin is a plugin that +applies a certain set of defaults to all builds that include that convention. We define one such plugins in +[gradle/plugins/src/main/kotlin](../gradle/plugins/src/main/kotlin) using Gradle's Kotlin DSL notation. +If you need to adjust something in the build itself, this is the places where all configuration is located. +For details, see comments in the existing convention plugins (`*.gradle.kts` files). + diff --git a/hedera-node/docs/hedera-java-style-guide.md b/docs/hedera-java-style-guide.md similarity index 100% rename from hedera-node/docs/hedera-java-style-guide.md rename to docs/hedera-java-style-guide.md diff --git a/hedera-node/docs/intellij-quickstart.md b/docs/intellij-quickstart.md similarity index 57% rename from hedera-node/docs/intellij-quickstart.md rename to docs/intellij-quickstart.md index 922149e4c1d6..5bf7dd39072a 100644 --- a/hedera-node/docs/intellij-quickstart.md +++ b/docs/intellij-quickstart.md @@ -1,10 +1,5 @@ # IntelliJ quickstart -## JVM - -OpenJDK17 is strongly recommended. You can [download it from IntelliJ](https://www.jetbrains.com/help/idea/sdk.html) -if you don't have it already. - ## Preliminaries Clone this repository: @@ -15,63 +10,78 @@ git clone https://github.com/hashgraph/hedera-services.git From IntelliJ, choose `File -> Open` the _hedera-services/_ directory you just cloned. -Make sure you are using JDK17 as the project SDK: - -

- -

+### (optional) IntelliJ plugins -Run the Gradle `assemble` task in the root project (for example, via the IntelliJ Gradle tool window). +The following plugins add comfort features for working with certain parts of the code base: -This will both, -
    -
  1. Build the hedera-node/data/apps/HederaNode.jar; and, -
  2. Populate your hedera-node/data/config directory with - dev-appropriate versions of the application.properties, - api-permission.properties, and node.properties - files that are used to bootstrap a new Hedera Services network. - (To read more about how properties are sourced in Services, - please see [here](./services-configuration.md).) -
+- [Android IntelliJ plugin](https://plugins.jetbrains.com/plugin/22989-android) + adds additional code navigation capabilities for Dagger2 annotated interfaces. +- [JMH IntelliJ plugin](https://plugins.jetbrains.com/plugin/7529-jmh-java-microbenchmark-harness) + allows running selected JMH benchmarks directly from the IDE. -## Starting a local single-node network +## Configure the JDK used by Gradle -Now browse to `com.hedera.node.app.ServicesMain`. Its -`main` method starts a single node network of Hedera Service by -calling `com.swirlds.platform.Browser#main`, which is the -entrypoint to bootstrap the Platform app named by the -[_config.txt_](../hedera-node/config.txt) in the working -directory. +The project is imported as a Gradle project. Before you can use all features reliably, make sure that Gradle is +started with the JDK we use in the project, which currently is: +**Eclipse Temurin, 21.0.1** -Run `ServicesMain#main` with an IntelliJ configuration whose working -directory is the _hedera-node/_ directory of your clone of this repo: +You can use IntelliJ to download the JDK if you do not have it installed.

- +

-You can also run from Gradle below: + +## Reload Project with Gradle + +After everything is configured, and everytime you change something in the project setup, you should press the +**Reload All Gradle project** in IntelliJ. + +Changes to the project setup include, +- Changing `build.gradle.kts` files +- Changing dependencies in `src/main/java/module-info.java` files +- Changing build configuration in `gradle/plugins/src/main/kotlin`

- +

-You will see a single black pane appear, similar to: +See [gradle-quickstart](gradle-quickstart.md) for more details on modifying the project setup. + +## Build, test, run through Gradle tasks + +You can run all tasks described in [gradle-quickstart](gradle-quickstart.md) from the Gradle tool window.

- +

-This node's name is "Alice" because of [Line 26](../hedera-node/config.txt#L26) +## Running a services instance or example apps + +### Starting a local single-node network + +You can start the `ServicesMain` process via the `modrun` Gradle task. You will see an output like the following: + +``` +2024-04-25 11:00:26.066 INFO 169 NettyGrpcServerManager - Starting gRPC server on port 50211 +2024-04-25 11:00:26.118 INFO 292 NettyGrpcServerManager - Epoll not available, using NIO +2024-04-25 11:00:26.172 INFO 172 NettyGrpcServerManager - gRPC server listening on port 50211 +2024-04-25 11:00:26.172 INFO 180 NettyGrpcServerManager - Starting TLS gRPC server on port 50212 +2024-04-25 11:00:26.172 INFO 292 NettyGrpcServerManager - Epoll not available, using NIO +2024-04-25 11:00:26.172 WARN 325 NettyGrpcServerManager - Specified TLS cert 'hedera.crt' doesn't exist! +2024-04-25 11:00:26.173 WARN 187 NettyGrpcServerManager - Could not start TLS server, will continue without it: hedera.crt +``` + +This node's name is "Alice" because of [Line 25](../hedera-node/config.txt#L25) in the _config.txt_ present in your working directory. Looking closer at _config.txt_, you can see you are running Hedera Services -(and not some other app) because [Line 12](../hedera-node/config.txt#L12) +(and not some other app) because [Line 11](../hedera-node/config.txt#L11) points to the JAR file you just built; and there are three nodes in your network because you specified "Bob" and "Carol" as well as "Alice". If multiple nodes Alice, Bob, and Carol are set up to run locally by -uncommenting lines [27](../hedera-node/config.txt#L27) and [29](../hedera-node/config.txt#L29), +uncommenting lines [26](../hedera-node/config.txt#L26) and [28](../hedera-node/config.txt#L28), they will all be running on your local machine; and communicating via the loopback interface. But each still has a private instance of the Platform, and keeps its own state, just as it would in a @@ -90,19 +100,18 @@ Public: 0aa8e21064c61eab86e2a9c164565b4e7a9a4146106e0a6cd03a8c395a110e92 Private: 91132178e72057a1d7528025956fe39b0b847f200ab59b2fdd367017f3087137 ``` -## Submitting transactions to your local network +### Submitting transactions to your local network The _test-clients/_ directory in this repo contains a large number of end-to-end tests that Hedera engineering uses to validate the behavior of Hedera Services. Many of these tests are written in the style of a BDD specification. -Run `HelloWorldSpec#main` with an IntelliJ configuration whose working -directory is the _test-clients/_ directory of your clone of this repo: +Run `HelloWorldSpec` with the following Gradle command: -

- -

+``` +./gradlew runTestClient -PtestClient=com.hedera.services.bdd.suites.crypto.HelloWorldSpec +``` Because [`node=localhost`](../test-clients/src/main/resource/spec-default.properties) in the _spec-default.properties_ controlling the `HelloWorldSpec` test, this @@ -123,9 +132,9 @@ will run against your local network, culminating in logs similar to: keypair via its configuration in [_spec-default.properties_](../test-clients/src/main/resource/spec-default.properties) under the `startupAccounts.path` key). -## Stopping/restarting the network +### Stopping/restarting the network -Stop the `ServicesMain` process in IntelliJ to shut down the network. +Stop the `ServicesMain` process by stopping the `modrun` Gradle run. When you restart `ServicesMain`, the nodes will attempt to restore their state from the _hedera-node/data/saved_ directory tree. diff --git a/hedera-node/docs/maintainers-guide.md b/docs/maintainers-guide.md similarity index 87% rename from hedera-node/docs/maintainers-guide.md rename to docs/maintainers-guide.md index e86e4f6075b8..f5f66c8da1e6 100644 --- a/hedera-node/docs/maintainers-guide.md +++ b/docs/maintainers-guide.md @@ -2,38 +2,25 @@ This document outlines the process and keynotes for the core maintainers of this repository . ## IntelliJ set up + IntelliJ is used for most of the development lifecycle. Install IntelliJ using the [Jetbrains toolbox](https://www.jetbrains.com/lp/toolbox/), instead of installing directly. If you are on an M1/M2 chipset, select **.dmg (macOs Apple Silicon)**. Once you have the jetbrains toolbox installed, open it and Install **Intellij IDEA Ultimate**. -## JDK -Download JDK-17.0.3 for mac [here](https://adoptium.net/temurin/releases/). - ## Cloning repository + Clone this repository using: ``` git clone git@github.com:hashgraph/hedera-services.git ``` -From IntelliJ, choose `File -> Open` the _hedera-services/_ directory you just cloned. -Make sure you are using JDK17 as the project SDK in IntelliJ. - -

- -

- -Follow [intellij-quickstart](intellij-quickstart.md) guide to run network on your local machine and submit transactions. - -## Gradle -Once the repository is opened in IntelliJ, to build the project from Intellij, open the Gradle tool window and -run `Tasks/build/assemble` to on the root project. If you are using command line use `./gradlew assemble`. +## Work with the project -Start using either the Gradle command line -`(./gradlew spotlessApply)` or set up the [Google Java Format IntelliJ Plugin](https://github.com/google/google-java-format#intellij-android-studio-and-other-jetbrains-ides) -in IntelliJ to format your code to follow Google Code Style. +Follow the [gradle-quickstart.md](gradle-quickstart.md) and [intellij-quickstart.md](intellij-quickstart.md). ## GPG set up + Every commit being pushed to the repository should be verified and [signed off](#dco-sign-off). So it is important to set up GPG keys before contributing to the repository. @@ -52,6 +39,7 @@ Use the following tutorials to set up a GPG key. - [IntelliJ Official YouTube - GPG Commit Signing (10:59)](https://youtu.be/RBhz-8fZN9A?t=659) ## Development Model + The Git Source Code Management(SCM) system treats branches as one of the fundamental constructs for supporting concurrent development and assisting with merge conflict resolution. Due to the fundamental nature and flexibility offered by Git branches, it is critical to have a consistent branching workflow in order to reduce merge conflicts and @@ -75,6 +63,7 @@ Note especially the roles of the `main` and `develop` branches: - `main` is a tightly-controlled branch that release engineering uses for final tags deployed to production. ### Creating issues on GitHub + GitHub's [issues](https://github.com/hashgraph/hedera-services/issues) are used as the primary method for tracking project changes. Any actionable item that need to be addressed, should be associated to an issue in GitHub. @@ -109,18 +98,21 @@ The release engineering team will handle the following: ### User Stories #### As a developer, I would like to create a branch to work on the feature for the upcoming release + As per the development model, every developer should create a feature branch from `develop` branch for working on a change targeted for the current release. The created branch should follow [naming conventions](branch-naming-conventions.md). The `develop` branch should be up-to-date with all the features going into the next release. #### As a developer, I would like to create a branch to work on the feature NOT targeted for upcoming release + As per the development model, every developer should create a feature branch to work from `develop` branch. The created branch should follow [naming conventions](branch-naming-conventions.md). But, the feature branch should NOT be merged into `develop` until the decision is made if the feature is going into upcoming release. #### As a developer, I would like to merge my feature branch or bug fix for the upcoming release + Open a pull request (PR) from the feature branch to `develop` branch and add `hashgraph/hedera-services-team` as reviewers. Also add the following labels on the PR : @@ -133,14 +125,14 @@ PR should be merged after an approving review and all the checks are passed. NOTE: 1. Any feature that is not going into the upcoming release should stay in the feature branch and should not be merged to `develop`. -2. Please use either the Gradle command line`(./gradlew spotlessApply)` or the [Google Java Format IntelliJ Plugin](https://github.com/google/google-java-format#intellij-android-studio-and-other-jetbrains-ides) +2. Please use either the Gradle command line `./gradlew qualityGate` or the [Google Java Format IntelliJ Plugin](https://github.com/google/google-java-format#intellij-android-studio-and-other-jetbrains-ides) to format your code to avoid failing checks in CI pipeline. 3. The linked issues should be automatically closed when a PR is merged or closed manually. #### As a developer, I would like to merge a bugfix/hotfix after release branch is created Once the release branch is created, only bugfixes or hotfixes should be merged into release branch. To do that, create -a `hotfix` from the `release` branch. The created branch should follow [naming conventions](#naming-convention-for-branches). +a `hotfix` from the `release` branch. The created branch should follow [naming conventions](branch-naming-conventions.md). Once the fix is in the branch, open a PR to the release branch. Once the fix is merged into `release` branch, it should be cherry-picked into the `develop` branch. @@ -151,6 +143,7 @@ the branch, create a PR targeting to `main`. Once bugfix is merged into `main`an back into the current `release` branch(if the release branch is still open), and also into `develop`. ### DCO Sign Off + The sign-off is a simple line at the end of a commit message. All commits need to be signed. Your signature certifies that you wrote the code or otherwise have the right to contribute the material. First, read the [Developer Certificate of Origin (DCO)](https://developercertificate.org/) to fully understand its terms. diff --git a/hedera-node/docs/assets/gradle-build.gif b/hedera-node/docs/assets/gradle-build.gif deleted file mode 100644 index c3d72e3e8144..000000000000 Binary files a/hedera-node/docs/assets/gradle-build.gif and /dev/null differ diff --git a/hedera-node/docs/assets/gradle-eet.gif b/hedera-node/docs/assets/gradle-eet.gif deleted file mode 100644 index ff9f2c6258df..000000000000 Binary files a/hedera-node/docs/assets/gradle-eet.gif and /dev/null differ diff --git a/hedera-node/docs/assets/gradle-run.gif b/hedera-node/docs/assets/gradle-run.gif deleted file mode 100644 index 5e9f8d77a47d..000000000000 Binary files a/hedera-node/docs/assets/gradle-run.gif and /dev/null differ diff --git a/hedera-node/docs/assets/gradle-run.png b/hedera-node/docs/assets/gradle-run.png deleted file mode 100644 index b53574bb6587..000000000000 Binary files a/hedera-node/docs/assets/gradle-run.png and /dev/null differ diff --git a/hedera-node/docs/assets/jdk-17.png b/hedera-node/docs/assets/jdk-17.png deleted file mode 100644 index 7e8341870b33..000000000000 Binary files a/hedera-node/docs/assets/jdk-17.png and /dev/null differ diff --git a/hedera-node/docs/assets/node-config.png b/hedera-node/docs/assets/node-config.png deleted file mode 100644 index 35fc6acb8635..000000000000 Binary files a/hedera-node/docs/assets/node-config.png and /dev/null differ diff --git a/hedera-node/docs/assets/node-configuration.png b/hedera-node/docs/assets/node-configuration.png deleted file mode 100644 index 7d2870348633..000000000000 Binary files a/hedera-node/docs/assets/node-configuration.png and /dev/null differ diff --git a/hedera-node/docs/assets/node-startup.png b/hedera-node/docs/assets/node-startup.png deleted file mode 100644 index 497bba427569..000000000000 Binary files a/hedera-node/docs/assets/node-startup.png and /dev/null differ diff --git a/hedera-node/docs/assets/spec-configuration.png b/hedera-node/docs/assets/spec-configuration.png deleted file mode 100644 index c309d8dfb7d2..000000000000 Binary files a/hedera-node/docs/assets/spec-configuration.png and /dev/null differ diff --git a/hedera-node/docs/design/modules.md b/hedera-node/docs/design/modules.md index 7efa6e7c3ee8..df0b667b1a1d 100644 --- a/hedera-node/docs/design/modules.md +++ b/hedera-node/docs/design/modules.md @@ -12,20 +12,20 @@ Java modules. Each module needs a `build.gradle.kts` file that describes the module. General best practices for all our (Java) modules are defined in custom plugins that can be found -under `build-logic/src/main/kotlin`. For a Java module the `com.hedera.hashgraph.javaConventions` plugin should be used. +under `gradle/plugins/src/main/kotlin`. For a Java module the `com.hedera.gradle.java` plugin should be used. Next to this each module should have a description. Since nothing else is needed for a minimal module the most simple `build.gradle.kts` looks like this: ``` plugins { -id("com.hedera.hashgraph.javaConventions") + id("com.hedera.gradle.java") } description = "A minimal module without any dependecies" ``` -The `group`, `name` and `version` of the module should not be added here. The `group` is the same for all modules on the -repository (`com.hedera.hashgraph`) and its definition can be found in the `com.hedera.hashgraph.conventions` plugin. +The `group` and `version` of the module should not be added here. The `group` is the same for all modules on the +repository (`com.hedera.hashgraph`) and its definition can be found in the `com.hedera.gradle.conventions` plugin. For the `version` we use a global definition, too. The current `version` is defined in the `gradle.properties` file in the root folder of the project. The `name` of a module is simple created based on the folder name of the module. @@ -45,14 +45,14 @@ specific path below the `src/main/resources/com/hedera/node/app/services/foo` fo ## Tests -All modules can have different types of tests. The `com.hedera.hashgraph.javaConventions` plugin provides direct support +All modules can have different types of tests. The `com.hedera.gradle.java` plugin provides direct support for unit test, integration tests, and end-to-end tests. Next to this the test fixtures functionality of Gradle is supported. ### Test fixtures To create clean and readable unit tests it is best practice to provide common functionality for tests in the test -fixtures of a module. Based on the `com.hedera.hashgraph.javaConventions` plugin test fixtures are supported for all +fixtures of a module. Based on the `com.hedera.gradle.java` plugin test fixtures are supported for all Java modules. All Java sources for the test fixtures must be placed under `src/testFixtures/java`. Additional resources that should be diff --git a/hedera-node/docs/design/service-modules.md b/hedera-node/docs/design/service-modules.md index 9c013600ae47..a69ac066510a 100644 --- a/hedera-node/docs/design/service-modules.md +++ b/hedera-node/docs/design/service-modules.md @@ -45,7 +45,7 @@ Next to this the `build.gradle.kts` file should look like this: ``` plugins { - id("com.hedera.hashgraph.conventions") + id("com.hedera.gradle.services") } description = "Hedera Foo Service API" @@ -130,7 +130,7 @@ Based on the given definitions and constrains a minimalistic `build.gradle.kts` ``` plugins { - id("com.hedera.hashgraph.conventions") + id("com.hedera.gradle.services") } description = "Default Hedera Foo Service Implementation" diff --git a/hedera-node/docs/gradle-quickstart.md b/hedera-node/docs/gradle-quickstart.md deleted file mode 100644 index 3d4ae5481762..000000000000 --- a/hedera-node/docs/gradle-quickstart.md +++ /dev/null @@ -1,129 +0,0 @@ -# Gradle quickstart - -## Installation - -The repo contains a shell script called gradlew (or gradlew.bat on windows). This script will automatically download the -right version of Gradle for this project, scoped to this project. This means you never need to have Gradle installed -manually on your computer. It also means, as the project updates to newer versions of Gradle, you will pick them up -automatically. And, the version of Gradle is checked in, meaning you will always have the right version for whatever -commit you are building from. - -Further, Gradle has the concept of a "toolchain". In our case, this toolchain is Java. Each individual developer needs -some semi-recent version of Java installed for running Gradle itself, but within the Gradle build it defines what -version of Java to download and use for building the project (compile, test, etc). This means you do not have to have a -specific version of Java pre-installed on your machine, but will always get the right version for the commit/branch you -are working on. - -## Libraries and Dependencies - -The `settings.gradle.kts` file in the root of the repo defines the "library catalog" -- the set of libraries from which -subprojects should select. By rule (which is not enforceable in Gradle itself), we should never declare a dependency for -the first time in a subproject. Instead, we first define the library, its version, and any library bundles in -`settings.gradle.kts`. Each subproject then declares its dependencies based on those libraries. In this way, we have a -single master-list of all libraries that have been approved for the project, including their versions. - -## Versions - -Our Gradle build has a single version number for all projects. It is defined in gradle.properties. Changing this version -number will automatically apply to every subproject. - -## build-logic - -Gradle has plugins. One type of plugin is the "convention plugin". A convention plugin is a plugin that applies a -certain set of defaults to all builds that include that convention. We define one such `hedera-convention` in build-logic. -It is then used by each of the subprojects to reduce the amount of boilerplate. We can create additional conventions in -the future if need be. 'build-logic' is a special directory in Gradle for hosting custom project plugins. - -## Sub Projects - -Each subproject has its own `build.gradle.kts` where the dependencies for that project are defined. Some subprojects ( -like `hedera-node`) have additional build logic. - -## Usage - -### Running an Instance - -To run an instance, you can use `./gradlew run`. It will compile (but not test) if needed before running the project. -This is the simplest way to get started. Try it out! After you finish the run, you can clean up all the files produced -by the run with the `./gradlew cleanRun` command. - -![./gradlew run](assets/gradle-run.gif) - -### Basic Build & Unit Test - -To build, simply `./gradlew build`. This is the most comprehensive task. It will compile, assemble, and test the -project. If you only want to test a specific subproject, you can do `./gradlew hedera-node:build` for example, or if you -are in the hedera-build subdirectory you can use `../gradlew build`. Gradle is smart and builds everything your project -depends on before building your project, but doesn't rebuild anything it doesn't need to. You can use -`./gradlew assemble` if you just want to compile and not test. The `test` task is for running unit tests. - -![./gradlew build](assets/gradle-build.gif) - -### Cleaning - -Gradle projects put all build artifacts into `build` directories (whereas Maven put things into `target`). To clean -your workspace of all these build artifacts, use `./gradlew clean`. - -### Testing - -#### Unit Tests - -The Gradle build supports our different types of tests, including unit, integration, hammer, micro-benchmark, and -end-to-end. You can use `./gradlew test` if you want to test (it will also compile things if needed). The `test` task is -for unit tests. Each subproject also supports integration tests. Unit tests will **always** be executed and must pass -before merging into **main**. The vast majority of our tests should be unit tests (measured in the 10's of thousands). -These tests are never flaky and should avoid arbitrary waits and timeouts at all costs. The full body of unit tests -should execute in roughly 5 minutes. - -#### Integration Tests - -We define integration tests as those that involve several components, but not an entire working instance. These use -JUnit. Integration tests take longer to execute than unit tests. These should be the second most plentiful type of test. -They are designed to ensure two or more components work together. We recommend -using [Testcontainers](https://www.testcontainers.org/) for databases, mirror nodes, explorers, or other components that -live in different repos. These tests should be written carefully to avoid flakiness. If a test fails, it should -**always** mean that there is a real problem. Per module or subproject, integration tests should take no more than 10 -minutes to execute. Across the entire repo, there should be thousands of integration tests. - -Integration tests must **all pass** before merging to **main**, so they must be fast and reliable. - -Integration tests can be executed with the `itest` task. You can run all integration tests by `./gradlew itest` or run a -specific module's integration tests by scoping it like `./gradlew hedera-node:itest`. The source code for integration -tests are found in the `src/itest/java` and `src/itest/resources` directories within the subproject. - -#### Hammer Tests - -Each subproject also supports "hammer" tests. A hammer test is a unit test that "hammers" the code. A more common and -less visceral name for this type of test is a "fuzzing" test. These usually take the form of pseudo-random tests that -run for an extended period of time and attempt to use a component in as many ways as possible. These can be run for all -subprojects with `./gradlew hammer` or run for a specific subproject. Hammer tests are JUnit tests and can be found -in `src/hammer/java` and `src/hammer/resources`. - -Hammer tests by their nature take longer to execute. These are run on a nightly basis. They have concrete pass/fail -behavior. If any hammer test fails, this should mean there is **definitely** a bug that needs to be triaged. - -#### Micro-benchmarks - -Micro-benchmarks are like the unit-tests of performance testing. They should be used liberally for establishing -metric-driven decisions about different designs. The specific numbers produced by a microbenchmark are not themselves -very useful because different hardware under different conditions can give different numbers. But they are useful when -comparing A/B implementations on the same hardware. These tests also take a significant amount of time to execute, -and are not very good at giving pass/fail criteria after execution. - -Rather, micro-benchmarks exist to help developers verify the impact of their changes in a particular part of the system. -Appropriate benchmarks should be run prior to creation of a PR. These are run nightly, and we record the results, so -we can do trend analysis over time. - -We use the [Java Micro-benchmarking Harness](https://github.com/openjdk/jmh), or JMH, for writing and executing our -micro-benchmarks. These benchmarks are found with subprojects in `src/jmh/java`. They can be run simply by using -`./gradlew jmh`. But if you do run it this way, be prepared for a long wait while all benchmarks are executed! -You may rather want to supply an "includes" flag to only include the benchmark of your choosing. See the docs -for the [JMH plugin](https://github.com/melix/jmh-gradle-plugin) for details. - -#### End-to-End Tests - -Finally, end-to-end tests (defined in `test-clients`) can be run with `./gradlew eet`. These tests need a running -instance. You can start one in one terminal with `./gradlew run` and then execute the tests from a second terminal with -`./gradlew eet`. Or you can use JRS to start an instance, or use some existing environment. - -![./gradlew eet](assets/gradle-eet.gif) diff --git a/hedera-node/test-clients/build.gradle.kts b/hedera-node/test-clients/build.gradle.kts index 9a45f02f5588..83cd6f6db5f1 100644 --- a/hedera-node/test-clients/build.gradle.kts +++ b/hedera-node/test-clients/build.gradle.kts @@ -59,6 +59,14 @@ sourceSets { create("yahcli") } +tasks.register("runTestClient") { + group = "build" + description = "Run a test client via -PtestClient=" + + classpath = sourceSets.main.get().runtimeClasspath + files(tasks.jar) + mainClass = providers.gradleProperty("testClient") +} + val ciCheckTagExpressions = mapOf( "hapiTestCrypto" to "CRYPTO",