merge main into verkle (#7928)

Signed-off-by: Karim Taam <[email protected]>

.github/ISSUE_TEMPLATE/bug-report.md

@@ -0,0 +1,50 @@
+---
+name: Bug Report
+about: necessary information to help us resolve the issue
+title: ''
+labels: 'bug'
+assignees: ''
+
+---
+
+<!-- Have you done the following? -->
+<!--   * read the Code of Conduct? By filing an Issue, you are expected to -->  
+<!--     comply with it, including treating everyone with respect: -->
+<!--     https://github.com/hyperledger/besu/blob/main/CODE_OF_CONDUCT.md -->
+<!--   * Reproduced the issue in the latest version of the software -->
+<!--   * Read the debugging docs: https://besu.hyperledger.org/private-networks/how-to -->
+<!--   * Duplicate Issue check:  https://github.com/search?q=+is%3Aissue+repo%3Ahyperledger/Besu -->
+
+### Steps to Reproduce
+1. [Step 1]
+2. [Step 2]
+3. [Step ...]
+
+**Expected behavior:** [What you expect to happen]
+
+**Actual behavior:** [What actually happens]
+
+**Frequency:** [What percentage of the time does it occur?]
+
+### Logs
+Please post relevant logs from Besu (and the consensus client, if running proof of stake) from before and after the issue.
+
+### Versions (Add all that apply)
+* Software version: [`besu --version`]
+* Java version: [`java -version`]
+* OS Name & Version: [`cat /etc/*release`]
+* Kernel Version: [`uname -a`]
+* Virtual Machine software & version: [`vmware -v`]
+* Docker Version: [`docker version`]
+* Cloud VM, type, size: [Amazon Web Services I3-large]
+* Consensus Client & Version if using Proof of Stake: [e.g. Teku, Lighthouse, Prysm, Nimbus, Lodestar]
+
+### Smart contract information (If you're reporting an issue arising from deploying or calling a smart contract, please supply related information)
+* Solidity version [`solc --version`]
+* Repo with minimal set of deployable/reproducible contract code - please provide a link
+* Please include specifics on how you are deploying/calling the contract
+* Have you reproduced the issue on other eth clients
+
+### Additional Information (Add any of the following or anything else that may be relevant)
+* Besu setup info - genesis file, config options
+* System info - memory, CPU

.github/ISSUE_TEMPLATE/feature-request.md

@@ -0,0 +1,45 @@
+---
+name: Feature Request
+about: suggesting new or altered functionality
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+<!-- Have you done the following? -->
+<!--   * read the Code of Conduct? By filing an Issue, you are expected to -->  
+<!--     comply with it, including treating everyone with respect: -->
+<!--     https://github.com/hyperledger/besu/blob/main/CODE_OF_CONDUCT.md -->
+<!--   * Duplicate Issue check:  https://github.com/search?q=+is%3Aissue+repo%3Ahyperledger/Besu -->
+
+### Description
+As an [Actor], I want [feature] so that [why].
+
+### Acceptance Criteria
+* [Criteria 1]
+* [Criteria 2]
+* [Criteria 3...]
+
+### Logs (if a change in behavior)
+Please post relevant logs from Besu (and the consensus client, if running proof of stake) illustrating behavior to be altered
+
+### Versions (Add all that apply)
+* Software version: [`besu --version`]
+* Java version: [`java -version`]
+* OS Name & Version: [`cat /etc/*release`]
+* Kernel Version: [`uname -a`]
+* Virtual Machine software & version: [`vmware -v`]
+* Docker Version: [`docker version`]
+* Cloud VM, type, size: [Amazon Web Services I3-large]
+* Consensus Client & Version if using Proof of Stake: [e.g. Teku, Lighthouse, Prysm, Nimbus, Lodestar]
+
+### Smart contract information (If you're reporting an issue arising from deploying or calling a smart contract, please supply related information)
+* Solidity version [`solc --version`]
+* Repo with minimal set of deployable/reproducible contract code - please provide a link
+* Please include specifics on how you are deploying/calling the contract
+* Have you reproduced the issue on other eth clients
+
+### Additional Information (Add any of the following or anything else that may be relevant)
+* Besu setup info - genesis file, config options
+* System info - memory, CPU

.github/issue_template.md

@@ -2,30 +2,10 @@
 <!--   * read the Code of Conduct? By filing an Issue, you are expected to -->  
 <!--     comply with it, including treating everyone with respect: -->
 <!--     https://github.com/hyperledger/besu/blob/main/CODE_OF_CONDUCT.md -->
-<!--   * Reproduced the issue in the latest version of the software -->
-<!--   * Read the debugging docs: https://besu.hyperledger.org/private-networks/how-to -->
 <!--   * Duplicate Issue check:  https://github.com/search?q=+is%3Aissue+repo%3Ahyperledger/Besu -->
-<!-- Note:  Not all sections will apply to all issue types. -->
+<!--   * Considered using a specific template for submitting a Bug or Feature Request? -->
 
-### Description
-As an [Actor], I want [feature] so that [why]. 
 
-### Acceptance Criteria
-* [Criteria 1]
-
-### Steps to Reproduce (Bug)
-1. [Step 1]
-2. [Step 2]
-3. [Step ...]
-
-**Expected behavior:** [What you expect to happen]
-
-**Actual behavior:** [What actually happens]
-
-**Frequency:** [What percentage of the time does it occur?]
-
-### Logs (if a bug)
-Please post relevant logs from Besu (and the consensus client, if running proof of stake) from before and after the issue. 
 
 ### Versions (Add all that apply)
 * Software version: [`besu --version`]

.github/workflows/stale-issues.yml

@@ -0,0 +1,27 @@
+name: Close inactive issues
+on:
+  schedule:
+    - cron: "30 1 * * *"
+
+jobs:
+  close-issues:
+    runs-on: ubuntu-latest
+    permissions:
+      issues: write
+      pull-requests: write
+      contents: write
+    steps:
+      - uses: actions/stale@v5
+        with:
+          days-before-issue-stale: 180
+          days-before-issue-close: 14
+          stale-issue-label: "stale"
+          stale-issue-message: "This issue is stale because it has been open for 6 months with no activity."
+          close-issue-message: "This issue was closed because it has been inactive for 14 days since being marked as stale."
+          days-before-pr-stale: 30
+          days-before-pr-close: 14
+          stale-pr-message: "This pr is stale because it has been open for 30 days with no activity."
+          close-pr-message: "This pr was closed because it has been inactive for 14 days since being marked as stale."
+          remove-issue-stale-when-updated: true
+          remove-pr-stale-when-updated: true
+          repo-token: ${{ secrets.GITHUB_TOKEN }}

CHANGELOG.md

@@ -1,19 +1,35 @@
 # Changelog
 
 ## [Unreleased]
-- Added isLabelsObserved to LabelledGauge in plugin-api. Default implementation returns false.
 
 ### Breaking Changes
+- Removed Retesteth rpc service and commands [#7833](https://github.com/hyperledger/besu/pull/7783)
 
 ### Upcoming Breaking Changes
+- Plugin API will be deprecating the BesuContext interface to be replaced with the ServiceManager interface.
+- `MetricSystem::createLabelledGauge` is deprecated and will be removed in a future release, replace it with `MetricSystem::createLabelledSuppliedGauge`
+- k8s (KUBERNETES) Nat method is now deprecated and will be removed in a future release
+- `--host-whitelist` has been deprecated in favor of `--host-allowlist` since 2020 and will be removed in a future release
+- Sunsetting features - for more context on the reasoning behind the deprecation of these features, including alternative options, read [this blog post](https://www.lfdecentralizedtrust.org/blog/sunsetting-tessera-and-simplifying-hyperledger-besu)
+  - Tessera privacy
+  - Smart-contract-based permissioning
+  - Proof of Work consensus
+  - Fast Sync
 
 ### Additions and Improvements
 - Fine tune already seen txs tracker when a tx is removed from the pool [#7755](https://github.com/hyperledger/besu/pull/7755)
+- Support for enabling and configuring TLS/mTLS in WebSocket service. [#7854](https://github.com/hyperledger/besu/pull/7854)
 - Create and publish Besu BOM (Bill of Materials) [#7615](https://github.com/hyperledger/besu/pull/7615) 
 - Update Java dependencies [#7786](https://github.com/hyperledger/besu/pull/7786)
 - Add a method to get all the transaction in the pool, to the `TransactionPoolService`, to easily access the transaction pool content from plugins [#7813](https://github.com/hyperledger/besu/pull/7813)
+- Upgrade RocksDB JNI library from version 8.3.2 to 9.7.3 [#7817](https://github.com/hyperledger/besu/pull/7817)
+- Add a method to check if a metric category is enabled to the plugin API [#7832](https://github.com/hyperledger/besu/pull/7832)
+- Add a new metric collector for counters which get their value from suppliers [#7894](https://github.com/hyperledger/besu/pull/7894)
+- Add account and state overrides to `eth_call` [#7801](https://github.com/hyperledger/besu/pull/7801) and `eth_estimateGas` [#7890](https://github.com/hyperledger/besu/pull/7890)
 
 ### Bug fixes
+- Fix registering new metric categories from plugins [#7825](https://github.com/hyperledger/besu/pull/7825)
+- Fix CVE-2024-47535 [7878](https://github.com/hyperledger/besu/pull/7878)
 
 ## 24.10.0
 
@@ -99,6 +115,7 @@ This release version has been deprecated release due to CI bug
 - Remove long-deprecated `perm*whitelist*` methods [#7401](https://github.com/hyperledger/besu/pull/7401)
 
 ### Additions and Improvements
+- Allow optional loading of `jemalloc` (if installed) by setting the environment variable `BESU_USING_JEMALLOC` to true/false. It that env is not set at all it will behave as if it is set to `true`
 - Expose set finalized/safe block in plugin api BlockchainService. These method can be used by plugins to set finalized/safe block for a PoA network (such as QBFT, IBFT and Clique).[#7382](https://github.com/hyperledger/besu/pull/7382)
 - In process RPC service [#7395](https://github.com/hyperledger/besu/pull/7395)
 - Added support for tracing private transactions using `priv_traceTransaction` API. [#6161](https://github.com/hyperledger/besu/pull/6161)

CONTRIBUTING.md

@@ -8,7 +8,7 @@ Welcome to the Besu repository! The following links are a set of guidelines for
 Having the following accounts is necessary for contributing code/issues to Besu.  
 * If you want to contribute code, you can make a [github account here](https://github.com).  
 * If you want to raise an issue, do so [in the issues tab](https://github.com/hyperledger/besu/issues).
-* To ask questions or chat with us, join our [Discord](https://discord.gg/hyperledger)
+* To ask questions or chat with us, join our [Discord](https://discord.com/invite/hyperledger)
 * To edit pages in our wiki, you'll need a [Linux Foundation (LF) account].
 
 ### Useful contributing links

DCO.md

@@ -5,4 +5,4 @@ As per section 13.a of the [Hyperledger Charter](https://www.hyperledger.org/abo
 
 The sign off needs to be using your legal name, not a pseudonym.  Git has a built-in mechanism to allow this with the `-s` or `--signoff` argument to `git commit` command, providing your `user.name` and `user.email` have been setup correctly.
 
-If you have any questions, you can reach us on Besu chat; first, [join the Discord server](https://discord.gg/hyperledger/) then [join the Besu channel](https://discord.com/channels/905194001349627914/938504958909747250).
+If you have any questions, you can reach us on Besu chat; first, [join the Discord server](https://discord.com/invite/hyperledger) then [join the Besu channel](https://discord.com/channels/905194001349627914/938504958909747250).

MAINTAINERS.md

@@ -22,6 +22,7 @@
 | Luis Pinto       | lu-pinto         | lu-pinto         |
 | Lucas Saldanha   | lucassaldanha    | lucassaldanha    |
 | Sally MacFarlane | macfarla         | macfarla         |
+| Matilda Clerke   | Matilda-Clerke   | MatildaClerke    |
 | Karim Taam       | matkt            | matkt            |
 | Matthew Whitehead| matthew1001      | matthew.whitehead      |
 | Meredith Baxter  | mbaxter          | mbaxter          |
@@ -84,7 +85,7 @@ The following steps must occur for a contributor to be "upgraded" as a maintaine
   - The proposed maintainer accepts the nomination and expresses a willingness
     to be a long-term (more than 6 month) committer by adding a comment in the proposal PR.
   - The PR will be communicated in all appropriate communication channels
-    including at least [besu-contributors channel on Discord](https://discord.gg/hyperledger),
+    including at least [besu-contributors channel on Discord](https://discord.com/invite/hyperledger),
     the [mailing list](https://lists.hyperledger.org/g/besu)
     and any maintainer/community call.
 - Approval by at least 3 current maintainers within two weeks of the proposal or

README.md

@@ -3,7 +3,7 @@
  [![Documentation](https://img.shields.io/github/actions/workflow/status/hyperledger/besu-docs/publish-main-docs.yml?branch=main&label=docs)](https://github.com/hyperledger/besu-docs/actions/workflows/publish-main-docs.yml)
  [![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/3174/badge)](https://bestpractices.coreinfrastructure.org/projects/3174)
  [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://github.com/hyperledger/besu/blob/main/LICENSE)
- [![Discord](https://img.shields.io/discord/905194001349627914?logo=Hyperledger&style=plastic)](https://discord.gg/hyperledger)
+ [![Discord](https://img.shields.io/discord/905194001349627914?logo=Hyperledger&style=plastic)](https://discord.com/invite/hyperledger)
  [![Twitter Follow](https://img.shields.io/twitter/follow/HyperledgerBesu)](https://twitter.com/HyperledgerBesu)
 
 [Download](https://github.com/hyperledger/besu/releases)
@@ -67,5 +67,5 @@ and <a href="https://www.yourkit.com/youmonitor/">YourKit YouMonitor</a>.
 
 [Besu Issues]: https://github.com/hyperledger/besu/issues
 [Besu User Documentation]: https://besu.hyperledger.org
-[Besu channel on Discord]: https://discord.gg/hyperledger
+[Besu channel on Discord]: https://discord.com/invite/hyperledger
 [Contributing Guidelines]: CONTRIBUTING.md

SUPPORT.md

@@ -20,5 +20,5 @@ Having Github, Discord, and Linux Foundation accounts is necessary for obtaining
 
 
 [Besu User Documentation]: https://besu.hyperledger.org
-[Besu channel on Discord]: https://discord.gg/hyperledger
+[Besu channel on Discord]: https://discord.com/invite/hyperledger
 [Contributing Guidelines]: CONTRIBUTING.md

acceptance-tests/dsl/src/main/java/org/hyperledger/besu/tests/acceptance/dsl/node/BesuNode.java

@@ -27,7 +27,7 @@
 import org.hyperledger.besu.ethereum.api.jsonrpc.JsonRpcConfiguration;
 import org.hyperledger.besu.ethereum.api.jsonrpc.ipc.JsonRpcIpcConfiguration;
 import org.hyperledger.besu.ethereum.api.jsonrpc.websocket.WebSocketConfiguration;
-import org.hyperledger.besu.ethereum.core.MiningParameters;
+import org.hyperledger.besu.ethereum.core.MiningConfiguration;
 import org.hyperledger.besu.ethereum.core.PrivacyParameters;
 import org.hyperledger.besu.ethereum.core.Util;
 import org.hyperledger.besu.ethereum.eth.transactions.TransactionPoolConfiguration;
@@ -101,7 +101,7 @@ public class BesuNode implements NodeConfiguration, RunnableNode, AutoCloseable
   private final boolean revertReasonEnabled;
 
   private final String name;
-  private MiningParameters miningParameters;
+  private MiningConfiguration miningConfiguration;
   private TransactionPoolConfiguration txPoolConfiguration;
   private final List<String> runCommand;
   private PrivacyParameters privacyParameters = PrivacyParameters.DEFAULT;
@@ -139,7 +139,7 @@ public class BesuNode implements NodeConfiguration, RunnableNode, AutoCloseable
   public BesuNode(
       final String name,
       final Optional<Path> dataPath,
-      final MiningParameters miningParameters,
+      final MiningConfiguration miningConfiguration,
       final TransactionPoolConfiguration txPoolConfiguration,
       final JsonRpcConfiguration jsonRpcConfiguration,
       final Optional<JsonRpcConfiguration> engineRpcConfiguration,
@@ -191,7 +191,7 @@ public BesuNode(
         },
         () -> this.keyPair = KeyPairUtil.loadKeyPair(homeDirectory));
     this.name = name;
-    this.miningParameters = miningParameters;
+    this.miningConfiguration = miningConfiguration;
     this.txPoolConfiguration = txPoolConfiguration;
     this.jsonRpcConfiguration = jsonRpcConfiguration;
     this.engineRpcConfiguration = engineRpcConfiguration;
@@ -678,12 +678,12 @@ public void setBootnodes(final List<URI> bootnodes) {
     this.bootnodes.addAll(bootnodes);
   }
 
-  public MiningParameters getMiningParameters() {
-    return miningParameters;
+  public MiningConfiguration getMiningParameters() {
+    return miningConfiguration;
   }
 
-  public void setMiningParameters(final MiningParameters miningParameters) {
-    this.miningParameters = miningParameters;
+  public void setMiningParameters(final MiningConfiguration miningConfiguration) {
+    this.miningConfiguration = miningConfiguration;
   }
 
   public TransactionPoolConfiguration getTransactionPoolConfiguration() {

acceptance-tests/dsl/src/main/java/org/hyperledger/besu/tests/acceptance/dsl/node/ProcessBesuNodeRunner.java

@@ -17,9 +17,9 @@
 import static com.google.common.base.Preconditions.checkState;
 import static java.nio.charset.StandardCharsets.UTF_8;
 
+import org.hyperledger.besu.cli.options.NetworkingOptions;
 import org.hyperledger.besu.cli.options.TransactionPoolOptions;
 import org.hyperledger.besu.cli.options.storage.DataStorageOptions;
-import org.hyperledger.besu.cli.options.unstable.NetworkingOptions;
 import org.hyperledger.besu.ethereum.api.jsonrpc.ipc.JsonRpcIpcConfiguration;
 import org.hyperledger.besu.ethereum.eth.transactions.ImmutableTransactionPoolConfiguration;
 import org.hyperledger.besu.ethereum.p2p.rlpx.connections.netty.TLSConfiguration;