docs: tutorial for running hardware node

Documents the setup process for renting a hardware box (via Hetzner) and configuring a fullnode on it. Necessarily overlaps with the regular ol' install docs for pd, but tried to be explicit in the tutorial for maximum comprehension.
penumbra-zone · Jun 28, 2024 · d1fb894 · d1fb894
1 parent 9ba5a03
commit d1fb894
Show file tree

Hide file tree

Showing 12 changed files with 183 additions and 113 deletions.
diff --git a/deployments/systemd/README.md b/deployments/systemd/README.md
@@ -4,25 +4,6 @@ Here are example unit files for running Penumbra (and the required CometBFT inst
 via systemd. *You'll need to customize the service files*, particularly the `User` declaration
 and binary fullpath in `ExecStart`, and possibly the path to the home directory, depending on your system.
 
-## Installing
-Copy the service files to a system-wide location:
+See the [guide] for details on how to use these configs.
 
-```bash
-# use 'envsubst' to replace `$USER` with your local username
-envsubst < penumbra.service | sudo tee /etc/systemd/system/penumbra.service
-envsubst < cometbft.service | sudo tee /etc/systemd/system/cometbft.service
-sudo systemctl daemon-reload
-sudo systemctl restart penumbra cometbft
-
-# view logs to monitor for errors
-sudo journalctl -af -u penumbra -u cometbft
-```
-
-## Uninstalling
-To remove the configs, run:
-
-```bash
-sudo systemctl disable --now penumbra cometbft
-sudo rm /etc/systemd/system/{penumbra,cometbft}.service
-sudo systemctl daemon-reload
-```
+[guide]: https://guide.penumbra.zone
diff --git a/deployments/systemd/cometbft.service b/deployments/systemd/cometbft.service
@@ -2,9 +2,11 @@
 Description=CometBFT for Penumbra
 
 [Service]
-ExecStart=/usr/local/bin/cometbft start --home $HOME/.penumbra/testnet_data/node0/cometbft
+ExecStart=/usr/local/bin/cometbft start --home /home/penumbra/.penumbra/testnet_data/node0/cometbft
 Restart=no
-User=$USER
+User=penumbra
+# Raise filehandle limit for RPC and P2P connections.
+LimitNOFILE=65536
 
 [Install]
 WantedBy=default.target
diff --git a/deployments/systemd/penumbra.service b/deployments/systemd/penumbra.service
@@ -3,15 +3,21 @@ Description=Penumbra pd
 Wants=cometbft.service
 
 [Service]
-# If both 1) running pd as non-root; and 2) using auto-https logic, then
-# uncomment the capability declarations below to permit binding to 443/TCP for HTTPS.
-# CapabilityBoundingSet=CAP_NET_BIND_SERVICE
-# AmbientCapabilities=CAP_NET_BIND_SERVICE
+# Support binding to 443/TCP for HTTPS.
+CapabilityBoundingSet=CAP_NET_BIND_SERVICE
+AmbientCapabilities=CAP_NET_BIND_SERVICE
+
 ExecStart=/usr/local/bin/pd start
-# Consider overriding the home directory, e.g.
-# ExecStart=/usr/local/bin/pd start --home /var/www/.penumbra/testnet_data/node0/pd
+# Consider adding an HTTPS URL if you have DNS set up:
+# ExecStart=/usr/local/bin/pd start --grpc-auto-https www.example.com
+
+# Disable automatic restart, since governance votes can intentionally halt the chain.
+# If you're running a validator, you should configure uptime monitoring.
 Restart=no
-User=$USER
+
+# Assumes that the `penumbra` user exists.
+User=penumbra
+
 # Raise filehandle limit for tower-abci.
 LimitNOFILE=65536
 # Consider configuring logrotate if using debug logs

diff --git a/docs/guide/src/README.md b/docs/guide/src/README.md
@@ -16,8 +16,8 @@ end-to-end encrypted.
 
 ## Using Penumbra on the web
 
-The easiest way to get started using Penumbra is with the Penumbra web
-extension.  The [web extension][webext] runs entirely locally, and contains an
+The easiest way to get started using Penumbra is with the [Prax Wallet] web
+extension.  The web extension runs entirely locally, and contains an
 embedded ultralight node that syncs and decrypts only the data visible to your
 wallet.  Websites can request to connect to your wallet and query your data.
 
@@ -65,4 +65,4 @@ The [_Resources_](./resources.md) chapter has links to other resources about the
 [protocol]: https://protocol.penumbra.zone
 [rustdoc]: https://rustdoc.penumbra.zone
 [Penumbra]: https://github.com/penumbra-zone/penumbra
-[webext]: https://chrome.google.com/webstore/detail/penumbra-wallet/lkpmkhpnhknhmibgnmmhdhgdilepfghe
+[Prax Wallet]: https://chromewebstore.google.com/detail/prax-wallet/lkpmkhpnhknhmibgnmmhdhgdilepfghe
diff --git a/docs/guide/src/SUMMARY.md b/docs/guide/src/SUMMARY.md
@@ -34,4 +34,5 @@
   - [Testing IBC](./dev/ibc.md)
 - [Resources](./resources.md)
 - [Tutorials](./tutorials.md)
+  - [Running a fullnode](./tutorials/running-node.md)
   - [Running a frontend](./tutorials/running-frontend.md)
diff --git a/docs/guide/src/node/pd.md b/docs/guide/src/node/pd.md
@@ -3,6 +3,9 @@
 This section describes how to build and run `pd`, the node implementation for
 Penumbra:
 
+- [Requirements](./pd/requirements.md) describes the compute resource necessary to run a Penumbra node;
 - [Installing `pd`](./pd/install.md) describes how to build `pd`;
 - [Joining a Testnet](./pd/join-testnet.md) describes how to join the current testnet;
 - [Creating a Testnet](../dev/devnet-quickstart.md) describes how to create a custom testnet, for instance for local development.
+
+If you're looking for a lightweight solution to interact with the Penumbra network, see the [pclientd](../node/pclientd.md) docs.
diff --git a/docs/guide/src/node/pd/install.md b/docs/guide/src/node/pd/install.md
@@ -1,11 +1,15 @@
 # Installing pd
+There are many ways to configure and run Penumbra. The easiest is to download
+binaries for `pd` and `cometbft` on a Linux system. For alternatives, see
+[deployment strategies](./requirements.md#deployment-strategies).
+If you want a detailed guide, see the [tutorial on running a node](../../tutorials/running-node.md).
 
+## Quickstart
 Download prebuilt binaries from the [Penumbra releases page on Github](https://github.com/penumbra-zone/penumbra/releases).
 Make sure to use the most recent version available, as the version of `pd` must
-match the software currently running on the network.
+match the software currently running on the network, to choose the correct platform for your machine.
 
-Make sure to choose the correct platform for your machine. After downloading the `.tar.gz` file,
-extract it, and copy its contents to your `$PATH`. For example:
+After downloading the `.tar.gz` file, extract it, and copy its contents to your `$PATH`. For example:
 
 ```
 curl -sSfL -O https://github.com/penumbra-zone/penumbra/releases/download/{{ #include ../../penumbra_version.md }}/pd-x86_64-unknown-linux-gnu.tar.gz
@@ -17,17 +21,10 @@ pd --version
 ```
 
 There's also a one-liner install script available on the release page, which will install `pd` to `$HOME/.cargo/bin/`.
-As of v0.64.1 (released 2023-12-12), we build Linux binaries on Ubuntu 22.04. If these binaries don't work for you out of the box,
-you'll need to [build from source](../../dev/build.md), or use the container images.
 
 ### Installing CometBFT
 
-You'll need to have [CometBFT installed](https://docs.cometbft.com/v0.37/guides/install)
-on your system to join your node to the testnet.
-
+You'll need to have [CometBFT installed](https://docs.cometbft.com/v0.37/guides/install) on your system to join your node to the testnet.
 You must use a specific version of CometBFT, `{{ #include ../../cometbft_version.md }}`, which you can download
 [from the CometBFT releases page](https://github.com/cometbft/cometbft/releases/tag/{{ #include ../../cometbft_version.md }}).
 If you prefer to compile from source instead, make sure you are compiling version `{{ #include ../../cometbft_version.md }}`.
-
-Previous versions of Penumbra used Tendermint, but as of Testnet 62 (released 2023-10-10),
-only CometBFT is supported. **Do not use** any version of Tendermint, which will not work with `pd`.
diff --git a/docs/guide/src/node/pd/join-testnet.md b/docs/guide/src/node/pd/join-testnet.md
@@ -10,57 +10,25 @@ A regular validator will participate in voting and rewards, if it becomes part
 of the consensus set.  Of course, these rewards, like all other testnet tokens,
 have no value.
 
-## Joining as a fullnode
+## Generating configs
 
 To join a testnet as a fullnode, [install the most recent version of `pd`](install.md), run
 `pd testnet join` to generate configs, then use those configs to run `pd` and
-`cometbft`. In more detail:
-
-### Resetting state
-
-First, reset the testnet data from any prior testnet you may have joined:
-
-```shell
-pd testnet unsafe-reset-all
-```
-
-This will delete the entire testnet data directory.
-
-### Generating configs
-
-Next, generate a set of configs for the current testnet:
-
-<!--
-### Begin join customization
-
-The following section describes how to join a testnet chain *which has never upgraded*.
-Once a chain upgrade occurs, a new-joining node must have access to an archive
-of historical, migrated state. When we upgrade the chain, we should update these
-docs to switch to the archive-url version:
--->
+`cometbft`.
 
 ```shell
-pd testnet join --external-address IP_ADDRESS:26656 --moniker MY_NODE_NAME \
+pd testnet join \
+    --moniker MY_NODE_NAME \
+    --external-address IP_ADDRESS:26656 \
     --archive-url "https://snapshots.penumbra.zone/testnet/pd-migrated-state-77-78.tar.gz"
 ```
 
-where `IP_ADDRESS` (like `1.2.3.4`) is the public IP address of the node you're running,
-and `MY_NODE_NAME` is a moniker identifying your node. Other peers will try to connect
+where `MY_NODE_NAME` is a moniker identifying your node, and `IP_ADDRESS` (like `1.2.3.4`)
+is the public IP address of the node you're running. Other peers will try to connect
 to your node over port `26656/TCP`. Finally, the `--archive-url` flag will fetch
 a tarball of historical blocks, so that your newly joining node can understand transactions
 that occurred prior to the most recent chain upgrade.
 
-<!--
-```shell
-pd testnet join --external-address IP_ADDRESS:26656 --moniker MY_NODE_NAME
-```
-
-where `IP_ADDRESS` (like `1.2.3.4`) is the public IP address of the node you're running,
-and `MY_NODE_NAME` is a moniker identifying your node. Other peers will try to connect
-to your node over port `26656/TCP`.
-### End join customization
--->
-
 If your node is behind a firewall or not publicly routable for some other reason,
 skip the `--external-address` flag, so that other peers won't try to connect to it.
 You can also skip the `--moniker` flag to use a randomized moniker instead of selecting one.
@@ -72,27 +40,7 @@ the section above on resetting node state.
 
 ### Running `pd` and `cometbft`
 
-Next, run `pd`:
-
-```shell
-pd start
-```
-
-Then (perhaps in another terminal), run CometBFT, specifying `--home`:
-
-```shell
-cometbft start --home ~/.penumbra/testnet_data/node0/cometbft
-```
-
-Alternatively, `pd` and `cometbft` can be orchestrated with `docker-compose`:
-
-```shell
-cd deployments/compose/
-docker-compose pull
-docker-compose up --abort-on-container-exit
-```
-
-or via systemd:
+Copy the systemd service configs into place from the [project git repo](https://github.com/penumbra-zone/penumbra):
 
 ```
 cd deployments/systemd/
@@ -102,5 +50,17 @@ sudo systemctl daemon-reload
 sudo systemctl restart penumbra cometbft
 ```
 
-See the [`deployments/`](https://github.com/penumbra-zone/penumbra/tree/{{ #include ../../penumbra_version.md }}/deployments)
-directory for more examples on configuration scripts.
+In particular, if you have DNS configured for your node, you should edit the `ExecStart` line for `pd`
+to use the `--grpc-auto-https` option.
+
+### Resetting state
+
+If you have previously joined a network before, and want to purge those configs,
+use:
+
+```shell
+pd testnet unsafe-reset-all
+```
+
+This will delete the entire testnet data directory, after which you can re-join.
+You should only run this command after stopping `pd` and `cometbft`.
diff --git a/docs/guide/src/node/pd/requirements.md b/docs/guide/src/node/pd/requirements.md
@@ -39,10 +39,19 @@ for protecting the validator identity.
 ## Deployment strategies
 
 We expect node operators to manage the lifecycle of their Penumbra deployments.
-Some example configs for systemd, docker compose, and kubernetes helm charts
-can be found in the Penumbra repo's [`deployments/`](https://github.com/penumbra-zone/penumbra/tree/{{ #include ../../penumbra_version.md }}/deployments) directory.
+Some example configs for systemd can be found in the Penumbra repo's
+[`deployments/`](https://github.com/penumbra-zone/penumbra/tree/{{ #include ../../penumbra_version.md }}/deployments) directory.
+Other community solutions include:
+
+* [Cosmos Operator] by [Strangelove] for Kubernetes
+* [NixOS derivations](https://github.com/starlingcyber/infra) maintained by [Starling Cybernetics]
+
 You should consult these configurations as a reference, and write your own
 scripts to maintain your node.
 
 Consider [joining the Penumbra Discord](../../resources.md#discord) to receive announcements
 about new versions and required actions by node operators.
+
+[Cosmos Operator]: https://github.com/strangelove-ventures/cosmos-operator/
+[Strangelove]: https://strange.love/
+[Starling Cybernetics]: https://starlingcyber.net/
diff --git a/docs/guide/src/tutorials.md b/docs/guide/src/tutorials.md
@@ -2,4 +2,5 @@
 
 Detailed guides for explaining common operations in the Penumbra ecosystem.
 
+* [Setting up a fullnode on rented hardware](./tutorials/running-node.md)
 * [Self-hosting a frontend for using GUI applications](./tutorials/running-frontend.md)
diff --git a/docs/guide/src/tutorials/running-node.md b/docs/guide/src/tutorials/running-node.md
@@ -0,0 +1,109 @@
+# Running a fullnode
+
+In order to interact with the Penumbra network, users must provide an RPC URL
+so that client software can read chain state and submit transactions.
+This is true of the [Prax wallet], and of [pcli], as well as any other client.
+While users can select a publicly available RPC URL from a party they trust,
+this guide demonstrates how a user can self-host an RPC URL for use by themselves and others.
+
+## Renting a server
+
+There are a variety of cloud providers that provide dedicated hardware for a per-month cost basis.
+Generally, hardware-based solutions will have superior performance, particularly in storage latency,
+and also more reliable performance over time. One suitable option is the
+[Matrix AX52 by Hetzner](https://www.hetzner.com/dedicated-rootserver/ax52/).
+
+To get started with Hetzner, [create an account](https://accounts.hetzner.com/signUp), provide billing information,
+then request a dedicated hardware server. While preparing the server request,
+you'll need to provide an SSH public key for the root user account. You can use this command to generate one
+if you don't have one already:
+
+```
+ssh-keygen -t ed25519
+cat ~/.ssh/id_ed25519.pub
+```
+
+Choose the Debian Stable option for operating system.
+Shortly after requesting the server, you should receive an email notifying you that it's ready to accept logins.
+
+## Setting up DNS
+
+In order to use HTTPS over the web interface, you'll need to create an A record for the domain you want to use,
+pointing to the IPv4 address for the server. Visit the website for your DNS provider, and create the A record,
+using the IP address displayed on the server page for Hetzner.
+
+## Provisioning the server
+
+Log into the server like so:
+
+```
+ssh -l root <YOUR_DNS_DOMAIN>
+```
+
+If that command fails, you'll need to debug your access settings.
+
+First, clone the git repository:
+
+```
+apt-get install -y git git-lfs
+git clone --branch {{ #include ../penumbra_version.md }} https://github.com/penumbra-zone/penumbra
+```
+
+Use that repo to copy the service configurations into place:
+
+```
+cd penumbra/deployments/systemd/
+cp penumbra.service cometbft.service /etc/systemd/system/
+# edit /etc/systemd/system/penumbra.service,
+# and add your DNS domain to the `--grpc-auto-https` example.
+systemctl daemon-reload
+```
+
+Follow the guide to [install pd and cometbft](../node/pd/install.md) from their respective
+release pages. You should be able to run `pd --version` and see `{{ #include ../penumbra_version.md }}`
+displayed. 
+
+Next, create a user account for running the Penumbra software:
+
+```
+sudo useradd -m -d /home/penumbra penumbra -s /bin/bash
+```
+
+We'll use this account to configure the `pd` and `cometbft` data directories.
+
+```
+sudo su -l penumbra
+pd testnet join \
+       --moniker <MONIKER> \
+       --external-address <EXTERNAL_ADDRESS> \
+       <NODE_URL>
+```
+
+The value for `NODE_URL` should be the CometBFT RPC endpoint for the node whose network
+you want to join. Change `MONIKER` to the human-readable name for your node on the network.
+Finally, the `EXTERNAL_ADDRESS` should be the public IP address of the server, so that
+other peers on the network can initiate connections to it, to share blocks.
+
+## Running the node
+Finally, start the services:
+
+```
+# return to root user
+exit
+systemctl restart penumbra cometbft
+journalctl -af -u penumbra
+```
+
+The final command will display logs from the `pd` process. In a short while, you should see
+blocks streaming in. If not, see the [debugging steps](../node/pd/debugging.md)
+to figure out what went wrong.
+
+<!--
+Next it would be nice to recommend setting up access to metrics, but let's assume that no existing
+prom/graf setup exists for the user. There's a compose config in the repo that can be used,
+but do we want to go through the motions of setting up `docker compose` on this host?
+Probably that's best.
+-->
+
+[pcli]: ../pcli.md
+[Prax wallet]: https://chromewebstore.google.com/detail/prax-wallet/lkpmkhpnhknhmibgnmmhdhgdilepfghe