-
Notifications
You must be signed in to change notification settings - Fork 296
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
docs: tutorial for running hardware node (#4678)
## Describe your changes Documents the setup process for renting a hardware box (via Hetzner) and configuring a fullnode on it. Necessarily overlaps with the regular ol' install docs for pd, but tried to be explicit in the tutorial for maximum comprehension. Making that simplified change, I looked through the old deployment/ci configs and culled a lot that were gathering dust. Now the repo concisely recommends using the systemd config, or links out to other community-maintained options for configuring nodes. ## Issue ticket number and link None. ## Checklist before requesting a review - [x] If this code contains consensus-breaking changes, I have added the "consensus-breaking" label. Otherwise, I declare my belief that there are not consensus-breaking changes, for the following reason: > no application code changes, just improving docs and cleaning things up --------- Co-authored-by: Conor Schaefer <[email protected]>
- Loading branch information
Showing
124 changed files
with
183 additions
and
4,954 deletions.
There are no files selected for viewing
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,96 +1,16 @@ | ||
# Penumbra deployments | ||
|
||
This directory contains config management logic for managing | ||
Penumbra networks. As of 2023Q1, prior to mainnet, | ||
Penumbra Labs runs three (3) discrete networks: | ||
|
||
* "testnet", updated approximately weekly | ||
* "preview", updated on every push to `main` in the repo | ||
* "devnet", updated ad-hoc to serve as a sandbox debugging environment | ||
|
||
Those networks each have their own genesis and knowledge of peers. | ||
The networks are completely separate. | ||
See the [Penumbra Guide] for setup steps on running a Penumbra node. | ||
|
||
## Directory structure | ||
|
||
``` | ||
. | ||
├── ci.sh # runner script for executing a deploy against k8s | ||
├── charts/ # helm charts used to configure genesis, nodes, and metrics | ||
├── networks/ # logic specific to network, e.g. "testnet" or "preview" | ||
│ └── testnet/ | ||
└── terraform/ # server and cluster provisioning logic | ||
└── modules/ | ||
``` | ||
|
||
## Running custom devnets | ||
|
||
Sometimes you want to perform a deploy that mimics our preview and testnet setups, | ||
but has its own lifecycle. For example, you may want to test new cluster config logic, | ||
or try to reproduce a bug on an older version of the software. Behold: | ||
|
||
``` | ||
cd deployments/ | ||
export HELM_RELEASE=penumbra-devnet | ||
./ci.sh | ||
``` | ||
|
||
That will deploy a new network based on latest `main` (containers are rebuilt on every merge) | ||
and run it. You can destroy it when you're done with `helm uninstall penumbra-devnet`. | ||
You can also run an older version of the software: | ||
|
||
``` | ||
cd deployments/ | ||
export HELM_RELEASE=penumbra-devnet | ||
export PENUMBRA_VERSION=v0.53.1 | ||
./ci.sh | ||
``` | ||
|
||
You'll see a message about a "patch release"; you'll need to edit the ci script's main function to force | ||
a run of the full deploy logic. | ||
|
||
## Out of band config | ||
There are several DNS records that are not handled | ||
by the automation in this repo. Each testnet should have: | ||
|
||
* `rpc.<fqdn>` # the cometbft rpc service | ||
* `grpc.<fqdn>` # the pd grpc service | ||
* `grafana.<fqdn>` # web interface for metrics dashboards | ||
|
||
To find the IPv4 address for `{g,}rpc.<fqdn>`, use this command: | ||
|
||
``` | ||
kubectl get svc -n infra traefik --output jsonpath='{.status.loadBalancer.ingress[0].ip}' | ||
``` | ||
|
||
The Traefik reverse proxy is used for fronting pd's grpc service, because Traefik supports h2c. | ||
See for details: https://github.com/penumbra-zone/penumbra/issues/2341 | ||
|
||
## Generating and storing public IPs for P2P connections | ||
|
||
There's a chicken-or-egg problem when creating a new network: the deployment will trigger the creation | ||
of LoadBalancer objects with public IPv4 addresses. Those public IP addresses are needed at genesis | ||
creation time, so that the validator configs are generated with an external address field | ||
in the Tendermint configs. To resolve, there's a special var `only_lb_svc=true` that will | ||
deploy just the P2P LBs. You can then poll the IPs, store them as additional vars, and rerun | ||
with `only_lb_svc=false`. | ||
|
||
├── systemd/ # example systemd configs for node operators | ||
├── containerfiles/ # OCI container image manifests | ||
├── scripts/ # scripts related to testing penumbra in ci | ||
├── containerfiles/ # OCI container image manifests | ||
├── compose/ # example docker-compose configs, unmaintained | ||
``` | ||
helmfile sync -f helmfile.d/penumbra-devnet.yaml --args --set=only_lb_svc=true | ||
./scripts/get-lb-ips penumbra-devnet | ||
helmfile sync -f helmfile.d/penumbra-devnet.yaml | ||
``` | ||
|
||
This two-step process is only required the *first* time a given network is deployed. | ||
Thereafter, resource retention policies will preserve the LBs, so that the IPs remain reserved, | ||
and can be reused on subsequent deployments of that network. | ||
|
||
## Dude, where's my logs? | ||
|
||
There's web-based access for viewing logs from the testnet deployment: | ||
|
||
* [Top-level view of all deployments](https://console.cloud.google.com/kubernetes/workload/overview?project=penumbra-sl-testnet) | ||
* [Logs for the deployment with RPC endpoints exposed](https://console.cloud.google.com/kubernetes/deployment/us-central1/testnet/default/penumbra-testnet-fn-0/logs?project=penumbra-sl-testnet) | ||
|
||
You must authenticate with your PL Google account to view that information; | ||
ask a team member if you need a grant added for your account. | ||
[Penumbra Guide]: https://guide.penumbra.zone |
This file was deleted.
Oops, something went wrong.
Oops, something went wrong.