chore: pull in latest penumbra guide

There were a few recent changes made to the guide in the protocol repo,
as of [0], and this commit pulls those changes in. At the very least,
I noticed that changes from the following PRs were missing:

* penumbra-zone/penumbra#4732
* penumbra-zone/penumbra#4726

Tried to make sure to preserve unique content updates implemented
during the mdbook -> nextra conversion.

[0] https://github.com/penumbra-zone/penumbra/tree/a2ffd8a79f1b397e81df7b7e31b256d545cdf004/docs/guide

pages/_meta.json

@@ -6,4 +6,4 @@
   "frontend": "Running a frontend",
   "dev": "Development",
   "resources": "Resources"
-}
+}

pages/dev.md

@@ -1 +1,11 @@
-This section is for developers working on Penumbra source code.
+This section is for developers, either working on Penumbra source code itself,
+or writing services that interact with Penumbra.
+
+<!--
+Use cases to cover:
+
+  * new contributor to protocol repo, wants to write rust code
+  * external dev wants to make a protobuf change, how do?
+  * external dev wants to run local services to prototype against
+
+-->

pages/dev/_meta.json

@@ -1,11 +1,13 @@
 {
-  "build": "Compiling from source",
-  "devnet-quickstart": "Devnet Quickstart",
+  "dev-env": "Developer environment",
+  "devnet-quickstart": "Devnet quickstart",
   "sql": "Working with SQLite",
   "docs": "Building documentation",
   "protobuf": "Building protobuf",
   "metrics": "Metrics",
-  "parameter_setup": "Zero Knowledge Proofs",
+  "parameter_setup": "Zero-knowledge proofs",
   "rpc": "RPC access",
-  "ibc": "Testing IBC"
+  "ibc": "Testing IBC",
+  "build": "Compiling from source",
+  "privacy": "Privacy"
 }

pages/dev/build.md

@@ -1,8 +1,12 @@
-Penumbra is written in Rust. To build it, you will need a recent
+Penumbra is written in [Rust]. To build it, you will need a recent
 stable version of Rust, as well as a few OS-level dependencies.
 We don't support building on Windows. If you need to use Windows,
 consider using [WSL] instead.
 
+This page aims to describe the steps necessary to work on Penumbra when
+settings up the build environment manually, without using [Nix].
+If you want an easy-to-use setup, see the docs on [developer environments](./dev-env.md).
+
 ### Installing the Rust toolchain
 
 This requires that you install a recent (>= 1.75) stable version
@@ -119,7 +123,7 @@ git submodule update --init
 
 # Build snappy using cmake.
 mkdir build
-cd build && cmake .. && make
+cd build && cmake ../ -DSNAPPY_BUILD_BENCHMARKS=OFF && make
 
 # Add an environment variable pointing to the build/ directory.
 SNAPPY_LIB_DIR=`pwd`
@@ -130,6 +134,8 @@ SNAPPY_LIB_DIR=`pwd`
 Once you've built rocksdb and set the environment variable, the `librocksdb-sys` crate will search
 in that directory for the compiled `librocksdb.a` static library when it is rebuilt.
 
+[Rust]: https://www.rust-lang.org/
 [snappy-build]: https://github.com/google/snappy?tab=readme-ov-file#building
 [protoc-install]: https://grpc.io/docs/protoc-installation/
 [WSL]: https://learn.microsoft.com/en-us/windows/wsl/install
+[Nix]: https://nixos.org/

pages/dev/dev-env.md

@@ -0,0 +1,70 @@
+To get started working on Penumbra, you'll need a few dependencies on your workstation.
+Running tests and local services is more involved. The project uses [Nix] to automate
+the creation of developer environments with suitable tooling. If you'd prefer not to use
+Nix, and instead configure your environment manually, see the docs on
+[compiling from source](./build.md).
+
+## Installation OS-level packages
+You'll need `git` and `git-lfs` to clone the Penumbra protocol repository.
+Install these via your package manager of choice:
+
+```bash
+# for macos
+brew install git-lfs
+# for linux debian/ubuntu
+sudo apt install -y git-lfs
+# for linux fedora
+sudo dnf install -y git-lfs
+```
+
+Then, for all platforms, make sure to run `git lfs install`. Now you're ready to clone the
+Penumbra protocol repo:
+
+```
+git clone https://github.com/penumbra-zone/penumbra
+```
+
+## Using `nix develop` for project dependencies
+
+<!--
+The Informal Systems team has a Nix setup at https://github.com/informalsystems/cosmos.nix,
+which recommends using the NixOS installer, which is why we recommend using that installer, too.
+-->
+
+Install [Nix]. After restarting your shell, create a config file to enable
+Nix flakes:
+
+```
+mkdir -p ~/.config/nix
+echo 'experimental-features = nix-command flakes' >> ~/.config/nix/nix.conf
+```
+
+Now hop into the Penumbra directory and activate the env:
+
+```
+cd penumbra
+nix develop
+```
+
+You'll have to wait a bit for packages to be built and installed. Once it finishes,
+your active shell will have access to Penumbra project dependencies, like a compatible
+version of `cometbft`, and other dev tooling, like `grpcurl` and `mdbook`.
+You can run `exit` to return to your normal shell, without those tools, which have been installed to `/nix/store/`.
+
+## Using `direnv`
+
+If you use [direnv], you can copy the example `.envrc` file to automatically activate
+the Penumbra nix environment when you cd to the repository:
+
+```
+cp .envrc.example .envrc
+direnv allow
+```
+
+The `.envrc` path is intentionally git-ignored, so you can customize it as you see fit.
+If you don't use `direnv`, you'll need to run `nix develop` in any terminal window
+where you want access to the Penumbra dev env. Using `direnv` will make shell startup
+a bit slower, so choose what's best for you.
+
+[Nix]: https://nixos.org/
+[direnv]: https://direnv.net/

pages/dev/devnet-quickstart.md

@@ -1,46 +1,35 @@
-This page describes a quickstart method for running `pd`+`cometbft` to test
-changes during development.
+This page assumes you've set up a [Penumbra developer environment](./dev-env.md),
+as it references several commands like `cometbft` in order to work.
+It describes how to run a Penumbra fullnode on your local workstation, for building
+and testing Penumbra and related services.
 
-To start, you'll need to install a [specific version of CometBFT](../node/pd/install.md#installing-cometbft).
+## Running a local devnet
 
-## Generating configs
-
-To generate a clean set of configs, run
-
-```shell
-cargo run --release --bin pd -- network generate
-```
-
-This will write configs to `~/.penumbra/network_data/`.
-
-## Running `pd`
-
-You'll probably want to set `RUST_LOG`.  Here's one suggestion that's quite verbose:
-
-```shell
-# Optional. Expect about 20GB/week of log data for pd with settings below.
-export RUST_LOG="info,pd=debug,penumbra=debug,jmt=debug"
-```
-
-To run `pd`, run
+To generate a devnet genesis and run a Penumbra fullnode locally, run:
 
 ```shell
-cargo run --release --bin pd -- start
+just dev
 ```
 
-This will start but won't do anything yet, because CometBFT isn't running.
-
-## Running `cometbft`
+Running that command will: 
+
+  1. build the local rust code from th elatest version on-disk
+  2. generate a genesis file, writing configs to `~/.penumbra/network_data/`
+  3. run the locally-built version of `pd`
+  4. run `cometbft` alongside `pd`, communicating over ABCI
+  5. run a prometheus/grafana [metrics setup](./metrics.md)
 
-To run CometBFT, run
-
-```shell
-cometbft --home ~/.penumbra/network_data/node0/cometbft/ start
-```
-
-in another terminal window.
+You can use the [process-compose] interface to view logs from any individual service.
+Use `ctrl+c` to halt the setup, and run `just dev` to start it again from the latest
+local source code.
 
 ## Running `pcli`
+<!--
+TODO:
+The dev env should generate an ad-hoc wallet, and/or accept an env var of a Penumbra wallet address,
+and add that wallet addr to the generated devnet genesis. Then the user could immediately get
+started with read/write interactions via pcli.
+-->
 
 To interact with the chain, configure a wallet pointing at the localhost node:
 
@@ -66,7 +55,10 @@ edit the `genesis.json` to add your address.
 After making changes, you may want to reset and restart the devnet:
 
 ```shell
-cargo run --release --bin pd -- testnet unsafe-reset-all
+# stop the running node via process-compose:
+ctrl+c
+# destroy local network state:
+cargo run --release --bin pd -- network unsafe-reset-all
 ```
 
 You'll probably also want to reset your wallet state:
@@ -76,14 +68,14 @@ cargo run --release --bin pcli -- --home ~/.local/share/pcli-localhost view rese
 ```
 
 At this point you're ready to generate new configs, and restart both `pd` and
-`cometbft`.  The order they're started in doesn't particularly matter for
-correctness, because `cometbft` will retry connecting to the ABCI server until
-it succeeds.
+`cometbft`, which you can do by running `just dev` again. Note that running `just dev`
+will _reuse_ any existing state in `~/.penumbra/network_data/`. You must manually `unsafe-reset-all`
+to purge that pre-existing config.
 
-## Optional: running smoke-tests
-
-Once you have a working devnet running, you should be able to run the [smoke tests](https://en.wikipedia.org/wiki/Smoke_testing_(software)) successfully. This can be useful if you are looking to contribute to Penumbra, or if you need to check that your setup is correct.
+## Running smoke tests (optional)
 
+Once you have a working devnet running, you should be able to run the [smoke tests](https://en.wikipedia.org/wiki/Smoke_testing_(software))
+successfully. This can be useful if you are looking to contribute to Penumbra, or if you need to check that your setup is correct.
 To run the smoke tests:
 
 1. Make sure you have a devnet running (see previous steps)
@@ -92,8 +84,9 @@ To run the smoke tests:
 PENUMBRA_NODE_PD_URL=http://127.0.0.1:8080 PCLI_UNLEASH_DANGER=yes cargo test --package pcli -- --ignored --test-threads 1
 ```
 
-Find the exact commands for each binary's smoke tests in `deployments/compose/process-compose-smoke-test.yml`.
-You can also run the entire smoke test suite end-to-end via `just smoke`, including setup and teardown of the network.
+You can also run the entire smoke test suite with an automatic fullnode. If you do so, 
+make sure to stop any fullnode running via `just dev`, as they'll conflict. Then run `just smoke`.
 If you want to execute the tests against an already-running devnet, however, use manual invocations like
-the `cargo test` example above. You'll need to install [process-compose](https://github.com/F1bonacc1/process-compose/)
-to use the automated setup.
+the `cargo test` example above.
+
+[process-compose]: https://f1bonacc1.github.io/process-compose/launcher/

pages/dev/ibc.md

@@ -49,8 +49,7 @@ You can view account history for the shared Osmosis testnet account here:
 Change the address at the end of the URL to your account to confirm that your test transfer worked.
 
 ## Updating Hermes config for a new testnet
-See the [procedure in the wiki](https://github.com/penumbra-zone/penumbra/wiki/Updating-Hermes)
-for up to date information.
+See the [relayer config guide](../node/relayer/hermes.md) for up to date information.
 
 Use the [IBC user docs](../pcli/transaction.md#ibc-withdrawals) to make a test transaction,
 to ensure that relaying is working. In the future, we should consider posting the newly created

pages/dev/metrics.md

@@ -1,15 +1,5 @@
 Metrics are an important part of observability, allowing us to understand what
-the Penumbra software is doing. Penumbra Labs runs Grafana instances for the public deployments:
-
-  * https://grafana.testnet.penumbra.zone
-  * https://grafana.testnet-preview.penumbra.zone
-
-There's a more comprehensive WIP dashboard, gated by basic auth for PL team:
-
-  * https://metrics.penumbra.zone
-
-Check the usual place for credentials. Eventually those views should be exported
-as public references.
+the Penumbra software is doing.
 
 ## Adding Metrics
 
@@ -67,15 +57,7 @@ we use for maintaining our dashboards.
 
 ## Editing metrics locally
 
-To facilitate working with metrics locally, first run a `pd` node on your machine with the metrics endpoint
-exposed. Then, you can spin up a metrics sidecar deployment:
-
-```bash
-just metrics
-```
-
-Note that this setup only works on Linux hosts, due to the use of host networking, so the metrics
-containers can reach network ports on the host machine.
+A minimal metrics setup will be automatically provisioned as part of the [devnet quickstart](./devnet-quickstart.md).
 
-To add new Grafana visualizations, open http://localhost:3000 and edit the existing dashboards.
-When you're happy with what you've got, follow the "Backing up Grafana" instructions above to save your work.
+To add new Grafana visualizations, open [http://127.0.0.1:3000](http://127.0.0.1:3000) and edit the existing dashboards.
+When you're happy with what you've got, follow the [Backing up Grafana](./metrics.md#backing-up-grafana) instructions above to save your work.

pages/dev/privacy.md

@@ -0,0 +1,31 @@
+# Privacy
+
+This document describes the privacy features of the Penumbra protocol, as well as conditions that must be true for privacy to be preserved.
+
+## Transfers
+
+Transfers do not reveal the asset, amount, sender or recipient identity. Transfers are also unlinkable to each other.
+
+## Swaps
+
+Initiating a swap does not reveal the identity of the swapper or the pre-paid claim fee, but it does reveal the assets and amounts in the swap.
+
+Claiming a swap does not reveal the amounts, asset types or the identity of the claimant. 
+
+An observer of the chain will see that an anonymous account minted shielded outputs of a swap in a trading pair, but those outputs can't be linked to the claimant.
+
+## Governance Voting
+
+During a vote, the voting power (amount and asset type of the staked note that's used for voting),
+the vote itself, the identity of the validator (equivalent to the asset type), as well as the proposal being voted on are **revealed** during a delegator vote.
+However, the address of the voter is hidden, which renders the information revealed anonymous.
+
+<!--
+## Staking
+
+TODO
+
+## IBC
+
+TODO
+-->

pages/dev/protobuf.md

@@ -1,3 +1,5 @@
+# Maintaining protobuf specs
+
 The Penumbra project dynamically generates code for interfacing
 with [gRPC]. The following locations within the repository
 are relevant:

pages/dev/rpc.md

@@ -1,3 +1,5 @@
+# Working with gRPC for Penumbra
+
 The Penumbra [`pd`](../node/pd.md) application exposes a [gRPC] service for integration
 with other tools, such as [`pcli`](../pcli.md) or the [web extension](../web.md).
 A solid understanding of how the gRPC methods work is helpful when

pages/dev/sql.md

@@ -1,3 +1,5 @@
+# Working with SQLite
+
 The view server uses [SQLite3](https://sqlite.org) to store client state locally.
 During debugging, you may wish to interact with the sqlite db directly.
 To do so:

pages/node.md

@@ -1,3 +1,5 @@
+# Running a node
+
 Running a node is not necessary to use the protocol. Both the web extension and
 `pcli` are designed to operate with any RPC endpoint. However, we've tried to
 make it as easy as possible to run nodes so that users can host their own RPC.

pages/node/_meta.json

@@ -1,4 +1,4 @@
 {
   "pd": "Full node: pd",
   "pclientd": "Ultralight node: pclientd"
-}
+}

pages/node/pclientd.md

@@ -1,3 +1,5 @@
+# Local RPC with `pclientd`
+
 Penumbra's architecture separates public shared state from private per-user
 state.  Each user's state is known only to them and other parties they disclose
 it to.  While this provides many advantages -- and enables the core features of

pages/node/pclientd/build_transaction.md

@@ -1,3 +1,5 @@
+# Building Transactions
+
 Using the view and custody services to construct a transaction has four steps.
 
 ## Plan the Transaction

pages/node/pclientd/configure.md

@@ -1,3 +1,5 @@
+# Configuring `pclientd`
+
 First, install `pclientd` following the [instructions for installing
 `pcli`](../../pcli/install.md) but downloading `pclientd` rather than `pcli`.
 

pages/node/pclientd/rpc.md

@@ -1,3 +1,5 @@
+# Making RPC requests
+
 `pclientd` exposes a GRPC and GRPC-web endpoint at its `bind_addr`. Several
 services are available.
 

pages/node/pd/chain-upgrade.md

@@ -1,3 +1,5 @@
+# Performing chain upgrades
+
 When consensus-breaking changes are made to the Penumbra protocol,
 node operators must coordinate upgrading to the new version of the software
 at the same time. Penumbra uses a governance proposal for scheduling upgrades