From e87f98ffe0dc31053c8b3c82ade7aadd4d528bd7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?L=C3=BAc=C3=A1s=20Meier?= Date: Tue, 24 Sep 2024 09:32:35 -0700 Subject: [PATCH] Describe using pindexer (#32) * Describe using pindexer Closes #31 * fix: edit language in review --------- Co-authored-by: Conor Schaefer --- pages/node/pd/indexing-events.md | 51 ++++++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) diff --git a/pages/node/pd/indexing-events.md b/pages/node/pd/indexing-events.md index 38a2dcb..4d13832 100644 --- a/pages/node/pd/indexing-events.md +++ b/pages/node/pd/indexing-events.md @@ -3,6 +3,8 @@ The `pd` software emits ABCI events while processing blocks. By default, these blocks are stored in CometBFT's key-value database locally, but node operators can opt-in to writing the events to an external PostgreSQL database. +Furthermore, the [`pindexer`](#using-pindexer) software can be used to take these raw ABCI events, +and produce penumbra-specific "app views" with rich formatted data. ## Configuring a Penumbra node to write events to postgres @@ -18,6 +20,55 @@ The format for `DATABASE_URL` is specified in the [Postgres docs](https://www.po After the node is running, check the logs for errors. Query the database with `SELECT height FROM blocks ORDER BY height DESC LIMIT 10;` and confirm you're seeing the latest blocks being added to the database. +## Using Pindexer + +`pindexer` reads from a Postgres ABCI event database, as described in the section above, +and produces a derived database with rich tables for Penumbra-specific views. +This is useful because the raw events database isn't easily or efficiently queryable +for Penumbra-specific use cases, such as seeing the current status of the DEX, +or the total amount of fees paid so far on the life of the chain. + +To run `pindexer`, you'll need to have configured the Penumbra node to index events into Postgres. Then: + +1. Create a Postgres database for the output of `pindexer`, say `$PINDEXER_DB`. +2. Run `pindexer --chain-id --genesis-json --src-database-url postgresql://localhost:5432/$DATABASE_NAME --dst-database-url postgresql://localhost:5432/$PINDEXER_DB` + +This assumes that Postgres is running locally on port 5432; for another setup the URL should change. + +The `` must be the `genesis.json` file for the chain, **before any upgrades.** +Note, in particular, that after an upgrade, there will be a new genesis file containing only a checkpoint. +`pindexer` specifically needs the original genesis file, because it needs to read information +about the start of the chain, such as initial allocations, to track changes over time accurately. + +### Example: Total Supply Indexing + +For example, after running `pindexer`, here's a query to get the total amount of the native +staking token up to the current height: + +```sql +SELECT (staked_um + unstaked_um + auction + dex)::NUMERIC / 10^6 as total +FROM ( + SELECT SUM(um) as staked_um + FROM ( + SELECT * + FROM supply_validators + ) validators + LEFT JOIN LATERAL ( + SELECT um + FROM supply_total_staked + WHERE validator_id = id + ORDER BY height DESC + LIMIT 1 + ) ON TRUE +) staked +LEFT JOIN LATERAL ( + SELECT um as unstaked_um, auction, dex + FROM supply_total_unstaked + ORDER BY height DESC + LIMIT 1 +) on TRUE +``` + ## Running an archive node If you are joining the network after a chain upgrade, the events behind the upgrade boundary