Skip to content

Commit

Permalink
Describe using pindexer (#32)
Browse files Browse the repository at this point in the history
* Describe using pindexer

Closes #31

* fix: edit language in review

---------

Co-authored-by: Conor Schaefer <[email protected]>
  • Loading branch information
cronokirby and conorsch authored Sep 24, 2024
1 parent 40e8b61 commit e87f98f
Showing 1 changed file with 51 additions and 0 deletions.
51 changes: 51 additions & 0 deletions pages/node/pd/indexing-events.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
The `pd` software emits ABCI events while processing blocks. By default,
these blocks are stored in CometBFT's key-value database locally, but node operators
can opt-in to writing the events to an external PostgreSQL database.
Furthermore, the [`pindexer`](#using-pindexer) software can be used to take these raw ABCI events,
and produce penumbra-specific "app views" with rich formatted data.

## Configuring a Penumbra node to write events to postgres

Expand All @@ -18,6 +20,55 @@ The format for `DATABASE_URL` is specified in the [Postgres docs](https://www.po
After the node is running, check the logs for errors. Query the database with `SELECT height FROM blocks ORDER BY height DESC LIMIT 10;` and confirm
you're seeing the latest blocks being added to the database.

## Using Pindexer

`pindexer` reads from a Postgres ABCI event database, as described in the section above,
and produces a derived database with rich tables for Penumbra-specific views.
This is useful because the raw events database isn't easily or efficiently queryable
for Penumbra-specific use cases, such as seeing the current status of the DEX,
or the total amount of fees paid so far on the life of the chain.

To run `pindexer`, you'll need to have configured the Penumbra node to index events into Postgres. Then:

1. Create a Postgres database for the output of `pindexer`, say `$PINDEXER_DB`.
2. Run `pindexer --chain-id <CHAIN_ID> --genesis-json <ORIGINAL_GENESIS_FILE> --src-database-url postgresql://localhost:5432/$DATABASE_NAME --dst-database-url postgresql://localhost:5432/$PINDEXER_DB`

This assumes that Postgres is running locally on port 5432; for another setup the URL should change.

The `<ORIGINAL_GENESIS_FILE>` must be the `genesis.json` file for the chain, **before any upgrades.**
Note, in particular, that after an upgrade, there will be a new genesis file containing only a checkpoint.
`pindexer` specifically needs the original genesis file, because it needs to read information
about the start of the chain, such as initial allocations, to track changes over time accurately.

### Example: Total Supply Indexing

For example, after running `pindexer`, here's a query to get the total amount of the native
staking token up to the current height:

```sql
SELECT (staked_um + unstaked_um + auction + dex)::NUMERIC / 10^6 as total
FROM (
SELECT SUM(um) as staked_um
FROM (
SELECT *
FROM supply_validators
) validators
LEFT JOIN LATERAL (
SELECT um
FROM supply_total_staked
WHERE validator_id = id
ORDER BY height DESC
LIMIT 1
) ON TRUE
) staked
LEFT JOIN LATERAL (
SELECT um as unstaked_um, auction, dex
FROM supply_total_unstaked
ORDER BY height DESC
LIMIT 1
) on TRUE
```

## Running an archive node

If you are joining the network after a chain upgrade, the events behind the upgrade boundary
Expand Down

0 comments on commit e87f98f

Please sign in to comment.