v0.8 (#1235)

* cached transport supports all methods * add more actions to context.client * fix: required block parameters for some methods * start new filter impl * debug rpc types * withLog * add debug helper functions * syncTrace in sync-historical added with filter matching * syncTrace filtering modified, blockwise processing added * sync-realtime: added tx/transfer factories * sync: fragments and encoding for transaction/transfer filters * sync-realtime: transaction/trasfer filter added * sync-realtime: handleBlock is modified for new filters * sync-realtime: fetchBlockEventData changed to debug and new filters for traces * update historical sync * build checkpoint for traces * record failed traces * implement insert traces db method * . * cleanup realtime * corrections: minor changes * sync-historical: traceCache added * sync-realtime: trace traversal added in fetchBlockEventData and handleBlock * sync: buildEvents and decodeEvents modfied * fix debug type * merge all interval tables into one * fix typo * filter/filterinterval migration into intervals * sync-store: fragment_id modified * fix fragment tests * stricter fragment id types * multi range * disable cache skips interval table * cleanup * batched getIntervals * fix: insert empty intervals * cleanup * migrations * order request * new rpc request table * notes * new filter types * decodeEvents + buildEvents * fix: range_agg * fix: range merging * fix: nits * more adjacent filters * rpc request migrations * required block actions * docs * fix * snake case * chore: changeset * preserve arrays * fix: ordering object arrays * chore: changeset * remove down migration * fix tests * fix: migration bug * fix: parse rpc response from cache * sync-historical: transaction filter * getEvents * account sources * accounts config * sync-realtime: fetchBlockEventData and handleBlock modified to handle account filters * cleanup realtime logs * minor fix * build accounts * start historical sync tests * update sync tests * sync: handle factory address * cleanup * fix transport tests * realtime sync tests * fix typecheck * account specific test * accounts example * fix virtual types * cleanup sync * note * transaction includeReverted * cleanup * back * factory fn * fix * filter * fix: config * sync: factory filter * sync: factory filter cont. * include * build * set * feature-accounts * cleanup * fix type * nits * nits * fragments * sync and events * update docs * update examples * update docs * cleanup * internal * trace type * nits * nits * remove transactionReceipt.logs * fix type error * fix * cleanup * fix config * fix query * normalize event * fix interval migrations * @/generated * fix accounts example * fix normalization * cleanup * fix * try fix actions * try fix actions * start migration guide * fix transaction event type * update mg * fix type test * use request hash to avoid large index * cleanup migration guide * fix migrations rounding * cleanup mg * config reference * cleanup * fix transfer txr type * update indexing functions docs * skip migrating checkpoint in trace migration * fix changeset ignore * docs * call traces * database schema manual deployments (#1312) * start deploy docs * more docs * cli * . * start new database service * test * dev * fix e2e * update direct sql * cli * fix schema bug * meta migration * explicit pre-build * database parameters * merge results * log * start list * db list * docs * improve error messages * cleanup * db list * cleanup list * dev fast path * accounts docs * remove as of * database docs mostly done * dev drop tables * direct sql * migration * fix: migrations compatible with postgres 14 * docs * error message tweaks --------- Co-authored-by: typedarray <[email protected]> * migration guide * changeset --------- Co-authored-by: Work <[email protected]> Co-authored-by: typedarray <[email protected]> Co-authored-by: Kevin <[email protected]>
ponder-sh · Dec 11, 2024 · 37ed2e1 · 37ed2e1
1 parent 2f12876
commit 37ed2e1
Show file tree

Hide file tree

Showing 260 changed files with 12,768 additions and 9,647 deletions.
diff --git a/.changeset/config.json b/.changeset/config.json
@@ -2,9 +2,10 @@
   "$schema": "https://unpkg.com/@changesets/[email protected]/schema.json",
   "changelog": ["@changesets/changelog-github", { "repo": "ponder-sh/ponder" }],
   "commit": false,
-  "fixed": [["@ponder/core", "create-ponder", "eslint-config-ponder"]],
+  "fixed": [["ponder", "create-ponder", "eslint-config-ponder"]],
   "ignore": [
     "@ponder/common",
+    "ponder-examples-feature-accounts",
     "ponder-examples-feature-blocks",
     "ponder-examples-feature-factory",
     "ponder-examples-feature-filter",

diff --git a/.changeset/early-days-turn.md b/.changeset/early-days-turn.md
@@ -0,0 +1,5 @@
+---
+"ponder": minor
+---
+
+Released `0.8`. Visit the [migration guide](https://ponder.sh/docs/migration-guide) for a full list of changes.
diff --git a/.changeset/rotten-socks-agree.md b/.changeset/rotten-socks-agree.md
@@ -0,0 +1,5 @@
+---
+"@ponder/core": patch
+---
+
+Added support for caching all available rpc methods. Several new client actions are available, including `context.client.simulateCall()`, `context.client.getTransactionCount()`, `context.client.getTransactionReceipt()`, and `context.client.getBlockTransactionCount()`. See [docs](https://ponder.sh/docs/indexing/read-contract-data#supported-actions) for all actions.
diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md
@@ -93,7 +93,7 @@ In the root directory, run the build command:
 pnpm build
 ```
 
-After the build completes, pnpm links packages across the project for development. This means that if you run any of the projects in the `examples/` directory, they will use the local version of `@ponder/core`.
+After the build completes, pnpm links packages across the project for development. This means that if you run any of the projects in the `examples/` directory, they will use the local version of `ponder`.
 
 <div align="right">
   <a href="#get-started">&uarr; back to top</a></b>
@@ -169,7 +169,7 @@ Ponder is still in alpha, so all changes should be marked as a patch.
 ### Changesets workflow
 
 1. Write a PR that includes a public API change or bug fix.
-2. Create a changeset using `pnpm changeset`. The changesets CLI will ask you which package is affected (`@ponder/core` or `create-ponder`) and if the change is a patch, minor, or major release.
+2. Create a changeset using `pnpm changeset`. The changesets CLI will ask you which package is affected (`ponder` or `create-ponder`) and if the change is a patch, minor, or major release.
 3. The changesets CLI will generate a Markdown file in `.changeset/` that includes the details you provided. Commit this file to your PR branch (e.g. `git commit -m "chore: changeset"`).
 4. When you push this commit to remote, a GitHub bot will detect the changeset and add a comment to your PR with a preview of the changelog.
 5. Merge your PR. The changesets Github Action workflow will open (or update) a PR with the title `"chore: version packages"`. The changes in your PR **will not be released** until this PR is merged.

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
@@ -60,7 +60,7 @@ jobs:
 
       - name: Setup Postgres
         if: ${{ matrix.database == 'Postgres' }}
-        uses: ikalnytskyi/action-setup-postgres@v4
+        uses: ikalnytskyi/action-setup-postgres@v7
         id: postgres
 
       - name: Setup

diff --git a/README.md b/README.md
@@ -62,7 +62,7 @@ Ponder fetches event logs for the contracts added to `ponder.config.ts`, and pas
 ```ts
 // ponder.config.ts
 
-import { createConfig } from "@ponder/core";
+import { createConfig } from "ponder";
 import { http } from "viem";
 
 import { BaseRegistrarAbi } from "./abis/BaseRegistrar";
@@ -92,7 +92,7 @@ The `ponder.schema.ts` file contains the database schema, and defines the shape
 ```ts
 // ponder.schema.ts
 
-import { onchainTable } from "@ponder/core";
+import { onchainTable } from "ponder";
 
 export const ensName = onchainTable("ens_name", (t) => ({
   name: p.text().primaryKey(),
@@ -108,8 +108,8 @@ Files in the `src/` directory contain **indexing functions**, which are TypeScri
 ```ts
 // src/BaseRegistrar.ts
 
-import { ponder } from "@/generated";
-import * as schema from "../ponder.schema";
+import { ponder } from "ponder:registry";
+import schema from "ponder:schema";
 
 ponder.on("BaseRegistrar:NameRegistered", async ({ event, context }) => {
   const { name, owner } = event.params;
@@ -167,7 +167,7 @@ If you're interested in contributing to Ponder, please read the [contribution gu
 
 ## Packages
 
-- `@ponder/core`
+- `ponder`
 - `@ponder/utils`
 - `create-ponder`
 - `eslint-config-ponder`
@@ -180,7 +180,7 @@ Ponder is MIT-licensed open-source software.
 [ci-url]: https://github.com/ponder-sh/ponder/actions/workflows/main.yml
 [tg-badge]: https://img.shields.io/endpoint?color=neon&logo=telegram&label=Chat&url=https%3A%2F%2Fmogyo.ro%2Fquart-apis%2Ftgmembercount%3Fchat_id%3Dponder_sh
 [tg-url]: https://t.me/ponder_sh
-[license-badge]: https://img.shields.io/npm/l/@ponder/core?label=License
+[license-badge]: https://img.shields.io/npm/l/ponder?label=License
 [license-url]: https://github.com/ponder-sh/ponder/blob/main/LICENSE
-[version-badge]: https://img.shields.io/npm/v/@ponder/core
+[version-badge]: https://img.shields.io/npm/v/ponder
 [version-url]: https://github.com/ponder-sh/ponder/releases
diff --git a/benchmarks/apps/ponder-reth/ponder.config.ts b/benchmarks/apps/ponder-reth/ponder.config.ts
@@ -1,4 +1,4 @@
-import { createConfig } from "@ponder/core";
+import { createConfig } from "ponder";
 import { http } from "viem";
 
 import { RocketTokenRETHAbi } from "./abis/RocketTokenRETH";

diff --git a/benchmarks/apps/ponder-reth/ponder.schema.ts b/benchmarks/apps/ponder-reth/ponder.schema.ts
@@ -1,4 +1,4 @@
-import { createSchema } from "@ponder/core";
+import { createSchema } from "ponder";
 
 export default createSchema((p) => ({
   Account: p.createTable({

diff --git a/benchmarks/apps/ponder-reth/src/index.ts b/benchmarks/apps/ponder-reth/src/index.ts
@@ -2,7 +2,7 @@
 import { ponder } from "@/generated";
 
 // biome-ignore lint/suspicious/noRedeclare: :)
-declare const ponder: import("@ponder/core").PonderApp<
+declare const ponder: import("ponder").PonderApp<
   typeof import("../ponder.config.js").default,
   typeof import("../ponder.schema.js").default
 >;

diff --git a/benchmarks/apps/ponder-univ2/ponder.config.ts b/benchmarks/apps/ponder-univ2/ponder.config.ts
@@ -1,4 +1,4 @@
-import { createConfig } from "@ponder/core";
+import { createConfig } from "ponder";
 import { http, getAbiItem } from "viem";
 
 import { FactoryAbi } from "./abis/FactoryAbi";

diff --git a/benchmarks/apps/ponder-univ2/ponder.schema.ts b/benchmarks/apps/ponder-univ2/ponder.schema.ts
@@ -1,4 +1,4 @@
-import { createSchema } from "@ponder/core";
+import { createSchema } from "ponder";
 
 export default createSchema((p) => ({
   UniswapFactory: p.createTable({

diff --git a/benchmarks/apps/ponder-univ2/src/index.ts b/benchmarks/apps/ponder-univ2/src/index.ts
@@ -2,7 +2,7 @@
 import { ponder } from "@/generated";
 
 // biome-ignore lint/suspicious/noRedeclare: :)
-declare const ponder: import("@ponder/core").PonderApp<
+declare const ponder: import("ponder").PonderApp<
   typeof import("../ponder.config.js").default,
   typeof import("../ponder.schema.js").default
 >;

diff --git a/benchmarks/package.json b/benchmarks/package.json
@@ -11,7 +11,7 @@
   "devDependencies": {
     "@graphprotocol/graph-cli": "0.61.0",
     "@graphprotocol/graph-ts": "^0.31.0",
-    "@ponder/core": "workspace:*",
+    "ponder": "workspace:*",
     "@types/node": "^20.10.0",
     "execa": "^8.0.1",
     "tsup": "^8.0.1",

diff --git a/docs/pages/docs/_meta.ts b/docs/pages/docs/_meta.ts
@@ -16,6 +16,7 @@ export default {
     title: "Config",
   },
   "contracts-and-networks": "Contracts & networks",
+  "accounts": "Accounts",
   "block-intervals": "Block intervals",
   "call-traces": "Call traces",
 

diff --git a/docs/pages/docs/accounts.mdx b/docs/pages/docs/accounts.mdx
@@ -0,0 +1,98 @@
+---
+title: "Accounts"
+description: "Learn how to index accounts in a Ponder app"
+---
+
+import { Callout, Steps } from "nextra/components";
+
+# Accounts
+
+Account indexing is useful for activity that is not associated with a specific contract, like transactions and native transfers to and from an address. The key difference between `contracts` and `accounts` in `ponder.config.ts` is that accounts do not have an ABI.
+
+<Callout type="info">
+  The standard Ethereum RPC does not support filtering for transactions & native
+  transfers. So, account indexing uses a block-by-block approach which is often
+  slower than log indexing.
+</Callout>
+
+## Quick example
+
+In this example, we'll index transactions sent by the [Beaver](https://beaverbuild.org/) block builder account.
+
+<Steps>
+
+### Add an account
+
+Add the network, address, and start block to the `accounts` field in `ponder.config.ts`.
+
+```ts filename="ponder.config.ts"
+import { createConfig } from "ponder";
+
+export default createConfig({
+  // ... more config
+  accounts: {
+    BeaverBuild: {
+      network: "mainnet",
+      address: "0x95222290DD7278Aa3Ddd389Cc1E1d165CC4BAfe5",
+      startBlock: 20000000,
+    },
+  },
+});
+```
+
+### Register indexing functions
+
+Register an indexing function for the `transaction:from` event. The framework will fetch all transactions where `transaction.from` matches the account address, then process each one using your indexing function.
+
+```ts filename="src/index.ts" {4}
+import { ponder } from "ponder:registry";
+import { transactions } from "ponder:schema";
+
+ponder.on("BeaverBuilder:transaction:from", async ({ event, context }) => {
+  await context.db.insert(transactions).values({
+    from: event.transaction.from,
+    to: event.transaction.to,
+    value: event.transaction.value,
+    gasUsed: event.transactionReceipt.gasUsed,
+    // ... more fields
+  });
+});
+```
+
+Account indexing also supports the `transaction:to`, `transfer:from`, and `transfer:to` events. [Read more](/docs/api-reference/config#accounts) about event types.
+
+</Steps>
+
+## Account name
+
+Every account must have a unique name, provided as a key to the `accounts` object. Names must be unique across accounts, contracts, and block intervals.
+
+```ts filename="ponder.config.ts" {9}
+import { createConfig } from "ponder";
+import { http } from "viem";
+
+export default createConfig({
+  networks: {
+    mainnet: { chainId: 1, transport: http(process.env.PONDER_RPC_URL_1) },
+  },
+  accounts: {
+    BeaverBuilder: {
+      network: "mainnet",
+      address: "0x95222290DD7278Aa3Ddd389Cc1E1d165CC4BAfe5",
+      startBlock: 12439123,
+    },
+  },
+});
+```
+
+## Network
+
+The `network` option for accounts works the same way as it does for contracts. [Read more](/docs/contracts-and-networks#network).
+
+## Address
+
+The `address` option for accounts works the same way as it does for contracts. You can provide a single address, a list of addresses, or an address factory. You can also specify network-specific overrides. [Read more](/docs/contracts-and-networks#address).
+
+## Block range
+
+The `startBlock` and `endBlock` options for accounts work the same way as it does for contracts. [Read more](/docs/contracts-and-networks#block-range).
diff --git a/docs/pages/docs/advanced/foundry.mdx b/docs/pages/docs/advanced/foundry.mdx
@@ -20,12 +20,12 @@ Ponder's RPC request cache works well for live networks where the chain is gener
 Use the `disableCache` option to **disable RPC request caching** for the Anvil network. With this option set to true, Ponder will clear the cache on start up and between hot reloads.
 
 ```ts filename="ponder.config.ts" {9}
-import { createConfig } from "@ponder/core";
+import { createConfig } from "ponder";
 import { http } from "viem";
 
 export default createConfig({
   networks: {
-    anvil: { 
+    anvil: {
       chainId: 31337,
       transport: http("http://127.0.0.1:8545"),
       disableCache: true,
@@ -43,7 +43,11 @@ We recommend using `31337` (the default Anvil chain ID) even when forking a live
 
 We recommend using [interval mining](https://book.getfoundry.sh/reference/anvil/#mining-modes) with a block time of ~2 seconds. This better simulates a live network.
 
-<Callout type="warning">Known issue: When indexing Anvil with auto mining enabled in an app with multiple networks, indexing progress will get "stuck" at the timestamp of the latest Anvil block. </Callout>
+<Callout type="warning">
+  Known issue: When indexing Anvil with auto mining enabled in an app with
+  multiple networks, indexing progress will get "stuck" at the timestamp of the
+  latest Anvil block.{" "}
+</Callout>
 
 ## Generate ABI files
 
@@ -74,7 +78,11 @@ export default defineConfig({
 
 Foundry scripts write transaction inputs and receipts to JSON files in the `broadcast` directory. You can import these files directly into `ponder.config.ts` to automate address management and enable hot reloading.
 
-<Callout type="info">Remember to enable [broadcast](https://book.getfoundry.sh/tutorials/solidity-scripting?highlight=deploy#deploying-locally) so that `forge script` submits transactions to Anvil.</Callout>
+<Callout type="info">
+  Remember to enable
+  [broadcast](https://book.getfoundry.sh/tutorials/solidity-scripting?highlight=deploy#deploying-locally)
+  so that `forge script` submits transactions to Anvil.
+</Callout>
 
 ### Automate address management
 
@@ -83,7 +91,7 @@ To read the contract address and deployment block number from a broadcast file,
 The `ponder.config.ts` file from the Foundry [example project](https://github.com/ponder-sh/ponder/tree/main/examples/with-foundry) demonstrates this pattern. Here, the first transaction in the broadcast file deployed the `Counter.sol` contract. The location of the contract address and start block within the broadcast file depends on the order and number of transactions in your deployment script.
 
 ```ts filename="ponder.config.ts" {4, 6-7, 21-22}
-import { createConfig } from "@ponder/core";
+import { createConfig } from "ponder";
 import { http, getAddress, hexToNumber } from "viem";
 import { counterABI } from "../abis/CounterAbi";
 import CounterDeploy from "../foundry/broadcast/Deploy.s.sol/31337/run-latest.json";
@@ -115,12 +123,12 @@ export default createConfig({
 If you import a JSON broadcast file in `ponder.config.ts`, the dev server will reload each time that file changes. This is a simple way to ensure that Ponder reloads every time you run a Foundry deployment script.
 
 ```ts filename="ponder.config.ts" {3-4}
-import { createConfig } from "@ponder/core";
+import { createConfig } from "ponder";
 import { http } from "viem";
 import CounterDeploy from "../foundry/broadcast/Deploy.s.sol/31337/run-latest.json";
 // ^ The development server detects changes to this file and triggers a hot reload.
 
 export default createConfig({
   // ...
 });
-```
+```
diff --git a/docs/pages/docs/advanced/telemetry.mdx b/docs/pages/docs/advanced/telemetry.mdx
@@ -7,7 +7,7 @@ import { Callout } from "nextra/components";
 
 # Telemetry
 
-As of version `0.0.79`, Ponder collects **completely anonymous** telemetry data about general usage. The developers use this data to prioritize new feature development, identify bugs, and improve performance & stability.
+Ponder collects **completely anonymous** telemetry data about general usage. The developers use this data to prioritize new feature development, identify bugs, and improve performance & stability.
 
 ## Opt out
 
@@ -20,6 +20,6 @@ PONDER_TELEMETRY_DISABLED = true
 
 ## Implementation
 
-Ponder's telemetry implementation is 100% open-source. The [telemetry service](https://github.com/ponder-sh/ponder/blob/main/packages/core/src/common/telemetry.ts#L47) (part of `@ponder/core`) runs on the user's device and submits event data via HTTP POST requests to the [telemetry collection endpoint](https://github.com/ponder-sh/ponder/blob/main/docs/pages/api/telemetry/index.ts) hosted at `https://ponder.sh/api/telemetry`.
+Ponder's telemetry implementation is 100% open-source. The [telemetry service](https://github.com/ponder-sh/ponder/blob/main/packages/core/src/common/telemetry.ts#L47) (part of `ponder`) runs on the user's device and submits event data via HTTP POST requests to the [telemetry collection endpoint](https://github.com/ponder-sh/ponder/blob/main/docs/pages/api/telemetry/index.ts) hosted at `https://ponder.sh/api/telemetry`.
 
 The implementation generates a stable anonymous unique identifier for the user's device and stores it at the [system default user config directory](https://github.com/sindresorhus/env-paths#pathsconfig). This config also stores the user's opt-out preference and a stable salt used to hash potentially sensitive data such as file paths and the git remote URL.