feat: use websockets to ingest blocks near chain tip

[kyscott18]

Description

Explore using websockets to watch for new blocks in sync-realtime rather than relying on polling. This would be an improvement for latency critical application, and save on rpc credits for some apps as well.

Specifically, I think we could use the "eth_subscribe" method in replacement of the eth_getBlockByNumber and setInterval.

ponder/packages/core/src/sync-realtime/index.ts

Lines 810 to 857 in e6a0ab8

	const enqueue = async () => {
	try {
	const block = await _eth_getBlockByNumber(args.requestQueue, {
	blockTag: "latest",
	});
	const latestBlock = getLatestUnfinalizedBlock();
	// We already saw and handled this block. No-op.
	if (latestBlock.hash === block.hash) {
	args.common.logger.trace({
	service: "realtime",
	msg: `Skipped processing '${args.network.name}' block ${hexToNumber(block.number)}, already synced`,
	});
	return;
	}
	const blockWithEventData = await fetchBlockEventData(block);
	consecutiveErrors = 0;
	return queue.add(blockWithEventData);
	} catch (_error) {
	if (isKilled) return;
	const error = _error as Error;
	args.common.logger.warn({
	service: "realtime",
	msg: `Failed to fetch latest '${args.network.name}' block`,
	error,
	});
	// After a certain number of attempts, emit a fatal error.
	if (++consecutiveErrors === ERROR_TIMEOUT.length) {
	args.common.logger.error({
	service: "realtime",
	msg: `Fatal error: Unable to fetch latest '${args.network.name}' block after ${ERROR_TIMEOUT.length} attempts.`,
	error,
	});
	args.onFatalError(error);
	}
	}
	};
	interval = setInterval(enqueue, args.network.pollingInterval);

After this upgrade and #1115, Ponder would be close to the minimum latency possible. It would be good to start to get an intuition around the short comings of the "eth_subscribe" rpc method and what work arounds we would need.

This would most likely require changes to ponder.config.ts. I think it could look like:

{
  chains: [mainnet, optimism],
  rpcUrls: {
    [mainnet.id]: ["https://mainnet.alchemy.com/...", ...],
    [optimism.id]: "https://mainnet.optimism.io",
  },
}

We probably would still need to preserve the polling option when there is no available websocket connection.

Related

• https://viem.sh/docs/actions/public/watchBlocks#watchblocks