test(jsonrpc): add jsonrpc compatibility test #419

cloudgray · 2025-08-11T14:23:34Z

Description

Summary

This PR implements a comprehensive JSON-RPC compatibility test suite that validates API compatibility between our evmd implementation and standard Ethereum clients (geth). The test suite addresses issue #101 by providing automated testing infrastructure to ensure our JSON-RPC endpoints behave consistently with Ethereum standards.

Testing

Quick Start

make test-rpc-compat

Manual Testing

Start individual components

# start evmd & geth node
./tests/jsonrpc/scripts/start-networks.sh

# build simulator
cd ./tests/jsonrpc/simulator
go build .

# Run simulator
./simulator

Configuration

Environment Variables

EVMD_URL: HTTP endpoint for evmd (default: http://localhost:8545)
EVMD_WS_URL: WebSocket endpoint for evmd (auto-derived from HTTP URL)
GETH_URL: HTTP endpoint for geth (default: http://localhost:8547)
GETH_WS_URL: WebSocket endpoint for geth (auto-derived from HTTP URL)

Test Accounts

Pre-configured test accounts with known private keys for deterministic testing:

dev0: Primary test account with substantial balance
dev1-dev3: Additional accounts for multi-account scenarios

Closes: #101

Author Checklist

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.

I have...

tackled an existing issue or discussed with a team member
left instructions on how to review the changes
targeted the main branch

zsystm · 2025-08-14T06:15:42Z

tests/jsonrpc/simulator/namespaces/engine.go

+	// Engine API namespace (not applicable for Cosmos chains)
+	MethodNameEngineNewPayloadV1        types.RpcName = "engine_newPayloadV1"
+	MethodNameEngineNewPayloadV2        types.RpcName = "engine_newPayloadV2"
+	MethodNameEngineNewPayloadV3        types.RpcName = "engine_newPayloadV3"


I couldn’t find any existing tests for this API.

zsystm · 2025-08-14T06:15:56Z

tests/jsonrpc/simulator/namespaces/engine.go

+	MethodNameEngineForkchoiceUpdatedV3 types.RpcName = "engine_forkchoiceUpdatedV3"
+	MethodNameEngineGetPayloadV1        types.RpcName = "engine_getPayloadV1"
+	MethodNameEngineGetPayloadV2        types.RpcName = "engine_getPayloadV2"
+	MethodNameEngineGetPayloadV3        types.RpcName = "engine_getPayloadV3"


I couldn’t find any existing tests for this API.

zsystm · 2025-08-14T06:15:59Z

tests/jsonrpc/simulator/namespaces/engine.go

+	MethodNameEngineNewPayloadV3        types.RpcName = "engine_newPayloadV3"
+	MethodNameEngineForkchoiceUpdatedV1 types.RpcName = "engine_forkchoiceUpdatedV1"
+	MethodNameEngineForkchoiceUpdatedV2 types.RpcName = "engine_forkchoiceUpdatedV2"
+	MethodNameEngineForkchoiceUpdatedV3 types.RpcName = "engine_forkchoiceUpdatedV3"


I couldn’t find any existing tests for this API.

zsystm · 2025-08-14T06:22:37Z

tests/jsonrpc/simulator/namespaces/debug.go

+)
+
+const (
+	NamespaceDebug = "debug"


It seems that for the debug namespace, this PR doesn’t compare results from Geth and from evmd.
To ensure JSON-RPC compatibility, every test should treat Geth’s result as the source of truth and verify whether evmd matches it or if anything is missing.

If that’s the case, could you create a follow-up issue for this? We should clarify the test methodology used in this PR and the exact coverage scope.

zsystm · 2025-08-14T12:02:28Z

tests/jsonrpc/scripts/evmd/start-evmd.sh

It looks like we have two start scripts serving similar purposes but in different contexts:

container-start-evmd.sh: runs inside the container (no Docker CLI), handles init/genesis tweaks, then evmd start.

start-evmd.sh: runs on the host, manages Docker lifecycle, port exposure, and applies additional genesis/app.toml tweaks (prometheus, pruning, static precompiles, STRv2 pair, etc.).

To avoid drift and reduce maintenance, could we consolidate shared logic into a single container-side bootstrap (e.g., scripts/evmd/bootstrap.sh) controlled via env vars (ENABLE_PROM, ENABLE_STRV2, RPC_BIND_ALL, BLOCK_MAX_GAS, …)?
The host script would then be a thin wrapper that only runs Docker and passes env/ports/volumes.

If consolidation is out of scope here, can we at least document the intentional differences and extract the common init (keys/genesis/params) into a helper that both call?

zsystm · 2025-08-14T12:06:11Z

tests/jsonrpc/simulator/namespaces/eth.go

+	MethodNameEthUnsubscribe types.RpcName = "eth_unsubscribe"
+)
+
+func EthCoinbase(rCtx *types.RPCContext) (*types.RpcResult, error) {


This should be removed after #456 is merged, since even Geth no longer supports this API. Other legacy APIs should also be removed — something we should keep in mind going forward.

@zsystm good to remove now then?

aljo242 · 2025-08-14T21:24:52Z

@cloudgray let's not commit a binary into the codebase. Can we just build this binary instead in the process?

cloudgray · 2025-08-18T07:32:26Z

@cloudgray let's not commit a binary into the codebase. Can we just build this binary instead in the process?

My bad. I removed. (a32dfe6)

aljo242 · 2025-08-18T16:05:42Z

tests/jsonrpc/simulator/go.mod

+go 1.23.8
+
+require (
+	github.com/ethereum/go-ethereum v1.15.10


do we want to update the go-ethereum dep to the latest

cloudgray · 2025-08-19T01:20:24Z

@aljo242 Oh, this PR was merged just after added to merge queue, before I modify code from code-review comment.
I'll create new PR that fixes items from code review comments.

cloudgray added 30 commits August 1, 2025 19:58

test(rpc-compat): add test simulator

f28c933

refactor(tests): jsonrpc test

d2892a1

test(jsonprc): refactor test suite

d389b8e

test(jsonrpc): refactor test suite

ea68a9d

test(jsonrpc): refactor test suite

4a068bc

test(jsonrpc): add network start/stop scripts of evmd and geth

46b4434

test(jsonrpc): fix api counting in report

39c6c10

test(jsonrpc): refactor result format

8f32e50

test(jsonrpc): refactor

92b26e9

test(jsonrpc): add state generator

e125461

test(jsonrpc): refactor simulator

aa42107

test(jsonrpc): refactor

60185dd

test(jsonrpc): setup geth dev mode

44fdef2

test(jsonrpc): clean unused code

e154a07

test(jsonrpc): setup initial state

540e38b

test(jsonrpc): add tx metadata for test

b2a5cfa

WIP: test(jsonrpc): fix some failing tests of eth_ apis

9be672d

test(jsonrpc): fix eth test

d020b7f

WIP: test(jsonrpc): fix debug api test cases

89876e1

test(jsonrpc): set pruning config as nothing for testing

58b814d

test(jsonrpc): enhance debug testing

f4bed6e

test(jsonrpc): fix net test case

111e866

chore: update working history

056bfa8

test(jsonrpc): skip personal apis

25081d5

test(jsonrpc): add websocket api test

086f067

jsonrpc(test): fix personal apis

7fdb7cb

test(jsonrpc): add missing debug apis

ef4d773

test(jsonrpc): update working history

81dba15

test(jsonrpc): fix eth_sign / eth_signTransaction test

d9df2e4

test(jsonrpc): refactor simulator

1ef72e2

cloudgray added 10 commits August 13, 2025 21:52

fix: github-ci

63cc892

fix: github-ci

ed7be1b

fix: github-ci

f62ef68

test(jsonrpc): modify test script

d2e7984

fix: github-ci

01b397f

test(jsonrpc): refactor test suite

32b1d3c

test(jsonrpc): refactor test suite

efef814

test(jsonrpc): fix websocket api tests

f474a53

chore: update README.md for jsonrpc test

ec26ad7

Merge branch 'main' into test/rpc-compat

47da06d

cloudgray marked this pull request as ready for review August 13, 2025 19:51

cloudgray requested review from a team as code owners August 13, 2025 19:51

zsystm reviewed Aug 14, 2025

View reviewed changes

aljo242 added the v0.5.x targeting v0.5.x label Aug 14, 2025

Merge branch 'main' into test/rpc-compat

0fa72b7

aljo242 self-assigned this Aug 14, 2025

cloudgray added 2 commits August 18, 2025 16:31

chore: remove binary file

a32dfe6

Merge branch 'main' into test/rpc-compat

310c720

Merge branch 'main' into test/rpc-compat

416d264

aljo242 enabled auto-merge August 18, 2025 15:49

Merge branch 'main' into test/rpc-compat

b76f2f6

aljo242 approved these changes Aug 18, 2025

View reviewed changes

aljo242 added this pull request to the merge queue Aug 18, 2025

Merged via the queue into cosmos:main with commit ab1cd0c Aug 18, 2025
18 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

test(jsonrpc): add jsonrpc compatibility test #419

test(jsonrpc): add jsonrpc compatibility test #419

Uh oh!

cloudgray commented Aug 11, 2025 •

edited

Loading

Uh oh!

zsystm Aug 14, 2025

Uh oh!

zsystm Aug 14, 2025

Uh oh!

zsystm Aug 14, 2025

Uh oh!

zsystm Aug 14, 2025

Uh oh!

zsystm Aug 14, 2025

Uh oh!

zsystm Aug 14, 2025

Uh oh!

aljo242 Aug 14, 2025

Uh oh!

aljo242 commented Aug 14, 2025

Uh oh!

cloudgray commented Aug 18, 2025

Uh oh!

aljo242 Aug 18, 2025

Uh oh!

Uh oh!

cloudgray commented Aug 19, 2025

Uh oh!

Uh oh!

test(jsonrpc): add jsonrpc compatibility test #419

test(jsonrpc): add jsonrpc compatibility test #419

Uh oh!

Conversation

cloudgray commented Aug 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Summary

Testing

Quick Start

Manual Testing

Configuration

Environment Variables

Test Accounts

Author Checklist

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aljo242 commented Aug 14, 2025

Uh oh!

cloudgray commented Aug 18, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cloudgray commented Aug 19, 2025

Uh oh!

Uh oh!

cloudgray commented Aug 11, 2025 •

edited

Loading