Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vm: adjust test runner with verkle state management #3716

Draft
wants to merge 79 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
79 commits
Select commit Hold shift + click to select a range
34a5f08
vm: verkle testing wip
gabrocheleau Sep 29, 2024
ac621b7
Add missing getStateRoot method
acolytec3 Sep 30, 2024
618074c
vm: add option for state manager
gabrocheleau Sep 30, 2024
a932790
Add expanded logging
acolytec3 Oct 1, 2024
a6bd723
fix account creation in preconditions step
acolytec3 Oct 1, 2024
eff3d6f
Add statemanager arg to runner args
acolytec3 Oct 1, 2024
8459c6c
add setStateRoot to verkleStateful
acolytec3 Oct 1, 2024
f971e70
vm: fix test runner
gabrocheleau Oct 1, 2024
b29c56a
vm: update generalStateTestRunner with verkle stateManager option
gabrocheleau Oct 2, 2024
bf7b152
Add verkle specific common setup
acolytec3 Oct 3, 2024
9694e21
Merge remote-tracking branch 'origin/master' into vm/verkle-testing
acolytec3 Oct 8, 2024
f83a1c5
lint
acolytec3 Oct 8, 2024
635648d
Merge remote-tracking branch 'origin/master' into vm/verkle-testing
acolytec3 Oct 24, 2024
b6e0ac7
Add fixes to allow verkle blocks to run
acolytec3 Oct 28, 2024
f7f8f73
Add comments on block constructor
acolytec3 Oct 28, 2024
f07055c
Merge remote-tracking branch 'origin/master' into vm/verkle-testing
acolytec3 Oct 28, 2024
91d3dd3
initialize witness
acolytec3 Oct 28, 2024
0e9f438
fix typo in postState verification check
acolytec3 Oct 28, 2024
c61cf3d
Reorganize verkle checks
acolytec3 Oct 28, 2024
3d86e4f
fix code chunking size calculation
acolytec3 Oct 28, 2024
4fe493f
clean up
acolytec3 Oct 28, 2024
5543970
Merge remote-tracking branch 'origin/master' into vm/verkle-testing
acolytec3 Oct 29, 2024
f29f458
Add trie walking helper
acolytec3 Oct 29, 2024
e0adfb6
remove rlp encoding of storage values
acolytec3 Oct 29, 2024
e3a614a
Merge branch 'master' into vm/verkle-testing
gabrocheleau Oct 30, 2024
0ab7ba2
cspell fix
gabrocheleau Oct 30, 2024
c5433d1
typo
acolytec3 Oct 30, 2024
b137c41
update logger
acolytec3 Oct 30, 2024
e05d41e
logic bugs
acolytec3 Oct 30, 2024
72f3bfd
Merge branch 'vm/verkle-testing' of https://github.com/ethereumjs/eth…
gabrocheleau Oct 31, 2024
66d9f04
Move verkle crypto loading out of run function
acolytec3 Oct 31, 2024
549d21b
Merge branch 'vm/verkle-testing' of https://github.com/ethereumjs/eth…
gabrocheleau Nov 1, 2024
22098c1
Merge branch 'master' into vm/verkle-testing
gabrocheleau Nov 1, 2024
063c873
Merge branch 'vm/verkle-testing' of https://github.com/ethereumjs/eth…
gabrocheleau Nov 1, 2024
e6dcdad
vm: re add validation logic for stateful verkle
gabrocheleau Nov 1, 2024
2d93031
Merge branch 'master' into vm/verkle-testing
gabrocheleau Nov 2, 2024
da060f8
verkle:improve type of createDefaultLeafVerkleValues
gabrocheleau Nov 3, 2024
a0c3536
Add new util method for dumping node hashes
acolytec3 Nov 4, 2024
d074be6
Partial fix for null child nodes
acolytec3 Nov 4, 2024
2f76957
Collapse internal nodes when deleting values
acolytec3 Nov 5, 2024
5d6b85d
Merge remote-tracking branch 'origin/master' into vm/verkle-testing
acolytec3 Nov 5, 2024
1178633
update packages
acolytec3 Nov 6, 2024
a444162
Merge remote-tracking branch 'origin/master' into vm/verkle-testing
acolytec3 Nov 6, 2024
344979e
fix code stem and code chunk suffix generation
acolytec3 Nov 7, 2024
f6e0656
Merge branch 'master' into vm/verkle-testing
gabrocheleau Nov 7, 2024
d922a8d
vm: update test runners with updated verkle crypto
gabrocheleau Nov 7, 2024
8def8a5
vm: update package lock
gabrocheleau Nov 7, 2024
c218c8e
fix suffix counts
acolytec3 Nov 7, 2024
760b3f8
Merge remote-tracking branch 'origin/master' into vm/verkle-testing
acolytec3 Nov 8, 2024
5ccea5f
Merge branch 'master' into vm/verkle-testing
gabrocheleau Nov 11, 2024
0545b03
fix verkle setup
acolytec3 Nov 14, 2024
6594acf
Let the robots rewrite chunkifyCode
acolytec3 Nov 15, 2024
519f5d2
Merge remote-tracking branch 'origin/master' into vm/verkle-testing
acolytec3 Nov 15, 2024
6302863
Merge branch 'master' into vm/verkle-testing
gabrocheleau Nov 17, 2024
1157db0
vm: fix test setup
gabrocheleau Nov 17, 2024
690eb32
vm: use common instead of verklecrypto
gabrocheleau Nov 20, 2024
52e4359
merge: fix merge conflicts
gabrocheleau Nov 21, 2024
2bb30bc
vm: fix historyAddress
gabrocheleau Nov 21, 2024
17ce748
vm: add 2935
gabrocheleau Nov 21, 2024
6d2ff2e
vm: some improvements for 2935 handling
gabrocheleau Nov 24, 2024
fa157ca
statemanageR: clarify post state error messages
gabrocheleau Nov 26, 2024
6ff4943
evm: access witness methods renaming, addition of system verkle acces…
gabrocheleau Nov 27, 2024
27615b3
evm: minor verkle testnet related adjustments
gabrocheleau Dec 3, 2024
3ece78d
evm: remove unused clear method2
gabrocheleau Dec 4, 2024
4ec2a97
evm: typo
gabrocheleau Dec 7, 2024
74647c6
Add simple cache to access witness (#3810)
acolytec3 Dec 9, 2024
2e8015b
evm: debug verkle access witness cost helper
gabrocheleau Dec 17, 2024
8854f6c
Fix step logging so dynamic gas doesn't include state accesses
acolytec3 Dec 19, 2024
50d7a99
merge with master
gabrocheleau Dec 20, 2024
d93016e
statemanager: fix getCode logic for last chunk
gabrocheleau Dec 21, 2024
2d3a02f
evm: access witness withdrawals handling
gabrocheleau Dec 21, 2024
98bd9c6
Merge branch 'master' into vm/verkle-testing
gabrocheleau Dec 21, 2024
216650b
vm: don't include access witness if miner value is 0
gabrocheleau Dec 23, 2024
b2dd43d
Merge branch 'vm/verkle-testing' of https://github.com/ethereumjs/eth…
gabrocheleau Dec 23, 2024
6449b5e
evm: fix witness skipping for precompiles and system contracts
gabrocheleau Dec 23, 2024
ec286f9
Add miner basic data and codehash to system access witness
acolytec3 Dec 24, 2024
cc351fd
evm: minor fixes
gabrocheleau Dec 24, 2024
ee00f50
fix rewardAccount to write account into trie
acolytec3 Dec 24, 2024
92ff519
Merge branch 'vm/verkle-testing' of https://github.com/ethereumjs/eth…
gabrocheleau Dec 24, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 79 additions & 0 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion packages/blockchain/src/helpers.ts
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ export async function getGenesisStateRoot(chainId: Chain, common: Common): Promi
return chainGenesis !== undefined ? chainGenesis.stateRoot : genGenesisStateRoot({}, common)
}

/*
/*
The code below calculates the empty requests hash as of devnet-4 for EIP 7685
Note: it is not possible to calculate this directly in the blockchain package,
this introduces the `ethereum-cryptography` dependency.
Expand Down
1 change: 0 additions & 1 deletion packages/client/bin/cli.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1176,7 +1176,6 @@ async function run() {
dnsNetworks: args.dnsNetworks,
extIP: args.extIP,
key,

logger,
maxPeers: args.maxPeers,
maxPerRequest: args.maxPerRequest,
Expand Down
4 changes: 4 additions & 0 deletions packages/client/src/execution/vmexecution.ts
Original file line number Diff line number Diff line change
Expand Up @@ -212,6 +212,10 @@
} else throw new Error('EIP-6800 active and no verkle execution mode specified')
await mcl.init(mcl.BLS12_381)
const rustBN = await initRustBN()
console.log(
'this.config.execCommon is activated 2935',
this.config.execCommon.isActivatedEIP(2935),
)

Check warning on line 218 in packages/client/src/execution/vmexecution.ts

View check run for this annotation

Codecov / codecov/patch

packages/client/src/execution/vmexecution.ts#L215-L218

Added lines #L215 - L218 were not covered by tests
this.verkleVM = await createVM({
common: this.config.execCommon,
blockchain: this.chain.blockchain,
Expand Down
4 changes: 4 additions & 0 deletions packages/common/src/chains.ts
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,10 @@ export const Mainnet: ChainConfig = {
name: 'osaka',
block: null,
},
{
name: 'verkle',
block: null,
},
],
bootstrapNodes: [
{
Expand Down
2 changes: 1 addition & 1 deletion packages/common/src/common.ts
Original file line number Diff line number Diff line change
Expand Up @@ -317,7 +317,7 @@ export class Common {
for (const hfChanges of this.HARDFORK_CHANGES) {
// EIP-referencing HF config (e.g. for berlin)
if ('eips' in hfChanges[1]) {
const hfEIPs = hfChanges[1]['eips'] ?? []
const hfEIPs = hfChanges[1].eips ?? []
for (const eip of hfEIPs) {
this._mergeWithParamsCache(this._params[eip] ?? {})
}
Expand Down
1 change: 1 addition & 0 deletions packages/evm/src/evm.ts
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,7 @@ export class EVM implements EVMInterface {
public blockchain: EVMMockBlockchainInterface
public journal: Journal
public verkleAccessWitness?: VerkleAccessWitness
public systemVerkleAccessWitness?: VerkleAccessWitness

public readonly transientStorage: TransientStorage

Expand Down
22 changes: 14 additions & 8 deletions packages/evm/src/opcodes/gas.ts
Original file line number Diff line number Diff line change
Expand Up @@ -173,7 +173,8 @@
let charge2929Gas = true
if (
common.isActivatedEIP(6800) &&
runState.interpreter._evm.getPrecompile(address) === undefined
runState.interpreter._evm.getPrecompile(address) === undefined &&
!address.equals(createAddressFromStackBigInt(common.param('systemAddress')))

Check warning on line 177 in packages/evm/src/opcodes/gas.ts

View check run for this annotation

Codecov / codecov/patch

packages/evm/src/opcodes/gas.ts#L176-L177

Added lines #L176 - L177 were not covered by tests
) {
let coldAccessGas = BIGINT_0
coldAccessGas += runState.env.accessWitness!.touchAddressOnReadAndComputeGas(
Expand Down Expand Up @@ -210,7 +211,8 @@
let charge2929Gas = true
if (
common.isActivatedEIP(6800) &&
runState.interpreter._evm.getPrecompile(address) === undefined
runState.interpreter._evm.getPrecompile(address) === undefined &&
!address.equals(createAddressFromStackBigInt(common.param('systemAddress')))

Check warning on line 215 in packages/evm/src/opcodes/gas.ts

View check run for this annotation

Codecov / codecov/patch

packages/evm/src/opcodes/gas.ts#L214-L215

Added lines #L214 - L215 were not covered by tests
) {
let coldAccessGas = BIGINT_0
coldAccessGas += runState.env.accessWitness!.touchAddressOnReadAndComputeGas(
Expand Down Expand Up @@ -281,7 +283,11 @@
const address = createAddressFromStackBigInt(runState.stack.peek()[0])
let charge2929Gas = true

if (common.isActivatedEIP(6800)) {
if (
common.isActivatedEIP(6800) &&
runState.interpreter._evm.getPrecompile(address) === undefined &&
!address.equals(createAddressFromStackBigInt(common.param('systemAddress')))
) {

Check warning on line 290 in packages/evm/src/opcodes/gas.ts

View check run for this annotation

Codecov / codecov/patch

packages/evm/src/opcodes/gas.ts#L286-L290

Added lines #L286 - L290 were not covered by tests
let coldAccessGas = BIGINT_0
coldAccessGas += runState.env.accessWitness!.touchAddressOnReadAndComputeGas(
address,
Expand Down Expand Up @@ -591,7 +597,6 @@
common.isActivatedEIP(6800) &&
runState.interpreter._evm.getPrecompile(toAddress) === undefined
) {
// TODO: add check if toAddress is not a precompile
const coldAccessGas = runState.env.accessWitness!.touchAndChargeMessageCall(toAddress)
if (value !== BIGINT_0) {
const contractAddress = runState.interpreter.getAddress()
Expand Down Expand Up @@ -738,7 +743,6 @@
common.isActivatedEIP(6800) &&
runState.interpreter._evm.getPrecompile(toAddress) === undefined
) {
// TODO: add check if toAddress is not a precompile
const coldAccessGas = runState.env.accessWitness!.touchAndChargeMessageCall(toAddress)

gas += coldAccessGas
Expand Down Expand Up @@ -947,9 +951,11 @@
gas += subMemUsage(runState, outOffset, outLength, common)

let charge2929Gas = true
if (common.isActivatedEIP(6800)) {
const toAddress = createAddressFromStackBigInt(toAddr)
// TODO: add check if toAddress is not a precompile
const toAddress = createAddressFromStackBigInt(toAddr)
if (
common.isActivatedEIP(6800) &&
runState.interpreter._evm.getPrecompile(toAddress) === undefined

Check warning on line 957 in packages/evm/src/opcodes/gas.ts

View check run for this annotation

Codecov / codecov/patch

packages/evm/src/opcodes/gas.ts#L957

Added line #L957 was not covered by tests
) {
const coldAccessGas = runState.env.accessWitness!.touchAndChargeMessageCall(toAddress)

gas += coldAccessGas
Expand Down
2 changes: 1 addition & 1 deletion packages/evm/src/params.ts
Original file line number Diff line number Diff line change
Expand Up @@ -272,7 +272,7 @@ export const paramsEVM: ParamsDict = {
*/
2935: {
// evm
historyStorageAddress: '0x0aae40965e6800cd9b1f4b05ff21581047e3f91e', // The address where the historical blockhashes are stored
historyStorageAddress: '0xfffffffffffffffffffffffffffffffffffffffe', // The address where the historical blockhashes are stored
historyServeWindow: 8192, // The amount of blocks to be served by the historical blockhash contract
systemAddress: '0xfffffffffffffffffffffffffffffffffffffffe', // The system address
},
Expand Down
28 changes: 18 additions & 10 deletions packages/statemanager/src/statefulVerkleStateManager.ts
Original file line number Diff line number Diff line change
Expand Up @@ -318,7 +318,7 @@
chunkStems[0],
chunkSuffixes.slice(
0,
codeChunks.length <= VERKLE_CODE_OFFSET ? codeChunks.length : VERKLE_CODE_OFFSET,
chunkSuffixes.length <= VERKLE_CODE_OFFSET ? chunkSuffixes.length : VERKLE_CODE_OFFSET,
),
codeChunks.slice(
0,
Expand Down Expand Up @@ -398,13 +398,16 @@
for (let x = 0; x < chunks.length; x++) {
if (chunks[x] === undefined) throw new Error(`expected code chunk at ID ${x}, got undefined`)

let lastChunkByteIndex = VERKLE_CODE_CHUNK_SIZE
// Determine code ending byte (if we're on the last chunk)
let sliceEnd = 32
if (x === chunks.length - 1) {
// On the last chunk, the end of the slice is either codeSize (if only one chunk) or codeSize % chunkSize
sliceEnd = (x === 0 ? codeSize : codeSize % VERKLE_CODE_CHUNK_SIZE) + 1
// On the last chunk, the slice either ends on a partial chunk (if codeSize doesn't exactly fit in full chunks), or a full chunk
lastChunkByteIndex = codeSize % VERKLE_CODE_CHUNK_SIZE || VERKLE_CODE_CHUNK_SIZE
}
code.set(chunks[x]!.slice(1, sliceEnd), code.byteOffset + x * VERKLE_CODE_CHUNK_SIZE)
code.set(
chunks[x]!.slice(1, lastChunkByteIndex + 1),
code.byteOffset + x * VERKLE_CODE_CHUNK_SIZE,
)
}
this._caches?.code?.put(address, code)

Expand Down Expand Up @@ -598,10 +601,14 @@
// we can only compare the actual code because to compare the first byte would
// be very tricky and impossible in certain scenarios like when the previous code chunk
// was not accessed and hence not even provided in the witness
// We are left-padding with two zeroes to get a 32-byte length, but these bytes should not be considered reliable
return bytesToHex(
setLengthRight(
code.slice(codeOffset, codeOffset + VERKLE_CODE_CHUNK_SIZE),
VERKLE_CODE_CHUNK_SIZE,
setLengthLeft(
setLengthRight(
code.slice(codeOffset, codeOffset + VERKLE_CODE_CHUNK_SIZE),
VERKLE_CODE_CHUNK_SIZE,
),
VERKLE_CODE_CHUNK_SIZE + 1,

Check warning on line 611 in packages/statemanager/src/statefulVerkleStateManager.ts

View check run for this annotation

Codecov / codecov/patch

packages/statemanager/src/statefulVerkleStateManager.ts#L606-L611

Added lines #L606 - L611 were not covered by tests
),
)
}
Expand Down Expand Up @@ -645,7 +652,7 @@

const { chunkKey } = accessedState
accessedChunks.set(chunkKey, true)
const computedValue: PrefixedHexString | null | undefined =
let computedValue: PrefixedHexString | null | undefined =

Check warning on line 655 in packages/statemanager/src/statefulVerkleStateManager.ts

View check run for this annotation

Codecov / codecov/patch

packages/statemanager/src/statefulVerkleStateManager.ts#L655

Added line #L655 was not covered by tests
await this.getComputedValue(accessedState)
if (computedValue === undefined) {
this.DEBUG &&
Expand All @@ -670,7 +677,7 @@
// if the access type is code, then we can't match the first byte because since the computed value
// doesn't has the first byte for push data since previous chunk code itself might not be available
if (accessedState.type === VerkleAccessedStateType.Code) {
// computedValue = computedValue !== null ? `0x${computedValue.slice(4)}` : null
computedValue = computedValue !== null ? `0x${computedValue.slice(4)}` : null

Check warning on line 680 in packages/statemanager/src/statefulVerkleStateManager.ts

View check run for this annotation

Codecov / codecov/patch

packages/statemanager/src/statefulVerkleStateManager.ts#L680

Added line #L680 was not covered by tests
canonicalValue = canonicalValue !== null ? `0x${canonicalValue.slice(4)}` : null
} else if (
accessedState.type === VerkleAccessedStateType.Storage &&
Expand All @@ -680,6 +687,7 @@
canonicalValue = ZEROVALUE
}

this._debug(`computed ${computedValue} canonical ${canonicalValue}`)

Check warning on line 690 in packages/statemanager/src/statefulVerkleStateManager.ts

View check run for this annotation

Codecov / codecov/patch

packages/statemanager/src/statefulVerkleStateManager.ts#L690

Added line #L690 was not covered by tests
if (computedValue !== canonicalValue) {
if (type === VerkleAccessedStateType.BasicData) {
this.DEBUG &&
Expand Down
1 change: 1 addition & 0 deletions packages/vm/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@
"devDependencies": {
"@ethereumjs/blockchain": "^8.0.0-alpha.1",
"@ethereumjs/ethash": "^4.0.0-alpha.1",
"@ethereumjs/verkle": "^0.2.0-alpha.1",
"@paulmillr/trusted-setups": "^0.1.2",
"@types/benchmark": "^1.0.33",
"@types/core-js": "^2.5.0",
Expand Down
Loading
Loading