Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VM/SM: Bundle Optimizations (Default wo Caches+EVMMockBlockchain, SM Code put() Fix, VerkleSM out, runTx()+Code Opts) #3601

Merged
merged 25 commits into from
Aug 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
57d382f
Use shallowCopy() Caches copy() optimization also for VerkleSM to avo…
holgerd77 Aug 17, 2024
c3eb295
Fix default state manager missing direct code write when used without…
holgerd77 Aug 17, 2024
bef2953
Do not initialize caches for default VM state manager
holgerd77 Aug 17, 2024
57cad15
Remove copy test not making sense any more under generalized cache/no…
holgerd77 Aug 17, 2024
b903ad4
Some more solid/qualified EVM dummy blockchain + interface naming to …
holgerd77 Aug 17, 2024
185d005
Use EVMMockBlockchain(Interface) as default for the VM, adjust some t…
holgerd77 Aug 17, 2024
8567b46
Move @ethereumjs/blockchain to dev dependencies in VM
holgerd77 Aug 17, 2024
86dcb21
Rebuild package-lock.json
holgerd77 Aug 17, 2024
01e6266
Adjust/fix some client tests
holgerd77 Aug 17, 2024
1692654
Lint fix
holgerd77 Aug 17, 2024
83db446
Add Verkle SM methods as optional methods to interface, replace Verkl…
holgerd77 Aug 17, 2024
78939cb
Also align client (no real effect yet, but generally try to work more…
holgerd77 Aug 17, 2024
6a803a1
Initialize runTx() default block with simpler constructor to avoid dr…
holgerd77 Aug 17, 2024
599bcca
Fully switch to DEFAULT_HEADER in VM.runTx() to avoid drawing in bloc…
holgerd77 Aug 17, 2024
8aa1e27
Opcode list size optimization
holgerd77 Aug 17, 2024
5ee5244
More optimizations
holgerd77 Aug 17, 2024
bf22d21
Precompile code optimizations
holgerd77 Aug 17, 2024
e337575
More optimizations (precompile index.ts file)
holgerd77 Aug 17, 2024
fe3813b
Some more
holgerd77 Aug 17, 2024
a96f926
Some doc compatification
holgerd77 Aug 17, 2024
9d25fde
Add CSpell checker to CI and fix typos (#3590)
jochem-brouwer Aug 17, 2024
f399286
Fix spell check
holgerd77 Aug 19, 2024
5064200
Merge branch 'master' into vm-statemanager-bundle-optimizations
holgerd77 Aug 19, 2024
79f7672
Remove accidentally committed examples/test.ts file
holgerd77 Aug 19, 2024
f613b0e
Merge branch 'vm-statemanager-bundle-optimizations' of https://github…
holgerd77 Aug 19, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

12 changes: 12 additions & 0 deletions packages/block/src/block/constructors.ts
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,18 @@ export function createBlock(blockData: BlockData = {}, opts?: BlockOptions) {
)
}

/**
* Simple static constructor if only an empty block is needed
* (tree shaking advantages since it does not draw all the tx constructors in)
*
* @param headerData
* @param opts
*/
export function createEmptyBlock(headerData: HeaderData, opts?: BlockOptions) {
const header = createBlockHeader(headerData, opts)
return new Block(header)
}

/**
* Static constructor to create a block from an array of Bytes values
*
Expand Down
4 changes: 4 additions & 0 deletions packages/block/test/block.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ import {
createBlockFromBytesArray,
createBlockFromRLPSerializedBlock,
createBlockFromRPC,
createEmptyBlock,
paramsBlock,
} from '../src/index.js'

Expand All @@ -46,6 +47,9 @@ describe('[Block]: block functions', () => {
'should use custom parameters provided',
)

const emptyBlock = createEmptyBlock({}, { common })
assert.ok(bytesToHex(emptyBlock.hash()), 'block should initialize')

// test default freeze values
// also test if the options are carried over to the constructor
block = createBlock({})
Expand Down
80 changes: 24 additions & 56 deletions packages/client/src/execution/vmexecution.ts
Original file line number Diff line number Diff line change
Expand Up @@ -221,7 +221,7 @@ export class VMExecution extends Execution {
}

async transitionToVerkle(merkleStateRoot: Uint8Array, assignToVM: boolean = true): Promise<void> {
if (this.vm.stateManager instanceof StatelessVerkleStateManager) {
if (typeof this.vm.stateManager.initVerkleExecutionWitness === 'function') {
return
}

Expand All @@ -230,15 +230,19 @@ export class VMExecution extends Execution {
await this.setupMerkleVM()
}
const merkleVM = this.merkleVM!
const merkleStateManager = merkleVM.stateManager as DefaultStateManager
const merkleStateManager = merkleVM.stateManager

if (this.verkleVM === undefined) {
await this.setupVerkleVM()
}
const verkleVM = this.verkleVM!
const verkleStateManager = verkleVM.stateManager as StatelessVerkleStateManager
const verkleStateManager = verkleVM.stateManager

const verkleStateRoot = await verkleStateManager.getTransitionStateRoot(
// TODO: can we please implement this in a different way and not introduce a method
// *inside* one state manager which takes another state manager?
// That bloats the interface too much, this should be minimally a separate util method
// or fully move to client
const verkleStateRoot = await (verkleStateManager as any).getTransitionStateRoot(
merkleStateManager,
merkleStateRoot,
)
Expand Down Expand Up @@ -416,7 +420,8 @@ export class VMExecution extends Execution {
vm = this.verkleVM
}

const needsStatelessExecution = vm.stateManager instanceof StatelessVerkleStateManager
const needsStatelessExecution =
typeof this.vm.stateManager.initVerkleExecutionWitness === 'function'
if (needsStatelessExecution && block.executionWitness === undefined) {
throw Error(`Verkle blocks need executionWitness for stateless execution`)
} else {
Expand Down Expand Up @@ -576,7 +581,7 @@ export class VMExecution extends Execution {
this.config.logger.warn(
`Setting execution head to hash=${short(jumpToHash)} number=${jumpToNumber}`,
)
await this.vm.blockchain.setIteratorHead('vm', jumpToHash)
await this.chain.blockchain.setIteratorHead('vm', jumpToHash)
})
}

Expand All @@ -595,19 +600,9 @@ export class VMExecution extends Execution {
this.running = true
let numExecuted: number | null | undefined = undefined

const { blockchain } = this.vm
if (typeof blockchain.getIteratorHead !== 'function') {
throw new Error('cannot get iterator head: blockchain has no getIteratorHead function')
}
let startHeadBlock = await blockchain.getIteratorHead()
let startHeadBlock = await this.chain.blockchain.getIteratorHead()
await this.checkAndReset(startHeadBlock)

if (typeof blockchain.getCanonicalHeadBlock !== 'function') {
throw new Error(
'cannot get iterator head: blockchain has no getCanonicalHeadBlock function',
)
}
let canonicalHead = await blockchain.getCanonicalHeadBlock()
let canonicalHead = await this.chain.blockchain.getCanonicalHeadBlock()

this.config.logger.debug(
`Running execution startHeadBlock=${startHeadBlock?.header.number} canonicalHead=${canonicalHead?.header.number} loop=${loop}`,
Expand Down Expand Up @@ -637,14 +632,14 @@ export class VMExecution extends Execution {
headBlock = undefined
parentState = undefined
errorBlock = undefined
this.vmPromise = blockchain
this.vmPromise = this.chain.blockchain
.iterator(
'vm',
async (block: Block, reorg: boolean) => {
// determine starting state for block run
// if we are just starting or if a chain reorg has happened
if (headBlock === undefined || reorg) {
headBlock = await blockchain.getBlock(block.header.parentHash)
headBlock = await this.chain.blockchain.getBlock(block.header.parentHash)
parentState = headBlock.header.stateRoot

if (reorg) {
Expand All @@ -664,11 +659,6 @@ export class VMExecution extends Execution {
// run block, update head if valid
try {
const { number, timestamp } = block.header
if (typeof blockchain.getTotalDifficulty !== 'function') {
throw new Error(
'cannot get iterator head: blockchain has no getTotalDifficulty function',
)
}

const hardfork = this.config.execCommon.getHardforkBy({
blockNumber: number,
Expand All @@ -692,7 +682,7 @@ export class VMExecution extends Execution {
}
if (
(!this.config.execCommon.gteHardfork(Hardfork.Osaka) &&
this.vm.stateManager instanceof StatelessVerkleStateManager) ||
typeof this.vm.stateManager.initVerkleExecutionWitness === 'function') ||
(this.config.execCommon.gteHardfork(Hardfork.Osaka) &&
this.vm.stateManager instanceof DefaultStateManager)
) {
Expand Down Expand Up @@ -800,7 +790,7 @@ export class VMExecution extends Execution {
backStepToHash ?? 'na',
)} hasParentStateRoot=${short(backStepToRoot ?? 'na')}:\n${error}`,
)
await this.vm.blockchain.setIteratorHead('vm', backStepToHash)
await this.chain.blockchain.setIteratorHead('vm', backStepToHash)
} else {
this.config.logger.error(
`${errorMsg}, couldn't back step to vmHead number=${backStepTo} hash=${short(
Expand Down Expand Up @@ -855,14 +845,7 @@ export class VMExecution extends Execution {

numExecuted = await this.vmPromise
if (numExecuted !== null) {
let endHeadBlock
if (typeof this.vm.blockchain.getIteratorHead === 'function') {
endHeadBlock = await this.vm.blockchain.getIteratorHead('vm')
} else {
throw new Error(
'cannot get iterator head: blockchain has no getIteratorHead function',
)
}
const endHeadBlock = await this.chain.blockchain.getIteratorHead('vm')

if (typeof numExecuted === 'number' && numExecuted > 0) {
const firstNumber = startHeadBlock.header.number
Expand Down Expand Up @@ -891,12 +874,7 @@ export class VMExecution extends Execution {
)
}
startHeadBlock = endHeadBlock
if (typeof this.vm.blockchain.getCanonicalHeadBlock !== 'function') {
throw new Error(
'cannot get iterator head: blockchain has no getCanonicalHeadBlock function',
)
}
canonicalHead = await this.vm.blockchain.getCanonicalHeadBlock()
canonicalHead = await this.chain.blockchain.getCanonicalHeadBlock()
}
}

Expand All @@ -917,19 +895,12 @@ export class VMExecution extends Execution {
this.STATS_INTERVAL,
)

const { blockchain } = this.vm
if (this.running || !this.started) {
return false
}

if (typeof blockchain.getIteratorHead !== 'function') {
throw new Error('cannot get iterator head: blockchain has no getIteratorHead function')
}
const vmHeadBlock = await blockchain.getIteratorHead()
if (typeof blockchain.getCanonicalHeadBlock !== 'function') {
throw new Error('cannot get iterator head: blockchain has no getCanonicalHeadBlock function')
}
const canonicalHead = await blockchain.getCanonicalHeadBlock()
const vmHeadBlock = await this.chain.blockchain.getIteratorHead()
const canonicalHead = await this.chain.blockchain.getCanonicalHeadBlock()

const infoStr = `vmHead=${vmHeadBlock.header.number} canonicalHead=${
canonicalHead.header.number
Expand Down Expand Up @@ -984,7 +955,7 @@ export class VMExecution extends Execution {
async executeBlocks(first: number, last: number, txHashes: string[]) {
this.config.logger.info('Preparing for block execution (debug mode, no services started)...')

const block = await this.vm.blockchain.getBlock(first)
const block = await this.chain.blockchain.getBlock(first)
const startExecutionHardfork = this.config.execCommon.getHardforkBy({
blockNumber: block.header.number,
timestamp: block.header.timestamp,
Expand All @@ -1000,13 +971,10 @@ export class VMExecution extends Execution {
const vm = await this.vm.shallowCopy(false)

for (let blockNumber = first; blockNumber <= last; blockNumber++) {
const block = await vm.blockchain.getBlock(blockNumber)
const parentBlock = await vm.blockchain.getBlock(block.header.parentHash)
const block = await this.chain.blockchain.getBlock(blockNumber)
const parentBlock = await this.chain.blockchain.getBlock(block.header.parentHash)
// Set the correct state root
const root = parentBlock.header.stateRoot
if (typeof vm.blockchain.getTotalDifficulty !== 'function') {
throw new Error('cannot get iterator head: blockchain has no getTotalDifficulty function')
}
vm.common.setHardforkBy({
blockNumber,
timestamp: block.header.timestamp,
Expand Down
9 changes: 4 additions & 5 deletions packages/client/src/miner/miner.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ import type { Config } from '../config.js'
import type { VMExecution } from '../execution/index.js'
import type { FullEthereumService } from '../service/index.js'
import type { FullSynchronizer } from '../sync/index.js'
import type { CliqueConsensus } from '@ethereumjs/blockchain'
import type { Blockchain, CliqueConsensus } from '@ethereumjs/blockchain'
import type { CliqueConfig } from '@ethereumjs/common'
import type { Miner as EthashMiner, Solution } from '@ethereumjs/ethash'

Expand Down Expand Up @@ -244,10 +244,9 @@ export class Miner {
const [signerAddress, signerPrivKey] = this.config.accounts[0]
cliqueSigner = signerPrivKey
// Determine if signer is INTURN (2) or NOTURN (1)
inTurn = await (vmCopy.blockchain.consensus as CliqueConsensus).cliqueSignerInTurn(
signerAddress,
number,
)
inTurn = await (
(vmCopy.blockchain as Blockchain).consensus as CliqueConsensus
).cliqueSignerInTurn(signerAddress, number)
difficulty = inTurn ? 2 : 1
}

Expand Down
3 changes: 0 additions & 3 deletions packages/client/src/miner/pendingBlock.ts
Original file line number Diff line number Diff line change
Expand Up @@ -104,9 +104,6 @@ export class PendingBlock {
const { timestamp, mixHash, parentBeaconBlockRoot, coinbase } = headerData
let { gasLimit } = parentBlock.header

if (typeof vm.blockchain.getTotalDifficulty !== 'function') {
throw new Error('cannot get iterator head: blockchain has no getTotalDifficulty function')
}
vm.common.setHardforkBy({
blockNumber: number,
timestamp,
Expand Down
10 changes: 7 additions & 3 deletions packages/client/test/miner/miner.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ import { wait } from '../integration/util.js'

import type { FullSynchronizer } from '../../src/sync/index.js'
import type { Block } from '@ethereumjs/block'
import type { CliqueConsensus } from '@ethereumjs/blockchain'
import type { Blockchain, CliqueConsensus } from '@ethereumjs/blockchain'
import type { VM } from '@ethereumjs/vm'

const A = {
Expand Down Expand Up @@ -242,7 +242,9 @@ describe('assembleBlocks() -> with a single tx', async () => {
await txPool.add(txA01)

// disable consensus to skip PoA block signer validation
;(vm.blockchain.consensus as CliqueConsensus).cliqueActiveSigners = () => [A.address] // stub
;((vm.blockchain as Blockchain).consensus as CliqueConsensus).cliqueActiveSigners = () => [
A.address,
] // stub

chain.putBlocks = (blocks: Block[]) => {
it('should include tx in new block', () => {
Expand Down Expand Up @@ -280,7 +282,9 @@ describe('assembleBlocks() -> with a hardfork mismatching tx', async () => {
})

// disable consensus to skip PoA block signer validation
;(vm.blockchain.consensus as CliqueConsensus).cliqueActiveSigners = () => [A.address] // stub
;((vm.blockchain as Blockchain).consensus as CliqueConsensus).cliqueActiveSigners = () => [
A.address,
] // stub

chain.putBlocks = (blocks: Block[]) => {
it('should not include tx', () => {
Expand Down
Loading
Loading