Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RSKIP 351 + RSKIP 144 #1930

Merged
merged 112 commits into from
Oct 9, 2024
Merged
Show file tree
Hide file tree
Changes from 26 commits
Commits
Show all changes
112 commits
Select commit Hold shift + click to select a range
7302420
Header version based on activation block
ilanolkies Oct 3, 2022
8bb8859
Use byte type for header version
ilanolkies Oct 6, 2022
2a1186f
Test genesis version
ilanolkies Oct 6, 2022
e24f599
Fix merge
ilanolkies Oct 27, 2022
0ead12c
RSKIP 351 implementation
ilanolkies Nov 1, 2022
d9a0537
Add tests
ilanolkies Nov 14, 2022
79685df
Merge branch 'master' into rskip351
ilanolkies Nov 14, 2022
0d0e7d8
Fix build
ilanolkies Nov 14, 2022
febccd6
Block header addition (#1790)
julianlen Jun 1, 2022
89f9471
Modified the remasc payment.
julianlen Jun 1, 2022
5d00cce
Research/parallel-tx/header generation from master (#1792)
julianlen Jun 15, 2022
9516448
Research/parallel-tx/block validation (#1804)
julianlen Jun 15, 2022
3a23caa
Created a Constant with the number of threads (#1815)
julianlen Jun 22, 2022
2398add
Research/refactoring code (#1817)
julianlen Jun 27, 2022
74f9c8c
Research/parallel-tx/optimize writes (#1811)
ilanolkies Jun 27, 2022
fba1e7a
added validation for the rskip144 (#1824)
julianlen Jul 6, 2022
29b92e2
Fix write the same value - avoid write on add balance 0 (#1833)
ilanolkies Jul 19, 2022
ecb2a65
Research/parallel-tx/Refactor validblock (#1794)
ilanolkies Jul 19, 2022
90e5ea8
Research/main parallel tx revision (#1865)
julianlen Sep 23, 2022
5a63b2e
Research/main parallel tx remasc (#1895)
julianlen Nov 14, 2022
6de53a3
Merge branch 'research/main_parallel_tx' into rskip351
ilanolkies Nov 17, 2022
194fa4b
Move edges to block header extension
ilanolkies Nov 18, 2022
ccf4e1f
Merge branch 'master' into rskip351+rskip144
ilanolkies Nov 22, 2022
e532933
Add RSKIP 144 to fingerroot500
ilanolkies Nov 22, 2022
51608e3
General fixes + refactor getSublistCandidate
ilanolkies Nov 23, 2022
4306c44
Fixing addBalance with zero (#1932)
julianlen Nov 24, 2022
016d57c
Fix writes after writing to parallel (#1949)
ilanolkies Dec 14, 2022
60172eb
Merge branch 'master' into rskip351+rskip144
ilanolkies Dec 15, 2022
7dd23fe
Merge branch 'master' into rskip351+rskip144
ilanolkies Dec 26, 2022
8f3b752
Merge branch 'master' into rskip351+rskip144
ilanolkies Feb 8, 2023
d8bb353
RSKIP 144 - Fix state after executing parallel (#1973)
ilanolkies Feb 22, 2023
a690e36
RSKIP 144 tests (#1983)
ilanolkies Mar 1, 2023
2c9a6e8
RSKIP 351 updates (#1938)
ilanolkies Mar 13, 2023
735ead4
Remove unused import
ilanolkies Mar 13, 2023
ac3884f
Apart from testing, when RSKIP144 is deactivated, the txExecutionSubl…
julianlen Apr 10, 2023
b45f75b
Rskip351+rskip144+precompiled contracts (#1994)
julianlen Apr 10, 2023
0353532
The gas limit provided to the TransactionHandler is now divided into …
julianlen Apr 12, 2023
9b90491
Now the gas limit considered is the sublist gas limit
julianlen Apr 19, 2023
9689245
Refactored TxExecutorTest so there is no extra constructor in Transac…
julianlen Apr 20, 2023
3e1b9de
Changed the gas limits for all the tests since the new considered gas…
julianlen Apr 20, 2023
40e3ebf
Tested that even with the sublists full the used gas states below or …
julianlen Apr 25, 2023
fe1ba90
Renaming of variables
julianlen Apr 27, 2023
31eca8a
Merge pull request #2025 from rsksmart/rskip351+rskip144+gaslimit
javicid May 3, 2023
6b21bb6
Fixes for sonarcloud
julianlen May 3, 2023
7078ec9
Merge branch 'master' into rskip351+rskip144-even
ilanolkies May 9, 2023
61402c6
Merge branch 'master' into rskip351+rskip144
ilanolkies May 23, 2023
bda76ae
Change in the gas limit in the addBlockToBlockchain method for the Lo…
julianlen May 23, 2023
b19eecc
Merge remote-tracking branch 'origin/master' into rskip351+rskip144
Vovchyk Jul 7, 2023
c5cece8
Updated activation hf for rskips; Fixed unit tests
Vovchyk Jul 7, 2023
f70e601
Fixed integration tests
Vovchyk Jul 10, 2023
4592e1a
Added missing getEncoded(bool,bool) to BlockHeader
Vovchyk Jul 12, 2023
449d28f
Removed unused 144and351.conf file
Vovchyk Jul 12, 2023
693bb52
Fixed long sync
Vovchyk Jul 21, 2023
9321aa3
Using compressed header encoding for calculating HashForMergedMining
Vovchyk Jul 25, 2023
39275f1
Merge branch 'master' into rskip351+rskip144
Vovchyk Sep 6, 2023
c188238
Fixed testPush0 unit test
Vovchyk Sep 6, 2023
2fb6693
Enabled precompiles in parallel list (with option to disable if neede…
Vovchyk Sep 11, 2023
28bf995
Added posibility to specify in code which precompiles are allowed to …
Vovchyk Sep 13, 2023
306f80e
Implemented reusable thread pools; Added flag to force tx list execut…
Vovchyk Sep 15, 2023
82ba683
Reworked thread scheduling for parallel tx execution
Vovchyk Sep 18, 2023
07a1186
Merge branch 'master' into rskip351+rskip144
Vovchyk Sep 18, 2023
228a266
Used in-line tx execution if num of edges is less than 2
Vovchyk Sep 19, 2023
6ef1560
Merge remote-tracking branch 'origin/master' into rskip351+rskip144
Vovchyk Oct 2, 2023
e8b9910
Merge branch 'master' into rskip351+rskip144
Vovchyk Oct 4, 2023
027ee5b
Added tx gas limit validation before adding to mempool
Vovchyk Oct 19, 2023
501d5d8
Merge remote-tracking branch 'origin/master' into rskip351+rskip144
Vovchyk Nov 15, 2023
27f2a82
Merge remote-tracking branch 'origin/master' into master-rskip351+rsk…
Vovchyk Jan 30, 2024
1e133fb
Merge pull request #2240 from rsksmart/master-rskip351+rskip144-merge
Vovchyk Jan 30, 2024
33590c6
Refactor PTE gas distribution logic
rmoreliovlabs Feb 2, 2024
a214bff
Change refactor approach and update affected tests
rmoreliovlabs Feb 6, 2024
48df218
Merge pull request #2246 from rsksmart/pte-gas-distribution-refactor
Vovchyk Feb 7, 2024
964eb45
Merge pull request #2251 from rsksmart/master
Vovchyk Feb 7, 2024
e5f9086
Disabled PTE feature by default
Vovchyk Feb 7, 2024
0f61489
Merge pull request #2252 from rsksmart/pte-disabled-by-default
Vovchyk Feb 7, 2024
2f35b8f
Update tx execution threads constant value
rmoreliovlabs Feb 8, 2024
cb03dd4
Fix failing test
rmoreliovlabs Feb 8, 2024
ef9fdfb
Code cleaning
rmoreliovlabs Feb 8, 2024
ca111b6
Merge pull request #2253 from rsksmart/update-parallel-threads-amount
Vovchyk Feb 8, 2024
30ebfb8
Cover cases with uneven division in getSublistGasLimit
rmoreliovlabs Feb 16, 2024
5161f65
Add javadoc and inner comments
rmoreliovlabs Feb 16, 2024
ecf6522
Merge pull request #2257 from rsksmart/refactor-get-sublist-gas-limit
rmoreliovlabs Feb 22, 2024
e1dfd1d
Merge remote-tracking branch 'origin/master' into master-rskip351+rsk…
Vovchyk Apr 16, 2024
8f9ed2a
Fixed bridgeDelegateCall unit test
Vovchyk Apr 16, 2024
a84de30
Merge pull request #2288 from rsksmart/master-rskip351+rskip144-merge
Vovchyk Apr 16, 2024
594b1b0
Merge branch 'master' into rskip351+rskip144
Vovchyk May 27, 2024
79547e0
Fixed sonar complains / Removed unused imports
Vovchyk May 27, 2024
9eb6d75
Merge branch 'master' into rskip351+rskip144
Vovchyk May 27, 2024
f714316
Adding custom config
nagarev Jun 11, 2024
e389b5a
Adding test (wip)
nagarev Jun 11, 2024
66cd486
Adding pte edges field
nagarev Jun 12, 2024
e299f56
Updating test WIP
nagarev Jun 12, 2024
18f2c0f
Updating tests
nagarev Jun 17, 2024
d79f2a7
Updating package name
nagarev Jun 19, 2024
36a2d76
Removing duplicated stacktrace print
nagarev Jun 20, 2024
cc2b39a
Updating test
nagarev Jun 24, 2024
5cb310f
Merge branch 'master' into rskip351+rskip144
Vovchyk Jun 26, 2024
51ba9e1
Merge pull request #2477 from rsksmart/add-int-test
Vovchyk Jul 1, 2024
b639740
Merge branch 'master' into rskip351+rskip144
Vovchyk Jul 4, 2024
d894940
fixed executedContractWithDelegateCallToNonExistentContract test
Vovchyk Jul 4, 2024
559de16
Merge branch 'master' into rskip351+rskip144
Vovchyk Jul 18, 2024
398faa5
Merge remote-tracking branch 'origin/master' into vovchyk/master-rski…
Vovchyk Sep 4, 2024
baf4c9a
Merge pull request #2708 from rsksmart/vovchyk/master-rskip351+rskip1…
Vovchyk Sep 4, 2024
3856836
fix(pte): fixing sonar complaints
Vovchyk Sep 5, 2024
731751d
Merge pull request #2709 from rsksmart/vovchyk/rskip351+rskip144-fix-…
Vovchyk Sep 5, 2024
79523ca
fix(pte): suppress sonar warnings
Vovchyk Sep 5, 2024
3850591
Merge pull request #2710 from rsksmart/vovchyk/rskip351+rskip144-supp…
Vovchyk Sep 5, 2024
3059c4c
Merge remote-tracking branch 'origin/master' into vovchyk/master-rski…
Vovchyk Sep 25, 2024
ecbb0ad
feat(pte): fix unit tests after merge from master
Vovchyk Sep 26, 2024
ebc5664
Merge pull request #2755 from rsksmart/vovchyk/master-rskip351+rskip1…
Vovchyk Sep 26, 2024
b6227a3
Merge branch 'master' into rskip351+rskip144
fmacleal Sep 27, 2024
f46f15f
Merge branch 'master' into rskip351+rskip144
Vovchyk Oct 8, 2024
63c08da
Merge branch 'master' into rskip351+rskip144
Vovchyk Oct 8, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions rskj-core/src/main/java/co/rsk/RskContext.java
Original file line number Diff line number Diff line change
Expand Up @@ -474,7 +474,8 @@ public synchronized BlockExecutor getBlockExecutor() {
blockExecutor = new BlockExecutor(
getRskSystemProperties().getActivationConfig(),
getRepositoryLocator(),
getTransactionExecutorFactory()
getTransactionExecutorFactory(),
getRskSystemProperties().isRemascEnabled()
);
}

Expand Down Expand Up @@ -1061,7 +1062,7 @@ public synchronized BlockValidationRule getBlockValidationRule() {
rskSystemProperties.getActivationConfig(),
rskSystemProperties.getNetworkConstants()
);
blockValidationRule = new BlockValidatorRule(
blockValidationRule = new BlockCompositeRule(
new TxsMinGasPriceRule(),
new BlockTxsMaxGasPriceRule(rskSystemProperties.getActivationConfig()),
new BlockUnclesValidationRule(
Expand All @@ -1088,7 +1089,8 @@ public synchronized BlockValidationRule getBlockValidationRule() {
blockTimeStampValidationRule,
new GasLimitRule(commonConstants.getMinGasLimit()),
new ExtraDataRule(commonConstants.getMaximumExtraDataSize()),
getForkDetectionDataRule()
getForkDetectionDataRule(),
new ValidTxExecutionSublistsEdgesRule()
);
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ private void executeBlocks(String[] args, BlockExecutor blockExecutor, BlockStor
Block block = blockStore.getChainBlockByNumber(n);
Block parent = blockStore.getBlockByHash(block.getParentHash().getBytes());

BlockResult blockResult = blockExecutor.execute(block, parent.getHeader(), false, false, true);
BlockResult blockResult = blockExecutor.execute(null, 0, block, parent.getHeader(), false, false, true);

Keccak256 stateRootHash = stateRootHandler.translate(block.getHeader());
if (!Arrays.equals(blockResult.getFinalState().getHash().getBytes(), stateRootHash.getBytes())) {
Expand Down
225 changes: 225 additions & 0 deletions rskj-core/src/main/java/co/rsk/core/TransactionListExecutor.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,225 @@
package co.rsk.core;

import co.rsk.crypto.Keccak256;
import org.ethereum.core.*;
import org.ethereum.vm.DataWord;
import org.ethereum.vm.PrecompiledContracts;
import org.ethereum.vm.program.ProgramResult;
import org.ethereum.vm.trace.ProgramTraceProcessor;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import javax.annotation.Nullable;
import java.util.*;
import java.util.concurrent.Callable;

public class TransactionListExecutor implements Callable<Boolean> {

private static final Logger logger = LoggerFactory.getLogger("transactionlistexecutor");

private final TransactionExecutorFactory transactionExecutorFactory;
private final List<Transaction> transactions;
private final Block block;
private final Repository track;
private final boolean vmTrace;
private final int vmTraceOptions;
private final Set<DataWord> deletedAccounts;
private final boolean discardInvalidTxs;
private final boolean acceptInvalidTransactions;
private final Map<Integer, Transaction> executedTransactions;
private final Map<Integer, TransactionReceipt> receipts;
private final Map<Keccak256, ProgramResult> transactionResults;
private final ProgramTraceProcessor programTraceProcessor;
private final boolean remascEnabled;
private long totalGas;
private int i;
private final boolean registerProgramResults;
private Coin totalPaidFees;

public TransactionListExecutor(
List<Transaction> transactions,
Block block,
TransactionExecutorFactory transactionExecutorFactory,
Repository track,
boolean vmTrace,
int vmTraceOptions,
Set<DataWord> deletedAccounts,
boolean discardInvalidTxs,
boolean acceptInvalidTransactions,
Map<Integer, TransactionReceipt> receipts,
Map<Integer, Transaction> executedTransactions,
Map<Keccak256, ProgramResult> transactionResults,
boolean registerProgramResults,
@Nullable ProgramTraceProcessor programTraceProcessor,
int firstTxIndex,
Coin totalPaidFees,
boolean remascEnabled) {
this.block = block;
this.transactionExecutorFactory = transactionExecutorFactory;
this.track = track;
this.vmTrace = vmTrace;
this.vmTraceOptions = vmTraceOptions;
this.transactions = new ArrayList<>(transactions);
this.deletedAccounts = deletedAccounts;
this.discardInvalidTxs = discardInvalidTxs;
this.acceptInvalidTransactions = acceptInvalidTransactions;
this.executedTransactions = executedTransactions;
this.receipts = receipts;
this.registerProgramResults = registerProgramResults;
this.transactionResults = transactionResults;
this.programTraceProcessor = programTraceProcessor;
this.totalGas = 0L;
this.i = firstTxIndex;
this.totalPaidFees = totalPaidFees;
this.remascEnabled = remascEnabled;
}

@Override
public Boolean call() {
long totalGasUsed = 0;

for (Transaction tx : transactions) {

int numberOfTransactions = block.getTransactionsList().size();
boolean isRemascTransaction = tx.isRemascTransaction(this.i, numberOfTransactions);

addFeesToRemascIfEnabled(isRemascTransaction);

TransactionExecutor txExecutor = transactionExecutorFactory.newInstance(
tx,
i,
block.getCoinbase(),
track,
block,
totalGasUsed,
vmTrace,
vmTraceOptions,
deletedAccounts
);
boolean transactionExecuted = txExecutor.executeTransaction();

if (!acceptInvalidTransactions && !transactionExecuted) {
if (discardIfInvalid(tx, numberOfTransactions, isRemascTransaction)) return false;
continue;
}

executedTransactions.put(i, tx);

if (this.registerProgramResults) {
this.transactionResults.put(tx.getHash(), txExecutor.getResult());
}

if (vmTrace) {
txExecutor.extractTrace(programTraceProcessor);
}

logger.trace("tx[{}] executed", i + 1);
logger.trace("track commit");

long txGasUsed = txExecutor.getGasUsed();
totalGasUsed += txGasUsed;

addPaidFeesToToal(txExecutor);

// It's used just for testing, the last tx should be always the REMASC.
payToRemascWhenThereIsNoRemascTx(numberOfTransactions, isRemascTransaction);

deletedAccounts.addAll(txExecutor.getResult().getDeleteAccounts());

TransactionReceipt receipt = createTransactionReceipt(totalGasUsed, tx, txExecutor, txGasUsed);

logger.trace("block: [{}] executed tx: [{}]", block.getNumber(), tx.getHash());

i++;

logger.trace("tx[{}].receipt", i);

receipts.put(i, receipt);

logger.trace("tx[{}] done", i);
}
totalGas += totalGasUsed;
return true;
}

private boolean discardIfInvalid(Transaction tx, int numberOfTransactions, boolean isRemascTransaction) {
// It's used just for testing, the last tx should be always the REMASC.
payToRemascWhenThereIsNoRemascTx(numberOfTransactions, isRemascTransaction);
if (!discardInvalidTxs) {
logger.warn("block: [{}] execution interrupted because of invalid tx: [{}]",
block.getNumber(), tx.getHash()
);
return true;
}

logger.warn("block: [{}] discarded tx: [{}]", block.getNumber(), tx.getHash());
return false;
}

private TransactionReceipt createTransactionReceipt(long totalGasUsed, Transaction tx, TransactionExecutor txExecutor, long txGasUsed) {
TransactionReceipt receipt = new TransactionReceipt();
receipt.setGasUsed(txGasUsed);
receipt.setCumulativeGas(totalGasUsed);

receipt.setTxStatus(txExecutor.getReceipt().isSuccessful());
receipt.setTransaction(tx);
receipt.setLogInfoList(txExecutor.getVMLogs());
receipt.setStatus(txExecutor.getReceipt().getStatus());
return receipt;
}

private void addPaidFeesToToal(TransactionExecutor txExecutor) {
Coin txPaidFees = txExecutor.getPaidFees();
if (txPaidFees != null) {
totalPaidFees = totalPaidFees.add(txPaidFees);
}
}

private void addFeesToRemascIfEnabled(boolean isRemascTransaction) {
if (this.remascEnabled && isRemascTransaction) {
addFeesToRemasc();
}
}

private void payToRemascWhenThereIsNoRemascTx(int numberOfTransactions, boolean isRemascTransaction) {
boolean isLastTx = this.i == numberOfTransactions - 1;
if (this.remascEnabled && isLastTx && !isRemascTransaction) {
addFeesToRemasc();
}
}

private void addFeesToRemasc() {
if (this.totalPaidFees.compareTo(Coin.ZERO) > 0) {
logger.trace("Adding fee to remasc contract account");
track.addBalance(PrecompiledContracts.REMASC_ADDR, this.totalPaidFees);
}
}

public Repository getRepository() {
return this.track;
}

public Set<DataWord> getDeletedAccounts() {
return new HashSet<>(this.deletedAccounts);
}

public Map<Integer, TransactionReceipt> getReceipts() {
return this.receipts;
}

public Map<Integer, Transaction> getExecutedTransactions() {
return this.executedTransactions;
}

public Map<Keccak256, ProgramResult> getTransactionResults() {
return this.transactionResults;
}

public Coin getTotalFees() {
return this.totalPaidFees;
}

public long getTotalGas() {
return this.totalGas;
}
}
2 changes: 1 addition & 1 deletion rskj-core/src/main/java/co/rsk/core/bc/BlockChainImpl.java
Original file line number Diff line number Diff line change
Expand Up @@ -249,7 +249,7 @@ private ImportResult internalTryToConnect(Block block) {
long saveTime = System.nanoTime();
logger.trace("execute start");

result = blockExecutor.execute(block, parent.getHeader(), false, noValidation, true);
result = blockExecutor.execute(null, 0, block, parent.getHeader(), false, noValidation, true);

logger.trace("execute done");

Expand Down
Loading