RPC to dry run of transaction

[ilblackdragon]

This is similar to view call, but allows to call state changing functions, attach deposit and sender.

This can be used to estimate gas used, find issues in arguments and debugging in general.

This is needed for proper charging of meta transactions #3439 and near/near-cli#561

CC @frol @nearmax

[frol]

Can we call testnet to be the dry-run environment? It seems that standalone runtime should be a much nicer facility to solve the gas estimation

[ilblackdragon]

It's not really easily possible to get the same inputs in testnet and especially on standalone runtime.

MainNet is constantly changing and if you want to get the current amount of gas for a function call (like cross contract calls, etc). Measuring storage can be even weirder.

So we do need some form of dry run on the given network to estimate gas and storage usage. This will allow for API libraries to define how much gas / deposit to attach.

[frol]

I can imagine we can hack together the execution of a single receipt, and basically return all the outputs (the state diff and the outgoing receipts). I don't see a way to do cross-contract calls since they would most certainly rely on the data stored by the previous dry-run call which did not persist the state.

The full solution I can imagine is the following: we build a standalone app with hooks to automagically pull the deployed code and the state of the requested account, and thus calls can persist the changes in its local snapshot of the state, and progress through the cross-contract calls.

I also see some relation to simulation testing in near-sdk-rs.

/cc @willemneal @evgenykuzyakov @olonho @bowenwang1996 @ailisp

[bowenwang1996]

The full solution I can imagine is the following: we build a standalone app with hooks to automagically pull the deployed code and the state of the requested account, and thus calls can persist the changes in its local snapshot of the state, and progress through the cross-contract calls.

It is not as easy because you need to pull the state of all the contracts touched by the call and all the branches of trie that they are on to somewhat accurately compute the cost. Also given the current state implementation, pulling state of a contract is costly and we might be better off just implement this as a rpc.

[ilblackdragon]

I would love what you @frol described (pulling a local clone of some part of substate of the blockchain), because it would allow to test things like testing upgrades with other apps involved and profiling things for real: near/near-cli#501

That said, it is indeed way more complex than described here feature.

Ideally, having this dry run that includes cross-contract calls (without them it doesn't really make much sense) on nodes with all shards tracked.

[bowenwang1996]

It seems that we decide to go with near-sandbox instead of pursuing this option. @ailisp @DiscRiskandBisque can you confirm?

[ilblackdragon]

I'm not sure what near-sandbox is, but I wouldn't expect something with this name to dry run a transaction on top of just deployed contract to MainNet with all the dependencies and state of accounts. Will near-sandbox have a sync with MainNet state somehow?

…

On Mon, Jun 28, 2021 at 7:37 PM Bowen Wang ***@***.***> wrote: It seems that we decide to go with near-sandbox instead of pursuing this option. @ailisp <https://github.com/ailisp> @DiscRiskandBisque <https://github.com/DiscRiskandBisque> can you confirm? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#3448 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABK27Q7YQVF4DVVEYFXKODTVEWWNANCNFSM4SEN3TAA> .

-- Best regards, Illia Polosukhin

[bowenwang1996]

@ilblackdragon my understanding is that it is expected to have some functionality that spoons mainnet state

[ilblackdragon]

Hm, so you expect the Wallet frontend to run this sandbox? Or how you are imagining it implementing requested here functionality?

…

On Tue, Jun 29, 2021 at 1:16 PM Bowen Wang ***@***.***> wrote: @ilblackdragon <https://github.com/ilblackdragon> my understanding is that it is expected to have some functionality that spoons mainnet state — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3448 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABK27RF5HJKPFBQHQBO4VLTVIS2FANCNFSM4SEN3TAA> .

-- Best regards, Illia Polosukhin

[bowenwang1996]

Hm, so you expect the Wallet frontend to run this sandbox?

Not sure. @mikedotexe @DiscRiskandBisque I think you are more involved in the sandbox design. Do you know the plan there?

[stale]

This issue has been automatically marked as stale because it has not had recent activity in the last 2 months.
It will be closed in 7 days if no further activity occurs.
Thank you for your contributions.

[OnlyOneJMJQ]

The sandbox could solve the dry run needs of developers, but for wallet we'll need another solution. That's based on the assumption that the wallet will need quicker estimates than I think we could get with a cluster of sandboxes doing dry-runs in the background. All that said, this solution allows for simple horizontal scaling in that more servers running more sandboxes equals more speed, so perhaps it could be a temporary solution.

Overall I strongly agree that this DX is essential. For high-value contracts and especially upgrades devs will want this functionality. It was a popular feature of Truffle for the reassurance it offers.

[ilblackdragon]

@DiscRiskandBisque will sandbox maintain literal copy of the mainnet up to date? E.g. be syncing with current blocks?
If yes, than it make sense to send dry run to those, though I'm not sure how state changes will be rolled back.

It seems like we either way need a dry run mode in the node or in our RPC infra, independent of it's running on sandbox or real nodes.

I do feel like adding dry run mode into RPC is way simpler, given we already running a bunch of servers with up to date state that are accepting requests (RPC servers) and all the tooling is already pointing at them.

Alternatively to modifying nearcore to add this, we actually design a blockchain-less node for serving query RPCs faster.

Meaning that we have a node just receives and stores final blockchain data and is able to execute WASM/lookup state extremely fast by storing data in the flat DB. Then dry run can be executed there as well.

[bowenwang1996]

@DiscRiskandBisque will sandbox maintain literal copy of the mainnet up to date? E.g. be syncing with current blocks?
If yes, than it make sense to send dry run to those, though I'm not sure how state changes will be rolled back.

No I believe the approach it uses is to first download the relevant pieces of state and then execute the transaction, which is going to be slow for large contracts.

Alternatively to modifying nearcore to add this, we actually design a blockchain-less node for serving query RPCs faster.

Yeah that is the goal :) but it will take a while to accomplish.

[stale]

This issue has been automatically marked as stale because it has not had recent activity in the last 2 months.
It will be closed in 7 days if no further activity occurs.
Thank you for your contributions.

[frol]

I believe we can close those in favor of using near-workspaces

[ilblackdragon]

@frol how would one use near-workspace as way to evaluate if given transaction would go through when building wallet / application?

[frol]

@ilblackdragon Building an app, I would use near-workspaces to setup sandbox, spoon the state from my mainnet contract at some specific height, and run the tests from there. Here is how we test migrations for DevHub contract (we don't spoon the state, we only download the deployed contract, but we could spoon the state as well): https://github.com/near/neardevhub-contract/blob/daced931a82437248298413b746a08f724fab5e0/tests/migration.rs

Building a wallet, I may benefit from dry run saving users' tokens for otherwise failing transactions, but there is no guarantee that the transaction will succeed if dry run succeeded (the state of the system may change since the dry-run time), so the win is marginal.