feat(compatibility): add zkSync support

[nbaztec]

Component

Forge

Describe the feature you would like

Foundry zkSync Support

To enable Foundry compatibility with zkSync, we created the fork https://github.com/matter-labs/foundry-zksync. This fork introduces changes to support smart contract compilation, deployment, testing, and interaction on zkSync Era.

This feature request is a proposal to integrate zkSync support into Foundry. It details an approach designed to be minimally invasive, complemented by a subsequent request for zksolc support in foundry-compilers. For more info related to zksolc please refer to here, and here.

Objective

The goal is to integrate zkSync support into Foundry, facilitating community and maintainer dialogue on how best to have these changes accepted.

Below is a brief outline on the differences that are accounted for in the proposed approach.

Differences

zkSync VM

The zkSync VM is fundamentally different from the EVM, in it, it uses an entirely different register-based architecture.

Storage

The Storage is as well different from the EVM's account based trie. In the zkSync VM there's a single global trie. Account balances, nonces, etc. are stored under the hashes slots for special system contracts.

Bytecode

Owing to a different VM, the corresponding bytecode is as well different than what is compiled with solc. In comparison, the zkSync VM is compiled with zksolc, which supports all opcodes, with a few exceptions.

Proposed Approach

1. Compilation

The foundry-identified files are passed through zksolc to generate zkSync VM bytecode. This was then matched using the contract's name with its respective EVM/solc counterpart - finally giving us a DualCompiledContract containing both evm and zk bytecodes and their respective hashes (which also differ in how they are computed). This "registry" of compiled contracts was propagated down to the Executors and to the tracers to perform specialized operations in the context of interoperability between the two scopes, namely EVM and ZK.

We're currently in the process of migrating the compilation logic to foundry-compilers, but the concept of compiling the same contract for both environments and passing it down to the internals would probably remain.

2. forge

The proposed approach leverages the CheatcodeTracer and an early implementation can be reviewed on the dev branch of foundry-zksync.

Consider the following contract:

contract Counter {
    uint256 value;

    function setValue(uint256 _value) {
        value = _value;
    }

    function getValue() view returns (uint256) {
        return value;
    }
}

contract CounterTest is Test {
    Counter counter;

    function setUp() {
        counter = new Counter();
    }

    function testSetValue() {
        counter.setValue(10);
        assertEq(10, counter.getValue());
    }
}

We assume the test contract is being tested for deployment on zkSync.

1. Foundry compiles the test suite with solc and then zksolc. Both bytecodes are bundled as DualCompiledContract and passed down to the Executor, till the CheatcodeTracer.
2. All interactions from the default CALLER account to the deployed test contract (including ensure_success calls) are handled as normal EVM calls. Except,
   1. address.balance , which are intercepted as opcodes and retrieve the data from ZK-storage
   2. block.timestamp, block.number, which are updated with the ZK-specific context on the Env directly.
3. Any CALL or CREATE is intercepted in the tracer (with call and create hooks) and translated into a ZK transactions. This includes
   1. Fetching ZK-equivalent bytecode
   2. Fetching correct account nonce, etc.
   3. Marking the callee as EOA (in ZK terms) to bypass EIP-3607 restriction
4. The transaction is then sent to ZK-VM where it returns the statediff, which is applied on the journaled_state, and the result returned back.
5. Any console.log() in the ZK-VM execution is translated back for foundry to pick up

This forms the basic premise of our implementation that foundry gets to do foundry-specific operations, and only at the necessary stage with invoke the zkSync VM when passing the --zksync flag.

Challenges

• The above use-case assumes forge test case. A similar strategy would be required in forge script.
• An executor must be context-aware to set_balance and set_nonce on the correct form of storage - EVM or ZK
• Certain cheatcodes like warp, deal, etc. must be context aware to set correct storage - EVM or ZK
• Certain functionalities like expectCall must be supported by the ZK tracer to return the appropriate data back to foundry's CheatcodeTracer to evaluate their success.

Plan

Feature Flag

It is proposed to put all zkSync related features behind a zksync feature flag. Rust nightly would be required to compile with this flag, as the zksync libraries currently depend on nightly rust.

Foundry zkSync

A single foundry-zksync module will contain all zkSync specific logic, and translations. Other parts of foundry source will simply use this module.

Foundry Compilers

zksolc compilation would initially be added to foundry-compilers that would later be used in foundry.

Forge Commands

Forge commands like test, create, build, script would be incrementally supported as pull requests.

Conclusion

This is obviously the result of work and testing over several months, and is offered as the best case from our current perspective to integrate foundry into the zkSync ecosystem, while maintaining the same level of foundry's user experience. On that front, we'd like to get the thoughts of foundry developers on anything we may have missed, misunderstood, or vaguely underestimated, in our proposed implementation.

Contributors

@aon, @Deniallugo, @dutterbutter, @HermanObst, @Jrigada, @Karrq, @nbaztec

Additional context

No response

[grandizzy]

@nbaztec we're supportive in adding such and rolling it in post v1, reviewed https://gist.github.com/nbaztec/205ac47e8e68bbb3264402965816a413 and would like to continue discussion here on impl.
CC @zerosnacks @klkvr @DaniPopes

[zerosnacks]

My preference goes to strategy 1a using generics: https://gist.github.com/nbaztec/205ac47e8e68bbb3264402965816a413#method-1-strategy for performance reasons + type safety at compile time over 1b given that it is not necessary to switch between strategies at runtime. One of my concerns is that the change would be quite invasive making it challenging to roll out in parts. A practical next step could be for ZKSync to implement an example in a draft PR exploring the integration as well as verifying the proposed workaround of Arc/Arc<Mutex<_>> containers with an example.

[popzxc]

A practical next step could be for ZKSync to implement an example in a draft PR exploring the integration as well as verifying the proposed workaround of Arc/Arc<Mutex<_>> containers with an example.

Yup, that's our plan. We're currently analyzing the areas where the implementation paths diverge, and want to initially build a PoC that would at least demonstrate the required abstractions. For that PoC we may probably use dynamic dispatch, as it's easier to implement, and then it will be rewritten to use static typing.

[nbaztec]

We have an example PR that I made in the making of the abstraction doc using dynamic dispatch (as it involved the least amount of code changes) here https://github.com/matter-labs/foundry-zksync/pull/692/files

It's currently not tracking the latest main but captures the essence of the implementation.

[nbaztec]

I gave the process a go with generics and encountered the issues/limitations that there were extensive code changes across the foundry code due to the introduction of strategy objects S via Backend<S>, Executor<S>, etc. that would render hard to review changes. Additionally, there were tertiary changes that included things like changing dyn Trait objects to Boxs as they aren't object safe anymore. This compared with splitting up override-able code and moving it around to put it into strategy objects would make this a huge big bang PR.

As a result, in the interest of simply demonstrating the approach, and focusing instead on abstractions and moving things around to avoid cyclic dependencies, we'd like to propose that we proceed with the trait object approach. This would not only be a smaller code footprint but also help the reviewers focus on the proposed abstraction boundaries without additional noise generated due to generic and trait propagation.

[popzxc]

@nbaztec is it possible to prepare a benchmark that would show whether trait objects involve any kind of performance penalty? E.g. with your PR above, you can write a benchmark and run it against upstream code and trait-based implementation.

[nbaztec]

@popzxc Yes I believe the foundry team also has a benchmarking which I saw being done on certain PRs. We'd definitely be running the dynamic dispatch approach with native foundry to figure out the performance hit.