feat(fuzz): ast-seeded dictionary

[0xrusowsky]

Motivation

closes #10233

Solution

1. leverage solar::sema::Compiler to collect all relevant AST literals found in the sources (excluding libs and scripts) and seed the FuzzerDictionary with them at initialization.
2. modify the strategies to source from a new pool of values (AST literals), when available
3. modify the string strategy to not always generate random strings, but also source from the string literals pool

Future improvements

• explore literal value folding as suggested in: feat(fuzz): ast-seeded dictionary #12015 (comment). Tracked on a separate issue: feat(fuzz): impl constant folding for typical expressions #12044
• use HIR rather than AST to figure out the type of the numeric literals, so that we can minimize dict size

PR Checklist

• Added Tests
• Added Documentation
• Breaking changes

[0xalpharush]

Not blocking but I would recommend implementing constant folding to some degree i.e. evaluate 2 * 2 ether, evaluate bytes32 IMPLEMENTATION_SLOT = bytes32(uint256(keccak256('eip1967.proxy.implementation')) - 1);, evaluate uint(-2). Arguably, solar will need this and the code could live there

See crytic/echidna#636

[grandizzy]

Not blocking but I would recommend implementing constant folding to some degree i.e. evaluate 2 * 2 ether, perform the keccack hash of a string, evaluate uint(-2).

See crytic/echidna#636

thanks! One thing here - this means we should collect from tests too which we don't do in PR, is this correct?

[0xalpharush]

AFAIK neither Echidna or slither's printer filters tests out. I think not including forge-std makes sense.

Also, I am not sure how the push/pop/log dictionary is managed currently in Foundry, but I think Echidna will always keep the constant pool around and eject the dynamically collected values after running a full sequence. For example, a user's balance that is emitted in one run may help within the same sequence but probably unlikely to help in a totally unrelated sequence.

[grandizzy]

AFAIK neither Echidna or slither's printer filters tests out. I think not including forge-std makes sense.

👍 @0xrusowsky let's include too

Also, I am not sure how the push/pop/log dictionary is managed currently in Foundry, but I think Echidna will always keep the constant pool around and eject the dynamically collected values after running a full sequence. For example, a user's balance that is emitted in one run may help within the same sequence but probably unlikely to help in a totally unrelated sequence.

• The push / pop dictionary + db addresses / storage values are populated when test starts and used across all runs, without being evicted.
  These are defined as
  

  foundry/crates/evm/fuzz/src/strategies/state.rs

  Lines 129 to 134 in b823ae0

  	/// Number of state values initially collected from db.
  	/// Used to revert new collected values at the end of each run.
  	db_state_values: usize,
  	/// Number of address values initially collected from db.
  	/// Used to revert new collected addresses at the end of each run.
  	db_addresses: usize,

  
  and collected when dict is created
  

  foundry/crates/evm/fuzz/src/strategies/state.rs

  Lines 52 to 54 in b823ae0

  	// Create fuzz dictionary and insert values from db state.
  	let mut dictionary = FuzzDictionary::new(config);
  	dictionary.insert_db_values(accs);

• We also maintain a dict of so called sample values, dynamically collected from logs, return values & state changes of runs up to a limit (set rn to the test depth) - these are also reused across all runs
  

  foundry/crates/evm/fuzz/src/strategies/state.rs

  Lines 135 to 136 in b823ae0

  	/// Sample typed values that are collected from call result and used across invariant runs.
  	sample_values: HashMap<DynSolType, B256IndexSet>,

• Then there are the regular values dynamically collected from runs that are not shared between runs
  

  foundry/crates/evm/fuzz/src/strategies/state.rs

  Lines 123 to 124 in b823ae0

  	/// Collected state values.
  	state_values: B256IndexSet,

Please let us know if you see any redundant data / ways to improve the dict. Thank you!

[0xrusowsky]

^ note that AST literals are injected into sample_values

[grandizzy]

lgtm! left a comment re tests, please check if worth adding them. thanks!

[grandizzy]

is this still a TODO or good to be removed?

[0xrusowsky]

good to be removed, always passes locally

[grandizzy]

that's awesome! I assume this one should pass now quite quickly and we can include too?

foundry/crates/forge/tests/cli/test_cmd/invariant/storage.rs

Lines 3 to 6 in e98bff5

	forgetest_init!(
	#[ignore = "slow"]
	storage,
	|prj, cmd| {

Since should_fuzz_literals test is only for stateless fuzz, would it make sense to add an invariant one as well? I am thinking in particular to replicate this one

foundry/crates/forge/tests/cli/test_cmd/invariant/common.rs

Line 440 in e98bff5

forgetest_init!(invariant_fixtures, |prj, cmd| {

which we weren't able to break without providing the fixtures hints.

[0xrusowsky]

@grandizzy all passes as expected, see 4d9ee25 (#12015)