Fix infinite isoltest expectation update loop on values not taking full slots #14500
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I ran into this while working on #12668. Isoltest falls into an infinite expectation update loop on some tests, e.g.
malformed_panic_2.sol
. This happens when the test fails for whatever reason and you accept the expectation update. The updated value does not really match what is returned from the EVM and it asks you to update again, ad infinitum.Example of the problem
Let's take a look at
malformed_panic_2.sol
. The expectation that triggers the problem is this:// b() -> FAILURE, hex"4e487b710000"
Isoltest will reformat it as something like this:
the specific number is different each time and is the original expectation followed by some random garbage. Note also that the integer does not match the bytes shown below it. Its hex value is instead:
The cause
There are two separate bugs here.
The first one is that when formatting the expectation,
BytesUtils
assumes a specific format for the output (described by theParameterList
type). If the output happens to be shorter than the format, it increments an iterator past the end of the vector containing it, grabbing some random garbage from memory. That's what we see in the expectations.Identical bug exists in both
BytesUtils::formatBytesRange()
andBytesUtils::formatRawBytes()
. The first one we use to format the expectation comment, the second to display the array of individual bytes. That's why they don't match in the example I have shown above.The second bug is that by default the expectation is always formatted as a sequence of 32-byte integers. When isoltest rereads such an expectation, it sees each integer as 32 bytes (because integers are right-aligned). That will never match output whose length is not a multiple of 32 bytes.
The only reason it does not happen more often is that we have some predefined "templates" for what expectations should look like and we only use default ones when the output length does not match those expected ones. For example if the function returns a proper
Panic
orError
, it will match the parameter format returned byContractABIUtils::failureParameters()
and we'll use that. To trigger it you need a weird example like the above, where something that looks like aPanic
is returned (same selector), but the length of revert data is different than it would be in an actualPanic
. Still, it happens often enough to be noticeable. I ran into this quite a few times in the past, never knowing why it's happening.