forge coverage w/ hardhat

[HrikB]

Component

Forge

Describe the feature you would like

Given that foundry supports hardhat integration, it would be great if forge coverage result was calculated using hardhat js tests as well.

Additional context

No response

[onbjerg]

This is not planned as it increases complexity of coverage a lot - more cases to cover on project structure, coverage would need to be inside of Anvil that would need to generate a seperate report with raw data we could merge into forge coverage reports for Forge tests etc.

Anvil should be 1:1 with Hardhat node (or close to), so it should be possible to use solidity-coverage (although slower than Anvil) to generate coverage reports for Hardhat projects that use Anvil

[ZeroEkkusu]

@onbjerg, this issue is related one I wanted to open.

We have a hybrid Hardhat/Foundry project and I tried merging the Hardhat and Forge coverage reports:

genhtml foundry-lcov.info hardhat-lcov.info -o hybrid-coverage --branch-coverage

From what I can tell, they are not compatible. It works for merging multiple Hardhat or Forge coverage reports.

The issues I identified:

• Hardhat counts unchecked, lines that revert, and maybe something else, while Forge does not, so the Total number of lines is bigger in Hardhat reports
• There are 2x the number of Total lines, functions, and branches
• Sometimes, the number of Hit functions is the expected + 1
• Otherwise, seems okay

Is it possible to produce a report that is compatible with Hardhat?

[onbjerg]

Try using a merge-specific tool for lcov files such as lcov --add-tracefile .... Otherwise, it is not going to be in scope for a while since it would be highly dependent on how Hardhat outputs its report in terms of file names etc. and that's not really something we can do much about.

For the function hits, it's probably because I opted to name them <contract>::<function> to avoid name clashes whereas Hardhat might just do <function>.

[ZeroEkkusu]

Thank you for the feedback and the suggestion, I tested it out. I'm going to document the results here for future reference:

• No branch coverage (I'm probably doing something wrong)
• Total lines are OK
• 2x Total functions (maybe because you named them differently)
• unchecked, lines that revert, etc are reported as not covered (because of Hardhat)
• Hit functions are expected + 1 sometimes

Link sc-forks/solidity-coverage#734