Forge does not always compute necessary recompilations correctly

[aathan]

I've noticed intermittent cases where forge doesn't seem to be recompiling contracts I expect need recompilation. I've also seen this strange error:

$ forge test -vvv --fork-url http://127.0.0.1:8545/ -m testAllPublic
ignoring ``: invalid filter directive

[onbjerg]

What platform are you on and what version? Can you also provide an example we can test off of?

[aathan]

So I'm hitting this again. This time I have a little more information. So, I'm doing:

forge test -vvv --fork-url http://127.0.0.1:8545/ -m testX
[⠢] Compiling...
[⠃] Compiling 1 files with 0.8.13
[⠒] Solc finished in 683.97ms
Compiler run successful

... notice the test does not run. A prior compilation emitted some warnings in the contract being testing (not Contract.t.sol) ...

forge test -vvv --fork-url http://127.0.0.1:8545/ -m testX
[⠒] Compiling...
No files changed, compilation skipped

Here I just "touched" the test file, but I guess the compilation is based on content delta, so I had to go in and add a space somewhere to force the recompilation. Once I did that, the next repeat of that same forge command above, ran the test.

I'm wondering if there's a bug where if there's a compilation error (or warning) in the core (non test) contracts but no error in the test contract, then you fix the core contract, somehow the test contract is in a state that doesn't run correctly and/or fails silently until you recompile the test. E.g., maybe it is incorrectly being marked as not-dirty even though it was further down the compilation chain when the core error was detected, and then doesn't get recompiled until you also touch it.

... hmm, but I just tried that chain of events and that wasn't it. Scratching my head.

[aathan]

This may be my "user error" (my fault) but combined with the delta mechanism, it creates somewhat unexpected results ... the sequence of events could be explained by there being an error in the test contract AND the core contract. If you then fix the core contract, and fix the test contract but don't hit save, your next forge test... will recompile the core contract but not the test contract (because it has not changed).

The net result is that your forge test emits no test output, instead of having retried the compilation of the test contract. Follow?

Arguably, failed compilations should leave those contracts marked as dirty so they will be recompiled again on subsequent forge commands. If you're trying to minimize emitting the same errors repeatedly, maybe there can be two flags -f to force recompile everything and -F to force recompilation on contracts previously not compiled due to errors.

[onbjerg]

Can you add this context to #1244 and possibly a separate issue about the compiler run successful -> no test are run (unless another issue is already present, then add context to that)? Discussions are primarily to discuss new features/user help and it seems like you might have found an actual bug

[aathan]

Sure. I didn't want to do the converse and discuss non-bugs in a bug report. If you think this is a bug I'll open a bug/add to one.

[onbjerg]

I think it is - going to mark a dummy answer as the solution so we can continue the discussion in the issue tracker instead :)

[aathan]

Hey, so:

$ export RUST_LOG=
$ forge --version
ignoring ``: invalid filter directive
forge 0.2.0 (64087b5 2022-04-13T00:10:42.429318924+00:00)

Empty RUST_LOG is the problem.

[onbjerg]

After discussion in #1299 (comment) this seems like a bug, possibly related to #1244