Add support to run unit tests in a multithreaded context

[eduar-hte]

what

The unit_test program has been updated to accept the mtstress argument to run operator/transformation tests in a multithreaded context.

why

The goal is to detect if the operator/transformation fails in this context.

misc

• In this mode, the test will be executed 5'000 times in 50 threads concurrently.
• Allocation & initialization of the operator/transformation is performed once in the main thread, while the evaluation is executed in the threads.
  • This is consistent with the library's support for multithreading, where initialization and loading of rules is expected to run once. See issue Review and document multi-threading support and limitations #3215.

references

This work is related to #3215, to check whether the optional mutex on the Pm operator to access acmp trees is needed.

This was introduced in commit 119a6fc & 7d786b3 likely because of issue #1573. This option is off by default and it's not clear whether it's necessary, as stated here.

[eduar-hte]

All operator & transformation unit tests were executed using this feature and no issue was detected.

NOTE: This doesn't guarantee that there's no multithreading issue lurking in there, though.

[sonarcloud]

Quality Gate passed

Issues
2 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

[airween]

Thanks, @eduar-hte!

A reminder to me: 11 tests are failed, I have to check the reason.

[eduar-hte]

A reminder to me: 11 tests are failed, I have to check the reason.

I think you're referring to these unit tests: test-cases/secrules-language-tests/transformations/phpArgsNames.json.

I remember looking into this while working on the Windows port (PR #3132), because when I run unit_tests manually they'd fail (they're not part of the make check test suite (test/test-suite.in) and that's why the builds don't fail.)

These tests were added to in owasp-modsecurity/secrules-language-tests@a3d4405 and are related to PR #2387, which I understand is a v3.1 feature.

[airween]

I think you're referring to these unit tests: test-cases/secrules-language-tests/transformations/phpArgsNames.json.

yes, exactly.

I remember looking into this while working on the Windows port (PR #3132), because when I run unit_tests manually they'd fail (they're not part of the make check test suite (test/test-suite.in) and that's why the builds don't fail.)

yes, I also wondered why this haven't appeared yet

These tests were added to in owasp-modsecurity/secrules-language-tests@a3d4405 and are related to PR #2387, which I understand is a v3.1 feature.

ah, thanks - I hadn't time to review the history, but yes, this explains the behavior. Seems like the tests had already added but the transformation not. I think we can remove this test file.

Thanks for helping discovery the situation.

[eduar-hte]

ah, thanks - I hadn't time to review the history, but yes, this explains the behavior. Seems like the tests had already added but the transformation not. I think we can remove this test file.

If you don't want to remove it, you could update the submodule reference in test/test-cases/secrules-language-tests to reference the previous commit (owasp-modsecurity/secrules-language-tests@d03f4c1), as the commit with the phpArgsNames tests is the latest commit in that repository.

[eduar-hte]

ah, thanks - I hadn't time to review the history, but yes, this explains the behavior. Seems like the tests had already added but the transformation not. I think we can remove this test file.

Submitted PR owasp-modsecurity/secrules-language-tests#10 to remove the tests associated to this missing transformation.