It should be possible to run tests from a specific module with runtest

[andreypopp]

I think it is useful to be able to run tests defined in a specific module with jbuilder runtest command, right now it seems the only invocation which is possible is to specify a directory: jbuilder runtest ./dir.

Also it would be useful to make jbuilder runtest to try its best efforts to run tests even if not all code from the specified package/dir can be compiled.

From a discussion with @rgrinberg on OCaml Discord:

@andreypopp: how do I run a ppx_inline_test within a module with dune?

@rgrinberg: you should make a ticket for this. basically, some parameters will need to be passed to the inline tests binary to make this work
but it should be possible
You can also experiment with the binary yourself, see if it has some option for that
Basically, it's the job of ppx_inline_test to split the test suite within a library
but we should be able to pass the appropriate command line args in dune

[KevinHock]

Is there another way to do this? Aside from deleting all the other tests, there's a lot of iteration time wasted in having to run a whole directory of tests vs. just one file.

[rgrinberg]

@KevinHock that seems like a separate problem. I would expect the tests for a module to re-run only when its cmo/cmx changes. @diml would it be possible to make that work?

Btw, another useful thing would be to run the tests in bytecode mode. I find that that for tests, the compilation vs. runtime is skewed heavily towards compilation for library tests. It would be nice to have this option.

[KevinHock]

Oh 💡 💡 💡 that's why I always get Done: 0/0 and wonder what's going on 😁 Thank you @rgrinberg 👍

[ghost]

@KevinHock that seems like a separate problem. I would expect the tests for a module to re-run only when its cmo/cmx changes. @diml would it be possible to make that work?

You have to take into account not only the cmo/cmx of the module, but also the ones of the dependent modules recursively. Additionally, this won't work well when some modules are linked only for their side effects.

However, we could run the tests for the various modules in parallel. ppx_inline_test supports running a subset of the tests, so we could just wire that in dune. Technically, it just means that we have to create intermediate aliases for each module.

[andreypopp]

ppx_inline_test supports running a subset of the tests, so we could just wire that in dune.

I'd like to have this — for now I have to resort to running tests via

esy dune exec -- semver/test/.TestSemver.inline-tests/run.exe inline-test-runner TestSemver -show-counts

when I need to "focus" tests to some module/test case inside the module. This, as I understand, uses knowledge about implementation details of inline tests.

I don't think having aliases for running tests in specific modules is enough, I think inline tests allows also to select a specific test case by line number which is also very useful.

[maiste]

No activity on this for months. We have closed it as unplanned. Feel free to reopen if you think this is important.

[rgrinberg]

No activity for years in fact. I do think this one is important though, judging by the number of user votes.

[Alizter]

I have some unfinished work on this. I haven't been able to polish it up yet due to other obligations.

[hhugo]

X-link #11109