Added ability to crosscompile doctests #6892
Conversation
|
Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @ehuss (or someone else) soon. If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes. Please see the contribution instructions for more information. |
rust-highfive
added
the
S-waiting-on-review
|
Thanks for the PR, but shouldn't the rustdoc PR also have support for passing arguments to the runtool? |
Yes it does already, rustdoc can take a --runtool argument which it will use for running tests. It is in the other PR. For example |
|
Er sorry my point is that there's a comment here that says |
|
How? Compiletest runtool support just doesn't allow passing arguments with spaces in them, I guess that could also be done here for consistency? But I don't think compiletest should really be setting any precedents for a public API. |
|
I have fixed some tests that were causing the checks to fail. At this point I am waiting on advisement for the runtool argument passing. |
|
@Goirad I'm not sure there's much advice to give? The arguments specified need to be handled and passed to rustdoc, but they currently aren't. There's not really a preference of how to handle them, but they need to be handled. |
|
@alexcrichton I suppose the question is how to pass arguments with spaces? |
|
I don't really understand the question in the sense that there's not any code to talk about to see where a bug might be or comment on that. If arguments are passed as process arguments it doesn't matter, it's only if one end does string parsing or something like that. |
|
The main question is how to get consistent behavior across all tools?
How is |
|
I don't really know nor do I have a preference. I don't personally have time to help design this feature, but I can provide review feedback that this isn't handled and needs to be handled. |
|
@ehuss As per Alex's suggestion, Cargo now passes arguments too, and the companion PR has been updated so that rustdoc accepts and uses them |
|
I have modified the companion PR for rustdoc to feature-gate ignore-foo attributes in doctests. |
|
I think it's fine to skip feature gates here since we'll just inherit the feature gates enabled in rustdoc. |
|
@alexcrichton can you elaborate? I'm not sure how that would work. In your proposal, would |
|
One option would be to just opt in to unstable arguments as well as per-target-ignores when a runtool is found. Although that leads to the situation where to be able to opt in to per-target ignores you need to have a runtool defined for the current target. |
|
Actually thinking about this it's already stable behavior today that we're not running doctests on cross-compiled scenarios with a runner configured. That can't regress in Cargo (and would if this patch lands as-is) so it probably means that we'll need a feature gate of some form in Cargo to enable this behavior. |
|
This seems reasonable enough to me, and it's just blocked on the rustc PR landing. Before landing this can you open a tracking issue in this repository explaining the feature, what's unstable, etc? |
|
@alexcrichton Yes of course, tracking issue here. |
|
☔ The latest upstream changes (presumably #7137) made this pull request unmergeable. Please resolve the merge conflicts. |
|
Could the tests also be updated to assert the right flags (like |
ehuss
added
S-waiting-on-author
Goirad
force-pushed
the
doctest-xcompile
branch
from
240cf7f to
f6ef349
Compare
|
@alexcrichton Tests have been updated |
|
☔ The latest upstream changes (presumably #7311) made this pull request unmergeable. Please resolve the merge conflicts. |
Goirad
force-pushed
the
doctest-xcompile
branch
from
f6ef349 to
26d3e7e
Compare
Goirad
force-pushed
the
doctest-xcompile
branch
2 times, most recently
from
d54bdd6 to
8731579
Compare
Goirad
force-pushed
the
doctest-xcompile
branch
from
8731579 to
a2209fc
Compare
|
@alexcrichton I have removed all OS restrictions from the tests |
|
@bors: r+ awesome, thanks! |
|
📌 Commit a2209fc has been approved by |
bors
added
S-waiting-on-bors
Added ability to crosscompile doctests This commit adds the ability to cross-compile and run doctests. Like before cargo checks if target == host, the difference is that if there is a runtool defined in config.toml, it passes the information forward to rustdoc so that it can run the doctests with that tool. If no tool is defined and the target != host, cargo instead displays a message that doctests will not be compiled because of the missing runtool. See [here](rust-lang/rust#60387) for the companion PR in the rust project that modifies rustdoc to accept the relevant options as well as allow ignoring doctests on a per target level. Partially resolves [#6460](#6460) See [here](#7040) for the tracking issue.
|
☀️ Test successful - checks-azure |
This stabilizes the doctest-xcompile feature by unconditionally enabling it. Closes #7040 Closes #12118 ## What is being stabilized? This changes it so that cargo will run doctests when using the `--target` flag for a target that is not the host. Previously, cargo would ignore doctests (and show a note if passing `--verbose`). A wrapper for running the doctest can be specified with the [`target.*.runner`](https://doc.rust-lang.org/cargo/reference/config.html#targettriplerunner) configuration option (which is powered by the `--test-runtool` rustdoc flag). This would typically be something like qemu to run under emulation. It is my understanding that this should work just like running other kinds of tests. Additionally, the [`target.*.linker`](https://doc.rust-lang.org/cargo/reference/config.html#targettriplelinker) config option is honored for using a custom linker. Already stabilized in rustdoc is the ability to [ignore tests per-target](https://doc.rust-lang.org/nightly/rustdoc/write-documentation/documentation-tests.html#ignoring-targets). ## Motivation The lack of doctest cross-compile support has always been simply due to the lack of functionality in rustdoc to support this. Rustdoc gained the ability to cross-compile doctests some time ago, but there were additional flags like the test runner that were not stabilized until just recently. This is intended to ensure that projects have full test coverage even when doing cross-compilation. It can be [surprising](#12118) to some that this was not happening, particularly since cargo is silent about it. ## Risks The cargo team had several conversations about how to roll out this feature. Ultimately we decided to enable it unconditionally with the understanding that most projects will probably want to have their doctests covered, and that any breakage will be a local concern that can be resolved by either fixing the test or ignoring the target. Tests in rust-lang/rust run into this issue, [particularly on android](rust-lang/rust#119147 (comment)), and those will need to be fixed before this reaches beta. This is something I am looking into. Some cross-compiling scenarios may need codegen flags that are not supported. It's not clear how common this will be, or if ignoring will be a solution, or how difficult it would be to update rustdoc and cargo to support these. Additionally, the split between RUSTFLAGS and RUSTDOCFLAGS can be cumbersome. ## Implementation history - rust-lang/rust#60387 -- Support added to rustdoc to support the `--target` flag and runtool and per-target-ignores. - #6892 -- Initial support in cargo. - #7391 -- Added unstable documentation. - #8094 -- Fix target for doc test cross compilation - #8358 -- Fixed regression with `--target=HOST` not working on stable. - #10132 -- Added note about doctests not running (under `--verbose`). - rust-lang/rust#112751 -- Fixed `--test-run-directory` interaction with `--test-runtool`. - rust-lang/rust#137096 -- Stabilization (and rename) of the rustdoc `--test-runtool` and `--test-runtool-arg` CLI args, and drops `--enable-per-target-ignores` unconditionally enabling it. ## Test coverage Cargo tests: - [artifact_dep::cross_doctests_works_with_artifacts](https://github.com/ehuss/cargo/blob/56c08f84e28d3653ae0c842f331a226738108188/tests/testsuite/artifact_dep.rs#L1248-L1326) -- Checks that doctest has access to the artifact dependencies. - [build_script::duplicate_script_with_extra_env](https://github.com/ehuss/cargo/blob/56c08f84e28d3653ae0c842f331a226738108188/tests/testsuite/build_script.rs#L5514-L5614) -- Checks that build-script env and cfg values are correctly handled on host versus target when cross running doctests. - [cross_compile::cross_tests](https://github.com/ehuss/cargo/blob/56c08f84e28d3653ae0c842f331a226738108188/tests/testsuite/cross_compile.rs#L416-L502) -- Basic test that cross-compiled tests work. - [cross_compile::doctest_xcompile_linker](https://github.com/ehuss/cargo/blob/56c08f84e28d3653ae0c842f331a226738108188/tests/testsuite/cross_compile.rs#L1139-L1182) -- Checks that the linker config argument works. - [custom_target::custom_target_minimal](https://github.com/ehuss/cargo/blob/56c08f84e28d3653ae0c842f331a226738108188/tests/testsuite/custom_target.rs#L39-L71) -- Checks that `.json` targets work with rustdoc cross tests. - [test::cargo_test_doctest_xcompile_ignores](https://github.com/ehuss/cargo/blob/56c08f84e28d3653ae0c842f331a226738108188/tests/testsuite/test.rs#L4743-L4777) -- Checks the `ignore-*` syntax works. - [test::cargo_test_doctest_xcompile_runner](https://github.com/ehuss/cargo/blob/2603268cda3e32565ac27ee642f2b755fa590bac/tests/testsuite/test.rs#L4783-L4863) -- Checks runner with cross doctests. - [test::cargo_test_doctest_xcompile_no_runner](https://github.com/ehuss/cargo/blob/2603268cda3e32565ac27ee642f2b755fa590bac/tests/testsuite/test.rs#L4869-L4907) -- Checks cross doctests without a runner. Rustdoc tests: - [run-make/doctest-runtool](https://github.com/rust-lang/rust/tree/25cdf1f67463c9365d8d83778c933ec7480e940b/tests/run-make/doctests-runtool) -- Tests behavior of `--test-run-directory` with relative paths of the runner. - [rustdoc/doctest/doctest-runtool](https://github.com/rust-lang/rust/blob/25cdf1f67463c9365d8d83778c933ec7480e940b/tests/rustdoc/doctest/doctest-runtool.rs) -- Tests for `--test-runtool` and `--test-runtool-arg`. ## Future concerns There have been some discussions (rust-lang/testing-devex-team#5) about changing how doctests are driven. My understanding is that stabilizing this should not affect those plans, since if cargo becomes the driver, it will simply need to build things with `--target` and use the appropriate runner. ## Change notes This PR changed tests a little: - artifact_dep::no_cross_doctests_works_with_artifacts was changed now that doctests actually work. - cross_compile::cross_tests was changed to properly check doctests. - cross_compile::no_cross_doctests dropped since it is no longer relevant. - standard_lib::doctest didn't need `-Zdoctest-xcompile` since `-Zbuild-std` no longer uses a target. - test::cargo_test_doctest_xcompile was removed since it is a duplicate of cross_compile::cross_tests I think this should probably wait until the next release cutoff, moving this to 1.89 (will update the PR accordingly if that happens).
This commit adds the ability to cross-compile and run doctests.
Like before cargo checks if target == host, the difference is that if there is a runtool defined in config.toml, it passes the information forward to rustdoc so that it can run the doctests with that tool. If no tool is defined and the target != host, cargo instead displays a message that doctests will not be compiled because of the missing runtool.
See here for the companion PR in the rust project that modifies rustdoc to accept the relevant options as well as allow ignoring doctests on a per target level.
Partially resolves #6460
See here for the tracking issue.