Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow running doctests via rust.doctest() #13933

Open
wants to merge 15 commits into
base: master
Choose a base branch
from

Conversation

bonzini
Copy link
Contributor

@bonzini bonzini commented Nov 20, 2024

Most of the series is refactoring that avoids duplicating the code that creates arguments to rustc. The integration with build.BuildTarget is a bit rust-specific, so I am willing to change things if you guys don't like it.

mesonbuild/build.py Fixed Show fixed Hide fixed
@@ -42,7 +42,7 @@
from .backend.backends import Backend
from .compilers import Compiler
from .interpreter.interpreter import SourceOutputs, Interpreter
from .interpreter.interpreterobjects import Test
from .interpreter.interpreterobjects import Test, Doctest

Check failure

Code scanning / CodeQL

Module-level cyclic import Error

'Test' may not be defined if module
mesonbuild.interpreter.interpreterobjects
is imported before module
mesonbuild.build
, as the
definition
of Test occurs after the cyclic
import
of mesonbuild.build.
'Test' may not be defined if module
mesonbuild.interpreter.interpreterobjects
is imported before module
mesonbuild.build
, as the
definition
of Test occurs after the cyclic
import
of mesonbuild.build.
'Test' may not be defined if module
mesonbuild.interpreter.interpreterobjects
is imported before module
mesonbuild.build
, as the
definition
of Test occurs after the cyclic
import
of mesonbuild.build.
'Test' may not be defined if module
mesonbuild.interpreter.interpreterobjects
is imported before module
mesonbuild.build
, as the
definition
of Test occurs after the cyclic
import
of mesonbuild.build.
'Test' may not be defined if module
mesonbuild.interpreter.interpreterobjects
is imported before module
mesonbuild.build
, as the
definition
of Test occurs after the cyclic
import
of mesonbuild.build.
'Test' may not be defined if module
mesonbuild.interpreter.interpreterobjects
is imported before module
mesonbuild.build
, as the
definition
of Test occurs after the cyclic
import
of mesonbuild.build.
'Test' may not be defined if module
mesonbuild.interpreter.interpreterobjects
is imported before module
mesonbuild.build
, as the
definition
of Test occurs after the cyclic
import
of mesonbuild.build.
'Test' may not be defined if module
mesonbuild.interpreter.interpreterobjects
is imported before module
mesonbuild.build
, as the
definition
of Test occurs after the cyclic
import
of mesonbuild.build.
'Test' may not be defined if module
mesonbuild.interpreter.interpreterobjects
is imported before module
mesonbuild.build
, as the
definition
of Test occurs after the cyclic
import
of mesonbuild.build.
'Test' may not be defined if module
mesonbuild.interpreter.interpreterobjects
is imported before module
mesonbuild.build
, as the
definition
of Test occurs after the cyclic
import
of mesonbuild.build.
'Test' may not be defined if module
mesonbuild.interpreter.interpreterobjects
is imported before module
mesonbuild.build
, as the
definition
of Test occurs after the cyclic
import
of mesonbuild.build.
'Test' may not be defined if module
mesonbuild.interpreter.interpreterobjects
is imported before module
mesonbuild.build
, as the
definition
of Test occurs after the cyclic
import
of mesonbuild.build.
@@ -42,7 +42,7 @@
from .backend.backends import Backend
from .compilers import Compiler
from .interpreter.interpreter import SourceOutputs, Interpreter
from .interpreter.interpreterobjects import Test
from .interpreter.interpreterobjects import Test, Doctest

Check failure

Code scanning / CodeQL

Module-level cyclic import Error

'Doctest' may not be defined if module
mesonbuild.interpreter.interpreterobjects
is imported before module
mesonbuild.build
, as the
definition
of Doctest occurs after the cyclic
import
of mesonbuild.build.
'Doctest' may not be defined if module
mesonbuild.interpreter.interpreterobjects
is imported before module
mesonbuild.build
, as the
definition
of Doctest occurs after the cyclic
import
of mesonbuild.build.
'Doctest' may not be defined if module
mesonbuild.interpreter.interpreterobjects
is imported before module
mesonbuild.build
, as the
definition
of Doctest occurs after the cyclic
import
of mesonbuild.build.
'Doctest' may not be defined if module
mesonbuild.interpreter.interpreterobjects
is imported before module
mesonbuild.build
, as the
definition
of Doctest occurs after the cyclic
import
of mesonbuild.build.
'Doctest' may not be defined if module
mesonbuild.interpreter.interpreterobjects
is imported before module
mesonbuild.build
, as the
definition
of Doctest occurs after the cyclic
import
of mesonbuild.build.
'Doctest' may not be defined if module
mesonbuild.interpreter.interpreterobjects
is imported before module
mesonbuild.build
, as the
definition
of Doctest occurs after the cyclic
import
of mesonbuild.build.
'Doctest' may not be defined if module
mesonbuild.interpreter.interpreterobjects
is imported before module
mesonbuild.build
, as the
definition
of Doctest occurs after the cyclic
import
of mesonbuild.build.
'Doctest' may not be defined if module
mesonbuild.interpreter.interpreterobjects
is imported before module
mesonbuild.build
, as the
definition
of Doctest occurs after the cyclic
import
of mesonbuild.build.
'Doctest' may not be defined if module
mesonbuild.interpreter.interpreterobjects
is imported before module
mesonbuild.build
, as the
definition
of Doctest occurs after the cyclic
import
of mesonbuild.build.
'Doctest' may not be defined if module
mesonbuild.interpreter.interpreterobjects
is imported before module
mesonbuild.build
, as the
definition
of Doctest occurs after the cyclic
import
of mesonbuild.build.
'Doctest' may not be defined if module
mesonbuild.interpreter.interpreterobjects
is imported before module
mesonbuild.build
, as the
definition
of Doctest occurs after the cyclic
import
of mesonbuild.build.
'Doctest' may not be defined if module
mesonbuild.interpreter.interpreterobjects
is imported before module
mesonbuild.build
, as the
definition
of Doctest occurs after the cyclic
import
of mesonbuild.build.
@bonzini bonzini force-pushed the rustdoc branch 4 times, most recently from 06d0967 to 625e503 Compare November 20, 2024 16:28
Copy link
Member

@dcbaker dcbaker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't read things too closely, but here's some initial review.

mesonbuild/interpreter/interpreter.py Outdated Show resolved Hide resolved
mesonbuild/interpreter/interpreterobjects.py Outdated Show resolved Hide resolved
docs/markdown/Rust-module.md Outdated Show resolved Hide resolved
mesonbuild/compilers/rust.py Outdated Show resolved Hide resolved
mesonbuild/modules/rust.py Show resolved Hide resolved
mesonbuild/modules/rust.py Outdated Show resolved Hide resolved
mesonbuild/modules/rust.py Outdated Show resolved Hide resolved
mesonbuild/modules/rust.py Outdated Show resolved Hide resolved
mesonbuild/modules/rust.py Outdated Show resolved Hide resolved
@bonzini bonzini force-pushed the rustdoc branch 8 times, most recently from 3817dbd to 7b380be Compare December 12, 2024 17:30
@bonzini bonzini marked this pull request as ready for review December 19, 2024 17:53
@dcbaker
Copy link
Member

dcbaker commented Dec 19, 2024

As much to help me keep track of things: patches 1-5 (remove cratetype variable) are r-b

mesonbuild/backend/ninjabackend.py Outdated Show resolved Hide resolved
mesonbuild/modules/rust.py Outdated Show resolved Hide resolved
mesonbuild/modules/rust.py Outdated Show resolved Hide resolved
mesonbuild/modules/rust.py Show resolved Hide resolved
mesonbuild/backend/ninjabackend.py Outdated Show resolved Hide resolved
mesonbuild/compilers/rust.py Outdated Show resolved Hide resolved
mesonbuild/backend/ninjabackend.py Outdated Show resolved Hide resolved
mesonbuild/backend/ninjabackend.py Show resolved Hide resolved
mesonbuild/backend/ninjabackend.py Show resolved Hide resolved
os.path.abspath of the target subdir is not guaranteed to give a sensible
answer; depending on what directory you run meson from, it could build
an absolute path relative the source directory or the build directory
or possibly neither one.  In fact, using os.path.abspath is quite rare
in Meson's code and generally only done in code like

    subdir = os.path.abspath(os.path.join(self.sourcedir, target['subdir']))

or

    ndir1 = os.path.abspath(os.path.realpath(dir1))

While at it, don't use getattr unnecessarily, the cratetype is available.

Reviewed-by: Dylan Baker <[email protected]>
Signed-off-by: Paolo Bonzini <[email protected]>
Reviewed-by: Dylan Baker <[email protected]>
Signed-off-by: Paolo Bonzini <[email protected]>
Implement RustCompiler.build_rpath_args, so that more code can
be shared between non-Rust and Rust targets.  Then, RustCompiler
can override it to convert the arguments to "-C link-arg=" and
add the rustup sysroot.

Reviewed-by: Dylan Baker <[email protected]>
Signed-off-by: Paolo Bonzini <[email protected]>
Since the introduction of dep-info=... it is possible to move the depfile
away from the main build directory without using --out-dir.  This is
less surprising, since the rules for mixing --emit, --out-dir and -o
are not really documented.

Reviewed-by: Dylan Baker <[email protected]>
Signed-off-by: Paolo Bonzini <[email protected]>
Add functions to RustCompiler() to account for differences
between rustc and "rustdoc --test": rustdoc always generates
a binary, does not support -g, and does not need --emit.

Reviewed-by: Dylan Baker <[email protected]>
Signed-off-by: Paolo Bonzini <[email protected]>
Since we're going to split generate_rust_target() in multiple functions,
eliminate the only variable that spans large parts of it.

The cratetype ninja variable has been unused since Meson started invoking
rustc directly nine years ago (commit d952812, "Fix Rust to work with 1.3
release. Closes mesonbuild#277.", 2015-10-11).

Reviewed-by: Dylan Baker <[email protected]>
Signed-off-by: Paolo Bonzini <[email protected]>
Allow reusing the code for doctests.  In particular, the sources are
shared between the two cases.

Signed-off-by: Paolo Bonzini <[email protected]>
This will be used by rustdoc tests because the Test objects takes a
single string for the command and everything else goes in the args.
But apart from this, the need to split the executable from the
arguments is common so create new methods to do it.

Signed-off-by: Paolo Bonzini <[email protected]>
A doctest target is a separate build target (with its own linker
arguments, including dependencies) that is built and added as a
unit test whenever the parent target is built.  The doctest's
target is not accessible via ninja.

Signed-off-by: Paolo Bonzini <[email protected]>
Adjust get_rust_compiler_args() to accept the crate-type externally, because
rustdoc tests are an executable but are compiled with the parent target's
--crate-type.

Apart from that, the rustdoc arguments are very similar to the parent target, and
are handled by the same functions that were split out of generate_rust_target.

This concludes the backend implementation of doctests, only leaving the
implementation of a doctest() function in the rust module.

Signed-off-by: Paolo Bonzini <[email protected]>
rust.doctest() will have to typecheck that to a list of strings,
no other argument types are allowed.  Extract the field out of
BaseTest, placing it in FuncBenchmark and the rust modules's
FuncTest.

Signed-off-by: Paolo Bonzini <[email protected]>
mesonbuild/modules/rust.py Outdated Show resolved Hide resolved
mesonbuild/modules/rust.py Outdated Show resolved Hide resolved
Signed-off-by: Paolo Bonzini <[email protected]>
QEMU needs it in its integration tests (in order to run global constructors),
and therefore in rust.doctest too.  With this change I could do:

  # Rust executables do not support objects, so add an intermediate step.
  rust_qemu_api_objs = static_library(
    'rust_qemu_api_objs',
    objects: [libqom.extract_all_objects(recursive: false),
              libhwcore.extract_all_objects(recursive: false)])

  rust.doctest('rust-qemu-api-doc', _qemu_api_rs,
    dependencies: [qemu_api, qemu_api_macros],
    link_with: libqemuutil,
    link_whole: [rust_qemu_api_objs],
    suite: ['doc', 'rust'])

followed by "meson test --suite doc".

For completeness, add it to rust.test as well.

Signed-off-by: Paolo Bonzini <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants