Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

missing build dependencies in the rust build #19714

Closed
maribu opened this issue Jun 7, 2023 · 3 comments
Closed

missing build dependencies in the rust build #19714

maribu opened this issue Jun 7, 2023 · 3 comments
Labels
Area: build system Area: Build system Area: Rust Area: Rust wrapper Type: bug The issue reports a bug / The PR fixes a bug (including spelling errors)

Comments

@maribu
Copy link
Member

maribu commented Jun 7, 2023

Description

preprocessor_successor.h is an auto generated header file included by e.g. auto_init_utils.h. Hence, it must be generated before any C code is processed. Therefore, it is added to BUILDDEPS

Steps to reproduce the issue

Run a rust build with many jobs.

Expected results

The build succeeds, no race conditions in the build process.

Actual results

-- running on worker mobi3 thread 1, build number 1902.
make: Entering directory '/tmp/dwq.0.21595906214010607/3cea9104ba1ac36ffc6676a6b06c190d/examples/rust-hello-world'
make: Leaving directory '/tmp/dwq.0.21595906214010607/3cea9104ba1ac36ffc6676a6b06c190d/examples/rust-hello-world'
make: Entering directory '/tmp/dwq.0.21595906214010607/3cea9104ba1ac36ffc6676a6b06c190d/examples/rust-hello-world'
Building application "hello-world" for "native" with MCU "native".

make[2]: Nothing to be done for 'compile-commands'.
make[3]: Nothing to be done for 'compile-commands'.
make[2]: Nothing to be done for 'compile-commands'.
make[3]: Nothing to be done for 'compile-commands'.
make[3]: Nothing to be done for 'compile-commands'.
make[3]: Nothing to be done for 'compile-commands'.
make[3]: Nothing to be done for 'compile-commands'.
make[3]: Nothing to be done for 'compile-commands'.
   Compiling proc-macro2 v1.0.56
   Compiling unicode-ident v1.0.8
   Compiling quote v1.0.26
   Compiling memchr v2.5.0
   Compiling syn v1.0.109
   Compiling libc v0.2.142
   Compiling glob v0.3.1
   Compiling serde_derive v1.0.160
   Compiling cfg-if v1.0.0
   Compiling minimal-lexical v0.2.1
   Compiling log v0.4.17
   Compiling clang-sys v1.6.1
   Compiling serde v1.0.160
   Compiling libloading v0.7.4
   Compiling semver v1.0.17
   Compiling serde_json v1.0.96
   Compiling regex-syntax v0.7.1
   Compiling nom v7.1.3
   Compiling aho-corasick v1.0.1
   Compiling either v1.8.1
   Compiling syn v2.0.15
   Compiling bindgen v0.64.0
   Compiling which v4.4.0
   Compiling rustc-hash v1.1.0
   Compiling ryu v1.0.13
   Compiling peeking_take_while v0.1.2
   Compiling itoa v1.0.6
   Compiling shlex v1.1.0
   Compiling bitflags v1.3.2
   Compiling lazycell v1.3.0
   Compiling lazy_static v1.4.0
   Compiling autocfg v1.1.0
   Compiling regex v1.8.1
   Compiling rustc_version v0.4.0
   Compiling nb v1.1.0
   Compiling cexpr v0.6.0
   Compiling byteorder v1.4.3
   Compiling heapless v0.7.16
   Compiling nb v0.1.3
   Compiling num-traits v0.2.15
   Compiling void v1.0.2
   Compiling shlex v0.1.1
   Compiling embedded-hal v0.2.7
   Compiling riot-wrappers v0.8.1 (https://github.com/RIOT-OS/rust-riot-wrappers#fbc6cfe8)
   Compiling hash32 v0.2.1
   Compiling c2rust-asm-casts v0.2.0
   Compiling cty v0.2.2
   Compiling stable_deref_trait v1.2.0
   Compiling switch-hal v0.4.0
   Compiling embedded-graphics v0.6.2
   Compiling cstr v0.2.11
   Compiling pin-utils v0.1.0
   Compiling mutex-trait v0.2.0
   Compiling hex v0.4.3
   Compiling bare-metal v1.0.0
   Compiling rust_riotmodules v0.1.0 (/tmp/dwq.0.21595906214010607/3cea9104ba1ac36ffc6676a6b06c190d/sys/rust_riotmodules)
   Compiling c2rust-bitfields-derive v0.2.1
   Compiling c2rust-bitfields v0.3.0
   Compiling riot-sys v0.7.9 (https://github.com/RIOT-OS/rust-riot-sys#b4bd4bde)
error: failed to run custom build command for `riot-sys v0.7.9 (https://github.com/RIOT-OS/rust-riot-sys#b4bd4bde)`

Caused by:
  process didn't exit successfully: `/tmp/dwq.0.21595906214010607/3cea9104ba1ac36ffc6676a6b06c190d/build/target/release/build/riot-sys-9b63ae16468e8507/build-script-build` (exit status: 101)
  --- stdout
  cargo:rerun-if-env-changed=BUILDING_RIOT_RS
  cargo:rerun-if-env-changed=RIOT_CC
  cargo:rerun-if-env-changed=RIOT_CFLAGS
  cargo:rerun-if-env-changed=RIOT_COMPILE_COMMANDS_JSON
  cargo:rerun-if-changed=/tmp/dwq.0.21595906214010607/3cea9104ba1ac36ffc6676a6b06c190d/build/cargo-compile-commands.json
  cargo:rerun-if-env-changed=RIOT_USEMODULE
  cargo:CC=clang
  cargo:CFLAGS=-DDEVELHELP -Werror -Wall -Wextra -pedantic -Og -U_FORTIFY_SOURCE "-std=gnu11" -m32 -fstack-protector-all -ffunction-sections -fdata-sections -DDEBUG_ASSERT_VERBOSE "-DRIOT_APPLICATION=\"hello-world\"" "-DBOARD_NATIVE=\"native\"" "-DRIOT_BOARD=BOARD_NATIVE" "-DCPU_NATIVE=\"native\"" "-DRIOT_CPU=CPU_NATIVE" "-DMCU_NATIVE=\"native\"" "-DRIOT_MCU=MCU_NATIVE" -fwrapv -Wstrict-overflow -fno-common -ffunction-sections -fdata-sections -Wall -Wextra -Wmissing-include-dirs -fno-delete-null-pointer-checks -Wstrict-prototypes -Wold-style-definition "-Wformat=2" -Wformat-overflow -Wformat-truncation "-fmacro-prefix-map=/tmp/dwq.0.21595906214010607/3cea9104ba1ac36ffc6676a6b06c190d/=" -Wcast-align -include /tmp/dwq.0.21595906214010607/3cea9104ba1ac36ffc6676a6b06c190d/build/riotbuild/riotbuild.h -Wno-unknown-warning-option -DNATIVE_INCLUDES -I/tmp/dwq.0.21595906214010607/3cea9104ba1ac36ffc6676a6b06c190d/boards/native/include -I/tmp/dwq.0.21595906214010607/3cea9104ba1ac36ffc6676a6b06c190d/core/lib/include -I/tmp/dwq.0.21595906214010607/3cea9104ba1ac36ffc6676a6b06c190d/core/include -I/tmp/dwq.0.21595906214010607/3cea9104ba1ac36ffc6676a6b06c190d/drivers/include -I/tmp/dwq.0.21595906214010607/3cea9104ba1ac36ffc6676a6b06c190d/cpu/native/include -I/tmp/dwq.0.21595906214010607/3cea9104ba1ac36ffc6676a6b06c190d/sys/include -DMODULE_AUTO_INIT -DMODULE_BOARD -DMODULE_BOARD_COMMON_INIT -DMODULE_CORE -DMODULE_CORE_IDLE_THREAD -DMODULE_CORE_INIT -DMODULE_CORE_LIB -DMODULE_CORE_MSG -DMODULE_CORE_PANIC -DMODULE_CORE_THREAD -DMODULE_CPU -DMODULE_LIBC -DMODULE_NATIVE_DRIVERS -DMODULE_PERIPH -DMODULE_PERIPH_COMMON -DMODULE_PERIPH_GPIO -DMODULE_PERIPH_GPIO_LINUX -DMODULE_PERIPH_INIT -DMODULE_PERIPH_INIT_GPIO -DMODULE_PERIPH_INIT_GPIO_LINUX -DMODULE_PERIPH_INIT_LED0 -DMODULE_PERIPH_INIT_LED1 -DMODULE_PERIPH_INIT_LED2 -DMODULE_PERIPH_INIT_LED3 -DMODULE_PERIPH_INIT_LED4 -DMODULE_PERIPH_INIT_LED5 -DMODULE_PERIPH_INIT_LED6 -DMODULE_PERIPH_INIT_LED7 -DMODULE_PERIPH_INIT_LEDS -DMODULE_PERIPH_INIT_PM -DMODULE_PERIPH_INIT_UART -DMODULE_PERIPH_PM -DMODULE_PERIPH_UART -DMODULE_PREPROCESSOR -DMODULE_PREPROCESSOR_SUCCESSOR -DMODULE_STDIO_NATIVE -DMODULE_SYS
  cargo:rerun-if-changed=riot-bindgen.h

  --- stderr
  /tmp/dwq.0.21595906214010607/3cea9104ba1ac36ffc6676a6b06c190d/sys/include/auto_init_utils.h:35:10: fatal error: 'preprocessor_successor.h' file not found
  thread 'main' panicked at 'Unable to generate bindings: ClangDiagnostic("/tmp/dwq.0.21595906214010607/3cea9104ba1ac36ffc6676a6b06c190d/sys/include/auto_init_utils.h:35:10: fatal error: 'preprocessor_successor.h' file not found\n")', /data/riotbuild/.cargo/git/checkouts/rust-riot-sys-d12733b89271907c/b4bd4bd/build.rs:224:10
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
make: *** [/tmp/dwq.0.21595906214010607/3cea9104ba1ac36ffc6676a6b06c190d/makefiles/cargo-targets.inc.mk:53: /tmp/dwq.0.21595906214010607/3cea9104ba1ac36ffc6676a6b06c190d/build/target/i686-unknown-linux-gnu/release/librust_hello_world.a] Error 101
make: Leaving directory '/tmp/dwq.0.21595906214010607/3cea9104ba1ac36ffc6676a6b06c190d/examples/rust-hello-world'
cat: /tmp/dwq.0.21595906214010607/3cea9104ba1ac36ffc6676a6b06c190d/build/test-input-hash.sha1: No such file or directory
cat: /tmp/dwq.0.21595906214010607/3cea9104ba1ac36ffc6676a6b06c190d/build/test-input-hash.sha1: No such file or directory
{"build/": 258840}

Versions

Current master, see https://ci.riot-os.org/details/b083b1ea3ed14081a7ae5f3ac9f38cf4
@maribu maribu added Type: bug The issue reports a bug / The PR fixes a bug (including spelling errors) Area: build system Area: Build system Area: Rust Area: Rust wrapper labels Jun 7, 2023
@kaspar030 kaspar030 mentioned this issue Jun 8, 2023
Open
@maribu
Copy link
Member Author

maribu commented Jun 8, 2023

This is strange. Makefile target $(CARGO_LIB) depends on $(BUILDDEPS), which in turn depends on $(BUILDDEPS).

Adding

diff --git a/makefiles/cargo-targets.inc.mk b/makefiles/cargo-targets.inc.mk
index 118cb83f7a..24d485be9a 100644
--- a/makefiles/cargo-targets.inc.mk
+++ b/makefiles/cargo-targets.inc.mk
@@ -40,6 +40,10 @@ $(CARGO_COMPILE_COMMANDS): $(BUILDDEPS)
             -e 's/"riscv-none-embed"/"riscv32"/g' \
       | $(LAZYSPONGE) $@
 
+$(info BUILDDEPS=$(BUILDDEPS))
+ifeq (,$(filter %preprocessor_successor.h,$(BUILDDEPS)))
+  $(error BUILDDEPS invalid)
+endif
 
 $(CARGO_LIB): $(RIOTBUILD_CONFIG_HEADER_C) $(BUILDDEPS) $(CARGO_COMPILE_COMMANDS) FORCE
     $(Q)command -v cargo >/dev/null || ($(COLOR_ECHO) \

should blow if $(BUILDDEPS) doesn't include the preprocessor_successor.h as dependency, so it should really be there.

Now I wonder if rather than a dependency issue this is an issue with the CFLAGS being wrong and -I$(BINDIR)/preprocessor not being present.

@maribu
Copy link
Member Author

maribu commented May 28, 2024

@chrysn if I recall correctly this is fixed, right?

@chrysn
Copy link
Member

chrysn commented May 28, 2024

Yes. It was fixed by RIOT-OS/rust-riot-sys#36. The cause was that some -I CFLAGS that were not set in all compilation units were dropped based on their order of occurrence in compile-commands.json, which in turn is influenced by the OS's unsorted reporting of files in a directory, which was why this was so hard to debug: If the sequence is good on one system, it stays good and doesn't turn bad, and only when the files are all created in a go, it was random whether or not it would occur.

Thanks for pinging me; I had forgotten about where this was tracked.

@maribu maribu closed this as completed May 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Area: build system Area: Build system Area: Rust Area: Rust wrapper Type: bug The issue reports a bug / The PR fixes a bug (including spelling errors)
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@chrysn @maribu