Host Path Included in Build Output Causes Yocto Build Failure #1087

surainbow · 2024-12-23T06:19:25Z

Description
Following the provided CMake build instructions:

mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX=$HOME/.local/lapack ..
cmake --build . -j --target install

Upon inspecting the build.make file, the generated build command appears as follows:
cd /buildarea6/pzha1-cn/upstream/lapack-buil/SRC && /folk/pzha1-cn/homebrew/bin/gfortran $(Fortran_DEFINES) $(Fortran_INCLUDES) $(Fortran_FLAGS) -cpp -c /buildarea6/pzha1-cn/upstream/lapack/SRC/xerbla.f -o CMakeFiles/lapack_obj.dir/xerbla.f.o
Checking the resulting xerbla.f.o file using strings and readelf reveals that the compiled object file contains host paths:

$strings BLAS/SRC/CMakeFiles/blas_64_obj.dir/blas_64_obj/xerbla.f.o | grep buildarea6
/buildarea6/pzha1-cn/upstream/lapack-buil/BLAS/SRC/blas_64_obj/xerbla.f

$readelf -x .rodata.str1.8 BLAS/SRC/CMakeFiles/blas_64_obj.dir/blas_64_obj/xerbla.f.o
Hex dump of section '.rodata.str1.8':
  0x00000000 2f627569 6c646172 6561362f 707a6861 /buildarea6/pzha
  0x00000010 312d636e 2f757073 74726561 6d2f6c61 1-cn/upstream/la
  0x00000020 7061636b 2d627569 6c2f424c 41532f53 pack-buil/BLAS/S
  0x00000030 52432f62 6c61735f 36345f6f 626a2f78 RC/blas_64_obj/x
  0x00000040 6572626c 612e6600 28202720 2a2a204f erbla.f.( ' ** O
  0x00000050 6e20656e 74727920 746f2027 2c20412c n entry to ', A,
  0x00000060 20272070 6172616d 65746572 206e756d  ' parameter num
  0x00000070 62657220 272c2049 322c2027 20686164 ber ', I2, ' had
  0x00000080 20272c20 20202020 20202761 6e20696c  ',       'an il
  0x00000090 6c656761 6c207661 6c756527 202900   legal value' ).

The inclusion of host paths (e.g., /buildarea6/pzha1-cn/upstream/lapack-buil/...) in the .o file leads to a failure in the Yocto build process during the do_package_qa step.

Host paths embedded in object files violate Yocto's reproducibility and deterministic build requirements, causing the build to fail.

Can anyone assist in resolving this issue of host paths being included in the build artifacts? Any guidance or patches to ensure a clean and reproducible build without embedded host-specific paths would be greatly appreciated.

The text was updated successfully, but these errors were encountered:

martin-frbg · 2024-12-23T08:34:28Z

can you check with objdump that the string is actually in a functional part, and not just a comment left by your compiler ?

surainbow · 2024-12-24T02:15:36Z

this string is not included in .text segment, but stored in .rodata.str1.8 segment only.

❯ objdump -d BLAS/SRC/CMakeFiles/blas_64_obj.dir/blas_64_obj/xerbla.f.o
BLAS/SRC/CMakeFiles/blas_64_obj.dir/blas_64_obj/xerbla.f.o:     file format elf64-x86-64
Disassembly of section .text:
0000000000000000 <xerbla_64_>:
   0:   41 55                   push   %r13
   2:   48 8d 05 00 00 00 00    lea    0x0(%rip),%rax        # 9 <xerbla_64_+0x9>
   9:   49 89 d5                mov    %rdx,%r13
   c:   41 54                   push   %r12
   e:   49 89 f4                mov    %rsi,%r12
  11:   55                      push   %rbp
  12:   48 89 fd                mov    %rdi,%rbp
  15:   53                      push   %rbx
  16:   48 81 ec 18 02 00 00    sub    $0x218,%rsp
  1d:   48 89 44 24 08          mov    %rax,0x8(%rsp)
  22:   48 8d 05 00 00 00 00    lea    0x0(%rip),%rax        # 29 <xerbla_64_+0x29>
  29:   48 89 e7                mov    %rsp,%rdi
  2c:   48 89 44 24 50          mov    %rax,0x50(%rsp)
  31:   48 8b 05 00 00 00 00    mov    0x0(%rip),%rax        # 38 <xerbla_64_+0x38>
  38:   c7 44 24 10 4d 00 00    movl   $0x4d,0x10(%rsp)
  3f:   00
  40:   48 89 04 24             mov    %rax,(%rsp)
  44:   48 c7 44 24 58 56 00    movq   $0x56,0x58(%rsp)
  4b:   00 00
  4d:   e8 00 00 00 00          call   52 <xerbla_64_+0x52>
  52:   48 89 ee                mov    %rbp,%rsi
  55:   4c 89 ef                mov    %r13,%rdi
  58:   e8 00 00 00 00          call   5d <xerbla_64_+0x5d>
  5d:   31 d2                   xor    %edx,%edx
  5f:   48 89 ee                mov    %rbp,%rsi
  62:   48 89 e7                mov    %rsp,%rdi
  65:   48 85 c0                test   %rax,%rax
  68:   48 0f 49 d0             cmovns %rax,%rdx
  6c:   e8 00 00 00 00          call   71 <xerbla_64_+0x71>
  71:   ba 08 00 00 00          mov    $0x8,%edx
  76:   4c 89 e6                mov    %r12,%rsi
  79:   48 89 e7                mov    %rsp,%rdi
  7c:   e8 00 00 00 00          call   81 <xerbla_64_+0x81>
  81:   48 89 e7                mov    %rsp,%rdi
  84:   e8 00 00 00 00          call   89 <xerbla_64_+0x89>
  89:   31 d2                   xor    %edx,%edx
  8b:   31 f6                   xor    %esi,%esi
  8d:   31 ff                   xor    %edi,%edi
  8f:   e8 00 00 00 00          call   94 <.LC1+0x4c>

❯ objdump -s -j .rodata.str1.8 BLAS/SRC/CMakeFiles/blas_64_obj.dir/blas_64_obj/xerbla.f.o
BLAS/SRC/CMakeFiles/blas_64_obj.dir/blas_64_obj/xerbla.f.o:     file format elf64-x86-64
Contents of section .rodata.str1.8:
 0000 2f627569 6c646172 6561362f 707a6861  /buildarea6/pzha
 0010 312d636e 2f757073 74726561 6d2f6c61  1-cn/upstream/la
 0020 7061636b 2d627569 6c2f424c 41532f53  pack-buil/BLAS/S
 0030 52432f62 6c61735f 36345f6f 626a2f78  RC/blas_64_obj/x
 0040 6572626c 612e6600 28202720 2a2a204f  erbla.f.( ' ** O
 0050 6e20656e 74727920 746f2027 2c20412c  n entry to ', A,
 0060 20272070 6172616d 65746572 206e756d   ' parameter num
 0070 62657220 272c2049 322c2027 20686164  ber ', I2, ' had
 0080 20272c20 20202020 20202761 6e20696c   ',       'an il
 0090 6c656761 6c207661 6c756527 202900    legal value' ).
❯ objdump -h BLAS/SRC/CMakeFiles/blas_64_obj.dir/blas_64_obj/xerbla.f.o
BLAS/SRC/CMakeFiles/blas_64_obj.dir/blas_64_obj/xerbla.f.o:     file format elf64-x86-64
Sections:
Idx Name          Size      VMA               LMA               File off  Algn
  0 .text         00000094  0000000000000000  0000000000000000  00000040  2**4
                  CONTENTS, ALLOC, LOAD, RELOC, READONLY, CODE
  1 .data         00000000  0000000000000000  0000000000000000  000000d4  2**0
                  CONTENTS, ALLOC, LOAD, DATA
  2 .bss          00000000  0000000000000000  0000000000000000  000000d4  2**0
                  ALLOC
  3 .rodata.str1.8 0000009f  0000000000000000  0000000000000000  000000d8  2**3
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  4 .rodata.cst8  00000008  0000000000000000  0000000000000000  00000178  2**3
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  5 .comment      00000023  0000000000000000  0000000000000000  00000180  2**0
                  CONTENTS, READONLY
  6 .note.GNU-stack 00000000  0000000000000000  0000000000000000  000001a3  2**0
                  CONTENTS, READONLY
  7 .eh_frame     00000048  0000000000000000  0000000000000000  000001a8  2**3
                  CONTENTS, ALLOC, LOAD, RELOC, READONLY, DATA

my suspection is gfortran add the string into .rodata segment,(see the generated complie command "cd /buildarea6/pzha1-cn/upstream/lapack-buil/SRC && /folk/pzha1-cn/homebrew/bin/gfortran $(Fortran_DEFINES) $(Fortran_INCLUDES) $(Fortran_FLAGS) -cpp -c /buildarea6/pzha1-cn/upstream/lapack-buil/BLAS/SRC/blas_64_obj/xerbla.f -o CMakeFiles/lapack_obj.dir/xerbla.f.o"), when i try to compile the manually by using relevant path, the relevant path stored in .rodata segement.

Only detect the xerbla.o included the path that i am not expect.

shr-project · 2025-01-21T17:42:42Z

I've tried to change SOURCES_64 and SOURCES variables to use paths relative to ${CMAKE_CURRENT_BINARY_DIR} as in:

From 4aaae99a6311534c0d8677ed21ab6eb036ec45dc Mon Sep 17 00:00:00 2001
From: Martin Jansa <[email protected]>
Date: Fri, 17 Jan 2025 15:30:09 +0000
Subject: [PATCH] CMakeLists.txt: use relative path to .f files

https://github.com/Reference-LAPACK/lapack/issues/1087

Signed-off-by: Martin Jansa <[email protected]>
---
 BLAS/SRC/CMakeLists.txt       | 4 ++--
 CBLAS/src/CMakeLists.txt      | 4 ++--
 SRC/CMakeLists.txt            | 8 ++++----
 TESTING/MATGEN/CMakeLists.txt | 8 ++++----
 4 files changed, 12 insertions(+), 12 deletions(-)

diff --git a/BLAS/SRC/CMakeLists.txt b/BLAS/SRC/CMakeLists.txt
index b9e6f7c4a..c96c1119d 100644
--- a/BLAS/SRC/CMakeLists.txt
+++ b/BLAS/SRC/CMakeLists.txt
@@ -117,7 +117,7 @@ if(BUILD_INDEX64_EXT_API)
   # Copy files so we can set source property specific to /${BLASLIB}_64_obj target
   file(MAKE_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/${BLASLIB}_64_obj)
   file(COPY ${SOURCES} DESTINATION ${CMAKE_CURRENT_BINARY_DIR}/${BLASLIB}_64_obj)
-  file(GLOB SOURCES_64_F ${CMAKE_CURRENT_BINARY_DIR}/${BLASLIB}_64_obj/*.f*)
+  file(GLOB SOURCES_64_F RELATIVE ${CMAKE_CURRENT_BINARY_DIR} ${CMAKE_CURRENT_BINARY_DIR}/${BLASLIB}_64_obj/*.f*)
   add_library(${BLASLIB}_64_obj OBJECT ${SOURCES_64_F})
   target_compile_options(${BLASLIB}_64_obj PRIVATE ${FOPT_ILP64})
   set_target_properties(${BLASLIB}_64_obj PROPERTIES POSITION_INDEPENDENT_CODE ON)
@@ -128,7 +128,7 @@ if(BUILD_INDEX64_EXT_API)
     else()
       set_source_files_properties(${F} PROPERTIES COMPILE_FLAGS "-cpp")
     endif()
-    file(STRINGS ${F} ${F}.lst)
+    file(STRINGS ${CMAKE_CURRENT_BINARY_DIR}/${F} ${F}.lst)
     list(FILTER ${F}.lst INCLUDE REGEX "subroutine|SUBROUTINE|external|EXTERNAL|function|FUNCTION")
     list(FILTER ${F}.lst EXCLUDE REGEX "^!.*")
     list(FILTER ${F}.lst EXCLUDE REGEX "^[*].*")
diff --git a/CBLAS/src/CMakeLists.txt b/CBLAS/src/CMakeLists.txt
index 8dcb2f293..e5ea96e62 100644
--- a/CBLAS/src/CMakeLists.txt
+++ b/CBLAS/src/CMakeLists.txt
@@ -137,7 +137,7 @@ if(BUILD_INDEX64_EXT_API)
   # Copy files so we can set source property specific to /${CBLASLIB}_64_obj target
   file(MAKE_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/${CBLASLIB}_64_fobj)
   file(COPY ${SOURCES_64_F} DESTINATION ${CMAKE_CURRENT_BINARY_DIR}/${CBLASLIB}_64_fobj)
-  file(GLOB SOURCES_64_F ${CMAKE_CURRENT_BINARY_DIR}/${CBLASLIB}_64_fobj/*)
+  file(GLOB SOURCES_64_F RELATIVE ${CMAKE_CURRENT_BINARY_DIR} ${CMAKE_CURRENT_BINARY_DIR}/${CBLASLIB}_64_fobj/*)
   add_library(${CBLASLIB}_64_cobj OBJECT ${SOURCES_64_C})
   add_library(${CBLASLIB}_64_fobj OBJECT ${SOURCES_64_F})
   set_target_properties(${CBLASLIB}_64_cobj ${CBLASLIB}_64_fobj PROPERTIES
@@ -152,7 +152,7 @@ if(BUILD_INDEX64_EXT_API)
   #Add suffix to all Fortran functions via macros
   foreach(F IN LISTS SOURCES_64_F)
       set(COPT_64_F)
-      file(STRINGS ${F} ${F}.lst)
+      file(STRINGS ${CMAKE_CURRENT_BINARY_DIR}/${F} ${F}.lst)
       list(FILTER ${F}.lst INCLUDE REGEX "subroutine|external")
       foreach(FUNC IN LISTS ${F}.lst)
         string(REGEX REPLACE "[ ]*(subroutine|external)[ ]*" "" FUNC ${FUNC})
diff --git a/SRC/CMakeLists.txt b/SRC/CMakeLists.txt
index be426cecd..c39edf5d1 100644
--- a/SRC/CMakeLists.txt
+++ b/SRC/CMakeLists.txt
@@ -545,13 +545,13 @@ if(BUILD_INDEX64_EXT_API)
     set(SOURCES_64)
     file(MAKE_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/${LAPACKLIB}_64_obj)
     file(COPY ${SOURCES} DESTINATION ${CMAKE_CURRENT_BINARY_DIR}/${LAPACKLIB}_64_obj)
-    file(GLOB SOURCES_64 ${CMAKE_CURRENT_BINARY_DIR}/${LAPACKLIB}_64_obj/*.*)
+    file(GLOB SOURCES_64 RELATIVE ${CMAKE_CURRENT_BINARY_DIR} ${CMAKE_CURRENT_BINARY_DIR}/${LAPACKLIB}_64_obj/*.*)
     list(REMOVE_ITEM SOURCES_64 la_xisnan.F90)
     foreach(F IN LISTS SOURCES_64)
       set(FFILE "")
-      file(READ ${F} FFILE)
-      file(WRITE ${F} "#include \"lapack_64.h\"\n")
-      file(APPEND ${F} "${FFILE}")
+      file(READ ${CMAKE_CURRENT_BINARY_DIR}/${F} FFILE)
+      file(WRITE ${CMAKE_CURRENT_BINARY_DIR}/${F} "#include \"lapack_64.h\"\n")
+      file(APPEND ${CMAKE_CURRENT_BINARY_DIR}/${F} "${FFILE}")
     endforeach()
     file(COPY lapack_64.h DESTINATION ${CMAKE_CURRENT_BINARY_DIR}/${LAPACKLIB}_64_obj)
     add_library(${LAPACKLIB}_64_obj OBJECT ${SOURCES_64})
diff --git a/TESTING/MATGEN/CMakeLists.txt b/TESTING/MATGEN/CMakeLists.txt
index 02e05a86d..97d502c57 100644
--- a/TESTING/MATGEN/CMakeLists.txt
+++ b/TESTING/MATGEN/CMakeLists.txt
@@ -65,12 +65,12 @@ if(BUILD_INDEX64_EXT_API)
     set(SOURCES_64)
     file(MAKE_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/${TMGLIB}_64_obj)
     file(COPY ${SOURCES} DESTINATION ${CMAKE_CURRENT_BINARY_DIR}/${TMGLIB}_64_obj)
-    file(GLOB SOURCES_64 ${CMAKE_CURRENT_BINARY_DIR}/${TMGLIB}_64_obj/*.*)
+    file(GLOB SOURCES_64 RELATIVE ${CMAKE_CURRENT_BINARY_DIR} ${CMAKE_CURRENT_BINARY_DIR}/${TMGLIB}_64_obj/*.*)
     foreach(F IN LISTS SOURCES_64)
       set(FFILE "")
-      file(READ ${F} FFILE)
-      file(WRITE ${F} "#include \"matgen_64.h\"\n")
-      file(APPEND ${F} "${FFILE}")
+      file(READ ${CMAKE_CURRENT_BINARY_DIR}/${F} FFILE)
+      file(WRITE ${CMAKE_CURRENT_BINARY_DIR}/${F} "#include \"matgen_64.h\"\n")
+      file(APPEND ${CMAKE_CURRENT_BINARY_DIR}/${F} "${FFILE}")
     endforeach()
     file(COPY matgen_64.h DESTINATION ${CMAKE_CURRENT_BINARY_DIR}/${TMGLIB}_64_obj)
     add_library(${TMGLIB}_64_obj OBJECT ${SOURCES_64})

but unfortunately that's not enough, because CMake will then still convert them to absolute paths as described https://gitlab.kitware.com/cmake/community/-/wikis/FAQ#why-does-cmake-use-full-paths-or-can-i-copy-my-build-tree and it seems to happen with Unix Makefiles as well as Ninja generator (https://gitlab.kitware.com/cmake/cmake/-/issues/13894).

I haven't found any way for gfortran not to embed the source file in .rodata (I've tried -fcanon-prefix-map -fdebug-prefix-map -ffile-prefix-map -fno-working-directory -P and whole DEBUG_PREFIX_MAP from Yocto which causes warnings about -fmacro-prefix-map not supported by fortran). As well as -gno-record-gcc-switches from https://gcc.gnu.org/pipermail/fortran/2024-October/061206.html.

martin-frbg · 2025-01-21T18:12:59Z

Can anyone please clarify if this is a problem with/for Yocto that did not exist with previous releases of LAPACK (compiled with the same version of gfortran) ? To me this looks as if it is just gfortran (or binutils, linker,...) producing a spurious comment about the original location of the format string in the rodata segment.

shr-project · 2025-01-21T18:24:37Z

I can confirm the same behavior with "hello world" fortran built with gfortran from Gentoo.

$ cat hello.f90 
PROGRAM TEST_HAVE_PRINT
  PRINT *, 'Hello'
END
$ gfortran --version
GNU Fortran (Gentoo 14.2.1_p20241221 p6) 14.2.1 20241221
Copyright (C) 2024 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
$ gfortran hello.f90 -o hello
$ objdump -s -j .rodata hello

hello:     file format elf64-x86-64

Contents of section .rodata:
 2000 01000200 00000000 00000000 00000000  ................
 2010 68656c6c 6f2e6639 30004865 6c6c6f00  hello.f90.Hello.
 2020 44280000 ff3f0000 00000000 01000000  D(...?..........
 2030 01000000 00000000 1f000000           ............
$ cp hello.f90 /tmp/
$ gfortran /tmp/hello.f90 -o hello
$ objdump -s -j .rodata hello

hello:     file format elf64-x86-64

Contents of section .rodata:
 2000 01000200 00000000 00000000 00000000  ................
 2010 2f746d70 2f68656c 6c6f2e66 39300048  /tmp/hello.f90.H
 2020 656c6c6f 00000000 00000000 00000000  ello............
 2030 44280000 ff3f0000 00000000 01000000  D(...?..........
 2040 01000000 00000000 1f000000           ............

The only Yocto related part is that Yocto enables reproducible builds by default and has QE check which errors out when it finds path to build work directory inside built packages (normally stripped by -fdebug-prefix-map and similar flags, which unfortunately doesn't work for gfortran).

https://git.openembedded.org/openembedded-core/commit/meta-selftest/recipes-test/fortran/fortran-helloworld.bb?id=8f1cfaf2be834217cd1cf5be98f44270c36cc31a
solves the same issue by using relative path (so that the the QE check isn't triggered by path to work directory - but in this case it was simple as FC was called directly and CMake wasn't involved)

surainbow added the Type: Bug label Dec 23, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Host Path Included in Build Output Causes Yocto Build Failure #1087

Host Path Included in Build Output Causes Yocto Build Failure #1087

surainbow commented Dec 23, 2024

martin-frbg commented Dec 23, 2024

surainbow commented Dec 24, 2024 •

edited

Loading

shr-project commented Jan 21, 2025 •

edited

Loading

martin-frbg commented Jan 21, 2025

shr-project commented Jan 21, 2025 •

edited

Loading

Host Path Included in Build Output Causes Yocto Build Failure #1087

Host Path Included in Build Output Causes Yocto Build Failure #1087

Comments

surainbow commented Dec 23, 2024

martin-frbg commented Dec 23, 2024

surainbow commented Dec 24, 2024 • edited Loading

shr-project commented Jan 21, 2025 • edited Loading

martin-frbg commented Jan 21, 2025

shr-project commented Jan 21, 2025 • edited Loading

surainbow commented Dec 24, 2024 •

edited

Loading

shr-project commented Jan 21, 2025 •

edited

Loading

shr-project commented Jan 21, 2025 •

edited

Loading