Linker Errors in Windows (intel toolchain with msvc)

[koutoui]

Hi all,
I've been trying to compile mumps v5.3.3.7 with the intel toolchain on Windows using cmake, but I'm getting linker errors when I'm trying to compile the c_example.c (that's under the example folder). (NOTE: All is done in the intel specific cmd with msvc2019 integration)

I first run:

cmake -B build -Wdev -G"Visual Studio 16 2019" -T"Intel C++ Compiler 19.1" -DCMAKE_INSTALL_PREFIX=C:/Users/koutoui/Documents/mumps/ -Darith=d -Dparallel=false

that outputs:

-- The C compiler identification is Intel 19.1.0.20200306
-- The Fortran compiler identification is Intel 19.1.0.20200306
-- Check for working C compiler: C:/Program Files (x86)/IntelSWTools/compilers_and_libraries_2020/windows/bin/intel64/icl.exe
-- Check for working C compiler: C:/Program Files (x86)/IntelSWTools/compilers_and_libraries_2020/windows/bin/intel64/icl.exe -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working Fortran compiler: C:/Program Files (x86)/IntelSWTools/compilers_and_libraries_2020/windows/bin/intel64/icl.exe
-- Check for working Fortran compiler: C:/Program Files (x86)/IntelSWTools/compilers_and_libraries_2020/windows/bin/intel64/icl.exe  -- works
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Determine Intel Fortran Compiler Implicit Link Path
-- Determine Intel Fortran Compiler Implicit Link Path -- done
-- Checking whether C:/Program Files (x86)/IntelSWTools/compilers_and_libraries_2020/windows/bin/intel64/icl.exe supports Fortran 90
-- Checking whether C:/Program Files (x86)/IntelSWTools/compilers_and_libraries_2020/windows/bin/intel64/icl.exe supports Fortran 90 -- yes
-- CMake 3.16.7
-- Looking for pthread.h
-- Looking for pthread.h - not found
-- Found Threads: TRUE
-- Found LAPACK: C:/Program Files (x86)/IntelSWTools/compilers_and_libraries_2020.1.216/windows/mkl/lib/intel64/mkl_intel_lp64.lib;C:/Program Files (x86)/IntelSWTools/compilers_and_libraries_2020.1.216/windows/mkl/lib/intel64/mkl_sequential.lib;C:/Program Files (x86)/IntelSWTools/compilers_and_libraries_2020.1.216/windows/mkl/lib/intel64/mkl_core.lib  found components: MKL
-- Performing Test LAPACK_real64_link
-- Performing Test LAPACK_real64_link - Success
KOUTOU--- a that was given is: d
-- Visual Studio programs linking Fortran and C code require special configuration, see:
-- https://software.intel.com/en-us/articles/configuring-visual-studio-for-mixed-language-applications
-- Consider using Ninja backend instead of VS backend if possible for your project.
-- Configuring done
-- Generating done
-- Build files have been written to: C:/Users/koutoui/Documents/repos/mumps.git_2/tags/v5.3.3.7/build

which I then build like that:

cmake --build build --clean-first

which outputs:

Microsoft (R) Build Engine version 16.6.0+5ff7b0c9e for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.

Microsoft (R) Build Engine version 16.6.0+5ff7b0c9e for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.

  Checking Build System
  Building Custom Rule C:/Users/koutoui/Documents/repos/mumps.git_2/tags/v5.3.3.7/libseq/CMakeLists.txt
icl: : warning #10159: invalid argument for option '/fp' [C:\Users\koutoui\Documents\repos\mumps.git_2\tags\v5.3.3.7\build\libseq\mpiseq.vcxproj]
icl: : warning #10006: ignoring unknown option '/libs:dll' [C:\Users\koutoui\Documents\repos\mumps.git_2\tags\v5.3.3.7\build\libseq\mpiseq.vcxproj]
icl: : warning #10006: ignoring unknown option '/threads' [C:\Users\koutoui\Documents\repos\mumps.git_2\tags\v5.3.3.7\build\libseq\mpiseq.vcxproj]
icl: : warning #10006: ignoring unknown option '/warn:declarations' [C:\Users\koutoui\Documents\repos\mumps.git_2\tags\v5.3.3.7\build\libseq\mpiseq.vcxproj]
icl: : warning #10006: ignoring unknown option '/heap-arrays' [C:\Users\koutoui\Documents\repos\mumps.git_2\tags\v5.3.3.7\build\libseq\mpiseq.vcxproj]
icl: : warning #10006: ignoring unknown option '/dbglibs' [C:\Users\koutoui\Documents\repos\mumps.git_2\tags\v5.3.3.7\build\libseq\mpiseq.vcxproj]
  elapse.c
  mpic.c
  mpiseq.vcxproj -> C:\Users\koutoui\Documents\repos\mumps.git_2\tags\v5.3.3.7\build\libseq\Debug\mpiseq.lib
  Building Custom Rule C:/Users/koutoui/Documents/repos/mumps.git_2/tags/v5.3.3.7/PORD/CMakeLists.txt
  graph.c
  gbipart.c
  gbisect.c
  ddcreate.c
  ddbisect.c
  nestdiss.c
  multisector.c
  gelim.c
  bucket.c
  tree.c
  symbfac.c
  interface.c
  sort.c
  minpriority.c
  pord.vcxproj -> C:\Users\koutoui\Documents\repos\mumps.git_2\tags\v5.3.3.7\build\PORD\Debug\pord.lib
  Building Custom Rule C:/Users/koutoui/Documents/repos/mumps.git_2/tags/v5.3.3.7/src/CMakeLists.txt
icl: : warning #10159: invalid argument for option '/fp' [C:\Users\koutoui\Documents\repos\mumps.git_2\tags\v5.3.3.7\build\src\mumps_common.vcxproj]
icl: : warning #10006: ignoring unknown option '/libs:dll' [C:\Users\koutoui\Documents\repos\mumps.git_2\tags\v5.3.3.7\build\src\mumps_common.vcxproj]
icl: : warning #10006: ignoring unknown option '/threads' [C:\Users\koutoui\Documents\repos\mumps.git_2\tags\v5.3.3.7\build\src\mumps_common.vcxproj]
icl: : warning #10006: ignoring unknown option '/warn:declarations' [C:\Users\koutoui\Documents\repos\mumps.git_2\tags\v5.3.3.7\build\src\mumps_common.vcxproj]
icl: : warning #10006: ignoring unknown option '/heap-arrays' [C:\Users\koutoui\Documents\repos\mumps.git_2\tags\v5.3.3.7\build\src\mumps_common.vcxproj]
icl: : warning #10006: ignoring unknown option '/dbglibs' [C:\Users\koutoui\Documents\repos\mumps.git_2\tags\v5.3.3.7\build\src\mumps_common.vcxproj]
  mumps_pord.c
  mumps_common.c
  mumps_size.c
  mumps_io.c
  mumps_io_basic.c
  mumps_io_thread.c
  mumps_io_err.c
  mumps_numa.c
  mumps_thread.c
  mumps_save_restore_C.c
  mumps_config_file_C.c
  mumps_thread_affinity.c
  mumps_common.vcxproj -> C:\Users\koutoui\Documents\repos\mumps.git_2\tags\v5.3.3.7\build\src\Debug\mumps_common.lib
  Building Custom Rule C:/Users/koutoui/Documents/repos/mumps.git_2/tags/v5.3.3.7/src/CMakeLists.txt
icl: : warning #10159: invalid argument for option '/fp' [C:\Users\koutoui\Documents\repos\mumps.git_2\tags\v5.3.3.7\build\src\dmumps.vcxproj]
icl: : warning #10006: ignoring unknown option '/libs:dll' [C:\Users\koutoui\Documents\repos\mumps.git_2\tags\v5.3.3.7\build\src\dmumps.vcxproj]
icl: : warning #10006: ignoring unknown option '/threads' [C:\Users\koutoui\Documents\repos\mumps.git_2\tags\v5.3.3.7\build\src\dmumps.vcxproj]
icl: : warning #10006: ignoring unknown option '/warn:declarations' [C:\Users\koutoui\Documents\repos\mumps.git_2\tags\v5.3.3.7\build\src\dmumps.vcxproj]
icl: : warning #10006: ignoring unknown option '/heap-arrays' [C:\Users\koutoui\Documents\repos\mumps.git_2\tags\v5.3.3.7\build\src\dmumps.vcxproj]
icl: : warning #10006: ignoring unknown option '/dbglibs' [C:\Users\koutoui\Documents\repos\mumps.git_2\tags\v5.3.3.7\build\src\dmumps.vcxproj]
  mumps_c.c
  dmumps_gpu.c
  dmumps.vcxproj -> C:\Users\koutoui\Documents\repos\mumps.git_2\tags\v5.3.3.7\build\src\Debug\dmumps.lib
  Building Custom Rule C:/Users/koutoui/Documents/repos/mumps.git_2/tags/v5.3.3.7/CMakeLists.txt

And then I install it:

cmake --build build --target install

which outputs:

Microsoft (R) Build Engine version 16.6.0+5ff7b0c9e for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.

  mpiseq.vcxproj -> C:\Users\koutoui\Documents\repos\mumps.git_2\tags\v5.3.3.7\build\libseq\Debug\mpiseq.lib
  pord.vcxproj -> C:\Users\koutoui\Documents\repos\mumps.git_2\tags\v5.3.3.7\build\PORD\Debug\pord.lib
  mumps_common.vcxproj -> C:\Users\koutoui\Documents\repos\mumps.git_2\tags\v5.3.3.7\build\src\Debug\mumps_common.lib
  dmumps.vcxproj -> C:\Users\koutoui\Documents\repos\mumps.git_2\tags\v5.3.3.7\build\src\Debug\dmumps.lib
  -- Install configuration: "Debug"
  -- Installing: C:/Users/koutoui/Documents/mumps/lib/mpiseq.lib
  -- Installing: C:/Users/koutoui/Documents/mumps/include/elapse.h
  -- Installing: C:/Users/koutoui/Documents/mumps/include/mpi.h
  -- Installing: C:/Users/koutoui/Documents/mumps/include/mpif.h
  -- Installing: C:/Users/koutoui/Documents/mumps/lib/pord.lib
  -- Installing: C:/Users/koutoui/Documents/mumps/lib/mumps_common.lib
  -- Installing: C:/Users/koutoui/Documents/mumps/lib/dmumps.lib
  -- Installing: C:/Users/koutoui/Documents/mumps/include/dmumps_c.h
  -- Installing: C:/Users/koutoui/Documents/mumps/include/dmumps_root.h
  -- Installing: C:/Users/koutoui/Documents/mumps/include/dmumps_struc.h
  -- Installing: C:/Users/koutoui/Documents/mumps/include/mumps_int_def.h
  -- Installing: C:/Users/koutoui/Documents/mumps/include/mumps_c_types.h
  -- Installing: C:/Users/koutoui/Documents/mumps/include/mumps_compat.h

But, when I try to compile the example (which I've changed a bit).

mpiicc -o ms mumps_test_src.cpp /I"C:/Users/koutoui/Documents/mumps/include" /LIBPATH:"C:/Users/koutoui/Documents/mumps/lib" dmumps.lib mumps_common.lib pord.lib mpiseq.lib

and I get:

mpicc.bat for the Intel(R) MPI Library 2019 Update 7 for Windows*
Copyright 2007-2020 Intel Corporation.

Intel(R) C++ Intel(R) 64 Compiler for applications running on Intel(R) 64, Version 19.1.1.216 Build 20200306
Copyright (C) 1985-2020 Intel Corporation.  All rights reserved.

mumps_test_src.cpp
Microsoft (R) Incremental Linker Version 14.16.27043.0
Copyright (C) Microsoft Corporation.  All rights reserved.

-out:ms.exe
/LIBPATH:C:/Users/koutoui/Documents/mumps/lib
"/LIBPATH:C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2020.1.216\windows\mpi\intel64\bin\..\..\intel64\lib\release"
"/LIBPATH:C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2020.1.216\windows\mpi\intel64\bin\..\..\intel64\lib"
impi.lib
mumps_test_src.obj
dmumps.lib
mumps_common.lib
pord.lib
mpiseq.lib
LINK : warning LNK4098: defaultlib 'MSVCRT' conflicts with use of other libs; use /NODEFAULTLIB:library
dmumps.lib(mumps_c.obj) : warning LNK4217: locally defined symbol strncpy imported in function dmumps_c
dmumps.lib(mumps_c.obj) : error LNK2019: unresolved external symbol DMUMPS_SET_TMP_PTR referenced in function DMUMPS_SET_TMP_PTR_C
dmumps.lib(mumps_c.obj) : error LNK2019: unresolved external symbol DMUMPS_F77 referenced in function dmumps_c
ms.exe : fatal error LNK1120: 2 unresolved externals
ERROR in the compiling/linking [1120]

Which outputs a DMUMPS_SET_TMP_PTR and a DMUMPS_F77 linker error.
Could you point me in the right direction on how to fix that? (I'm not much of a windows expert).
Note: I've opened the visual studio solution to build it from there, since there was a url suggestion on your readme(https://software.intel.com/content/www/us/en/develop/articles/configuring-visual-studio-for-mixed-language-applications.html), but the IFORT_COMPILER/etc variables were already defined

Best Regards,
Ioannis

[scivision]

I would use the Ninja or MinGW Makefiles generator from Intel oneapi command prompt to use Visual Studio.

[dariomangoni]

@koutoui just for reference: the approach here followed mixes C and Fortran source files all at once (e.g. mumps_common).
Visual Studio then considers the target as C/C++ code, but actually the compiler options that are passed are for the Fortran compiler.
In order to overcome this issue I think that you have to compile (where "compile" is used in the strict sense i.e. without linking) C and Fortran file in separate object files, then link the two object files in a single library.
Something like:

ADD_LIBRARY(mumps_common
                        $<TARGET_OBJECTS:MUMPS_COMMON_OBJ_FORTRAN>
                        $<TARGET_OBJECTS:MUMPS_COMMON_OBJ_C>
                        ${MUMPS_COMMON_EXPORTS})

(here the MUMPS_COMMON_EXPORTS are used to allow the symbol exposition that was probably needed because I was compiling shared libraries and I don't know if MUMPS_CALL was enough to export all the symbols needed)
This allowed me to compile Mumps in Windows directly in VS and also as shared library at least for 5.1.2 (the last time I tried) but it is quite stressful operation.

[scivision]

Yes I tried this in "objects" branch but had problems with the cyclical links in Mumps.
I think this is a worthwhile improvement. To do this, I will probably have to start with the original Mumps Makefiles, mimicking their numerous object file creation. Once that works, I can combine into fewer object files perhaps, or just so some foreach() loops.

[dariomangoni]

If it can be of any help this is my full CMakeLists (dated back to Mumps 5.1.1) that I've used for a while to build Mumps for our software.
However, what I did is to merge all the libraries in just one shared library. Probably this overcame the problem, but then you won't have the standard set of files (mumps_common, dmumps, etc will be collapsed in a single libdmumps).
CMakeLists_5.1.1.txt

[scivision]

OK thank you!

[eremite-ever]

This project is too awesome！
but I also encountered this link problem in Windows(intel toochain with msvc), and I am looking forward to a good solution！

dmumps.lib(mumps_c.obj) : error LNK2019: unresolved external symbol DMUMPS_SET_TMP_PTR referenced in function DMUMPS_SET_TMP_PTR_C
dmumps.lib(mumps_c.obj) : error LNK2019: unresolved external symbol DMUMPS_F77 referenced in function dmumps_c

[scivision]

For both #1 and #2, despite recent MUMPS CMake enhancement re: shared libs, on Windows with Intel oneAPI (icl/ifort or icx/ifx) shared libs still don't work. This may be an upstream MUMPS issue. Using static libs (the default) does work.