Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kokkos: Don't let Kokkos set CMAKE_CXX_FLAGS #12572

Closed
wants to merge 3 commits into from

Conversation

masterleinad
Copy link
Contributor

@trilinos/kokkos

Motivation

Kokkos is currently setting CMAKE_CXX_FLAGS for Trilinos which is problematic.

  • the errors in Trilinos SYCL build failing: llvm-foreach: Error: Device name missing.  #12420 demonstrate that this breaks linking for anything that doesn't also use target_link_libraries(target Kokkos::kokkos) (or an alias for Kokkos::kokkos) since the compile option fsycl-targets=spir64_gen can only be used if -Xsycl-target-backend is used as linker flag.
  • CMAKE_CXX_FLAGS forces compile options on all targets that don't need it overriding possibly more suitable flags.

The lines modified here were last touched in kokkos/kokkos#2660 suggesting that Trilinos was relying on CMAKE_CXX_FLAGS being set by Kokkos at least at some point. I assume this has become obsolete by using target_link_libraries more consistently. @bartlettroscoe does that match your expectations?
Note that the changes here are also included in #11779 which makes Kokkos a TriBits-compatible raw CMake project snapshotted in Trilinos.

Related Issues

Testing

Compiling Trilinos with Kokkos+SYCL.

@masterleinad masterleinad requested a review from a team as a code owner December 1, 2023 15:54
@trilinos-autotester
Copy link
Contributor

Status Flag 'Pre-Test Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging
NO INSPECTION HAS BEEN PERFORMED ON THIS PULL REQUEST! - This PR must be inspected by setting label 'AT: PRE-TEST INSPECTED'.

@lucbv lucbv added the AT: PRE-TEST INSPECTED Required to test outside contributions. This label alone will not allow a PR to merge. label Dec 1, 2023
@lucbv lucbv added type: bug The primary issue is a bug in Trilinos code or tests pkg: Kokkos labels Dec 1, 2023
lucbv
lucbv previously approved these changes Dec 1, 2023
@trilinos-autotester trilinos-autotester removed the AT: PRE-TEST INSPECTED Required to test outside contributions. This label alone will not allow a PR to merge. label Dec 1, 2023
@trilinos-autotester
Copy link
Contributor

Status Flag 'Pre-Test Inspection' - SUCCESS: The last commit to this Pull Request has been INSPECTED by label AT: PRE-TEST INSPECTED! Autotester is Removing Label; this inspection will remain valid until a new commit to source branch is performed.

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:

Pull Request Auto Testing STARTING (click to expand)

Build Information

Test Name: Trilinos_PR_gcc-8.3.0

  • Build Num: 3382
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-8.3.0-openmpi-1.10.1-openmp_release-debug_static_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS type: bug;pkg: Kokkos
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA ff26745
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA f5bd08c

Build Information

Test Name: Trilinos_PR_gcc-8.3.0-serial

  • Build Num: 1881
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-v2-gnu-8.3.0-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS type: bug;pkg: Kokkos
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA ff26745
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA f5bd08c

Build Information

Test Name: Trilinos_PR_gcc-8.3.0-debug

  • Build Num: 1871
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-8.3.0-openmpi-1.10.1-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS type: bug;pkg: Kokkos
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA ff26745
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA f5bd08c

Build Information

Test Name: Trilinos_PR_clang-11.0.1

  • Build Num: 1870
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-clang-11.0.1-openmpi-1.10.1-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS type: bug;pkg: Kokkos
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA ff26745
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA f5bd08c

Build Information

Test Name: Trilinos_PR_python3

  • Build Num: 3069
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-7.2.0-anaconda3-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_pr-framework
PR_LABELS type: bug;pkg: Kokkos
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL ascic
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA ff26745
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA f5bd08c

Build Information

Test Name: Trilinos_PR_cuda-11.4.2-uvm-off

  • Build Num: 2873
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-cuda-11.4.2-sems-gnu-10.1.0-sems-openmpi-4.0.5_release_static_Volta70_no-asan_complex_no-fpic_mpi_pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS type: bug;pkg: Kokkos
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL GPU
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA ff26745
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA f5bd08c

Build Information

Test Name: Trilinos_PR_intel-2021.3

  • Build Num: 1512
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-intel-2021.3-sems-openmpi-4.0.5_release-debug_shared_no-kokkos-arch_no-asan_no-complex_fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-off_no-package-enables
PR_LABELS type: bug;pkg: Kokkos
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA ff26745
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA f5bd08c

Using Repos:

Repo: TRILINOS (masterleinad/Trilinos)
  • Branch: more_sycl_fixes
  • SHA: ff26745
  • Mode: TEST_REPO

Pull Request Author: masterleinad

Copy link
Member

@bartlettroscoe bartlettroscoe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@maartenarnst,

I agree, we should not be expecting CMAKE_CXX_FLAGS to be passed along to downstream CMake customers who pull in Kokkos (or any Trilinos package) with find_package(<upstreamPkg> ...). Instead, critical compiler options should be propagated using the INTEFACE_COMPILE_OPTIONS target property using target_link_libraries(<dowsntreamTarget> <INTERFACE|PUBLIC|PRIVIATE> <upstreamTarget>).

But then why not also remove setting CMAKE_CXX_FLAGS as a local var which will hide CMAKE_CXX_FLAGS set as a cache var and not allow the user to tweak any of these flags? I think this should be addressed before merging this PR (see below).

# In ProjectCompilerPostConfig.cmake, we capture the "global" flags Trilinos wants in
# TRILINOS_TOPLEVEL_CXX_FLAGS
SET(CMAKE_CXX_FLAGS "${TRILINOS_TOPLEVEL_CXX_FLAGS} ${KOKKOSCORE_CXX_FLAGS}" PARENT_SCOPE)
SET(CMAKE_CXX_FLAGS "${TRILINOS_TOPLEVEL_CXX_FLAGS} ${KOKKOSCORE_CXX_FLAGS}")
Copy link
Member

@bartlettroscoe bartlettroscoe Dec 1, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is CMAKE_CXX_FLAGS still being set as a local var? This line should either be removed or should be changed to:

SET(CMAKE_CXX_FLAGS "${TRILINOS_TOPLEVEL_CXX_FLAGS} ${KOKKOSCORE_CXX_FLAGS} ${CMAKE_CXX_FLAGS}") 

so that it will allow the user to override any of these flags (and therefore negate them if needed) or:

SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${TRILINOS_TOPLEVEL_CXX_FLAGS} ${KOKKOSCORE_CXX_FLAGS} ") 

so that the user's options don't override these Kokkos-set options.

Simply setting CMAKE_CXX_FLAGS as a local variable (and hiding the CMake cache var of the same name) is a big CMake no-no (see section 16.6.3. "Prefer Appending Over Replacing Flags" in the book "Professional CMake: 16th Edition").

But if Kokkos is setting critical flags through target_compile_options(Kokkos::<kokkoslib> ...), then why bother touching the CMAKE_CXX_FLAGS var at all? Just set those compiler options in the flags passed to target_compile_options(Kokkos::<kokkoslib> ...) and then let the user add additional options in the cache var CMAKE_CXX_FLAGS.

NOTE: Removing or changing that line of code could technically break backward compatibility but it seems unlikely that anyone will get bit by this (or problems will be rare and should be easy to recover from).

NOTE: There is a long discussion of these issues with Brad King (of which you were @mentioned) in:

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code path is only used for Trilinos anyway and I tried to be conservative. Hopefully, we can move forward with kokkos/kokkos#6164 soon and clean up the interaction between Trilinos and Kokkos some more.

I'm happy to give it a shot to remove the whole block and all related code already.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@masterleinad, if you want to defer final cleanup, then that is fine and we can go ahead and merge this. But given problems with the PR builds (see #12571), there is no hurry. I will go ahead and remove my "request changes" and you can decide what you want to do with this.

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Error: Jenkins Jobs - A user has pushed a change to the PR before testing completed. NEW EVENT 'committed', ID C_kwDOAsJyMdoAKDdmNTAxMTRhZTNhODNhYWU1MTUzMmQ2YzY4N2E3YjViNDlmMTM1ZDI... The Jenkins Jobs will be shutdown; Testing of this PR must occur again.

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Jenkins Testing: 1 or more Jobs FAILED

Note: Testing will normally be attempted again in approx. 2 Hrs 30 Mins. If a change to the PR source branch occurs, the testing will be attempted again on next available autotester run.

Pull Request Auto Testing has FAILED (click to expand)

Build Information

Test Name: Trilinos_PR_gcc-8.3.0

  • Build Num: 3382
  • Status: ERROR

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-8.3.0-openmpi-1.10.1-openmp_release-debug_static_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS type: bug;pkg: Kokkos
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA ff26745
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA f5bd08c

Build Information

Test Name: Trilinos_PR_gcc-8.3.0-serial

  • Build Num: 1881
  • Status: ERROR

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-v2-gnu-8.3.0-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS type: bug;pkg: Kokkos
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA ff26745
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA f5bd08c

Build Information

Test Name: Trilinos_PR_gcc-8.3.0-debug

  • Build Num: 1871
  • Status: ERROR

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-8.3.0-openmpi-1.10.1-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS type: bug;pkg: Kokkos
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA ff26745
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA f5bd08c

Build Information

Test Name: Trilinos_PR_clang-11.0.1

  • Build Num: 1870
  • Status: ERROR

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-clang-11.0.1-openmpi-1.10.1-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS type: bug;pkg: Kokkos
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA ff26745
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA f5bd08c

Build Information

Test Name: Trilinos_PR_python3

  • Build Num: 3069
  • Status: ERROR

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-7.2.0-anaconda3-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_pr-framework
PR_LABELS type: bug;pkg: Kokkos
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL ascic
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA ff26745
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA f5bd08c

Build Information

Test Name: Trilinos_PR_cuda-11.4.2-uvm-off

  • Build Num: 2873
  • Status: ERROR

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-cuda-11.4.2-sems-gnu-10.1.0-sems-openmpi-4.0.5_release_static_Volta70_no-asan_complex_no-fpic_mpi_pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS type: bug;pkg: Kokkos
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL GPU
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA ff26745
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA f5bd08c

Build Information

Test Name: Trilinos_PR_intel-2021.3

  • Build Num: 1512
  • Status: ERROR

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-intel-2021.3-sems-openmpi-4.0.5_release-debug_shared_no-kokkos-arch_no-asan_no-complex_fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-off_no-package-enables
PR_LABELS type: bug;pkg: Kokkos
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA ff26745
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA f5bd08c


CDash Test Results for PR# 12572.


Wiki: How to Reproduce PR Testing Builds and Errors.

Copy link
Member

@bartlettroscoe bartlettroscoe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This cleanup is not critical and since there is a more significant refactoring of the Kokkos build under Trilinos coming (see kokkos/kokkos#6164), I will defer to others if to address the last CMAKE_CXX_FLAGS issue in this PR or not.

@bartlettroscoe bartlettroscoe dismissed their stale review December 1, 2023 20:11

Let others decide, see above.

@bartlettroscoe
Copy link
Member

FYI: Don't bother putting on AT: RETEST until:

is resolved.

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pre-Test Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging
NO INSPECTION HAS BEEN PERFORMED ON THIS PULL REQUEST! - This PR must be inspected by setting label 'AT: PRE-TEST INSPECTED'.

@masterleinad
Copy link
Contributor Author

Can we retest this and come to a conclusion if we want both commits or only the first one? It seems this is also causing problems for HIP, see kokkos/kokkos#6736.

@bartlettroscoe
Copy link
Member

Can we retest this and come to a conclusion if we want both commits or only the first one? It seems this is also causing problems for HIP, see kokkos/kokkos#6736.

@masterleinad, let's pull the trigger on the suggestions in above in a new commit and then trigger the retest?

@masterleinad
Copy link
Contributor Author

masterleinad commented Jan 23, 2024

@masterleinad, let's pull the trigger on the suggestions in #12572 (review) in a new commit and then trigger the retest?

@bartlettroscoe I pushed a more conservative version than I had before (in https://github.com/masterleinad/Trilinos/pull/new/more_sycl_fixes_backup as backup now) that avoids setting CMAKE_CXX_FLAGS and deletes all related (now unused) variables.

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pre-Test Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging
NO INSPECTION HAS BEEN PERFORMED ON THIS PULL REQUEST! - This PR must be inspected by setting label 'AT: PRE-TEST INSPECTED'.

@ndellingwood
Copy link
Contributor

@masterleinad we'll need these changes applied to the kokkos repo, as well as applied to the release-candidate-4.2.01 branch if this PR makes it into Trilinos before the patch release (to avoid clobber)

bartlettroscoe
bartlettroscoe previously approved these changes Jan 24, 2024
Copy link
Member

@bartlettroscoe bartlettroscoe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Rip it out. Less is more :-)

STRING(REPLACE ";" " " KOKKOSCORE_CUDA_OPTIONS "${KOKKOS_CUDA_OPTIONS}")
FOREACH(CUDAFE_FLAG ${KOKKOS_CUDAFE_OPTIONS})
SET(KOKKOSCORE_CUDAFE_OPTIONS "${KOKKOSCORE_CUDAFE_OPTIONS} -Xcudafe ${CUDAFE_FLAG}")
LIST(APPEND KOKKOS_ALL_COMPILE_OPTIONS -Xcudafe ${CUDAFE_FLAG})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing this block of code seems to have the side effect that the cuda fe flags are no longer passed to Trilinos through KOKKOS_ALL_COMPILE_OPTIONS. May be good to doublecheck if that’s intended.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right. For now, it's better not to modify KOKKOS_ALL_COMPILE_OPTIONS. I pushed a corresponding commit.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK!

Copy link
Member

@bartlettroscoe bartlettroscoe Jan 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The handling of KOKKOS_ALL_COMPILE_OPTIONS is a bit confusing to me so I don't understand the logic here. From what I can tell, these options are written to a configured file KokkosTrilinosConfig.cmake and added to the Kokkos::kokkos target using target_compiler_options(). Then that file gets read in and written to the end of the generated file KokkosConfig.cmake which gets installed. Why not just directly add those compile options to the individual Kokkos libraries using target_compiler_options()? Then those options would be carried along with the direct Kokkos libraries no matter how they are used.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree but KOKKOS_ALL_COMPILE_OPTIONS doesn't cause the issue we are trying to fix here and we wanted to keep the changes here minimal.
Note that all this logic is just for Trilinos. A standalone Kokkos doesn't even go through these lines of code.

With the refactoring in #11779 we should eliminate all the weirdness we had in Kokkos just to make Trilinos work.

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Jenkins Testing: 1 or more Jobs FAILED

Note: Testing will normally be attempted again in approx. 2 Hrs 30 Mins. If a change to the PR source branch occurs, the testing will be attempted again on next available autotester run.

Pull Request Auto Testing has FAILED (click to expand)

Build Information

Test Name: Trilinos_PR_gcc-8.3.0

  • Build Num: 3578
  • Status: FAILED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-8.3.0-openmpi-1.10.1-openmp_release-debug_static_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS type: bug;pkg: Kokkos
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA 2d2078e
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e70daa4

Build Information

Test Name: Trilinos_PR_gcc-8.3.0-serial

  • Build Num: 2077
  • Status: FAILED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-v2-gnu-8.3.0-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS type: bug;pkg: Kokkos
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA 2d2078e
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e70daa4

Build Information

Test Name: Trilinos_PR_gcc-8.3.0-debug

  • Build Num: 2067
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-8.3.0-openmpi-1.10.1-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS type: bug;pkg: Kokkos
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA 2d2078e
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e70daa4

Build Information

Test Name: Trilinos_PR_clang-11.0.1

  • Build Num: 2066
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-clang-11.0.1-openmpi-1.10.1-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS type: bug;pkg: Kokkos
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA 2d2078e
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e70daa4

Build Information

Test Name: Trilinos_PR_python3

  • Build Num: 3247
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-7.2.0-anaconda3-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_pr-framework
PR_LABELS type: bug;pkg: Kokkos
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL ascic
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA 2d2078e
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e70daa4

Build Information

Test Name: Trilinos_PR_cuda-11.4.2-uvm-off

  • Build Num: 3069
  • Status: FAILED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-cuda-11.4.2-sems-gnu-10.1.0-sems-openmpi-4.0.5_release_static_Volta70_no-asan_complex_no-fpic_mpi_pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS type: bug;pkg: Kokkos
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL GPU
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA 2d2078e
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e70daa4

Build Information

Test Name: Trilinos_PR_intel-2021.3

  • Build Num: 1708
  • Status: FAILED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-intel-2021.3-sems-openmpi-4.0.5_release-debug_shared_no-kokkos-arch_no-asan_no-complex_fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-off_no-package-enables
PR_LABELS type: bug;pkg: Kokkos
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA 2d2078e
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e70daa4


CDash Test Results for PR# 12572.


Wiki: How to Reproduce PR Testing Builds and Errors.

@masterleinad
Copy link
Contributor Author

What do the test failures look like?

@bartlettroscoe
Copy link
Member

bartlettroscoe commented Jan 25, 2024

What do the test failures look like?

@masterleinad, the main error was building the object file:

  • packages/kokkos/simd/unit_tests/CMakeFiles/Kokkos_UnitTest_SIMD.dir/TestSIMD.cpp.o

in the build:

  • PR-12572-test-rhel7_sems-v2-gnu-8.3.0-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables-2077

which was:

In file included from ../Trilinos/packages/kokkos/simd[Kokkos_SIMD.hpp:29](https://github.com/trilinos/Trilinos//blob/master/Kokkos_SIMD.hpp#L29),
                 from ../Trilinos/packages/kokkos/simd/unit_tests/include/TestSIMD_MathOps.hpp:20,
                 from ../Trilinos/packages/kokkos/simd/unit_tests/TestSIMD.cpp:17:
../Trilinos/packages/kokkos/simd[Kokkos_SIMD_Scalar.hpp:227](https://github.com/trilinos/Trilinos//blob/master/Kokkos_SIMD_Scalar.hpp#L227):2: error: extra ';' [-Werror=pedantic]
 };
  ^
../Trilinos/packages/kokkos/simd[Kokkos_SIMD_Scalar.hpp:235](https://github.com/trilinos/Trilinos//blob/master/Kokkos_SIMD_Scalar.hpp#L235):2: error: extra ';' [-Werror=pedantic]
 };
  ^
../Trilinos/packages/kokkos/simd[Kokkos_SIMD_Scalar.hpp:243](https://github.com/trilinos/Trilinos//blob/master/Kokkos_SIMD_Scalar.hpp#L243):2: error: extra ';' [-Werror=pedantic]
 };
  ^
../Trilinos/packages/kokkos/simd[Kokkos_SIMD_Scalar.hpp:251](https://github.com/trilinos/Trilinos//blob/master/Kokkos_SIMD_Scalar.hpp#L251):2: error: extra ';' [-Werror=pedantic]
 };
  ^
[CTest: warning matched] cc1plus: all warnings being treated as errors

The test Phalanx_experimental_mdfield_refactor_MPI_1 also segfaulted in the build PR-12572-test-rhel7_sems-intel-2021.3-sems-openmpi-4.0.5_release-debug_shared_no-kokkos-arch_no-asan_no-complex_fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-off_no-package-enables-1708 (ascic114) but I would guess that is unrelated.

@dalg24
Copy link
Contributor

dalg24 commented Jan 25, 2024

These were fixed in kokkos/kokkos#6510 which is not in 4.2.00
@ndellingwood we might want to cherry-pick in 4.2.01

@masterleinad
Copy link
Contributor Author

I'm surprised that these problems only show up in this pull request. Are pull requests tested rebased on top of develop?
Otherwise, I can rebase manually and hope that that fixes it.

@masterleinad
Copy link
Contributor Author

I rebased and cherry-picked kokkos/kokkos#6510 so we can retest. The best solution would be to merge this as part of the upcoming Kokkos patch release anyway.

@bartlettroscoe
Copy link
Member

bartlettroscoe commented Jan 25, 2024

I'm surprised that these problems only show up in this pull request. Are pull requests tested rebased on top of develop? Otherwise, I can rebase manually and hope that that fixes it.

@masterleinad, that is a test file and that only gets built if enabling all of the Trilinos packages or when Kokkos is changed. Most PRs builds would not enable the Kokkos tests because they don't contain changes that could break Kokkos.

This is why I argue that Trilinos needs a post-merge CI build that will catch issues like this.

@ndellingwood ndellingwood added the AT: PRE-TEST INSPECTED Required to test outside contributions. This label alone will not allow a PR to merge. label Jan 25, 2024
@trilinos-autotester trilinos-autotester removed the AT: PRE-TEST INSPECTED Required to test outside contributions. This label alone will not allow a PR to merge. label Jan 26, 2024
@trilinos-autotester
Copy link
Contributor

Status Flag 'Pre-Test Inspection' - SUCCESS: The last commit to this Pull Request has been INSPECTED by label AT: PRE-TEST INSPECTED! Autotester is Removing Label; this inspection will remain valid until a new commit to source branch is performed.

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:

Pull Request Auto Testing STARTING (click to expand)

Build Information

Test Name: Trilinos_PR_gcc-8.3.0

  • Build Num: 3586
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-8.3.0-openmpi-1.10.1-openmp_release-debug_static_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS type: bug;pkg: Kokkos
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA 84fdfd5
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA d835e43

Build Information

Test Name: Trilinos_PR_gcc-8.3.0-serial

  • Build Num: 2085
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-v2-gnu-8.3.0-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS type: bug;pkg: Kokkos
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA 84fdfd5
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA d835e43

Build Information

Test Name: Trilinos_PR_gcc-8.3.0-debug

  • Build Num: 2075
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-8.3.0-openmpi-1.10.1-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS type: bug;pkg: Kokkos
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA 84fdfd5
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA d835e43

Build Information

Test Name: Trilinos_PR_clang-11.0.1

  • Build Num: 2074
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-clang-11.0.1-openmpi-1.10.1-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS type: bug;pkg: Kokkos
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA 84fdfd5
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA d835e43

Build Information

Test Name: Trilinos_PR_python3

  • Build Num: 3255
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-7.2.0-anaconda3-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_pr-framework
PR_LABELS type: bug;pkg: Kokkos
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL ascic
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA 84fdfd5
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA d835e43

Build Information

Test Name: Trilinos_PR_cuda-11.4.2-uvm-off

  • Build Num: 3077
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-cuda-11.4.2-sems-gnu-10.1.0-sems-openmpi-4.0.5_release_static_Volta70_no-asan_complex_no-fpic_mpi_pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS type: bug;pkg: Kokkos
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL GPU
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA 84fdfd5
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA d835e43

Build Information

Test Name: Trilinos_PR_intel-2021.3

  • Build Num: 1716
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-intel-2021.3-sems-openmpi-4.0.5_release-debug_shared_no-kokkos-arch_no-asan_no-complex_fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-off_no-package-enables
PR_LABELS type: bug;pkg: Kokkos
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA 84fdfd5
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA d835e43

Using Repos:

Repo: TRILINOS (masterleinad/Trilinos)
  • Branch: more_sycl_fixes
  • SHA: 84fdfd5
  • Mode: TEST_REPO

Pull Request Author: masterleinad

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Jenkins Testing: 1 or more Jobs FAILED

Note: Testing will normally be attempted again in approx. 2 Hrs 30 Mins. If a change to the PR source branch occurs, the testing will be attempted again on next available autotester run.

Pull Request Auto Testing has FAILED (click to expand)

Build Information

Test Name: Trilinos_PR_gcc-8.3.0

  • Build Num: 3586
  • Status: FAILED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-8.3.0-openmpi-1.10.1-openmp_release-debug_static_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS type: bug;pkg: Kokkos
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA 84fdfd5
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA d835e43

Build Information

Test Name: Trilinos_PR_gcc-8.3.0-serial

  • Build Num: 2085
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-v2-gnu-8.3.0-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS type: bug;pkg: Kokkos
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA 84fdfd5
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA d835e43

Build Information

Test Name: Trilinos_PR_gcc-8.3.0-debug

  • Build Num: 2075
  • Status: FAILED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-8.3.0-openmpi-1.10.1-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS type: bug;pkg: Kokkos
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA 84fdfd5
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA d835e43

Build Information

Test Name: Trilinos_PR_clang-11.0.1

  • Build Num: 2074
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-clang-11.0.1-openmpi-1.10.1-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS type: bug;pkg: Kokkos
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA 84fdfd5
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA d835e43

Build Information

Test Name: Trilinos_PR_python3

  • Build Num: 3255
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-7.2.0-anaconda3-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_pr-framework
PR_LABELS type: bug;pkg: Kokkos
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL ascic
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA 84fdfd5
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA d835e43

Build Information

Test Name: Trilinos_PR_cuda-11.4.2-uvm-off

  • Build Num: 3077
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-cuda-11.4.2-sems-gnu-10.1.0-sems-openmpi-4.0.5_release_static_Volta70_no-asan_complex_no-fpic_mpi_pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS type: bug;pkg: Kokkos
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL GPU
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA 84fdfd5
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA d835e43

Build Information

Test Name: Trilinos_PR_intel-2021.3

  • Build Num: 1716
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-intel-2021.3-sems-openmpi-4.0.5_release-debug_shared_no-kokkos-arch_no-asan_no-complex_fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-off_no-package-enables
PR_LABELS type: bug;pkg: Kokkos
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA 84fdfd5
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA d835e43


CDash Test Results for PR# 12572.


Wiki: How to Reproduce PR Testing Builds and Errors.

@masterleinad
Copy link
Contributor Author

What's the failure this time?

@ndellingwood
Copy link
Contributor

@masterleinad looks like some of the jobs did not complete, related to #12696 and possibly backlog of jobs waiting in the queue when the GPU nodes came back online. Retesting

@ndellingwood ndellingwood added the AT: RETEST Causes the PR autotester to run a new round of PR tests on the next iteration label Jan 26, 2024
@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - User Requested Retest - Label AT: RETEST will be reset after testing.

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:

Pull Request Auto Testing STARTING (click to expand)

Build Information

Test Name: Trilinos_PR_gcc-8.3.0

  • Build Num: 3593
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-8.3.0-openmpi-1.10.1-openmp_release-debug_static_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS type: bug;pkg: Kokkos;AT: RETEST
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA 84fdfd5
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA d835e43

Build Information

Test Name: Trilinos_PR_gcc-8.3.0-serial

  • Build Num: 2092
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-v2-gnu-8.3.0-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS type: bug;pkg: Kokkos;AT: RETEST
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA 84fdfd5
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA d835e43

Build Information

Test Name: Trilinos_PR_gcc-8.3.0-debug

  • Build Num: 2082
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-8.3.0-openmpi-1.10.1-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS type: bug;pkg: Kokkos;AT: RETEST
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA 84fdfd5
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA d835e43

Build Information

Test Name: Trilinos_PR_clang-11.0.1

  • Build Num: 2081
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-clang-11.0.1-openmpi-1.10.1-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS type: bug;pkg: Kokkos;AT: RETEST
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA 84fdfd5
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA d835e43

Build Information

Test Name: Trilinos_PR_python3

  • Build Num: 3261
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-7.2.0-anaconda3-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_pr-framework
PR_LABELS type: bug;pkg: Kokkos;AT: RETEST
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL ascic
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA 84fdfd5
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA d835e43

Build Information

Test Name: Trilinos_PR_cuda-11.4.2-uvm-off

  • Build Num: 3084
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-cuda-11.4.2-sems-gnu-10.1.0-sems-openmpi-4.0.5_release_static_Volta70_no-asan_complex_no-fpic_mpi_pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS type: bug;pkg: Kokkos;AT: RETEST
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL GPU
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA 84fdfd5
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA d835e43

Build Information

Test Name: Trilinos_PR_intel-2021.3

  • Build Num: 1723
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-intel-2021.3-sems-openmpi-4.0.5_release-debug_shared_no-kokkos-arch_no-asan_no-complex_fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-off_no-package-enables
PR_LABELS type: bug;pkg: Kokkos;AT: RETEST
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA 84fdfd5
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA d835e43

Using Repos:

Repo: TRILINOS (masterleinad/Trilinos)
  • Branch: more_sycl_fixes
  • SHA: 84fdfd5
  • Mode: TEST_REPO

Pull Request Author: masterleinad

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Jenkins Testing: 1 or more Jobs FAILED

Note: Testing will normally be attempted again in approx. 2 Hrs 30 Mins. If a change to the PR source branch occurs, the testing will be attempted again on next available autotester run.

Pull Request Auto Testing has FAILED (click to expand)

Build Information

Test Name: Trilinos_PR_gcc-8.3.0

  • Build Num: 3593
  • Status: FAILED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-8.3.0-openmpi-1.10.1-openmp_release-debug_static_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS type: bug;pkg: Kokkos;AT: RETEST
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA 84fdfd5
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA d835e43

Build Information

Test Name: Trilinos_PR_gcc-8.3.0-serial

  • Build Num: 2092
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-v2-gnu-8.3.0-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS type: bug;pkg: Kokkos;AT: RETEST
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA 84fdfd5
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA d835e43

Build Information

Test Name: Trilinos_PR_gcc-8.3.0-debug

  • Build Num: 2082
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-8.3.0-openmpi-1.10.1-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS type: bug;pkg: Kokkos;AT: RETEST
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA 84fdfd5
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA d835e43

Build Information

Test Name: Trilinos_PR_clang-11.0.1

  • Build Num: 2081
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-clang-11.0.1-openmpi-1.10.1-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS type: bug;pkg: Kokkos;AT: RETEST
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA 84fdfd5
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA d835e43

Build Information

Test Name: Trilinos_PR_python3

  • Build Num: 3261
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-7.2.0-anaconda3-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_pr-framework
PR_LABELS type: bug;pkg: Kokkos;AT: RETEST
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL ascic
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA 84fdfd5
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA d835e43

Build Information

Test Name: Trilinos_PR_cuda-11.4.2-uvm-off

  • Build Num: 3084
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-cuda-11.4.2-sems-gnu-10.1.0-sems-openmpi-4.0.5_release_static_Volta70_no-asan_complex_no-fpic_mpi_pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS type: bug;pkg: Kokkos;AT: RETEST
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL GPU
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA 84fdfd5
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA d835e43

Build Information

Test Name: Trilinos_PR_intel-2021.3

  • Build Num: 1723
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-intel-2021.3-sems-openmpi-4.0.5_release-debug_shared_no-kokkos-arch_no-asan_no-complex_fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-off_no-package-enables
PR_LABELS type: bug;pkg: Kokkos;AT: RETEST
PULLREQUESTNUM 12572
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_REPO https://github.com/masterleinad/Trilinos
TRILINOS_SOURCE_SHA 84fdfd5
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA d835e43


CDash Test Results for PR# 12572.


Wiki: How to Reproduce PR Testing Builds and Errors.

@trilinos-autotester trilinos-autotester removed the AT: RETEST Causes the PR autotester to run a new round of PR tests on the next iteration label Jan 28, 2024
@ndellingwood
Copy link
Contributor

Looks like the sems-gnu-8.3.0-openmpi test failed due to running out of space

/usr/bin/ld: final link failed: No space left on device

No failures reported in the other builds

@masterleinad
Copy link
Contributor Author

We merged this one on the Kokkos-side and will pull it in with the patch release, see kokkos/kokkos#6756.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pkg: Kokkos type: bug The primary issue is a bug in Trilinos code or tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants