Merge remote-tracking branch 'origin/develop' into feature/fortran-32…

…index-build
LLNL · Apr 25, 2024 · 79c7f1c · 79c7f1c
2 parents d0b3497 + 1d61bdd
commit 79c7f1c
Show file tree

Hide file tree

Showing 38 changed files with 879 additions and 171 deletions.
diff --git a/.github/workflows/check-clang-format.yml b/.github/workflows/check-clang-format.yml
@@ -12,7 +12,7 @@ jobs:
       image: ghcr.io/llnl/sundials_spack_cache:llvm-17.0.4-h4lflucc3v2vage45opbo2didtcuigsn.spack
     steps:
       - name: Install git
-        run: | 
+        run: |
           apt update
           apt install -y git
 
@@ -26,18 +26,18 @@ jobs:
 
       - name: Print clang-format version
         run: clang-format --version
-      
+
       - name: Run checker on code
         run: |
-          ./scripts/format.sh .
-      
+          ./scripts/format.sh benchmarks examples include src test
+
       - name: Run git diff to see if anything changed
         run: /usr/bin/git diff --name-only --exit-code
-      
+
       - name: Run git diff if we failed
         if: failure()
         run:  /usr/bin/git diff > diff
-      
+
       - name: Archive diff if we failed
         uses: actions/upload-artifact@v3
         if: failure()

diff --git a/.gitmodules b/.gitmodules
@@ -7,3 +7,6 @@
 [submodule "test/answers"]
 	path = test/answers
 	url = https://github.com/sundials-codes/answers.git
+[submodule "external/sundials-addon-example"]
+	path = external/sundials-addon-example
+	url = https://github.com/sundials-codes/sundials-addon-example.git
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -19,6 +19,15 @@ with the Cray Fortran compiler.
 
 Enabled the Fortran interfaces to build with 32-bit `sunindextype`.
 
+Fixed a bug where `MRIStepEvolve` would not handle a recoverable error produced
+from evolving the inner stepper.
+
+Added CMake infrastructure that enables externally maintained addons/plugins
+to be *optionally* built with SUNDIALS. See the [Contributing Guide](./CONTRIBUTING.md)
+for more details.
+
+Added support for Kokkos Kernels v4.
+
 ## Changes to SUNDIALS in release v7.0.0
 
 ### Major Feature

diff --git a/CMakeLists.txt b/CMakeLists.txt
@@ -221,6 +221,10 @@ if(SUNDIALS_TEST_UNITTESTS)
   add_subdirectory(test/unit_tests)
 endif()
 
+if(SUNDIALS_ENABLE_EXTERNAL_ADDONS)
+  add_subdirectory(external)
+endif()
+
 # ===============================================================
 # Install configuration header files and license file.
 # ===============================================================

diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -1,9 +1,16 @@
 # Contributing to SUNDIALS
 
-At this time, the SUNDIALS team does not have the resources to review and take
-in large additions to the code or significant new features. Contributions
-addressing bug fixes or minor changes are preferred via a pull request to the
-[SUNDIALS GitHub repository](https://github.com/LLNL/sundials).
+There are two primary ways of contributing to SUNDIALS. The first way is by particpating
+in the development of SUNDIALS directly through contributions of code to the primary
+[SUNDIALS repository](https://github.com/LLNL/sundials). This is the best way to contribute
+bug fixes and minor improvements. At this time, the SUNDIALS team does not have the resources
+to review and take in large additions to the code or significant new features.
+Larger additions can be contributed as a SUNDIALS "addon" which is a component that may be
+optionally downloaded by users and then compiled and installed with SUNDIALS.
+
+## Direct Contributions via Pull Requests
+
+Direct contributions to SUNDIALS are made by opening a Pull Request.
 
 All new contributions to SUNDIALS must be made under the BSD 3-clause license.
 See the [LICENSE](./LICENSE) and [NOTICE](./NOTICE) files for details. The
@@ -62,12 +69,11 @@ By making a contribution to this project, I certify that:
     this project or the open source license(s) involved.
 ```
 
-As discussed in the [Docker software project blog](https://blog.docker.com/2014/01/docker-code-contributions-require-developer-certificate-of-origin/)
-this DCO "lets us know that you are entitled to contribute this code to
-[SUNDIALS] and that you are willing to have it used in distributions and
-derivative works."
+The DCO lets us know that you are entitled to contribute this code to
+SUNDIALS and that you are willing to have it used in distributions and
+derivative works.
 
-"By including the DCO signature, you are stating that one or
+By including the DCO signature, you are stating that one or
 more of the following is true of your contribution:
 
 1.  You created this contribution/change and have the right to submit it
@@ -77,7 +83,7 @@ more of the following is true of your contribution:
 3.  This contribution/change has been provided to you by someone who did
     1 or 2 and you are submitting the contribution unchanged.
 4.  You understand this contribution is public and may be redistributed as
-    open source software" under the BSD license.
+    open source software under the BSD license.
 
 All commits submitted to the SUNDIALS project need to have the following sign
 off line in the commit message:
@@ -89,3 +95,31 @@ Replacing Jane Doe’s details with your name and email address.
 If you've set `user.name` and `user.email` in your Git configuration, you can
 automatically add a sign off line at the end of the commit message by using the
 `-s` option (e.g., `git commit -s`).
+
+## Contributions via SUNDIALS Addons
+
+SUNDIALS "addons" are community developed code additions for SUNDIALS that can be subsumed by the
+SUNDIALS build system so that they have full access to all internal SUNDIALS symbols.
+The intent is for SUNDIALS addons to function as if they are part of the SUNDIALS library,
+while allowing them to potentially have different licenses
+(although we encourage BSD-3-Clause still), code style
+(although we encourage them to follow the SUNDIALS style outlined :ref:`here <Style>`),
+and they **are not maintained by the SUNDIALS team**.
+
+### Creating an addon
+
+To create a SUNDIALS addon and use it there are a few things you need to do:
+
+1. In your addon project, ensure that you have a `CMakeLists.txt` that uses the
+   `sundials_add_library` CMake macro to create the library target. The best thing to do is simply
+   copy from, or refer to, a `CMakeLists.txt` in the SUNDIALS `src` directory.
+2. Follow the steps in the `README.md` file in the `external/` directory in the root of the SUNDIALS
+   source code.
+
+An example addon is available [here](https://github.com/sundials-codes/sundials-addon-example).
+
+### Friends of SUNDIALS
+
+The SUNDIALS team maintains a list of some SUNDIALS addons in our [Friends of SUNDIALS](https://github.com/sundials-codes/friends-of-sundials>) repository. These addons are not
+maintained by the SUNDIALS team, but have been developed in consultation with us.
+**Currently we are only adding projects for existing collaborations**. Please contact the development team if you are interested in collaborating on an addon.
diff --git a/cmake/SundialsBuildOptionsPre.cmake b/cmake/SundialsBuildOptionsPre.cmake
@@ -267,6 +267,16 @@ if(SUNDIALS_DEBUG_PRINTVEC AND SUNDIALS_LOGGING_LEVEL LESS 5)
   set(SUNDIALS_LOGGING_LEVEL "5" CACHE STRING "${DOCSTR}" FORCE)
 endif()
 
+# ---------------------------------------------------------------
+# Options for SUNDIALS external
+# ---------------------------------------------------------------
+
+sundials_option(SUNDIALS_ENABLE_EXTERNAL_ADDONS BOOL
+  "Enables including EXTERNALLY MAINTAINED addons in the SUNDIALS build." OFF)
+if(SUNDIALS_ENABLE_EXTERNAL_ADDONS)
+  message(WARNING "SUNDIALS_ENABLE_EXTERNAL_ADDONS=TRUE. External addons are not maintained by the SUNDIALS team. Use at your own risk.")
+endif()
+
 # ---------------------------------------------------------------
 # Options for SUNDIALS testing
 # ---------------------------------------------------------------

diff --git a/doc/README.md b/doc/README.md
@@ -1,10 +1,35 @@
 # SUNDIALS Documentation
 
 The SUNDIALS documentation is written using reStructuredText and
-[Sphinx](https://www.sphinx-doc.org/).
+[Sphinx](https://www.sphinx-doc.org/). We host the generated HTML documentation
+at https://sundials.readthedocs.io.
 
 To build the documentation with Sphinx you will need Python 3.9+. Sphinx and the
-necessary extensions can be installed using the requirements file i.e.,
-`pip install -r requirements.txt`. Additionally, building the developer
+necessary extensions can be installed using the requirements file i.e., `pip
+install -r requirements.txt`. Additionally, building the developer
 documentation requires [Graphviz](https://graphviz.org/) for generating
 flowcharts.
+
+Once you have the dependencies installed, you can choose what you want to build.
+To build the so-called "superbuild" of HTML docs that includes everything
+(like our readthedocs) then do
+
+```bash
+cd superbuild
+make -j4
+```
+
+If you want to build the docs separately for each of the individual SUNDIALS packages,
+then you should run
+
+```bash
+make -j4 html|latexpdf # build HTML docs or PDF (using latex)
+```
+
+Finally, if you just want to build the docs for one SUNDIALS package, then you can
+run
+
+```bash
+cd <package>
+make -j4 html|latexpdf # build HTML docs or PDF (using latex)
+```
diff --git a/doc/arkode/guide/source/Butcher.rst b/doc/arkode/guide/source/Butcher.rst
@@ -14,9 +14,9 @@
 
 .. _Butcher:
 
-=========================
-Appendix: Butcher tables
-=========================
+==============
+Butcher Tables
+==============
 
 Here we catalog the full set of Butcher tables included in ARKODE. We group
 these into four categories: *explicit*, *implicit*, *additive* and

diff --git a/doc/arkode/guide/source/Constants.rst b/doc/arkode/guide/source/Constants.rst
@@ -14,9 +14,9 @@
 
 .. _ARKODE.Constants:
 
-===========================
-Appendix: ARKODE Constants
-===========================
+================
+ARKODE Constants
+================
 
 Below we list all input and output constants used by the main solver,
 timestepper, and linear solver modules, together with a short

diff --git a/doc/arkode/guide/source/sundials/index.rst b/doc/arkode/guide/source/sundials/index.rst
@@ -12,9 +12,9 @@
 
 .. _SUNDIALS:
 
-**************
-Using SUNDIALS
-**************
+***************
+Getting Started
+***************
 
 The packages that make up SUNDIALS are built upon shared classes for vectors,
 matrices, and algebraic solvers. In addition, the packages all leverage some

diff --git a/doc/cvode/guide/source/sundials/index.rst b/doc/cvode/guide/source/sundials/index.rst
@@ -12,9 +12,9 @@
 
 .. _SUNDIALS:
 
-**************
-Using SUNDIALS
-**************
+***************
+Getting Started
+***************
 
 The packages that make up SUNDIALS are built upon shared classes for vectors,
 matrices, and algebraic solvers. In addition, the packages all leverage some

diff --git a/doc/cvodes/guide/source/sundials/index.rst b/doc/cvodes/guide/source/sundials/index.rst
@@ -12,9 +12,9 @@
 
 .. _SUNDIALS:
 
-**************
-Using SUNDIALS
-**************
+***************
+Getting Started
+***************
 
 The packages that make up SUNDIALS are built upon shared classes for vectors,
 matrices, and algebraic solvers. In addition, the packages all leverage some

diff --git a/doc/ida/guide/source/sundials/index.rst b/doc/ida/guide/source/sundials/index.rst
@@ -12,9 +12,9 @@
 
 .. _SUNDIALS:
 
-**************
-Using SUNDIALS
-**************
+***************
+Getting Started
+***************
 
 The packages that make up SUNDIALS are built upon shared classes for vectors,
 matrices, and algebraic solvers. In addition, the packages all leverage some

diff --git a/doc/idas/guide/source/sundials/index.rst b/doc/idas/guide/source/sundials/index.rst
@@ -12,9 +12,9 @@
 
 .. _SUNDIALS:
 
-**************
-Using SUNDIALS
-**************
+***************
+Getting Started
+***************
 
 The packages that make up SUNDIALS are built upon shared classes for vectors,
 matrices, and algebraic solvers. In addition, the packages all leverage some

diff --git a/doc/install_guide/source/conf.py b/doc/install_guide/source/conf.py
@@ -29,7 +29,10 @@
 # coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
 extensions = ['sphinx_rtd_theme', 'sphinx.ext.ifconfig', 'sphinx.ext.mathjax',
               'sphinxfortran.fortran_domain', 'sphinxcontrib.bibtex',
-              'sphinx_copybutton', 'sphinx_sundials']
+              'sphinx_copybutton', 'sphinx_sundials', 'sphinx.ext.intersphinx']
+
+intersphinx_mapping = {'sundials': (f'https://sundials.readthedocs.io/en/{doc_version}',
+                                    ('../../superbuild/build/html/objects.inv', None))}
 
 # References
 bibtex_bibfiles = ['../../shared/sundials.bib']

diff --git a/doc/kinsol/guide/source/sundials/index.rst b/doc/kinsol/guide/source/sundials/index.rst
@@ -12,9 +12,9 @@
 
 .. _SUNDIALS:
 
-**************
-Using SUNDIALS
-**************
+***************
+Getting Started
+***************
 
 The packages that make up SUNDIALS are built upon shared classes for vectors,
 matrices, and algebraic solvers. In addition, the packages all leverage some

diff --git a/doc/shared/RecentChanges.rst b/doc/shared/RecentChanges.rst
@@ -1,5 +1,8 @@
 **New Features**
 
+Added CMake infrastructure that enables externally maintained addons/plugins
+to be *optionally* built with SUNDIALS. See :ref:`Contributing` for details.
+
 **Bug Fixes**
 
 Updated the CMake variable ``HIP_PLATFORM`` default to ``amd`` as the previous
@@ -18,3 +21,8 @@ to a function pointer instead of ``c_null_funptr``. This caused compilation issu
 with the Cray Fortran compiler.
 
 Enabled the Fortran interfaces to build with 32-bit ``sunindextype``.
+
+Fixed a bug where :c:func:`MRIStepEvolve` would not handle a recoverable error
+produced from evolving the inner stepper.
+
+Added support for Kokkos Kernels v4.
diff --git a/doc/shared/sundials/Install.rst b/doc/shared/sundials/Install.rst
@@ -639,6 +639,17 @@ illustration only.
       Error checks will impact performance, but can be helpful for debugging.
 
 
+.. cmakeoption:: SUNDIALS_ENABLE_EXTERNAL_ADDONS
+
+   Build SUNDIALS with any external addons that you have put in ``sundials/external``.
+
+   Default: ``OFF``
+
+   .. warning::
+
+      Addons are not maintained by the SUNDIALS team. Use at your own risk.
+
+
 .. cmakeoption:: ENABLE_GINKGO
 
    Enable interfaces to the Ginkgo linear algebra library.
@@ -2176,4 +2187,27 @@ The following command builds and installs SUNDIALS with MPI, HIP, and the Fortra
    # Need an allocation to run the tests:
    salloc -A <account> -t 10 -N 1 -p batch
    make test
-   make test_install_all
+   make test_install_all
+
+
+Building with SUNDIALS Addons
+-----------------------------
+
+SUNDIALS "addons" are community developed code additions for SUNDIALS that can be subsumed by the
+SUNDIALS build system so that they have full access to all internal SUNDIALS symbols. The intent is
+for SUNDIALS addons to function as if they are part of the SUNDIALS library, while allowing them to
+potentially have different licenses (although we encourage BSD-3-Clause still), code style (although
+we encourage them to follow the SUNDIALS style outlined :ref:`here <Style>`).
+
+.. warning::
+
+   SUNDIALS addons are not maintained by the SUNDIALS team and may come with different
+   licenses. Use them at your own risk.
+
+To build with SUNDIALS addons,
+
+1. Clone/copy the addon(s) into ``<sundials root>/external/``
+2. Copy the ``sundials-addon-example`` block in the ``<sundials root>/external/CMakeLists.txt``,
+   paste it below the example block, and modify the path listed for your own external addon(s).
+3. When building SUNDIALS, set the CMake option :cmakeop:`SUNDIALS_ENABLE_EXTERNAL_ADDONS` to `ON`
+4. Build SUNDIALS as usual.
diff --git a/doc/shared/sundials/index.rst b/doc/shared/sundials/index.rst
@@ -12,9 +12,9 @@
 
 .. _SUNDIALS:
 
-==============
-Using SUNDIALS
-==============
+===============
+Getting Started
+===============
 
 As discussed in :numref:`Organization`, the six solver packages
 (CVODE(S), IDA(S), ARKODE, KINSOL) that make up SUNDIALS are built upon common