release 3.0 updates

CHANGELOG.md

@@ -1,5 +1,58 @@
 # CHANGELOG
 
+## version 3.0
+
+### Added
+
+- Sparse-dense Sylvester solver, see `mess_sylvester_sparse_dense`.
+- Krylov subspace projection methods for Lyapunov equations and AREs
+  using extended and rational Krylov subspaces, see `mess_KSM`, as
+  well as demonstration examples in `KSM_FDM`, `KSM_Rail`.
+- A new logging system allowing to print messages to both console
+  and files, and switch between various output file format. See `mess_log_*`.
+- `mess_tf_plot` as unified backend for `mess_sigma_plot`,
+  `mess_Frobenius_TF_errorplot`
+- `mess_version` a function to get the current version of M-M.E.S.S.
+
+### Changed
+
+- `mess_galerkin_projection_acceleration` has been replaced by
+  `mess_solve_projected_eqn`, which now covers both the acceleration
+  case and the case of projected equations in the context of `mess_KSM`.
+- `eqn.st` and `eqn.nd` from DAE usfs are now unified and called
+  `eqn.manifold_dim` everywhere.
+- the LDL^T ADI for Lyapunov now expects W  T  W^T rather than
+  G  S G^T as the constant term. Consequently, `eqn.S` and `eqn.G` are
+  now called `eqn.T` and `eqn.W`.
+- missing demonstration models are now loaded from the web on user
+  request. The first run with a model may, thus, need to be
+  interactive to confirm the download request.
+- usfs have been optimized further to not use unnecessary hidden data.
+- `mess_lrnm`, and `mess_lrradi` now require `eqn.R` and `eqn.Q` to be
+  set in LDL^T mode.
+- `operatormanager` now has a mandatory pass-through argument `opts`
+  for correct logging.
+
+### Fixed
+
+- many functions did not warn users about non-convergence
+- minor bugs in DRE methods were addressed
+- `get_ritz_vals` usfs and `mess_para` behavior has been unified
+- removed dead code in `mess_lrnm`
+- `lyap` in Octave and MATLAB had different behavior which is now
+  properly wrapped to give consistent results
+- BDF methods of higher order now use lower order methods with smaller
+  step sizes for a successive wind-up procedure to guarantee the
+  expected order of convergence.
+- exact line search in `mess_lrnm` now uses a more expensive, yet far
+  more numerically stable way to compute the relevant norms.
+
+### Deprecated
+
+- all functions in the `mor`folder are not going to be maintained in
+  future releases in favor of using M-M.E.S.S. as the sparse solver
+  backend for MORLAB in their version 6.0 and newer.
+
 ## version 2.2
 
 ### Added
@@ -107,30 +160,30 @@
   non-existent ones with a general `mess_do_nothing` function
 - renamed `opts.bdf.stage` to `opts.bdf.step`.
 - CI testing
-  - demos serve as system tests
-  - additional unit tests for the smaller building blocks and backend routines
+   - demos serve as system tests
+   - additional unit tests for the smaller building blocks and backend routines
 
 ### Changed
 
 - improved Riccati iteration
 - updated minimum required/recommended Matlab and Octave versions
   (see `DEPENDENCIES.md`)
 - unified function interfaces for top level calls
-- unified handling of low rank updated operators. Now always A+UV' is
+- unified handling of low-rank updated operators. Now always A+UV' is
   used. (Note the sign of the update and the transposition in V)
 - major updates in the MOR routines
 - some restructuring in the opts structure.
-  - `opts.adi.shifts` has moved to `opts.shifts` such that also RADI
-    can use it independent of ADI
-  - `opts.norm` now determines the norm for all methods rather than
+   - `opts.adi.shifts` has moved to `opts.shifts` such that also RADI
+     can use it independent of ADI
+   - `opts.norm` now determines the norm for all methods rather than
      having to consistently specify the same norm in each substructure
-  - initial feedbacks for the Riccati solvers are now stored in the
-    `opts` structure for the method rather than `eqn`
+   - initial feedbacks for the Riccati solvers are now stored in the
+     `opts` structure for the method rather than `eqn`
 - The projection shift routine uses the flag `opts.shifts.implicitVtAV`.
   Default is `true`. If set to `false` A*V is computed explicitly.
 - redesign of the demos
-  - turned scripts into actual demo functions
-  - new demos for indefinite AREs and H-infinity control
+   - turned scripts into actual demo functions
+   - new demos for indefinite AREs and H-infinity control
 
 ### Fixed
 
@@ -149,29 +202,30 @@
 
 - Minor consistency and bug fixes and improved integrity of metafiles.
 - CI testing
-  - demos serve as system tests
-  - additional unit tests for the smaller building blocks and backend routines
+   - demos serve as system tests
+   - additional unit tests for the smaller building blocks and backend routines
 
 ## version 1.0
 
 Compared to the predecessor LyaPack a couple of things have changed.
 
 - The user supplied functions are now managed by an operator manager
-- The low rank ADI now has:
-  - optimized treatment of E matrices in generalized equations
-  - more choices for shift selection, including completely automatic
-    generation of shifts
-  - improved stopping criteria based on low rank factors of the current residual
-  - automatic generation of real low rank factors also for complex shifts
+- The low-rank ADI now has:
+   - optimized treatment of E matrices in generalized equations
+   - more choices for shift selection, including completely automatic
+     generation of shifts
+   - improved stopping criteria based on low-rank factors of the
+     current residual
+   - automatic generation of real low-rank factors also for complex shifts
 - The Newton-Kleinman iteration features:
-  - optimized treatment of E matrices in generalized equations
-  - improved stopping criteria based on low rank factors of the current residual
-  - inexact Newton, line search and Galerkin projection acceleration
+   - optimized treatment of E matrices in generalized equations
+   - improved stopping criteria based on low-rank factors of the
+     current residual
+   - inexact Newton, line search and Galerkin projection acceleration
 - Examples have been extended
 - The Riccati iteration for H-infinity Riccati equations was added
 - DSPMR has not yet been ported to the new infrastructure
 - The SRM routine for balanced truncation is only available for
   none-DAE systems. Still, DAE versions are included in the
   corresponding DEMOS.
 - A tangential IRKA implementation for non-DAE systems was added
-

CITATION.cff

@@ -1,33 +1,20 @@
 # YAML 1.2
 ---
 abstract: |
-    "M-M.E.S.S.   provides low-rank solvers for large-scale symmetric matrix equations
-           with sparse or sparse + low rank coefficients. The main focus is on
-           differential and algebraic Riccati equations appearing in control and model
-           order reduction, as well as algebraic Lyapunov equations for, e.g., balanced
-           truncation.
-    
-           The  underlying  dynamical  system may be of first or second order and 
-           structured proper differential algebraic equations (DAEs) that allow for
-           implicit index reduction are also supported.
-    
-           The  solvers philosophy is to always work on the implicitly linearized (for
-           second order systems) and/or implicitly projected (in the DAE case) matrix
-           equations. That means the implicit Lyapunov or Riccati equation is always of
-           the form known for a standard first order ODE, that may have a non identity
-           but invertible E matrix.
-    
-           Further, M-M.E.S.S. provides functions for Balanced Truncation and
-           (tangential) iterative rational Krylov algorithm (IRKA) for model order 
-           reduction (MOR) of first order state space systems and some examples
-           demonstrate the use of the algorithms in MOR of second order systems and DAEs.
-    
-           In close relation to the predecessor LyaPack, we use user supplied functions
-           (usfs) that implement the actions of the system matrices E and A in 
-           multiplication  and  (shifted) solves. We provide those functions for
-           standard state space systems, second order systems, structured DAEs of
-           index 1 and 2, as well as second order DAEs of index 1, 2 and 3. For more
-           information on usfs see help mess_usfs."
+    "M-M.E.S.S.   The M-M.E.S.S. toolbox provides solvers for large-scale,
+     sparse, symmetric linear and quadratic matrix equations. These can be
+     algebraic and differential equations, and the solvers are in their core
+     all based on the low-rank ADI method.  M-M.E.S.S. can be seen as the
+     successor to the LyaPack toolbox with an improved formulation of the ADI,
+     that now properly supports generalized state-space systems, but also
+     special structured DAEs. It features additional solvers for differential
+     equations, improved shift parameter computation and a guarantee to compute
+     real low-rank factorization, but follows the same general philosophy of
+     user supplied functions that the LyaPack toolbox used.
+
+     Additionally, from version 3.0 on, also the first non-symmetric equations
+     are supported, and Krylov subpace projection methods for symmetric
+     algebraic matrix equations exist."
 authors: 
   -
     affiliation: "Max Planck Institute for Dynamics of Complex Technical Systems"
@@ -45,9 +32,9 @@ authors:
     given-names: Peter
     orcid: "https://orcid.org/0000-0003-3362-4103"
 cff-version: "1.2.0"
-doi: "10.5281/zenodo.5938237"
+doi: "10.5281/zenodo.7701424"
 license: "BSD 2-Clause"
 message: "If you use this software, please cite it using these metadata."
 title: "M-M.E.S.S. -- Matrix Equations Sparse Solvers for MATLAB and Octave"
-version: "2.2"
+version: "3.0"
 ...

CITATION.md

@@ -11,20 +11,20 @@ below.
 
 ## DOI
 
-The DOI for version 2.2 is
-[10.5281/zenodo.5938237](http://doi.org/10.5281/zenodo.5938237)
+The DOI for version 3.0 is
+[10.5281/zenodo.7701424](http://doi.org/10.5281/zenodo.7701424)
 
 ## BibTeX
 
 ```
-@Misc{SaaKB21-mmess-2.2,
+@Misc{SaaKB21-mmess-3.0,
   author =       {Saak, J. and K\"{o}hler, M. and Benner, P.},
-  title =        {{M-M.E.S.S.}-2.2 -- The Matrix Equations Sparse Solvers
+  title =        {{M-M.E.S.S.}-3.0 -- The Matrix Equations Sparse Solvers
                   library},
-  month =        feb,
-  year =         2022,
+  month =        aug,
+  year =         2023,
   note =         {see also:\url{https://www.mpi-magdeburg.mpg.de/projects/mess}},
-  doi =          {10.5281/zenodo.5938237},
+  doi =          {10.5281/zenodo.7701424},
   key =          {MMESS}
 }
 

CODE

@@ -1,8 +1,8 @@
 name: Matrix Equations Sparse Solvers (for Matlab and Octave)
 shortname: M-M.E.S.S.
-version: 2.2
-release-date: 2022-02-02
-id: 10.5281/zenodo.5938237
+version: 3.0
+release-date: 2023-XX-XX
+id: 10.5281/zenodo.7701424
 id-type: doi
 authors: MESS developer community
 copyright holders: Jens Saak, Martin Köhler, Peter Benner
@@ -17,4 +17,4 @@ languages: Matlab
 dependencies: GNU Octave >= 5.1, MATLAB >= 2014a
 systems: Linux, Windows, MacOS
 website: https://gitlab.mpi-magdeburg.mpg.de/mess/mmess-releases
-keywords: symmetric matrix equations, LR-ADI, Newton Kleinman, BDF methods, Rosenbrock methods, splitting methods, Riccati iteration, balanced trunation, IRKA
+keywords: symmetric matrix equations, LR-ADI, Newton Kleinman, BDF methods, Rosenbrock methods, splitting methods, Riccati iteration, balanced trunation, IRKA, EKSM, RKSM

CODE_OF_CONDUCT.md

@@ -128,4 +128,3 @@ For answers to common questions about this code of conduct, see the FAQ at
 [https://www.contributor-covenant.org/faq](https://www.contributor-covenant.org/faq).
 Translations are available at
 [https://www.contributor-covenant.org/translations](https://www.contributor-covenant.org/translations).
-

CONTRIBUTING.md

@@ -3,7 +3,7 @@
 Please note that this project is released with a Contributor [Code of
 Conduct](CODE_OF_CONDUCT.md). By participating in this project you
 agree to abide by its terms. Note further, that we follow
-[![Contributor Covenant](https://img.shields.io/badge/Contributor%20Covenant-v2.0%20adopted-ff69b4.svg)](code_of_conduct.md)
+[![Contributor Covenant](https://img.shields.io/badge/Contributor%20Covenant-v2.0%20adopted-ff69b4.svg)](CODE_OF_CONDUCT.md)
 and would like to emphasize the following additional examples of
 unacceptable behavior:
 
@@ -22,7 +22,7 @@ GIT history.
 
 ## Attribution
 
-When you have contributed code to M-M.E.S.S. you and the content of
+When you have contributed code to M-M.E.S.S., you and the content of
 your contribution will be mentioned in the project's [contributors
 file](CONTRIBUTORS.md).  Contributions are grouped by release, so if
 you have contributed code to multiple releases, you will be mentioned
@@ -44,15 +44,15 @@ guidelines below:
 **Naming of functions:**
 
 * to avoid shadowing of functions from other toolboxes/packages or
-  the MATLAB core, all M-M.E.S.S. solver functions start with `mess_`
+  the MATLAB/Octave core, all M-M.E.S.S. solver functions start with `mess_`
 * demonstration routines can have names deviating from the above
-  principle but should be descriptive enough such that a basic idea
+  principle, but should be descriptive enough such that a basic idea
   of their contents is evident.
 * user supplied functions need to have a special naming scheme
-  described in the documentation or found in the `operatormanager`
+  described in the documentation and found in the `operatormanager`
   function inside the `usfs` folder
 * Routines that do not obey the above have to be put into a
-  `private` folder.
+  `private` folder, or embedded into their calling functions source file.
 
 **Code Style:**
 
@@ -78,8 +78,8 @@ guidelines below:
   MATLAB's `lyap` and thus intended for matrices.
 * For the same abstraction reason, in core solvers, all operations
   with the system matrices have to be implemented via the `usfs`
-  system and access to `eqn.A_` or `eqn.E_` is prohibited. (see `help
-  mess_usfs` for details on the `usfs` system and philosophy)
+  system and access to, e.g., `eqn.A_` or `eqn.E_` is prohibited.
+  (see `help mess_usfs` for details on the `usfs` system and philosophy)
 * The options structure `opts` should contain a separate
   substructure for each algorithm/solver-function and options are
   only allowed on the top level when they are absolutely necessary
@@ -89,14 +89,32 @@ guidelines below:
   the shape of the factorization computed, or the norm that should
   consistently be used for measuring progress are allowed on the top
   level.
-* Avoid code duplication. If code blocks or variations thereof need to
+* Avoid code duplication. If code blocks, or variations thereof, need to
   be repeated, consider to put them into a function and call it
   where necessary. If a helper function lacks functionality consider
   extending rather than doubling it.
 * We prefer comments to start with a space for readability, i.e.
   `% comment` rather than `%comment`.
 * We use blanks around operators and after commas in argument lists,
   e.g. `y = a * x + b;` and not `y=a*x+b;`
+* Transposes, denoted by `'`, only occur on capital letters.
+  (this is currently required by our spellcheckers and may be dropped
+   in the future)
+
+**Warnings, Errors and Logging:**
+
+* We provide or own logging mechanisms. We, therefore, do not call
+  `error`, or `warning` directly, but via `mess_err` and `mess_warn`
+  found in the `logger` folder.
+* `mess_log_matrix` additionally allows to write matrices to `.mat`
+  files.
+* `mess_log_plot` can store the figures in image files.
+* All M-M.E.S.S. code must use these and demonstration examples have
+  to initialize and finalize the logger via `mess_log_initialize`,
+  `mess_log_finalize`.
+* It is mandatory to use `mess_fprintf` rather than `fprintf` or
+  `disp`.
+* CI tests in `_tests` can be an exception from this rule.
 
 ## Maintainability
 
@@ -108,30 +126,36 @@ follow these guidelines:
 * Work on a single issue per merge request or branch.
 * branch and merge often, since long living branches tend to become
   messy to handle and synchronize with the master.
-* Try to 'rebase' your work onto the master before merging or
-  requesting a merge, if possible
+* Try to 'rebase' your work onto the main branch before merging or
+  requesting a merge, if possible.
 
 ## Test Framework
 
 We use an extensive test system on our continuous integration (CI)
 server to ensure that new features do not break old functionality and
 all features work in both MATLAB and GNU Octave, both with and without
-toolboxes/packages installed. Currently CI time is limited to 2 hours
+toolboxes/packages installed. Currently CI time is limited to 6 hours
 and all tests should be finished in that time.
 
-We distinguish between so called unit tests and system tests. Here,
-unit tests are testing the smallest possible building blocks. As a
+We distinguish between so called unit tests, system tests, and extended tests.
+Here, unit tests are testing the smallest possible building blocks. As a
 rule of thumb, a routine that does not call external functions, or
 only private functions is a candidate for a unit test.
 
 Larger routines that run a hierarchy of other stuff, such as our
 demonstrator functions in the `DEMOS` folder should be used to perform
-the much bigger system tests.
+the bigger system tests.
+
+If runtimes become an issue, consider moving system tests to the extended
+tests category. While unit and system tests are executed on every push that
+affects them and included in our weekly pipelines, extended tests
+(that run exceptionally long) are executed only in our monthly extended
+pipelines.
 
 * All files in `DEMOS` should be accompanied by a system test
   calling them on an, ideally, small example. (See comment about the
   CI time above)
 * All `usfs` sets should be checked by unit tests. Use non-symmetric
   systems and system matrices where possible!
 * Consider to provide a unit test for every helper routine in your
-  method.
+  code.