Cleaned up regularization documentation and examples

README.rst

@@ -34,13 +34,15 @@ See manual.pdf or `here <https://numericalalgorithmsgroup.github.io/dfols/>`_.
 
 Citation
 --------
-If you use DFO-LS in a paper, please cite:
+The development of DFO-LS is outlined over several publications:
 
-Cartis, C., Fiala, J., Marteau, B. and Roberts, L., `Improving the Flexibility and Robustness of Model-Based Derivative-Free Optimization Solvers <https://doi.org/10.1145/3338517>`_, *ACM Transactions on Mathematical Software*, 45:3 (2019), pp. 32:1-32:41.
+1. C. Cartis, J. Fiala, B. Marteau and L. Roberts, `Improving the Flexibility and Robustness of Model-Based Derivative-Free Optimization Solvers <https://doi.org/10.1145/3338517>`_, *ACM Transactions on Mathematical Software*, 45:3 (2019), pp. 32:1-32:41 [`preprint <https://arxiv.org/abs/1804.00154>`_] . 
+2. M. Hough, and L. Roberts, `Model-Based Derivative-Free Methods for Convex-Constrained Optimization <https://doi.org/10.1137/21M1460971>`_, *SIAM Journal on Optimization*, 21:4 (2022), pp. 2552-2579 [`preprint <https://arxiv.org/abs/2111.05443>`_].
+3. Y. Liu, K. H. Lam and L. Roberts, `Black-box Optimization Algorithms for Regularized Least-squares Problems <http://arxiv.org/abs/2407.14915>`_, *arXiv preprint arXiv:arXiv:2407.14915*, 2024.
 
-If you use DFO-LS for problems with constraints, including bound constraints, please also cite:
-
-Hough, M. and Roberts, L., `Model-Based Derivative-Free Methods for Convex-Constrained Optimization <https://doi.org/10.1137/21M1460971>`_, *SIAM Journal on Optimization*, 21:4 (2022), pp. 2552-2579.
+If you use DFO-LS in a paper, please cite [1]. 
+If your problem has constraints, including bound constraints, please cite [1,2].
+If your problem includes a regularizer, please cite [1,3].
 
 Requirements
 ------------
@@ -70,27 +72,13 @@ For easy installation, use `pip <http://www.pip-installer.org/>`_ as root:
 
 .. code-block:: bash
 
-    $ [sudo] pip install DFO-LS
-
-or alternatively *easy_install*:
-
-.. code-block:: bash
-
-    $ [sudo] easy_install DFO-LS
-
-If you do not have root privileges or you want to install DFO-LS for your private use, you can use:
-
-.. code-block:: bash
-
-    $ pip install --user DFO-LS
-
-which will install DFO-LS in your home directory.
+    $ pip install DFO-LS
 
 Note that if an older install of DFO-LS is present on your system you can use:
 
 .. code-block:: bash
 
-    $ [sudo] pip install --upgrade DFO-LS
+    $ pip install --upgrade DFO-LS
 
 to upgrade DFO-LS to the latest version.
 
@@ -107,22 +95,14 @@ DFO-LS is written in pure Python and requires no compilation. It can be installe
 
  .. code-block:: bash
 
-    $ [sudo] pip install .
-
-If you do not have root privileges or you want to install DFO-LS for your private use, you can use:
-
- .. code-block:: bash
-
-    $ pip install --user .
-
-instead.
+    $ pip install .
 
 To upgrade DFO-LS to the latest version, navigate to the top-level directory (i.e. the one containing :code:`pyproject.toml`) and rerun the installation using :code:`pip`, as above:
 
  .. code-block:: bash
 
     $ git pull
-    $ [sudo] pip install .  # with admin privileges
+    $ pip install .
 
 Testing
 -------
@@ -145,7 +125,7 @@ If DFO-LS was installed using *pip* you can uninstall as follows:
 
  .. code-block:: bash
 
-    $ [sudo] pip uninstall DFO-LS
+    $ pip uninstall DFO-LS
 
 If DFO-LS was installed manually you have to remove the installed files by hand (located in your python site-packages directory).
 

dfols/__init__.py

@@ -39,7 +39,7 @@
 from __future__ import absolute_import, division, print_function, unicode_literals
 
 # DFO-LS version
-__version__ = '1.4.1'
+__version__ = '1.5.0'
 
 # Main solver & exit flags
 from .solver import *

docs/advanced.rst

@@ -2,7 +2,7 @@ Advanced Usage
 ==============
 This section describes different optional user parameters available in DFO-LS.
 
-In the last section (:doc:`userguide`), we introduced :code:`dfols.solve()`, which has the optional input :code:`user_params`. This is a Python dictionary of user parameters. We will now go through the settings which can be changed in this way. More details are available in the papers [CFMR2018]_ and [HR2022]_.
+In the last section (:doc:`userguide`), we introduced :code:`dfols.solve()`, which has the optional input :code:`user_params`. This is a Python dictionary of user parameters. We will now go through the settings which can be changed in this way. More details are available in the papers [CFMR2018]_, [HR2022]_ and [LLR2024]_.
 
 The default values, used if no override is given, in some cases vary depending on whether :code:`objfun` has stochastic noise; that is, whether evaluating :code:`objfun(x)` several times at the same :code:`x` gives the same result or not. Whether or not this is the case is determined by the :code:`objfun_has_noise` input to :code:`dfols.solve()` (and not by inspecting :code:`objfun`, for instance).
 
@@ -128,4 +128,7 @@ References
    Coralia Cartis, Jan Fiala, Benjamin Marteau and Lindon Roberts, `Improving the Flexibility and Robustness of Model-Based Derivative-Free Optimization Solvers <https://doi.org/10.1145/3338517>`_, *ACM Transactions on Mathematical Software*, 45:3 (2019), pp. 32:1-32:41 [`preprint <https://arxiv.org/abs/1804.00154>`_] 
 
 .. [HR2022]   
-   Hough, M. and Roberts, L., `Model-Based Derivative-Free Methods for Convex-Constrained Optimization <https://doi.org/10.1137/21M1460971>`_, *SIAM Journal on Optimization*, 21:4 (2022), pp. 2552-2579 [`preprint <https://arxiv.org/abs/2111.05443>`_].
+   Matthew Hough and Lindon Roberts, `Model-Based Derivative-Free Methods for Convex-Constrained Optimization <https://doi.org/10.1137/21M1460971>`_, *SIAM Journal on Optimization*, 21:4 (2022), pp. 2552-2579 [`preprint <https://arxiv.org/abs/2111.05443>`_].
+
+.. [LLR2024]   
+   Yanjun Liu, Kevin H. Lam and Lindon Roberts, `Black-box Optimization Algorithms for Regularized Least-squares Problems <http://arxiv.org/abs/2407.14915>`_, *arXiv preprint arXiv:2407.14915* (2024).

docs/build/html/.buildinfo

@@ -1,4 +1,4 @@
 # Sphinx build info version 1
 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
-config: 35e7e5a3a73f5accc3f3cb99e4006184
+config: 8fe9d19f5645917bc7a4a240ff52ff51
 tags: 645f666f9bcd5a90fca523b33c5a78b7

docs/build/html/_sources/advanced.rst.txt

@@ -2,7 +2,7 @@ Advanced Usage
 ==============
 This section describes different optional user parameters available in DFO-LS.
 
-In the last section (:doc:`userguide`), we introduced :code:`dfols.solve()`, which has the optional input :code:`user_params`. This is a Python dictionary of user parameters. We will now go through the settings which can be changed in this way. More details are available in the papers [CFMR2018]_ and [HR2022]_.
+In the last section (:doc:`userguide`), we introduced :code:`dfols.solve()`, which has the optional input :code:`user_params`. This is a Python dictionary of user parameters. We will now go through the settings which can be changed in this way. More details are available in the papers [CFMR2018]_, [HR2022]_ and [LLR2024]_.
 
 The default values, used if no override is given, in some cases vary depending on whether :code:`objfun` has stochastic noise; that is, whether evaluating :code:`objfun(x)` several times at the same :code:`x` gives the same result or not. Whether or not this is the case is determined by the :code:`objfun_has_noise` input to :code:`dfols.solve()` (and not by inspecting :code:`objfun`, for instance).
 
@@ -128,4 +128,7 @@ References
    Coralia Cartis, Jan Fiala, Benjamin Marteau and Lindon Roberts, `Improving the Flexibility and Robustness of Model-Based Derivative-Free Optimization Solvers <https://doi.org/10.1145/3338517>`_, *ACM Transactions on Mathematical Software*, 45:3 (2019), pp. 32:1-32:41 [`preprint <https://arxiv.org/abs/1804.00154>`_] 
 
 .. [HR2022]   
-   Hough, M. and Roberts, L., `Model-Based Derivative-Free Methods for Convex-Constrained Optimization <https://doi.org/10.1137/21M1460971>`_, *SIAM Journal on Optimization*, 21:4 (2022), pp. 2552-2579 [`preprint <https://arxiv.org/abs/2111.05443>`_].
+   Matthew Hough and Lindon Roberts, `Model-Based Derivative-Free Methods for Convex-Constrained Optimization <https://doi.org/10.1137/21M1460971>`_, *SIAM Journal on Optimization*, 21:4 (2022), pp. 2552-2579 [`preprint <https://arxiv.org/abs/2111.05443>`_].
+
+.. [LLR2024]   
+   Yanjun Liu, Kevin H. Lam and Lindon Roberts, `Black-box Optimization Algorithms for Regularized Least-squares Problems <http://arxiv.org/abs/2407.14915>`_, *arXiv preprint arXiv:2407.14915* (2024).

docs/build/html/_sources/contributors.rst.txt

@@ -3,8 +3,10 @@ Contributors
 
 Main author
 -----------
-* Lindon Roberts (University of Sydney)
+* `Lindon Roberts <https://lindonroberts.github.io/>`_ (University of Sydney)
 
 Contributors
 ------------
 * Matthew Hough (University of Waterloo): handle general convex constraints [version 1.3]
+* Yanjun Liu (Princeton University): nonsmooth regularizer [version 1.5]
+* Kevin Lam (Australian National University): nonsmooth regularizer [version 1.5]

docs/build/html/_sources/history.rst.txt

@@ -61,3 +61,8 @@ Version 1.4.1 (11 Apr 2024)
 ---------------------------
 * Migrate package setup to pyproject.toml (required for Python version 3.12)
 * Drop support for Python 2.7 and <=3.8 in line with SciPy >=1.11 dependency (introduced v1.4.0)
+
+Version 1.5.0 (11 Sep 2024)
+---------------------------
+* Add support for (possibly nonsmooth) regularizer term.
+* Drop warning about infeasible initial point if the point is on an upper/lower bound.

docs/build/html/_sources/index.rst.txt

@@ -9,7 +9,7 @@ DFO-LS: Derivative-Free Optimizer for Least-Squares Minimization
 
 **Date:** |today|
 
-**Author:** `Lindon Roberts <[email protected]>`_
+**Author:** `Lindon Roberts <https://lindonroberts.github.io/>`_
 
 DFO-LS is a flexible package for finding local solutions to nonlinear least-squares minimization problems (with optional regularizer and constraints), without requiring any derivatives of the objective. DFO-LS stands for Derivative-Free Optimizer for Least-Squares.
 

docs/build/html/_sources/install.rst.txt

@@ -59,7 +59,7 @@ To upgrade DFO-LS to the latest version, navigate to the top-level directory (i.
  .. code-block:: bash
 
     $ git pull
-    $ pip install .  # with admin privileges
+    $ pip install .
 
 Testing
 -------