Skip to content

Commit ea64e39

Browse files
committed
Updated documentation for version 1.2
1 parent 808e425 commit ea64e39

File tree

100 files changed

+22313
-2194
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

100 files changed

+22313
-2194
lines changed

README.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@ Requirements
3939
DFO-LS requires the following software to be installed:
4040

4141
* Python 2.7 or Python 3 (http://www.python.org/)
42-
* Fortran compiler (e.g. gfortran)
42+
* Fortran compiler (e.g. `gfortran <https://gcc.gnu.org/wiki/GFortran>`_), required by the `trustregion <https://github.com/lindonroberts/trust-region>`_ package.
4343

4444
Additionally, the following python packages should be installed (these will be installed automatically if using *pip*, see `Installation using pip`_):
4545

docs/build/doctrees/advanced.doctree

80.7 KB
Binary file not shown.
21.7 KB
Binary file not shown.
32.9 KB
Binary file not shown.

docs/build/doctrees/history.doctree

11.5 KB
Binary file not shown.

docs/build/doctrees/index.doctree

13 KB
Binary file not shown.

docs/build/doctrees/info.doctree

33.6 KB
Binary file not shown.

docs/build/doctrees/install.doctree

16 KB
Binary file not shown.

docs/build/doctrees/userguide.doctree

83.8 KB
Binary file not shown.

docs/build/html/.buildinfo

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
11
# Sphinx build info version 1
22
# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
3-
config: c455c8d539b1418eb7155481ab831294
3+
config: 925451d78dcb942ee18dd13a268c82ac
44
tags: 645f666f9bcd5a90fca523b33c5a78b7

docs/build/html/_sources/advanced.rst.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ Logging and Output
2222

2323
Initialization of Points
2424
------------------------
25-
* :code:`init.random_initial_directions` - Build the initial interpolation set using random directions (as opposed to coordinate directions). Default is :code:`True`.
25+
* :code:`init.random_initial_directions` - Build the initial interpolation set using random directions (as opposed to coordinate directions). Default as of version 1.2 is :code:`False`.
2626
* :code:`init.random_directions_make_orthogonal` - If building initial interpolation set with random directions, whether or not these should be orthogonalized. Default is :code:`True`.
2727
* :code:`init.run_in_parallel` - If using random directions, whether or not to ask for all :code:`objfun` to be evaluated at all points without any intermediate processing. Default is :code:`False`.
2828

docs/build/html/_sources/history.rst.txt

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,3 +24,12 @@ Version 1.1.1 (5 Apr 2019)
2424
--------------------------
2525
* Link code to Zenodo, to create DOI - no changes to the DFO-LS algorithm.
2626

27+
Version 1.2 (12 Feb 2020)
28+
-------------------------
29+
* Use deterministic initialisation by default (so it is no longer necessary to set a random seed for reproducibility of DFO-LS results).
30+
* Full model Hessian stored rather than just upper triangular part - this improves the runtime of Hessian-based operations.
31+
* Faster trust-region and geometry subproblem solutions in Fortran using the `trustregion <https://github.com/lindonroberts/trust-region>`_ package.
32+
* Faster interpolation solution for multiple right-hand sides.
33+
* Don't adjust starting point if it is close to the bounds (as long as it is feasible).
34+
* Option to stop default logging behavior and/or enable per-iteration printing.
35+
* Bugfix: correctly handle 1-sided bounds as inputs, avoid divide-by-zero warnings when auto-detecting restarts.

docs/build/html/_sources/index.rst.txt

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,8 @@ That is, DFO-LS solves
2020
\min_{x\in\mathbb{R}^n} &\quad f(x) := \sum_{i=1}^{m}r_{i}(x)^2 \\
2121
\text{s.t.} &\quad a \leq x \leq b
2222
23+
The upper and lower bounds on the variables are non-relaxable (i.e. DFO-LS will never ask to evaluate a point outside the bounds).
24+
2325
Full details of the DFO-LS algorithm are given in our paper: C. Cartis, J. Fiala, B. Marteau and L. Roberts, `Improving the Flexibility and Robustness of Model-Based Derivative-Free Optimization Solvers <https://arxiv.org/abs/1804.00154>`_, technical report, University of Oxford, (2018). DFO-LS is a more flexible version of `DFO-GN <https://github.com/numericalalgorithmsgroup/dfogn>`_.
2426

2527
If you are interested in solving general optimization problems (without a least-squares structure), you may wish to try `Py-BOBYQA <https://github.com/numericalalgorithmsgroup/pybobyqa>`_, which has many of the same features as DFO-LS.

docs/build/html/_sources/install.rst.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ Requirements
66
DFO-LS requires the following software to be installed:
77

88
* Python 2.7 or Python 3 (http://www.python.org/)
9+
* Fortran compiler (e.g. `gfortran <https://gcc.gnu.org/wiki/GFortran>`_), required by the `trustregion <https://github.com/lindonroberts/trust-region>`_ package.
910

1011
Additionally, the following python packages should be installed (these will be installed automatically if using *pip*, see `Installation using pip`_):
1112

docs/build/html/_sources/userguide.rst.txt

Lines changed: 67 additions & 60 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ DFO-LS is designed to solve the local optimization problem
1111
\min_{x\in\mathbb{R}^n} &\quad f(x) := \sum_{i=1}^{m}r_{i}(x)^2 \\
1212
\text{s.t.} &\quad a \leq x \leq b
1313
14-
where the bound constraints :math:`a \leq x \leq b` are optional.
14+
where the bound constraints :math:`a \leq x \leq b` are optional. The upper and lower bounds on the variables are non-relaxable (i.e. DFO-LS will never ask to evaluate a point outside the bounds).
1515

1616
DFO-LS iteratively constructs an interpolation-based model for the objective, and determines a step using a trust-region framework.
1717
For an in-depth technical description of the algorithm see the paper [CFMR2018]_.
@@ -68,7 +68,8 @@ The :code:`solve` function has several optional arguments which the user may pro
6868
dfols.solve(objfun, x0, args=(), bounds=None, npt=None, rhobeg=None,
6969
rhoend=1e-8, maxfun=None, nsamples=None,
7070
user_params=None, objfun_has_noise=False,
71-
scaling_within_bounds=False)
71+
scaling_within_bounds=False,
72+
do_logging=True, print_progress=False)
7273
7374
These arguments are:
7475

@@ -82,6 +83,8 @@ These arguments are:
8283
* :code:`user_params` - a Python dictionary :code:`{'param1': val1, 'param2':val2, ...}` of optional parameters. A full list of available options is given in the next section :doc:`advanced`.
8384
* :code:`objfun_has_noise` - a flag to indicate whether or not :code:`objfun` has stochastic noise; i.e. will calling :code:`objfun(x)` multiple times at the same value of :code:`x` give different results? This is used to set some sensible default parameters (including using multiple restarts), all of which can be overridden by the values provided in :code:`user_params`.
8485
* :code:`scaling_within_bounds` - a flag to indicate whether the algorithm should internally shift and scale the entries of :code:`x` so that the bounds become :math:`0 \leq x \leq 1`. This is useful is you are setting :code:`bounds` and the bounds have different orders of magnitude. If :code:`scaling_within_bounds=True`, the values of :code:`rhobeg` and :code:`rhoend` apply to the *shifted* variables.
86+
* :code:`do_logging` - a flag to indicate whether logging output should be produced. This is not automatically visible unless you use the Python `logging <https://docs.python.org/3/library/logging.html>`_ module (see below for simple usage).
87+
* :code:`print_progress` - a flag to indicate whether to print a per-iteration progress log to terminal.
8588

8689
In general when using optimization software, it is good practice to scale your variables so that moving each by a given amount has approximately the same impact on the objective function.
8790
The :code:`scaling_within_bounds` flag is designed to provide an easy way to achieve this, if you have set the bounds :code:`lower` and :code:`upper`.
@@ -116,9 +119,6 @@ A commonly-used starting point for testing purposes is :math:`x_0=(-1.2,1)`. The
116119
# Define the starting point
117120
x0 = np.array([-1.2, 1.0])
118121
119-
# Set random seed (for reproducibility)
120-
np.random.seed(0)
121-
122122
# Call DFO-LS
123123
soln = dfols.solve(rosenbrock, x0)
124124
@@ -130,12 +130,12 @@ Note that DFO-LS is a randomized algorithm: in its first phase, it builds an int
130130
.. code-block:: none
131131
132132
****** DFO-LS Results ******
133-
Solution xmin = [ 1. 1.]
134-
Residual vector = [ -2.22044605e-15 0.00000000e+00]
135-
Objective value f(xmin) = 4.930380658e-30
136-
Needed 36 objective evaluations (at 36 points)
137-
Approximate Jacobian = [[ -1.98957443e+01 1.00000000e+01]
138-
[ -1.00000000e+00 8.37285083e-16]]
133+
Solution xmin = [1. 1.]
134+
Residual vector = [0. 0.]
135+
Objective value f(xmin) = 0
136+
Needed 33 objective evaluations (at 33 points)
137+
Approximate Jacobian = [[-2.00180000e+01 1.00000000e+01]
138+
[-1.00000000e+00 8.19971362e-16]]
139139
Exit flag = 0
140140
Success: Objective is sufficiently small
141141
****************************
@@ -160,12 +160,12 @@ DFO-LS correctly finds the solution to the constrained problem:
160160
.. code-block:: none
161161
162162
****** DFO-LS Results ******
163-
Solution xmin = [ 0.9 0.81]
164-
Residual vector = [ 0. 0.1]
163+
Solution xmin = [0.9 0.81]
164+
Residual vector = [3.10862447e-14 1.00000000e-01]
165165
Objective value f(xmin) = 0.01
166-
Needed 65 objective evaluations (at 65 points)
167-
Approximate Jacobian = [[ -1.79999998e+01 9.99999990e+00]
168-
[ -9.99999998e-01 -2.53940698e-09]]
166+
Needed 58 objective evaluations (at 58 points)
167+
Approximate Jacobian = [[-1.79999999e+01 9.99999998e+00]
168+
[-1.00000000e+00 8.62398179e-10]]
169169
Exit flag = 0
170170
Success: rho has reached rhoend
171171
****************************
@@ -193,12 +193,12 @@ And we can now see each evaluation of :code:`objfun`:
193193
.. code-block:: none
194194
195195
Function eval 1 at point 1 has f = 39.65 at x = [-1.2 0.85]
196-
Initialising (random directions)
196+
Initialising (coordinate directions)
197197
Function eval 2 at point 2 has f = 14.337296 at x = [-1.08 0.85]
198198
Function eval 3 at point 3 has f = 55.25 at x = [-1.2 0.73]
199199
...
200-
Function eval 64 at point 64 has f = 0.0100000029949496 at x = [ 0.89999999 0.81 ]
201-
Function eval 65 at point 65 has f = 0.00999999999999993 at x = [ 0.9 0.81]
200+
Function eval 57 at point 57 has f = 0.010000001407575 at x = [0.89999999 0.80999999]
201+
Function eval 58 at point 58 has f = 0.00999999999999997 at x = [0.9 0.81]
202202
Did a total of 1 run(s)
203203
204204
If we wanted to save this output to a file, we could replace the above call to :code:`logging.basicConfig()` with
@@ -208,6 +208,20 @@ If we wanted to save this output to a file, we could replace the above call to :
208208
logging.basicConfig(filename="myfile.log", level=logging.INFO,
209209
format='%(message)s', filemode='w')
210210
211+
If you have logging for some parts of your code and you want to deactivate all DFO-LS logging, you can use the optional argument :code:`do_logging=False` in :code:`dfols.solve()`.
212+
213+
An alternative option available is to get DFO-LS to print to terminal progress information every iteration, by setting the optional argument :code:`print_progress=True` in :code:`dfols.solve()`. If we do this for the above example, we get
214+
215+
.. code-block:: none
216+
217+
Run Iter Obj Grad Delta rho Evals
218+
1 1 1.43e+01 1.61e+02 1.20e-01 1.20e-01 3
219+
1 2 4.35e+00 3.77e+01 4.80e-01 1.20e-01 4
220+
1 3 4.35e+00 3.77e+01 6.00e-02 1.20e-02 4
221+
...
222+
1 55 1.00e-02 2.00e-01 1.50e-08 1.00e-08 56
223+
1 56 1.00e-02 2.00e-01 1.50e-08 1.00e-08 57
224+
211225
Example: Noisy Objective Evaluation
212226
-----------------------------------
213227
As described in :doc:`info`, derivative-free algorithms such as DFO-LS are particularly useful when :code:`objfun` has noise. Let's modify the previous example to include random noise in our objective evaluation, and compare it to a derivative-based solver:
@@ -269,25 +283,25 @@ The output of this is:
269283
objfun(x0) = [-4.39545837 2.20903317]
270284
271285
****** DFO-LS Results ******
272-
Solution xmin = [ 1. 1.]
273-
Residual vector = [ 3.51006670e-08 2.00158313e-10]
274-
Objective value f(xmin) = 1.232096886e-15
275-
Needed 46 objective evaluations (at 46 points)
276-
Approximate Jacobian = [[ -2.04330578e+01 1.00296466e+01]
277-
[ -9.88260906e-01 -3.77364910e-03]]
286+
Solution xmin = [1. 1.00000003]
287+
Residual vector = [ 1.59634974e-07 -4.63036198e-09]
288+
Objective value f(xmin) = 2.550476524e-14
289+
Needed 53 objective evaluations (at 53 points)
290+
Approximate Jacobian = [[-1.98196347e+01 9.90335675e+00]
291+
[-1.01941978e+00 4.24991776e-05]]
278292
Exit flag = 0
279293
Success: Objective is sufficiently small
280294
****************************
281295
282296
283297
** SciPy results **
284-
Solution xmin = [-1.2 1. ]
285-
Objective value f(xmin) = 23.96809472
286-
Needed 5 objective evaluations
298+
Solution xmin = [-1.20000087 1.00000235]
299+
Objective value f(xmin) = 23.95535774
300+
Needed 6 objective evaluations
287301
Exit flag = 3
288302
`xtol` termination condition is satisfied.
289303
290-
DFO-LS is able to find the solution with only 10 more function evaluations than in the noise-free case. However SciPy's derivative-based solver, which has no trouble solving the noise-free problem, is unable to make any progress.
304+
DFO-LS is able to find the solution with 20 more function evaluations as in the noise-free case. However SciPy's derivative-based solver, which has no trouble solving the noise-free problem, is unable to make any progress.
291305

292306
As noted above, DFO-LS has an input parameter :code:`objfun_has_noise` to indicate if :code:`objfun` has noise in it, which it does in this case. Therefore we can call DFO-LS with
293307

@@ -300,12 +314,12 @@ Using this setting, we find the correct solution faster:
300314
.. code-block:: none
301315
302316
****** DFO-LS Results ******
303-
Solution xmin = [ 1. 1.]
304-
Residual vector = [ -5.80172077e-09 2.10781076e-09]
305-
Objective value f(xmin) = 3.810283004e-17
317+
Solution xmin = [1. 1.]
318+
Residual vector = [-4.06227943e-08 2.51525603e-10]
319+
Objective value f(xmin) = 1.650274685e-15
306320
Needed 29 objective evaluations (at 29 points)
307-
Approximate Jacobian = [[ -1.96671666e+01 9.88784341e+00]
308-
[ -1.00451147e+00 1.43596001e-04]]
321+
Approximate Jacobian = [[-1.99950530e+01 1.00670067e+01]
322+
[-9.96161167e-01 -2.41166495e-04]]
309323
Exit flag = 0
310324
Success: Objective is sufficiently small
311325
****************************
@@ -342,9 +356,6 @@ The code for this is:
342356
# Define the starting point
343357
x0 = np.array([100.0, -1.0])
344358
345-
# Set random seed (for reproducibility)
346-
np.random.seed(0)
347-
348359
# We expect exponential decay: set upper bound x[1] <= 0
349360
upper = np.array([1e20, 0.0])
350361
@@ -358,23 +369,22 @@ The output of this is (noting that DFO-LS moves :math:`x_0` to be far away enoug
358369

359370
.. code-block:: none
360371
361-
RuntimeWarning: x0 too close to upper bound, adjusting
362372
****** DFO-LS Results ******
363-
Solution xmin = [ 4.98830860e+02 -1.01256863e-01]
364-
Residual vector = [-0.18167084 0.06098401 0.76276294 0.11962349 -0.265898 -0.59788818
365-
-1.02611899 -1.51235371 -1.56145452 -1.63266662]
373+
Solution xmin = [ 4.98830861e+02 -1.01256863e-01]
374+
Residual vector = [-0.1816709 0.06098397 0.76276301 0.11962354 -0.26589796 -0.59788814
375+
-1.02611897 -1.5123537 -1.56145452 -1.63266662]
366376
Objective value f(xmin) = 9.504886892
367-
Needed 99 objective evaluations (at 99 points)
368-
Approximate Jacobian = [[ -9.12901557e-01 -4.09843510e+02]
369-
[ -8.59085471e-01 -6.42808522e+02]
370-
[ -2.47253894e-01 -1.70205399e+03]
371-
[ -1.34675403e-01 -1.33017159e+03]
372-
[ -8.71359818e-02 -1.04752827e+03]
373-
[ -5.75305576e-02 -8.09280563e+02]
374-
[ -2.83185322e-02 -4.97239478e+02]
375-
[ -2.22993603e-03 -6.70749492e+01]
376-
[ -5.24135530e-04 -1.95045149e+01]
377-
[ -2.65977795e-04 -1.07858009e+01]]
377+
Needed 79 objective evaluations (at 79 points)
378+
Approximate Jacobian = [[-9.12897463e-01 -4.09843514e+02]
379+
[-8.59085679e-01 -6.42808544e+02]
380+
[-2.47252555e-01 -1.70205419e+03]
381+
[-1.34676365e-01 -1.33017181e+03]
382+
[-8.71355033e-02 -1.04752848e+03]
383+
[-5.75304364e-02 -8.09280752e+02]
384+
[-2.83184867e-02 -4.97239623e+02]
385+
[-2.22992989e-03 -6.70749826e+01]
386+
[-5.24129962e-04 -1.95045269e+01]
387+
[-2.65956876e-04 -1.07858081e+01]]
378388
Exit flag = 0
379389
Success: rho has reached rhoend
380390
****************************
@@ -438,9 +448,6 @@ The code for this is:
438448
# DFO-LS returns will likely depend on x0!
439449
x0 = np.array([0.1, -2.0])
440450
441-
# Set random seed (for reproducibility)
442-
np.random.seed(0)
443-
444451
# Call DFO-LS
445452
soln = dfols.solve(nonlinear_system, x0)
446453
@@ -454,11 +461,11 @@ The output of this is
454461
455462
****** DFO-LS Results ******
456463
Solution xmin = [ 0.09777309 -2.32510588]
457-
Residual vector = [ 2.89990254e-13 3.31557004e-12]
458-
Objective value f(xmin) = 1.107709904e-23
459-
Needed 18 objective evaluations (at 18 points)
460-
Approximate Jacobian = [[ 3.32510429 0.90222738]
461-
[ 10.22774647 -0.9999939 ]]
464+
Residual vector = [-1.45394186e-09 -1.95108811e-08]
465+
Objective value f(xmin) = 3.827884295e-16
466+
Needed 13 objective evaluations (at 13 points)
467+
Approximate Jacobian = [[ 3.32499552 0.90216381]
468+
[10.22664908 -1.00061604]]
462469
Exit flag = 0
463470
Success: Objective is sufficiently small
464471
****************************

0 commit comments

Comments
 (0)