You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/build/html/_sources/advanced.rst.txt
+1-1Lines changed: 1 addition & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -22,7 +22,7 @@ Logging and Output
22
22
23
23
Initialization of Points
24
24
------------------------
25
-
* :code:`init.random_initial_directions` - Build the initial interpolation set using random directions (as opposed to coordinate directions). Default is :code:`True`.
25
+
* :code:`init.random_initial_directions` - Build the initial interpolation set using random directions (as opposed to coordinate directions). Default as of version 1.2 is :code:`False`.
26
26
* :code:`init.random_directions_make_orthogonal` - If building initial interpolation set with random directions, whether or not these should be orthogonalized. Default is :code:`True`.
27
27
* :code:`init.run_in_parallel` - If using random directions, whether or not to ask for all :code:`objfun` to be evaluated at all points without any intermediate processing. Default is :code:`False`.
The upper and lower bounds on the variables are non-relaxable (i.e. DFO-LS will never ask to evaluate a point outside the bounds).
24
+
23
25
Full details of the DFO-LS algorithm are given in our paper: C. Cartis, J. Fiala, B. Marteau and L. Roberts, `Improving the Flexibility and Robustness of Model-Based Derivative-Free Optimization Solvers <https://arxiv.org/abs/1804.00154>`_, technical report, University of Oxford, (2018). DFO-LS is a more flexible version of `DFO-GN <https://github.com/numericalalgorithmsgroup/dfogn>`_.
24
26
25
27
If you are interested in solving general optimization problems (without a least-squares structure), you may wish to try `Py-BOBYQA <https://github.com/numericalalgorithmsgroup/pybobyqa>`_, which has many of the same features as DFO-LS.
where the bound constraints :math:`a \leq x \leq b` are optional.
14
+
where the bound constraints :math:`a \leq x \leq b` are optional. The upper and lower bounds on the variables are non-relaxable (i.e. DFO-LS will never ask to evaluate a point outside the bounds).
15
15
16
16
DFO-LS iteratively constructs an interpolation-based model for the objective, and determines a step using a trust-region framework.
17
17
For an in-depth technical description of the algorithm see the paper [CFMR2018]_.
@@ -68,7 +68,8 @@ The :code:`solve` function has several optional arguments which the user may pro
* :code:`user_params` - a Python dictionary :code:`{'param1': val1, 'param2':val2, ...}` of optional parameters. A full list of available options is given in the next section :doc:`advanced`.
83
84
* :code:`objfun_has_noise` - a flag to indicate whether or not :code:`objfun` has stochastic noise; i.e. will calling :code:`objfun(x)` multiple times at the same value of :code:`x` give different results? This is used to set some sensible default parameters (including using multiple restarts), all of which can be overridden by the values provided in :code:`user_params`.
84
85
* :code:`scaling_within_bounds` - a flag to indicate whether the algorithm should internally shift and scale the entries of :code:`x` so that the bounds become :math:`0\leq x \leq1`. This is useful is you are setting :code:`bounds` and the bounds have different orders of magnitude. If :code:`scaling_within_bounds=True`, the values of :code:`rhobeg` and :code:`rhoend` apply to the *shifted* variables.
86
+
* :code:`do_logging` - a flag to indicate whether logging output should be produced. This is not automatically visible unless you use the Python `logging <https://docs.python.org/3/library/logging.html>`_ module (see below for simple usage).
87
+
* :code:`print_progress` - a flag to indicate whether to print a per-iteration progress log to terminal.
85
88
86
89
In general when using optimization software, it is good practice to scale your variables so that moving each by a given amount has approximately the same impact on the objective function.
87
90
The :code:`scaling_within_bounds` flag is designed to provide an easy way to achieve this, if you have set the bounds :code:`lower` and :code:`upper`.
@@ -116,9 +119,6 @@ A commonly-used starting point for testing purposes is :math:`x_0=(-1.2,1)`. The
116
119
# Define the starting point
117
120
x0 = np.array([-1.2, 1.0])
118
121
119
-
# Set random seed (for reproducibility)
120
-
np.random.seed(0)
121
-
122
122
# Call DFO-LS
123
123
soln = dfols.solve(rosenbrock, x0)
124
124
@@ -130,12 +130,12 @@ Note that DFO-LS is a randomized algorithm: in its first phase, it builds an int
If you have logging for some parts of your code and you want to deactivate all DFO-LS logging, you can use the optional argument :code:`do_logging=False` in :code:`dfols.solve()`.
212
+
213
+
An alternative option available is to get DFO-LS to print to terminal progress information every iteration, by setting the optional argument :code:`print_progress=True` in :code:`dfols.solve()`. If we do this for the above example, we get
214
+
215
+
.. code-block:: none
216
+
217
+
Run Iter Obj Grad Delta rho Evals
218
+
1 1 1.43e+01 1.61e+02 1.20e-01 1.20e-01 3
219
+
1 2 4.35e+00 3.77e+01 4.80e-01 1.20e-01 4
220
+
1 3 4.35e+00 3.77e+01 6.00e-02 1.20e-02 4
221
+
...
222
+
1 55 1.00e-02 2.00e-01 1.50e-08 1.00e-08 56
223
+
1 56 1.00e-02 2.00e-01 1.50e-08 1.00e-08 57
224
+
211
225
Example: Noisy Objective Evaluation
212
226
-----------------------------------
213
227
As described in :doc:`info`, derivative-free algorithms such as DFO-LS are particularly useful when :code:`objfun` has noise. Let's modify the previous example to include random noise in our objective evaluation, and compare it to a derivative-based solver:
DFO-LS is able to find the solution with only 10 more function evaluations than in the noise-free case. However SciPy's derivative-based solver, which has no trouble solving the noise-free problem, is unable to make any progress.
304
+
DFO-LS is able to find the solution with 20 more function evaluations as in the noise-free case. However SciPy's derivative-based solver, which has no trouble solving the noise-free problem, is unable to make any progress.
291
305
292
306
As noted above, DFO-LS has an input parameter :code:`objfun_has_noise` to indicate if :code:`objfun` has noise in it, which it does in this case. Therefore we can call DFO-LS with
293
307
@@ -300,12 +314,12 @@ Using this setting, we find the correct solution faster:
0 commit comments