A package to analyze and debug LBFGS linesearch behavior inside GRAPE.jl.
It works with either Optim.jl/LBFGS or LBFGSB.jl as a backend.
The package allows to explore how the value of the functional varies both in the direction of the gradient and in the search direction chosen by LBFGS for different step widths, around the step width α chosen by the optimizer
The GRAPELinesearchAnalysis package is not registered. Thus, you can only install it from Github
The package is designed to be used as a callback for a GRAPE.jl optimization. For example,
using GRAPELinesearchAnalysis
opt_result = optimize_grape(
problem,
callback=GRAPELinesearchAnalysis.plot_linesearch(
@__DIR__, samples=100, verbose=false, use_current_backend=false
),
)
will produce a set of images in the folder containing the script, one per iteration.
The plots are generated by temporarily switching to the GR backend, unless use_current_backend=true is given. If verbose=true, is given, information about where the files are written will be printed to screen. The samples is the number of points to sample. Each points requires a full time propagation.
An exemplary generated plot will look like this:
This images analyzes the linesearch in a particular iteration of the optimization. The top panel shows the guess and the optimized pulse for that iteration. The second panel shows the gradient, which would be the update (up to a step width α determined by linesearch) in a first-order gradient ascent. The third panel shows the search direction determined by LBFGS, in comparison to the gradient. The LBFGS search direction is the gradient scaled by the inverse of the approximate Hessian.
The fourth panel shows how the value of the function would change in the direction of the gradient for different step width α.
Finally the bottom panel shows the same information for the LBFGS search direction, in comparison to the gradient. The initial value of the functional (α=0), i.e. J_T for the guess pulse, is indicated by a bullet in the bottom two panels. Also, the value of the functional and the step width α that was LBFGS actually chose are also indicated by a bullet in the bottom panel.