Using preconditioners in the adjoints of a linear solve

[ChrisRackauckas]

From @avik-pal #449

Preconditioning. These need to be added to the Adjoint Sensitivity Struct. Can we use a left preconditioner for the forward problem as the transpose right preconditioner? Need someone with a better grasp of linear algebra to chime in here

@amontoison I think some Krylov methods only support pre-conditioners on one side though?

Maybe we need some way for someone to specify adjoint preconditioners?

[amontoison]

Hi @ChrisRackauckas, Krylov methods dedicated to symmetric linear systems only support centered preconditioning (special case of left + right preconditioning) to maintain the symmetry.
But it should not be an issue for the adjoint sensitivity because A = A', so you can reuse the same preconditioner.

If the system is not symmetric, the Krylov method supports both left and right preconditioning.

For the transpose of the preconditioner, I expect the user to provide adjoint(P) as input of the solver.
But it's not always easy to do that when P is a matrix-free preconditioner.