We continue to investigate inverse problems using Bayesian inference, still in the nonlinear case, but now for problems of very high-dimensionally, i.e. our uncertain input \(\alpha\) has dimension \(M \gg 10\). In these cases McMC will generally converge poorly (although this is an active area of research), and an alternative approach is needed. In particular we look at so-called variational approaches, in which we attempt to identify the maximum a posteriori (MAP) estimate of \(\alpha\).

The result is an optimization problem in a high-dimensional parameter space, with the posterior pdf as a cost-function. We consider gradient-based optimization methods, and spend most of this section examining how to efficiently find derivatives of the posterior in high-dimensions. The main challenge is the presence of the simulation code, and the need to obtain derivatives of it. After some unsatisfactory options are discussed, we introduce Lagrange multiplier, or adjoint approaches, which give us \(M\)-independent algorithms.

In Tutorial 5 we apply this method to the same inverse Poisson problem covered in Tutorial 4, but this time with the uncertain diffusion coefficient \(\alpha\) defined at every mesh point. This increases the dimensionality of the problem to the extent that McMC is no longer effective, whereas optimization to find the MAP estimate works quickly.

Smith does not cover this topic in detail, however talks about obtaining derivatives of simulation codes, and adjoints in particular in Chapter 14. He uses it for the purposes of "local sensitivity analysis" - i.e. assessing which input parameters are important based on their derivatives. Smith's treatment of adjoint is very different, covering the continuous adjoint, rather than the discrete (as I do). As such in this part of the course, Smith is very much complementary as opposed to supportive of the material.