In this part we take our first look at the inverse problem, namely: Given measurements at the output of a simulation code, how can I determine the inputs? Such inverse problems are ill-posed by default, so any numerical method must address the fact that there may be multiple valid solutions (usually infinitely many), or no (exact) solutions at all. We resolve such issues by using a stochastic framework, and asking: What probability distributions on the inputs are consistent with the (noisy) measurement of the outputs? This framework always gives a unique (probabilistic) solution to the problem.
To understand and use this framework, we need the ideas of conditional probability and Bayes' theorem. After giving the fundamentals, we introduce the Bayesian approach with several examples (Videos E-F, H and I). Each example requires some modelling: choices must be made for the prior and the statistical model.
Finally we introduce Gaussian processes, and use Bayes as a way of constructing a Gaussian process that regresses data points. In Tutorial 3 you'll implement and investigate the method.
In Smith, see the referenced sections below each video; but also Chapter 6 is a nice summary of alternative (i.e. non-Bayesian) approaches to the inverse problem.
1:40 - I should say "the variance of the first variable".
3:38 - Note that zero correlation (\(\rho = 0\)) implies independence of the random-variables in this setting (of multivariate Gaussians), but more generally it is possible to have dependent random-variables with \(\rho = 0\), notably when their dependence is nonlinear.
4:40 - It may also be interesting to ask yourself what happens if the covariance matrix is positive semi-definite.
13:58 - Should read \(\sigma_1^2 - \sigma_{12} \sigma_2^{-2} \sigma_{12}\).
14:24 - The minus should be a plus: \(\mu_{1\mid 2} = \mu_1 + \Sigma_{12} \Sigma_{22}^{-1} (x_2 - \mu_2)\).
11:10 - Usually I point out that the distribution looks increasingly Gaussian as more input uncertainties are added - and I hint that it might be due to the central-limit theorem :-)
