Robustness of Metropolis-Hastings algorithms

• Applicant: Professor Dr. Daniel Rudolf
• Subject Area: Mathematics
• Term: from 2017 to 2022
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 389483880

• Final Report Year: 2022

Final Report Abstract

Metropolis-Hastings (MH) algorithms, and Markov chain methods in general, are probably the most widely used methods for the approximate simulation of target distributions. Robustness properties regarding evaluation errors and concentration properties of the density determining the target distribution are therefore of particular interest. Evaluation errors always occur when the target density can only be approximated. By means of our further developed perturbation theory of Markov chains, we have proposed and investigated a new, modified form of the Monte Carlo within Metropolis (MCwM) approach in the case of having a non-evaluable density. This is based on an adapted restriction of the state space. Our theoretical results have shown that this implies new error estimates and minimal cost bounds (to obtain a certain small error). A stability analysis for a class of target distributions, namely those determined by doubly-intractable densities, also allows us to consider other Markov chain methods. In Bayesian inference when analysing data that contain particularly large amounts of parameter information, very concentrated posterior distributions occur. The high concentration is accompanied by a slow exploration of the state space in classical MH algorithms. We propose MH algorithms whose proposal kernels are adapted to the covariance of the target measure and introduce terminology to characterise concentration-robust behaviour. Using this terminology, and new techniques of proof, the proposed algorithms have been analysed for Gaussian and beyond target measures under regularity assumptions. Unexpectedly and surprisingly, the spectral gap of the transition operators of the proposed MH algorithms in the Gaussian case is independent of the level of concentration.

Publications

• On a Metropolis-Hastings importance sampling estimator, Electron. J. Stat., 14 (2020), 857-889
  D. Rudolf und B. Sprungk
  (See online at https://doi.org/10.1214/20-EJS1680)
• On the convergence of the Laplace approximation and noise-level-robustness of Laplace-based Monte Carlo methods for Bayesian inverse problems, Num. Math., 145 (2020), 915-971
  C. Schillings, B. Sprungk und P. Wacker
  (See online at https://doi.org/10.1007/s00211-020-01131-1)
• Perturbation bounds for Monte Carlo within Metropolis via restricted approximations, Stoch. Process. Their Appl., 130 (2020), 2200-2227
  F. Medina-Aguayo, D. Rudolf, N. Schweizer
  (See online at https://doi.org/10.1016/j.spa.2019.06.015)
• Stability of doubly-intractable distributions, Electron. Commun. Probab., 25 (2020), 1-13
  M. Habeck, D. Rudolf und B. Sprungk
  (See online at https://doi.org/10.1214/20-ECP341)
• Geometric convergence of elliptical slice sampling, Proceedings of the 38th ICML, PMLR, 139 (2021), 7969-7978
  V. Natarovskii, D. Rudolf und B. Sprungk
  (See online at https://doi.org/10.48550/arXiv.2105.03308)
• Robust random walk-like Metropolis-Hastings algorithms for concentrating posteriors
  D. Rudolf und B. Sprungk
  (See online at https://doi.org/10.48550/arXiv.2202.12127)