We present a selection criterion for the Euclidean metric adapted during warmup in a Hamiltonian Monte Carlo sampler that makes it possible for a sampler to automatically pick the metric based on the model and the availability of warmup draws. Additionally, we present a new adaptation inspired by the selection criterion that requires significantly fewer warmup draws to be effective. The effectiveness of the selection criterion and adaptation are demonstrated on a number of applied problems. An implementation for the Stan probabilistic programming language is provided.
And here’s their conclusion:
Adapting an effective metric is important for the performance of HMC. This paper outlines a criterion that can be used to automate the selection of an efficient metric from an array of options. In addition, we present a new low-rank adaptation scheme that makes it possible to sample effectively from highly correlated posteriors, even when few warmup draws are available. The selection criterion and the new adaptation are demonstrated to be effective on a number of different models.
All of the necessary eigenvalues and eigenvectors needed to evaluate the selection criterion and build the new adaptation can be computed efficiently with the Lanczos algorithm, making this method suitable for models with large numbers of parameters.
This research looks like it will have a big practical impact.