Horseshoe.png

Shrinkage priors

Posterior contraction theorems, results on uncertainty quantification, variable selection and more for shrinkage priors, in particular for the popular horseshoe prior.

See also Software for information about the horseshoe R package.

Summary

Shrinkage priors offer a Bayesian solution to 'needle-in-a-haystack' type problems, where we expect only a few signals amidst a lot of noise. We study these priors from a Bayesian point of view: if there is a true sparse vector underlying the data-generating mechanism, will we be able to estimate it accurately? Can we reliably separate the signal from the noise? What is the probability that the true vector is captured by a credible set derived from the posterior, and how big will that credible set be?

 

The horseshoe prior, introduced by Carvalho, Polson and Scott (2010), is a popular shrinkage prior with excellent empirical results. Its density on any individual coordinate combines a a pole at zero, representing our expectation that the coordinate could be zero (noise), with heavy tails, which prevents too strong shrinkage if the coordinate is nonzero (a signal). In the papers listed below, we study theoretical properties of the horseshoe prior in the normal means model. We show that the horseshoe has many desirable theoretical properties, and in essence behaves just as well as discrete mixture priors (spike-and-slab). From our theoretical results, we derive guidelines for practice, which are implemented in the R software package horseshoe (see Software). 

Going beyond the horseshoe, in Van der Pas, Salomond and Schmidt-Hieber (2017) we provide conditions on shrinkage priors under which posterior contraction at the minimax rate is guaranteed. In the book chapter Van der Pas (2021), results from the papers below and the broader literature on theoretical properties of continuous shrinkage priors are synthesized.

Related papers

Van der Pas, S. (2021). Theoretical guarantees for the horseshoe and other global-local shrinkage priors. In Handbook of Bayesian Variable Selection (pp. 133-160). Chapman and Hall/CRC. [link]

Van der Pas, S., Szabó, B., & van der Vaart, A. (2017). Uncertainty quantification for the horseshoe (with discussion). Bayesian Analysis, 12(4), 1221-1274. [link]

Van der Pas, S., Szabó, B., & van der Vaart, A. (2017). Adaptive posterior contraction rates for the horseshoe. Electronic Journal of Statistics, 11(2), 3196-3225. [link]

Van der Pas, S. L., Salomond, J. B., & Schmidt-Hieber, J. (2016). Conditions for posterior contraction in the sparse normal means problem. Electronic Journal of Statistics, 10(1), 976-1000. [link]

Van der Pas, S. L., Kleijn, B. J., & Van Der Vaart, A. W. (2014). The horseshoe estimator: Posterior concentration around nearly black vectors. Electronic Journal of Statistics, 8(2), 2585-2618. [link]