Shrinkage priors

Shrinkage priors offer a Bayesian solution to 'needle-in-a-haystack' type problems, where we expect only a few signals amidst a lot of noise. We study these priors from a Bayesian point of view: if there is a true sparse vector underlying the data-generating mechanism, will we be able to estimate it accurately? Can we reliably separate the signal from the noise? What is the probability that the true vector is captured by a credible set derived from the posterior, and how big will that credible set be?

The horseshoe prior, introduced by Carvalho, Polson and Scott (2010), is a popular shrinkage prior with excellent empirical results. Its density on any individual coordinate combines a a pole at zero, representing our expectation that the coordinate could be zero (noise), with heavy tails, which prevents too strong shrinkage if the coordinate is nonzero (a signal). In the papers listed below, we study theoretical properties of the horseshoe prior in the normal means model. We show that the horseshoe has many desirable theoretical properties, and in essence behaves just as well as discrete mixture priors (spike-and-slab). From our theoretical results, we derive guidelines for practice, which are implemented in the R software package horseshoe (see Software). 

​

Going beyond the horseshoe, in Van der Pas, Salomond and Schmidt-Hieber (2017) we provide conditions on shrinkage priors under which posterior contraction at the minimax rate is guaranteed. In the book chapter Van der Pas (2021), results from the papers below and the broader literature on theoretical properties of continuous shrinkage priors are synthesized.