G-prior

In statistics, the g-prior is an objective prior for the regression coefficients of a multiple regression. It was introduced by Arnold Zellner. It is a key tool in Bayes and empirical Bayes variable selection.

Definition

Consider a data set $(x_1,y_1),\dots ,(x_n,y_n)$, where the $x_i$ are Euclidean vectors and the $y_i$ are scalars. The multiple regression model is formulated as

$y_i=x_i^{\mathrm{\top }}\beta +{\epsilon }_i.$

where the ${\epsilon }_i$ are random errors. Zellner's g-prior for $\beta$ is a multivariate normal distribution with covariance matrix proportional to the inverse Fisher information matrix for $\beta$, similar to a Jeffreys prior.

Assume the ${\epsilon }_i$ are i.i.d. normal with zero mean and variance ${\psi }^{-1}$. Let $X$ be the matrix with $i$th row equal to $x_i^{\mathrm{\top }}$. Then the g-prior for $\beta$ is the multivariate normal distribution with prior mean a hyperparameter ${\beta }_0$ and covariance matrix proportional to ${\psi }^{-1}(X^{\mathrm{\top }}X{)}^{-1}$, i.e.,

$\beta |\psi \sim \text{N}[{\beta }_0,g{\psi }^{-1}(X^{\mathrm{\top }}X{)}^{-1}].$

where g is a positive scalar parameter.

Posterior distribution of beta

The posterior distribution of $\beta$ is given as

$\beta |\psi ,x,y\sim \text{N}[q\hat{\beta }+(1-q){\beta }_0,\frac{q}{\psi }(X^{\mathrm{\top }}X{)}^{-1}].$

where $q=g/(1+g)$ and

$\hat{\beta }=(X^{\mathrm{\top }}X{)}^{-1}{X}^{\mathrm{\top }}y.$

is the maximum likelihood (least squares) estimator of $\beta$. The vector of regression coefficients $\beta$ can be estimated by its posterior mean under the g-prior, i.e., as the weighted average of the maximum likelihood estimator and ${\beta }_0$,

$\tilde{\beta }=q\hat{\beta }+(1-q){\beta }_0.$

Clearly, as g →∞, the posterior mean converges to the maximum likelihood estimator.

Selection of g

Estimation of g is slightly less straightforward than estimation of $\beta$. A variety of methods have been proposed, including Bayes and empirical Bayes estimators.

Further reading

• Datta, Jyotishka; Ghosh, Jayanta K. (2015). "In Search of Optimal Objective Priors for Model Selection and Estimation". in Upadhyay, Satyanshu Kumar; Singh, Umesh; Dey, Dipak K. et al.. Current Trends in Bayesian Methodology with Applications. CRC Press. pp. 225–243. ISBN 978-1-4822-3511-1. 
• Marin, Jean-Michel; Robert, Christian P. (2007). "Regression and Variable Selection". Bayesian Core : A Practical Approach to Computational Bayesian Statistics. New York: Springer. pp. 47–84. doi:10.1007/978-0-387-38983-7_3. ISBN 978-0-387-38979-0.