Moment matrix
In mathematics, a moment matrix is a special symmetric square matrix whose rows and columns are indexed by monomials. The entries of the matrix depend on the product of the indexing monomials only (cf. Hankel matrices.)
Moment matrices play an important role in polynomial fitting, polynomial optimization (since positive semidefinite moment matrices correspond to polynomials which are sums of squares)[1] and econometrics.[2]
Application in regression
A multiple linear regression model can be written as
- [math]\displaystyle{ y = \beta_{0} + \beta_{1} x_{1} + \beta_{2} x_{2} + \dots \beta_{k} x_{k} + u }[/math]
where [math]\displaystyle{ y }[/math] is the explained variable, [math]\displaystyle{ x_{1}, x_{2} \dots, x_{k} }[/math] are the explanatory variables, [math]\displaystyle{ u }[/math] is the error, and [math]\displaystyle{ \beta_{0}, \beta_{1} \dots, \beta_{k} }[/math] are unknown coefficients to be estimated. Given observations [math]\displaystyle{ \left\{ y_{i}, x_{1i}, x_{2i}, \dots, x_{ki} \right\}_{i=1}^{n} }[/math], we have a system of [math]\displaystyle{ n }[/math] linear equations that can be expressed in matrix notation.[3]
- [math]\displaystyle{ \begin{bmatrix} y_{1} \\ y_{2} \\ \vdots \\ y_{n} \end{bmatrix} = \begin{bmatrix} 1 & x_{11} & x_{12} & \dots & x_{1k} \\ 1 & x_{21} & x_{22} & \dots & x_{2k} \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 1 & x_{n1} & x_{n2} & \dots & x_{nk} \\ \end{bmatrix} \begin{bmatrix} \beta_{0} \\ \beta_{1} \\ \vdots \\ \beta_{k} \end{bmatrix} + \begin{bmatrix} u_{1} \\ u_{2} \\ \vdots \\ u_{n} \end{bmatrix} }[/math]
or
- [math]\displaystyle{ \mathbf{y} = \mathbf{X} \boldsymbol{\beta} + \mathbf{u} }[/math]
where [math]\displaystyle{ \mathbf{y} }[/math] and [math]\displaystyle{ \mathbf{u} }[/math] are each a vector of dimension [math]\displaystyle{ n \times 1 }[/math], [math]\displaystyle{ \mathbf{X} }[/math] is the design matrix of order [math]\displaystyle{ N \times (k+1) }[/math], and [math]\displaystyle{ \boldsymbol{\beta} }[/math] is a vector of dimension [math]\displaystyle{ (k+1) \times 1 }[/math]. Under the Gauss–Markov assumptions, the best linear unbiased estimator of [math]\displaystyle{ \boldsymbol{\beta} }[/math] is the linear least squares estimator [math]\displaystyle{ \mathbf{b} = \left( \mathbf{X}^{\mathsf{T}} \mathbf{X} \right)^{-1} \mathbf{X}^{\mathsf{T}} \mathbf{y} }[/math], involving the two moment matrices [math]\displaystyle{ \mathbf{X}^{\mathsf{T}} \mathbf{X} }[/math] and [math]\displaystyle{ \mathbf{X}^{\mathsf{T}} \mathbf{y} }[/math] defined as
- [math]\displaystyle{ \mathbf{X}^{\mathsf{T}} \mathbf{X} = \begin{bmatrix} n & \sum x_{i1} & \sum x_{i2} & \dots & \sum x_{ik} \\ \sum x_{i1} & \sum x_{i1}^{2} & \sum x_{i1} x_{i2} & \dots & \sum x_{i1} x_{ik} \\ \sum x_{i2} & \sum x_{i1} x_{i2} & \sum x_{i2}^{2} & \dots & \sum x_{i2} x_{ik} \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ \sum x_{ik} & \sum x_{i1} x_{ik} & \sum x_{i2} x_{ik} & \dots & \sum x_{ik}^{2} \end{bmatrix} }[/math]
and
- [math]\displaystyle{ \mathbf{X}^{\mathsf{T}} \mathbf{y} = \begin{bmatrix} \sum y_{i} \\ \sum x_{i1} y_{i} \\ \vdots \\ \sum x_{ik} y_{i} \end{bmatrix} }[/math]
where [math]\displaystyle{ \mathbf{X}^{\mathsf{T}} \mathbf{X} }[/math] is a square normal matrix of dimension [math]\displaystyle{ (k+1) \times (k+1) }[/math], and [math]\displaystyle{ \mathbf{X}^{\mathsf{T}} \mathbf{y} }[/math] is a vector of dimension [math]\displaystyle{ (k+1 ) \times 1 }[/math].
See also
References
- ↑ Lasserre, Jean-Bernard, 1953- (2010). Moments, positive polynomials and their applications. World Scientific (Firm). London: Imperial College Press. ISBN 978-1-84816-446-8. OCLC 624365972. https://www.worldcat.org/oclc/624365972.
- ↑ Goldberger, Arthur S. (1964). "Classical Linear Regression". Econometric Theory. New York: John Wiley & Sons. pp. 156–212. ISBN 0-471-31101-4. https://books.google.com/books?id=KZq5AAAAIAAJ&pg=PA156.
- ↑ Huang, David S. (1970). Regression and Econometric Methods. New York: John Wiley & Sons. pp. 52–65. ISBN 0-471-41754-8. https://books.google.com/books?id=5IxRAAAAMAAJ&pg=PA52.
External links
- Hazewinkel, Michiel, ed. (2001), "Moment matrix", Encyclopedia of Mathematics, Springer Science+Business Media B.V. / Kluwer Academic Publishers, ISBN 978-1-55608-010-4, https://www.encyclopediaofmath.org/index.php?title=p/m130190
 |  |
