Completeness (statistics)
In statistics, completeness is a property of a statistic in relation to a parameterised model for a set of observed data.
A complete statistic T is one for which any proposed distribution on the domain of T is predicted by one or more prior distributions on the model parameter space. In other words, the model space is 'rich enough' that every possible distribution of T can be explained by some prior distribution on the model parameter space. In contrast, a sufficient statistic T is one for which any two prior distributions will yield different distributions on T. (This last statement assumes that the model space is identifiable, i.e. that there are no 'duplicate' parameter values. This is a minor point.)
Put another way: assume that we have an identifiable model space parameterised by [math]\displaystyle{ \theta }[/math], and a statistic [math]\displaystyle{ T }[/math] (which is effectively just a function of one or more i.i.d. random variables drawn from the model). Then consider the map [math]\displaystyle{ f : p_\theta \mapsto p_{T | \theta} }[/math] which takes each distribution on model parameter [math]\displaystyle{ \theta }[/math] to its induced distribution on statistic [math]\displaystyle{ T }[/math]. The statistic [math]\displaystyle{ T }[/math] is said to be complete when [math]\displaystyle{ f }[/math] is surjective, and sufficient when [math]\displaystyle{ f }[/math] is injective.
Definition
Consider a random variable X whose probability distribution belongs to a parametric model Pθ parametrized by θ.
Say T is a statistic; that is, the composition of a measurable function with a random sample X1,...,Xn.
The statistic T is said to be complete for the distribution of X if, for every measurable function g,:[1]
[math]\displaystyle{ \text{if }\operatorname{E}_\theta(g(T))=0\text{ for all }\theta\text{ then }\mathbf{P}_\theta(g(T)=0)=1\text{ for all }\theta. }[/math]
The statistic T is said to be boundedly complete for the distribution of X if this implication holds for every measurable function g that is also bounded.
Example 1: Bernoulli model
The Bernoulli model admits a complete statistic.[2] Let X be a random sample of size n such that each Xi has the same Bernoulli distribution with parameter p. Let T be the number of 1s observed in the sample, i.e. [math]\displaystyle{ \textstyle T = \sum_{i=1}^n X_i }[/math]. T is a statistic of X which has a binomial distribution with parameters (n,p). If the parameter space for p is (0,1), then T is a complete statistic. To see this, note that
- [math]\displaystyle{ \operatorname{E}_p(g(T)) = \sum_{t=0}^n {g(t){n \choose t}p^{t}(1-p)^{n-t}} = (1-p)^n \sum_{t=0}^n {g(t){n \choose t}\left(\frac{p}{1-p}\right)^t} . }[/math]
Observe also that neither p nor 1 − p can be 0. Hence [math]\displaystyle{ E_p(g(T)) = 0 }[/math] if and only if:
- [math]\displaystyle{ \sum_{t=0}^n g(t){n \choose t}\left(\frac{p}{1-p}\right)^t = 0. }[/math]
On denoting p/(1 − p) by r, one gets:
- [math]\displaystyle{ \sum_{t=0}^n g(t){n \choose t}r^t = 0 . }[/math]
First, observe that the range of r is the positive reals. Also, E(g(T)) is a polynomial in r and, therefore, can only be identical to 0 if all coefficients are 0, that is, g(t) = 0 for all t.
It is important to notice that the result that all coefficients must be 0 was obtained because of the range of r. Had the parameter space been finite and with a number of elements less than or equal to n, it might be possible to solve the linear equations in g(t) obtained by substituting the values of r and get solutions different from 0. For example, if n = 1 and the parameter space is {0.5}, a single observation and a single parameter value, T is not complete. Observe that, with the definition:
- [math]\displaystyle{ g(t) = 2(t-0.5), \, }[/math]
then, E(g(T)) = 0 although g(t) is not 0 for t = 0 nor for t = 1.
Relation to sufficient statistics
For some parametric families, a complete sufficient statistic does not exist (for example, see Galili and Meilijson 2016 [3]).
For example, if you take a sample sized n > 2 from a N(θ,θ2) distribution, then [math]\displaystyle{ \left(\sum_{i=1}^n X_i, \sum_{i=1}^n X_i^2\right) }[/math] is a minimal sufficient statistic and is a function of any other minimal sufficient statistic, but [math]\displaystyle{ 2\left(\sum_{i=1}^n X_i\right)^2-(n+1)\sum_{i=1}^nX_i^2 }[/math] has an expectation of 0 for all θ, so there cannot be a complete statistic.
If there is a minimal sufficient statistic then any complete sufficient statistic is also minimal sufficient. But there are pathological cases where a minimal sufficient statistic does not exist even if a complete statistic does.
Importance of completeness
The notion of completeness has many applications in statistics, particularly in the following two theorems of mathematical statistics.
Lehmann–Scheffé theorem
Completeness occurs in the Lehmann–Scheffé theorem,[4] which states that if a statistic that is unbiased, complete and sufficient for some parameter θ, then it is the best mean-unbiased estimator for θ. In other words, this statistic has a smaller expected loss for any convex loss function; in many practical applications with the squared loss-function, it has a smaller mean squared error among any estimators with the same expected value.
Examples exists that when the minimal sufficient statistic is not complete then several alternative statistics exist for unbiased estimation of θ, while some of them have lower variance than others.[5]
See also minimum-variance unbiased estimator.
Basu's theorem
Bounded completeness occurs in Basu's theorem,[6] which states that a statistic that is both boundedly complete and sufficient is independent of any ancillary statistic.
Bahadur's theorem
Bounded completeness also occurs in Bahadur's theorem. In the case where there exists at least one minimal sufficient statistic, a statistic which is sufficient and boundedly complete, is necessarily minimal sufficient. Another form of Bahadur's theorem states that any sufficient and boundedly complete statistic over a finite-dimensional coordinate space is also minimal sufficient.[7]
Notes
- ↑ Young, G. A. and Smith, R. L. (2005). Essentials of Statistical Inference. (p. 94). Cambridge University Press.
- ↑ Casella, G. and Berger, R. L. (2001). Statistical Inference. (pp. 285–286). Duxbury Press.
- ↑ Tal Galili; Isaac Meilijson (31 Mar 2016). "An Example of an Improvable Rao–Blackwell Improvement, Inefficient Maximum Likelihood Estimator, and Unbiased Generalized Bayes Estimator". The American Statistician 70 (1): 108–113. doi:10.1080/00031305.2015.1100683. PMID 27499547.
- ↑ Casella, George; Berger, Roger L. (2001). Statistical Inference (2nd ed.). Duxbury Press. ISBN 978-0534243128.
- ↑ Tal Galili; Isaac Meilijson (31 Mar 2016). "An Example of an Improvable Rao–Blackwell Improvement, Inefficient Maximum Likelihood Estimator, and Unbiased Generalized Bayes Estimator". The American Statistician 70 (1): 108–113. doi:10.1080/00031305.2015.1100683. PMID 27499547.
- ↑ Casella, G. and Berger, R. L. (2001). Statistical Inference. (pp. 287). Duxbury Press.
- ↑ "Statistical Inference Lecture Notes". July 7, 2022. https://www.math.kth.se/matstat/gru/Statistical%20inference/Lecture5.pdf.
References
- Basu, D. (1988). J. K. Ghosh. ed. Statistical information and likelihood : A collection of critical essays by Dr. D. Basu. Lecture Notes in Statistics. 45. Springer. ISBN 978-0-387-96751-6.
- {{Cite book
- E. L., Lehmann; Romano, Joseph P. (2005). Testing statistical hypotheses. Springer Texts in Statistics (Third ed.). New York: Springer. pp. xiv+784. ISBN 978-0-387-98864-1. http://www.springerlink.com/content/978-0-387-98864-1#section=545952&page=1.
- "Completeness, similar regions, and unbiased estimation. I.". Sankhyā: the Indian Journal of Statistics 10 (4): 305–340. 1950. doi:10.1007/978-1-4614-1412-4_23.
- "Completeness, similar regions, and unbiased estimation. II". Sankhyā: The Indian Journal of Statistics 15 (3): 219–236. 1955. doi:10.1007/978-1-4614-1412-4_24.
 |  |
