View source for U-statistic

{{Short description|Class of statistics in estimation theory}}
In [[Statistical theory|statistical theory]], a '''U-statistic''' is a class of statistics defined as the average over the application of a given function applied to all tuples of a fixed size. The letter "U" stands for unbiased.   In elementary statistics, U-statistics arise naturally in producing [[Minimum-variance unbiased estimator|minimum-variance unbiased estimator]]s.

The theory of U-statistics allows a [[Minimum-variance unbiased estimator|minimum-variance unbiased estimator]] to be derived from each unbiased estimator of an ''estimable parameter'' (alternatively, ''statistical [[Functional (mathematics)|functional]]'') for large classes of [[Probability distribution|probability distribution]]s.<ref>Cox & Hinkley (1974), p.&nbsp;200, p.&nbsp;258</ref><ref>Hoeffding (1948), between Eq's(4.3),(4.4)</ref> An estimable parameter is a [[Measurable function|measurable function]] of the population's [[Cumulative distribution function|cumulative probability distribution]]: For example, for every probability distribution, the population median is an estimable parameter. The theory of U-statistics applies to general classes of probability distributions.

==History==
Many statistics originally derived for particular parametric families have been recognized as U-statistics for general distributions. In non-parametric statistics, the theory of U-statistics is used to establish for statistical procedures (such as estimators and tests) and estimators relating to the asymptotic normality and to the variance (in finite samples) of such quantities.<ref>Sen (1992)</ref> The theory has been used to study more general statistics as well as [[Stochastic process|stochastic process]]es, such as [[Random graph|random graph]]s.<ref>Page 508 in {{cite book | last1=Koroljuk | first1=V. S. | last2=Borovskich | first2=Yu. V. | title=Theory of ''U''-statistics | edition=Translated by P. V. Malyshev and D. V. Malyshev from the 1989 Russian original | series=Mathematics and its Applications | volume=273 | publisher= Kluwer Academic Publishers Group | location=Dordrecht | year=1994 | pages=x+552 | isbn=0-7923-2608-3 | mr= 1472486 }}</ref><ref>Pages 381–382 in {{cite book  |last=Borovskikh | first=Yu. V. | title=''U''-statistics in Banach spaces | publisher=VSP | location=Utrecht | year=1996 | pages=xii+420 | isbn=90-6764-200-2 | mr=1419498 }}</ref><ref>Page xii in {{cite book  | last1=Kwapień | first1=Stanisƚaw | last2=Woyczyński | first2=Wojbor A. | title=Random series and stochastic integrals: Single and multiple |  series=Probability and its Applications | publisher=Birkhäuser Boston, Inc.| location=Boston, MA | year=1992 | pages=xvi+360 | isbn=0-8176-3572-6 | mr=1167198 }}</ref>

Suppose that a problem involves independent and identically-distributed random variables and that estimation of a certain parameter is required. Suppose that a simple unbiased estimate can be constructed based on only a few observations: this defines the basic estimator based on a given number of observations. For example, a single observation is itself an unbiased estimate of the mean and a pair of observations can be used to derive an unbiased estimate of the variance. The U-statistic based on this estimator is defined as the average (across all combinatorial selections of the given size from the full set of observations) of the basic estimator applied to the sub-samples.

[[Biography:Pranab K. Sen|Pranab K. Sen]] (1992) provides a review of the paper by [[Biography:Wassily Hoeffding|Wassily Hoeffding]] (1948), which introduced U-statistics and set out the theory relating to them, and in doing so Sen outlines the importance U-statistics have in statistical theory. Sen says,<ref>Sen (1992) p.&nbsp;307</ref> “The impact of Hoeffding (1948) is overwhelming at the present time and is very likely to continue in the years to come.” Note that the theory of U-statistics is not limited to<ref>Sen (1992), p306</ref> the case of independent and identically-distributed random variables or to scalar random-variables.<ref>Borovskikh's last chapter discusses U-statistics for [[Exchangeable random variables|exchangeable]] random elements taking values in a [[Vector space|vector space]] ([[Separable space|separable]] [[Banach space]]).</ref>

==Definition==
The term U-statistic, due to Hoeffding (1948), is defined as follows.

Let <math>K</math> be either the real or complex numbers, and let <math>f\colon (K^d)^r\to K</math> be a <math>K</math>-valued function of <math>r</math> <math>d</math>-dimensional variables.
For each <math>n\ge r</math> the associated U-statistic <math>f_n\colon (K^d)^n \to K</math> is defined to be the average of the values <math>f(x_{i_1}, \dotsc, x_{i_r})</math> over the set <math>I_{r, n}</math> of <math>r</math>-tuples of indices from <math>\{1, 2, \dotsc, n\}</math> with distinct entries.
Formally,
:<math>f_n(x_1,\dotsc, x_n) = \frac{1}{\prod_{i = 0}^{r - 1} (n - i)} \sum_{(i_1, \dotsc, i_r) \in I_{r, n}} f(x_{i_1},\dotsc, x_{i_r})</math>.
In particular, if <math>f</math> is symmetric the above is simplified to
:<math>f_n(x_1, \dotsc, x_n) = \frac{1}{\binom{n}{r}} \sum_{(i_1, \dotsc, i_r) \in J_{r, n}} f(x_{i_1}, \dotsc, x_{i_r})</math>,
where now <math>J_{r, n}</math> denotes the subset of <math>I_{r, n}</math> of ''increasing'' tuples.

Each U-statistic <math>f_n</math> is necessarily a [[Symmetric function|symmetric function]].

U-statistics are very natural in statistical work, particularly in Hoeffding's context of [[Independent and identically distributed random variables|independent and identically distributed random variables]], or more generally for exchangeable sequences, such as in simple random sampling from a finite population, where the defining property is termed ‘inheritance on the average’.

Fisher's ''k''-statistics and Tukey's [[Polykay|polykay]]s are examples of [[Homogeneous polynomial|homogeneous polynomial]] U-statistics (Fisher, 1929; Tukey, 1950).

For a simple random sample ''&phi;'' of size&nbsp;''n'' taken from a population of size&nbsp;''N'', the U-statistic has the property that the average over sample values&nbsp;''&fnof;''<sub>''n''</sub>(''x&phi;'') is exactly equal to the population value&nbsp;''&fnof;''<sub>''N''</sub>(''x'').{{Clarify|date=June 2022|reason=For lack of precision of the premises it is not possible to formalize this statement.}}

==Examples==

Some examples:
If <math>f(x) = x</math> the U-statistic <math>f_n(x) = \bar x_n = (x_1 + \cdots + x_n)/n</math> is the sample mean.

If <math>f(x_1, x_2) = |x_1 - x_2|</math>, the U-statistic is the mean pairwise deviation
<math>f_n(x_1,\ldots, x_n) = 2 / (n(n-1))\sum_{i> j} |x_i - x_j| </math>, defined for <math>n\ge 2</math>.

If <math>f(x_1, x_2) = (x_1 - x_2)^2/2</math>, the U-statistic is the [[Sample variance|sample variance]] 
<math>f_n(x) = \sum(x_i - \bar x_n)^2/(n-1)</math>
with divisor <math>n-1</math>, defined for <math>n\ge 2</math>.

The third <math>k</math>-statistic <math>k_{3,n}(x) = \sum(x_i - \bar x_n)^3 n/((n-1)(n-2))</math>,
the sample [[Skewness|skewness]] defined for <math>n\ge 3</math>,
is a U-statistic.

The following case highlights an important point. If <math>f(x_1, x_2, x_3)</math> is the [[Median|median]] of three values, <math>f_n(x_1,\ldots, x_n)</math> is not the median of <math>n</math> values. However, it is a minimum variance unbiased estimate of the expected value of the median of three values, not the median of the population. Similar estimates play a central role where the parameters of a family of [[Probability distribution|probability distribution]]s are being estimated by probability weighted moments or L-moments.

==See also==
* [[V-statistic]]

==Notes==
{{reflist}}

==References==

* {{cite book  |last=Borovskikh | first=Yu. V. | title=''U''-statistics in Banach spaces | publisher=VSP | location=Utrecht | year=1996 | pages=xii+420 | isbn=90-6764-200-2 | mr=1419498 }}
* Cox, D. R., Hinkley, D. V. (1974) ''Theoretical statistics''. Chapman and Hall. {{ISBN|0-412-12420-3}}
* [[Biography:Ronald Fisher|Fisher, R. A.]] (1929) Moments and product moments of sampling distributions.  ''Proceedings of the London Mathematical Society'', 2, 30:199&ndash;238.
* [[Biography:Wassily Hoeffding|Hoeffding, W.]] (1948) A class of statistics with asymptotically normal distributions. [[Annals of Statistics]], 19:293–325. (Partially reprinted in: Kotz, S., Johnson, N. L. (1992) ''Breakthroughs in Statistics'', Vol I, pp 308–334. Springer-Verlag. {{ISBN|0-387-94037-5}})
* {{cite book | last1=Koroljuk | first1=V. S. | last2=Borovskich | first2=Yu. V. | title=Theory of ''U''-statistics | edition=Translated by P. V. Malyshev and D. V. Malyshev from the 1989 Russian original | series=Mathematics and its Applications | volume=273 | publisher= Kluwer Academic Publishers Group | location=Dordrecht | year=1994 | pages=x+552 | isbn=0-7923-2608-3 | mr= 1472486 }}
* Lee, A. J. (1990) ''U-Statistics: Theory and Practice''. Marcel Dekker, New York. pp320 {{ISBN|0-8247-8253-4}}
* [[Biography:Pranab K. Sen|Sen, P. K.]] (1992) Introduction to Hoeffding (1948) A Class of Statistics with Asymptotically Normal Distribution. In: Kotz, S., Johnson, N. L. ''Breakthroughs in Statistics'', Vol I, pp 299–307. Springer-Verlag. {{ISBN|0-387-94037-5}}.
* {{cite book|last=Serfling|first=Robert J.|year=1980|title=Approximation theorems of mathematical statistics|url=https://archive.org/details/approximationthe0000serf|url-access=registration|location=New York|publisher=John Wiley and Sons|isbn=0-471-02403-1}}
* {{cite journal |last=Tukey |first=J. W. |year=1950 |title=Some Sampling Simplified |journal=[[Organization:Journal of the American Statistical Association|Journal of the American Statistical Association]] |volume=45 |issue=252 |pages=501–519 |doi=10.1080/01621459.1950.10501142 }}
* {{cite journal |last=Halmos |first=P. |year=1946 |title=The Theory of Unbiased Estimation |journal=[[Annals of Mathematical Statistics]] |volume=1 |issue=17 |pages=34–43 |doi=10.1214/aoms/1177731020 |doi-access=free }}

{{Statistics|inference}}

{{DEFAULTSORT:U-Statistic}}
[[Category:Estimation theory]]
[[Category:Nonparametric statistics]]
[[Category:Asymptotic theory (statistics)]]
[[Category:U-statistics]]

{{Sourceattribution|U-statistic}}