Cantelli's inequality
In probability theory, Cantelli's inequality (also called the Chebyshev-Cantelli inequality and the one-sided Chebyshev inequality) is an improved version of Chebyshev's inequality for one-sided tail bounds.[1][2][3] The inequality states that, for [math]\displaystyle{ \lambda \gt 0, }[/math]
- [math]\displaystyle{ \Pr(X-\mathbb{E}[X]\ge\lambda) \le \frac{\sigma^2}{\sigma^2 + \lambda^2}, }[/math]
where
- [math]\displaystyle{ X }[/math] is a real-valued random variable,
- [math]\displaystyle{ \Pr }[/math] is the probability measure,
- [math]\displaystyle{ \mathbb{E}[X] }[/math] is the expected value of [math]\displaystyle{ X }[/math],
- [math]\displaystyle{ \sigma^2 }[/math] is the variance of [math]\displaystyle{ X }[/math].
Applying the Cantelli inequality to [math]\displaystyle{ -X }[/math] gives a bound on the lower tail,
- [math]\displaystyle{ \Pr(X-\mathbb{E}[X]\le -\lambda) \le \frac{\sigma^2}{\sigma^2 + \lambda^2}. }[/math]
While the inequality is often attributed to Francesco Paolo Cantelli who published it in 1928,[4] it originates in Chebyshev's work of 1874.[5] When bounding the event random variable deviates from its mean in only one direction (positive or negative), Cantelli's inequality gives an improvement over Chebyshev's inequality. The Chebyshev inequality has "higher moments versions" and "vector versions", and so does the Cantelli inequality.
Comparison to Chebyshev's inequality
For one-sided tail bounds, Cantelli's inequality is better, since Chebyshev's inequality can only get
- [math]\displaystyle{ \Pr(X - \mathbb{E}[X] \geq \lambda) \leq \Pr(|X-\mathbb{E}[X]|\ge\lambda) \le \frac{\sigma^2}{\lambda^2}. }[/math]
On the other hand, for two-sided tail bounds, Cantelli's inequality gives
- [math]\displaystyle{ \Pr(|X-\mathbb{E}[X]|\ge\lambda) = \Pr(X-\mathbb{E}[X]\ge\lambda) + \Pr(X-\mathbb{E}[X]\le-\lambda) \le \frac{2\sigma^2}{\sigma^2 + \lambda^2}, }[/math]
which is always worse than Chebyshev's inequality (when [math]\displaystyle{ \lambda \geq \sigma }[/math]; otherwise, both inequalities bound a probability by a value greater than one, and so are trivial).
Proof
Let [math]\displaystyle{ X }[/math] be a real-valued random variable with finite variance [math]\displaystyle{ \sigma^2 }[/math] and expectation [math]\displaystyle{ \mu }[/math], and define [math]\displaystyle{ Y = X - \mathbb{E}[X] }[/math] (so that [math]\displaystyle{ \mathbb{E}[Y] = 0 }[/math] and [math]\displaystyle{ \operatorname{Var}(Y) = \sigma^2 }[/math]).
Then, for any [math]\displaystyle{ u\geq 0 }[/math], we have
- [math]\displaystyle{ \Pr( X-\mathbb{E}[X]\geq\lambda) = \Pr( Y \geq \lambda) = \Pr( Y + u \geq \lambda + u) \leq \Pr( (Y + u)^2 \geq (\lambda + u)^2 ) \leq \frac{\mathbb{E}[(Y + u)^2] }{(\lambda + u)^2} = \frac{\sigma^2 + u^2 }{(\lambda + u)^2}. }[/math]
the last inequality being a consequence of Markov's inequality. As the above holds for any choice of [math]\displaystyle{ u\in\mathbb{R} }[/math], we can choose to apply it with the value that minimizes the function [math]\displaystyle{ u \geq 0 \mapsto \frac{\sigma^2 + u^2 }{(\lambda + u)^2} }[/math]. By differentiating, this can be seen to be [math]\displaystyle{ u_\ast = \frac{\sigma^2}{\lambda} }[/math], leading to
- [math]\displaystyle{ \Pr( X-\mathbb{E}[X] \geq\lambda) \leq \frac{\sigma^2 + u_\ast^2 }{(\lambda + u_\ast)^2} = \frac{\sigma^2}{\lambda^2 + \sigma^2} }[/math] if [math]\displaystyle{ \lambda \gt 0 }[/math]
Generalizations
Various stronger inequalities can be shown. He, Zhang, and Zhang showed[6] (Corollary 2.3) when [math]\displaystyle{ \mathbb{E}[X]=0,\,\mathbb{E}[X^2]=1 }[/math] and [math]\displaystyle{ \lambda\ge0 }[/math]:
- [math]\displaystyle{ \Pr(X\ge\lambda) \le 1- (2\sqrt{3}-3)\frac{(1+\lambda^2)^2}{\mathbb{E}[X^4]+6\lambda^2+\lambda^4}. }[/math]
In the case [math]\displaystyle{ \lambda=0 }[/math] this matches a bound in Berger's "The Fourth Moment Method",[7]
- [math]\displaystyle{ \Pr(X\ge 0) \ge \frac{2\sqrt{3}-3}{\mathbb{E}[X^4]}. }[/math]
This improves over Cantelli's inequality in that we can get a non-zero lower bound, even when [math]\displaystyle{ \mathbb{E}[X]=0 }[/math].
See also
References
- ↑ Boucheron, Stéphane (2013). Concentration inequalities : a nonasymptotic theory of independence. Gábor Lugosi, Pascal Massart. Oxford: Oxford University Press. ISBN 978-0-19-953525-5. OCLC 829910957. https://www.worldcat.org/oclc/829910957.
- ↑ "Tail and Concentration Inequalities" by Hung Q. Ngo
- ↑ "Concentration-of-measure inequalities" by Gábor Lugosi
- ↑ Cantelli, F. P. (1928), "Sui confini della probabilita," Atti del Congresso Internazional del Matematici, Bologna, 6, 47-5
- ↑ Ghosh, B.K., 2002. Probability inequalities related to Markov's theorem. The American Statistician, 56(3), pp.186-190
- ↑ He, S.; Zhang, J.; Zhang, S. (2010). "Bounding probability of small deviation: A fourth moment approach". Mathematics of Operations Research 35 (1): 208–232. doi:10.1287/moor.1090.0438.
- ↑ Berger, Bonnie (August 1997). "The Fourth Moment Method" (in en). SIAM Journal on Computing 26 (4): 1188–1207. doi:10.1137/S0097539792240005. ISSN 0097-5397. http://epubs.siam.org/doi/10.1137/S0097539792240005.
 |  |
