View source for Information source (mathematics)

In [[Mathematics|mathematics]], an '''information source''' is a sequence of [[Random variable|random variable]]s ranging over a finite alphabet &Gamma;, having a [[Stationary distribution|stationary distribution]].
The '''uncertainty''', or '''[[Entropy rate|entropy rate]]''', of an information source is defined as

:<math>H\{\mathbf{X}\} = \lim_{n\to\infty} H(X_n | X_0, X_1, \dots, X_{n-1})</math>

where

:<math> X_0, X_1, \dots, X_n</math>

is the sequence of random variables defining the information source, and

:<math>H(X_n | X_0, X_1, \dots, X_{n-1})</math>

is the conditional information entropy of the sequence of random variables.  Equivalently, one has

:<math>H\{\mathbf{X}\} = \lim_{n\to\infty} 
\frac{H(X_0, X_1, \dots, X_{n-1}, X_n)}{n+1}.</math>

==See also==
* [[Markov information source]]
* [[Asymptotic equipartition property]]

==References==
* Robert B. Ash, ''Information Theory'', (1965) Dover Publications. {{ISBN|0-486-66521-6}}
zh-yue:資訊源
[[Category:Information theory]]
[[Category:Stochastic processes]]



{{Sourceattribution|Information source (mathematics)}}