Stationary process
In mathematics and statistics, a stationary process (or a strict/strictly stationary process or strong/strongly stationary process) is a stochastic process whose unconditional joint probability distribution does not change when shifted in time.^{[1]} Consequently, parameters such as mean and variance also do not change over time. If you draw a line through the middle of a stationary process then it should be flat; it may have 'seasonal' cycles around the trend line, but overall it does not trend up nor down.
Since stationarity is an assumption underlying many statistical procedures used in time series analysis, nonstationary data are often transformed to become stationary. The most common cause of violation of stationarity is a trend in the mean, which can be due either to the presence of a unit root or of a deterministic trend. In the former case of a unit root, stochastic shocks have permanent effects, and the process is not meanreverting. In the latter case of a deterministic trend, the process is called a trendstationary process, and stochastic shocks have only transitory effects after which the variable tends toward a deterministically evolving (nonconstant) mean.
A trend stationary process is not strictly stationary, but can easily be transformed into a stationary process by removing the underlying trend, which is solely a function of time. Similarly, processes with one or more unit roots can be made stationary through differencing. An important type of nonstationary process that does not include a trendlike behavior is a cyclostationary process, which is a stochastic process that varies cyclically with time.
For many applications strictsense stationarity is too restrictive. Other forms of stationarity such as widesense stationarity or Nthorder stationarity are then employed. The definitions for different kinds of stationarity are not consistent among different authors (see Other terminology).
Strictsense stationarity
Definition
Formally, let [math]\displaystyle{ \left\{X_t\right\} }[/math] be a stochastic process and let [math]\displaystyle{ F_{X}(x_{t_1 + \tau}, \ldots, x_{t_n + \tau}) }[/math] represent the cumulative distribution function of the unconditional (i.e., with no reference to any particular starting value) joint distribution of [math]\displaystyle{ \left\{X_t\right\} }[/math] at times [math]\displaystyle{ t_1 + \tau, \ldots, t_n + \tau }[/math]. Then, [math]\displaystyle{ \left\{X_t\right\} }[/math] is said to be strictly stationary, strongly stationary or strictsense stationary if^{[2]}^{:p. 155}
[math]\displaystyle{ F_{X}(x_{t_1+\tau} ,\ldots, x_{t_n+\tau}) = F_{X}(x_{t_1},\ldots, x_{t_n}) \quad \text{for all } \tau,t_1, \ldots, t_n \in \mathbb{R} \text{ and for all } n \in \mathbb{N}_{\gt 0} }[/math] 

( ) 
Since [math]\displaystyle{ \tau }[/math] does not affect [math]\displaystyle{ F_X(\cdot) }[/math], [math]\displaystyle{ F_{X} }[/math] is independent of time.
Examples
White noise is the simplest example of a stationary process.
An example of a discretetime stationary process where the sample space is also discrete (so that the random variable may take one of N possible values) is a Bernoulli scheme. Other examples of a discretetime stationary process with continuous sample space include some autoregressive and moving average processes which are both subsets of the autoregressive moving average model. Models with a nontrivial autoregressive component may be either stationary or nonstationary, depending on the parameter values, and important nonstationary special cases are where unit roots exist in the model.
Example 1
Let [math]\displaystyle{ Y }[/math] be any scalar random variable, and define a timeseries [math]\displaystyle{ \left\{X_t\right\} }[/math], by
 [math]\displaystyle{ X_t=Y \qquad \text{ for all } t. }[/math]
Then [math]\displaystyle{ \left\{X_t\right\} }[/math] is a stationary time series, for which realisations consist of a series of constant values, with a different constant value for each realisation. A law of large numbers does not apply on this case, as the limiting value of an average from a single realisation takes the random value determined by [math]\displaystyle{ Y }[/math], rather than taking the expected value of [math]\displaystyle{ Y }[/math].
The time average of [math]\displaystyle{ X_t }[/math] does not converge since the process is not ergodic.
Example 2
As a further example of a stationary process for which any single realisation has an apparently noisefree structure, let [math]\displaystyle{ Y }[/math] have a uniform distribution on [math]\displaystyle{ [0,2\pi] }[/math] and define the time series [math]\displaystyle{ \left\{X_t\right\} }[/math] by
 [math]\displaystyle{ X_t=\cos (t+Y) \quad \text{ for } t \in \mathbb{R}. }[/math]
Then [math]\displaystyle{ \left\{X_t\right\} }[/math] is strictly stationary since ([math]\displaystyle{ (t+ Y) }[/math] modulo [math]\displaystyle{ 2 \pi }[/math]) follows the same uniform distribution as [math]\displaystyle{ Y }[/math] for any [math]\displaystyle{ t }[/math].
Example 3
Keep in mind that a weakly white noise is not necessarily strictly stationary. Let [math]\displaystyle{ \omega }[/math] be a random variable uniformly distributed in the interval [math]\displaystyle{ (0, 2\pi) }[/math] and define the time series [math]\displaystyle{ \left\{z_t\right\} }[/math]
[math]\displaystyle{ z_t=\cos(t\omega) \quad (t=1,2,...) }[/math]
Then
 [math]\displaystyle{ \begin{align} \mathbb{E}(z_t) &= \frac{1}{2\pi} \int_0^{2\pi} \cos(t\omega) \,d\omega = 0,\\ \operatorname{Var}(z_t) &= \frac{1}{2\pi} \int_0^{2\pi} \cos^2(t\omega) \,d\omega = 1/2,\\ \operatorname{Cov}(z_t , z_j) &= \frac{1}{2\pi} \int_0^{2\pi} \cos(t\omega)\cos(j\omega) \,d\omega = 0 \quad \forall t\neq j. \end{align} }[/math]
So [math]\displaystyle{ \{z_t\} }[/math] is a white noise in the weak sense (the mean and crosscovariances are zero, and the variances are all the same), however it is not strictly stationary.
Nthorder stationarity
In Eq.1, the distribution of [math]\displaystyle{ n }[/math] samples of the stochastic process must be equal to the distribution of the samples shifted in time for all [math]\displaystyle{ n }[/math]. Nthorder stationarity is a weaker form of stationarity where this is only requested for all [math]\displaystyle{ n }[/math] up to a certain order [math]\displaystyle{ N }[/math]. A random process [math]\displaystyle{ \left\{X_t\right\} }[/math] is said to be Nthorder stationary if:^{[2]}^{:p. 152}
[math]\displaystyle{ F_{X}(x_{t_1+\tau} ,\ldots, x_{t_n+\tau}) = F_{X}(x_{t_1},\ldots, x_{t_n}) \quad \text{for all } \tau,t_1, \ldots, t_n \in \mathbb{R} \text{ and for all } n \in \{1,\ldots,N\} }[/math] 

( ) 
Weak or widesense stationarity
Definition
A weaker form of stationarity commonly employed in signal processing is known as weaksense stationarity, widesense stationarity (WSS), or covariance stationarity. WSS random processes only require that 1st moment (i.e. the mean) and autocovariance do not vary with respect to time and that the 2nd moment is finite for all times. Any strictly stationary process which has a finite mean and covariance is also WSS.^{[3]}^{:p. 299}
So, a continuous time random process [math]\displaystyle{ \left\{X_t\right\} }[/math] which is WSS has the following restrictions on its mean function [math]\displaystyle{ m_X(t) \triangleq \operatorname E[X_t] }[/math] and autocovariance function [math]\displaystyle{ K_{XX}(t_1, t_2) \triangleq \operatorname E[(X_{t_1}m_X(t_1))(X_{t_2}m_X(t_2))] }[/math]:
[math]\displaystyle{ \begin{align} & m_X(t) = m_X(t + \tau) & & \text{for all } \tau,t \in \mathbb{R} \\ & K_{XX}(t_1, t_2) = K_{XX}(t_1  t_2, 0) & & \text{for all } t_1,t_2 \in \mathbb{R} \\ & \operatorname E[X_t^2] \lt \infty & & \text{for all } t \in \mathbb{R} \end{align} }[/math] 

( ) 
The first property implies that the mean function [math]\displaystyle{ m_X(t) }[/math] must be constant. The second property implies that the autocovariance function depends only on the difference between [math]\displaystyle{ t_1 }[/math] and [math]\displaystyle{ t_2 }[/math] and only needs to be indexed by one variable rather than two variables.^{[2]}^{:p. 159} Thus, instead of writing,
 [math]\displaystyle{ \,\!K_{XX}(t_1  t_2, 0)\, }[/math]
the notation is often abbreviated by the substitution [math]\displaystyle{ \tau = t_1  t_2 }[/math]:
 [math]\displaystyle{ K_{XX}(\tau) \triangleq K_{XX}(t_1  t_2, 0) }[/math]
This also implies that the autocorrelation depends only on [math]\displaystyle{ \tau = t_1  t_2 }[/math], that is
 [math]\displaystyle{ \,\! R_X(t_1,t_2) = R_X(t_1t_2,0) \triangleq R_X(\tau). }[/math]
The third property says that the second moments must be finite for any time [math]\displaystyle{ t }[/math].
Motivation
The main advantage of widesense stationarity is that it places the timeseries in the context of Hilbert spaces. Let H be the Hilbert space generated by {x(t)} (that is, the closure of the set of all linear combinations of these random variables in the Hilbert space of all squareintegrable random variables on the given probability space). By the positive definiteness of the autocovariance function, it follows from Bochner's theorem that there exists a positive measure [math]\displaystyle{ \mu }[/math] on the real line such that H is isomorphic to the Hilbert subspace of L^{2}(μ) generated by {e^{−2πiξ⋅t}}. This then gives the following Fouriertype decomposition for a continuous time stationary stochastic process: there exists a stochastic process [math]\displaystyle{ \omega_\xi }[/math] with orthogonal increments such that, for all [math]\displaystyle{ t }[/math]
 [math]\displaystyle{ X_t = \int e^{ 2 \pi i \lambda \cdot t} \, d \omega_\lambda, }[/math]
where the integral on the righthand side is interpreted in a suitable (Riemann) sense. The same result holds for a discretetime stationary process, with the spectral measure now defined on the unit circle.
When processing WSS random signals with linear, timeinvariant (LTI) filters, it is helpful to think of the correlation function as a linear operator. Since it is a circulant operator (depends only on the difference between the two arguments), its eigenfunctions are the Fourier complex exponentials. Additionally, since the eigenfunctions of LTI operators are also complex exponentials, LTI processing of WSS random signals is highly tractable—all computations can be performed in the frequency domain. Thus, the WSS assumption is widely employed in signal processing algorithms.
Definition for complex stochastic process
In the case where [math]\displaystyle{ \left\{X_t\right\} }[/math] is a complex stochastic process the autocovariance function is defined as [math]\displaystyle{ K_{XX}(t_1, t_2) = \operatorname E[(X_{t_1}m_X(t_1))\overline{(X_{t_2}m_X(t_2))}] }[/math] and, in addition to the requirements in Eq.3, it is required that the pseudoautocovariance function [math]\displaystyle{ J_{XX}(t_1, t_2) = \operatorname E[(X_{t_1}m_X(t_1))(X_{t_2}m_X(t_2))] }[/math] depends only on the time lag. In formulas, [math]\displaystyle{ \left\{X_t\right\} }[/math] is WSS, if
[math]\displaystyle{ \begin{align} & m_X(t) = m_X(t + \tau) & & \text{for all } \tau,t \in \mathbb{R} \\ & K_{XX}(t_1, t_2) = K_{XX}(t_1  t_2, 0) & & \text{for all } t_1,t_2 \in \mathbb{R} \\ & J_{XX}(t_1, t_2) = J_{XX}(t_1  t_2, 0) & & \text{for all } t_1,t_2 \in \mathbb{R} \\ & \operatorname E[X(t)^2] \lt \infty & & \text{for all } t \in \mathbb{R} \end{align} }[/math] 

( ) 
Joint stationarity
The concept of stationarity may be extended to two stochastic processes.
Joint strictsense stationarity
Two stochastic processes [math]\displaystyle{ \left\{X_t\right\} }[/math] and [math]\displaystyle{ \left\{Y_t\right\} }[/math] are called jointly strictsense stationary if their joint cumulative distribution [math]\displaystyle{ F_{XY}(x_{t_1} ,\ldots, x_{t_m},y_{t_1^'} ,\ldots, y_{t_n^'}) }[/math] remains unchanged under time shifts, i.e. if
[math]\displaystyle{ F_{XY}(x_{t_1} ,\ldots, x_{t_m},y_{t_1^'} ,\ldots, y_{t_n^'}) = F_{XY}(x_{t_1+\tau} ,\ldots, x_{t_m+\tau},y_{t_1^'+\tau} ,\ldots, y_{t_n^'+\tau}) \quad \text{for all } \tau,t_1, \ldots, t_m, t_1^', \ldots, t_n^' \in \mathbb{R} \text{ and for all } m,n \in \mathbb{N} }[/math] 

( ) 
Joint (M + N)thorder stationarity
Two random processes [math]\displaystyle{ \left\{X_t\right\} }[/math] and [math]\displaystyle{ \left\{Y_t\right\} }[/math] is said to be jointly (M + N)thorder stationary if:^{[2]}^{:p. 159}
[math]\displaystyle{ F_{XY}(x_{t_1} ,\ldots, x_{t_m},y_{t_1^'} ,\ldots, y_{t_n^'}) = F_{XY}(x_{t_1+\tau} ,\ldots, x_{t_m+\tau},y_{t_1^'+\tau} ,\ldots, y_{t_n^'+\tau}) \quad \text{for all } \tau,t_1, \ldots, t_m, t_1^', \ldots, t_n^' \in \mathbb{R} \text{ and for all } m \in \{1,\ldots,M\}, n \in \{1,\ldots,N\} }[/math] 

( ) 
Joint weak or widesense stationarity
Two stochastic processes [math]\displaystyle{ \left\{X_t\right\} }[/math] and [math]\displaystyle{ \left\{Y_t\right\} }[/math] are called jointly widesense stationary if they are both widesense stationary and their crosscovariance function [math]\displaystyle{ K_{XY}(t_1, t_2) = \operatorname E[(X_{t_1}m_X(t_1))(Y_{t_2}m_Y(t_2))] }[/math] depends only on the time difference [math]\displaystyle{ \tau = t_1  t_2 }[/math]. This may be summarized as follows:
[math]\displaystyle{ \begin{align} & m_X(t) = m_X(t + \tau) & & \text{for all } \tau,t \in \mathbb{R} \\ & m_Y(t) = m_Y(t + \tau) & & \text{for all } \tau,t \in \mathbb{R} \\ & K_{XX}(t_1, t_2) = K_{XX}(t_1  t_2, 0) & & \text{for all } t_1,t_2 \in \mathbb{R} \\ & K_{YY}(t_1, t_2) = K_{YY}(t_1  t_2, 0) & & \text{for all } t_1,t_2 \in \mathbb{R} \\ & K_{XY}(t_1, t_2) = K_{XY}(t_1  t_2, 0) & & \text{for all } t_1,t_2 \in \mathbb{R} \end{align} }[/math] 

( ) 
Relation between types of stationarity
 If a stochastic process is Nthorder stationary, then it is also Mthorder stationary for all [math]\displaystyle{ M \le N }[/math].
 If a stochastic process is second order stationary ([math]\displaystyle{ N=2 }[/math]) and has finite second moments, then it is also widesense stationary.^{[2]}^{:p. 159}
 If a stochastic process is widesense stationary, it is not necessarily secondorder stationary.^{[2]}^{:p. 159}
 If a stochastic process is strictsense stationary and has finite second moments, it is widesense stationary.^{[3]}^{:p. 299}
 If two stochastic processes are jointly (M + N)thorder stationary, this does not guarantee that the individual processes are Mth respectively Nthorder stationary.^{[2]}^{:p. 159}
Other terminology
The terminology used for types of stationarity other than strict stationarity can be rather mixed. Some examples follow.
 Priestley uses stationary up to order m if conditions similar to those given here for wide sense stationarity apply relating to moments up to order m.^{[4]}^{[5]} Thus wide sense stationarity would be equivalent to "stationary to order 2", which is different from the definition of secondorder stationarity given here.
 Honarkhah and Caers also use the assumption of stationarity in the context of multiplepoint geostatistics, where higher npoint statistics are assumed to be stationary in the spatial domain.^{[6]}
 Tahmasebi and Sahimi have presented an adaptive Shannonbased methodology that can be used for modeling of any nonstationary systems.^{[7]}
Differencing
One way to make some time series stationary is to compute the differences between consecutive observations. This is known as differencing. Differencing can help stabilize the mean of a time series by removing changes in the level of a time series, and so eliminating trends. This can also remove seasonality, if differences are taken appropriately (e.g. differencing observations 1 year apart to remove yearlo).
Transformations such as logarithms can help to stabilize the variance of a time series.
One of the ways for identifying nonstationary times series is the ACF plot. Sometimes, patterns will be more visible in the ACF plot than in the original time series; however, this is not always the case.^{[8]}
Another approach to identifying nonstationarity is to look at the Laplace transform of a series, which will identify both exponential trends and sinusoidal seasonality (complex exponential trends). Related techniques from signal analysis such as the wavelet transform and Fourier transform may also be helpful.
See also
 Lévy process
 Stationary ergodic process
 Wiener–Khinchin theorem
 Ergodicity
 Statistical regularity
 Autocorrelation
 Whittle likelihood
References
 ↑ Gagniuc, Paul A. (2017). Markov Chains: From Theory to Implementation and Experimentation. USA, NJ: John Wiley & Sons. pp. 1–256. ISBN 9781119387558.
 ↑ ^{2.0} ^{2.1} ^{2.2} ^{2.3} ^{2.4} ^{2.5} ^{2.6} Park,Kun Il (2018). Fundamentals of Probability and Stochastic Processes with Applications to Communications. Springer. ISBN 9783319680743.
 ↑ ^{3.0} ^{3.1} Ionut Florescu (7 November 2014). Probability and Stochastic Processes. John Wiley & Sons. ISBN 9781118593202.
 ↑ Priestley, M. B. (1981). Spectral Analysis and Time Series. Academic Press. ISBN 0125649223.
 ↑ Priestley, M. B. (1988). Nonlinear and Nonstationary Time Series Analysis. Academic Press. ISBN 0125649118. https://archive.org/details/nonlinearnonstat0000prie.
 ↑ Honarkhah, M.; Caers, J. (2010). "Stochastic Simulation of Patterns Using DistanceBased Pattern Modeling". Mathematical Geosciences 42 (5): 487–517. doi:10.1007/s1100401092767.
 ↑ Tahmasebi, P.; Sahimi, M. (2015). "Reconstruction of nonstationary disordered materials and media: Watershed transform and crosscorrelation function" (PDF). Physical Review E 91 (3): 032401. doi:10.1103/PhysRevE.91.032401. PMID 25871117. http://journals.aps.org/pre/abstract/10.1103/PhysRevE.91.032401.
 ↑ "8.1 Stationarity and differencing  OTexts". https://www.otexts.org/fpp/8/1.
Further reading
 Enders, Walter (2010). Applied Econometric Time Series (Third ed.). New York: Wiley. pp. 53–57. ISBN 9780470505397.
 Jestrovic, I.; Coyle, J. L.; Sejdic, E (2015). "The effects of increased fluid viscosity on stationary characteristics of EEG signal in healthy adults". Brain Research 1589: 45–53. doi:10.1016/j.brainres.2014.09.035. PMID 25245522.
 Hyndman, Athanasopoulos (2013). Forecasting: Principles and Practice. Otexts. https://www.otexts.org/fpp/8/1
External links
Original source: https://en.wikipedia.org/wiki/Stationary process.
Read more 