Power transform
In statistics, a power transform is a family of functions applied to create a monotonic transformation of data using power functions. It is a data transformation technique used to stabilize variance, make the data more normal distribution-like, improve the validity of measures of association (such as the Pearson correlation between variables), and for other data stabilization procedures.
Power transforms are used in multiple fields, including multi-resolution and wavelet analysis,[1] statistical data analysis, medical research, modeling of physical processes,[2] geochemical data analysis,[3] epidemiology[4] and many other clinical, environmental and social research areas.
Definition
The power transformation is defined as a continuously varying function, with respect to the power parameter λ, in a piece-wise function form that makes it continuous at the point of singularity (λ = 0). For data vectors (y1,..., yn) in which each yi > 0, the power transform is
- [math]\displaystyle{ y_i^{(\lambda)} = \begin{cases} \dfrac{y_i^\lambda-1}{\lambda(\operatorname{GM}(y))^{\lambda -1}} , &\text{if } \lambda \neq 0 \\[12pt] \operatorname{GM}(y)\ln{y_i} , &\text{if } \lambda = 0 \end{cases} }[/math]
where
- [math]\displaystyle{ \operatorname{GM}(y) = \left(\prod_{i=1}^n y_i\right)^\frac{1}{n} = \sqrt[n]{y_1 y_2 \cdots y_n} \, }[/math]
is the geometric mean of the observations y1, ..., yn. The case for [math]\displaystyle{ \lambda = 0 }[/math] is the limit as [math]\displaystyle{ \lambda }[/math] approaches 0. To see this, note that [math]\displaystyle{ y_i^\lambda = \exp({\lambda \ln(y_i)}) = 1 + \lambda \ln(y_i) + O((\lambda \ln(y_i))^2) }[/math] - using Taylor series. Then [math]\displaystyle{ \dfrac{y_i^\lambda-1}\lambda = \ln(y_i) + O(\lambda) }[/math], and everything but [math]\displaystyle{ \ln(y_i) }[/math] becomes negligible for [math]\displaystyle{ \lambda }[/math] sufficiently small.
The inclusion of the (λ − 1)th power of the geometric mean in the denominator simplifies the scientific interpretation of any equation involving [math]\displaystyle{ y_i^{(\lambda)} }[/math], because the units of measurement do not change as λ changes.
Box and Cox (1964) introduced the geometric mean into this transformation by first including the Jacobian of rescaled power transformation
- [math]\displaystyle{ \frac{y^\lambda-1} \lambda. }[/math]
with the likelihood. This Jacobian is as follows:
- [math]\displaystyle{ J(\lambda; y_1, \ldots, y_n) = \prod_{i=1}^n |d y_i^{(\lambda)} / dy| = \prod_{i=1}^n y_i^{\lambda-1} = \operatorname{GM}(y)^{n(\lambda-1)} }[/math]
This allows the normal log likelihood at its maximum to be written as follows:
- [math]\displaystyle{ \begin{align} \log ( \mathcal{L} (\hat\mu,\hat\sigma)) & = (-n/2)(\log(2\pi\hat\sigma^2) +1) + n(\lambda-1) \log(\operatorname{GM}(y)) \\[5pt] & = (-n/2)(\log(2\pi\hat\sigma^2 / \operatorname{GM}(y)^{2(\lambda-1)}) + 1). \end{align} }[/math]
From here, absorbing [math]\displaystyle{ \operatorname{GM}(y)^{2(\lambda-1)} }[/math] into the expression for [math]\displaystyle{ \hat\sigma^2 }[/math] produces an expression that establishes that minimizing the sum of squares of residuals from [math]\displaystyle{ y_i^{(\lambda)} }[/math]is equivalent to maximizing the sum of the normal log likelihood of deviations from [math]\displaystyle{ (y^\lambda-1)/\lambda }[/math] and the log of the Jacobian of the transformation.
The value at Y = 1 for any λ is 0, and the derivative with respect to Y there is 1 for any λ. Sometimes Y is a version of some other variable scaled to give Y = 1 at some sort of average value.
The transformation is a power transformation, but done in such a way as to make it continuous with the parameter λ at λ = 0. It has proved popular in regression analysis, including econometrics.
Box and Cox also proposed a more general form of the transformation that incorporates a shift parameter.
- [math]\displaystyle{ \tau(y_i;\lambda, \alpha) = \begin{cases} \dfrac{(y_i + \alpha)^\lambda - 1}{\lambda (\operatorname{GM}(y+\alpha))^{\lambda - 1}} & \text{if } \lambda\neq 0, \\ \\ \operatorname{GM}(y+\alpha)\ln(y_i + \alpha)& \text{if } \lambda=0,\end{cases} }[/math]
which holds if yi + α > 0 for all i. If τ(Y, λ, α) follows a truncated normal distribution, then Y is said to follow a Box–Cox distribution.
Bickel and Doksum eliminated the need to use a truncated distribution by extending the range of the transformation to all y, as follows:
- [math]\displaystyle{ \tau(y_i;\lambda, \alpha) = \begin{cases} \dfrac{\operatorname{sgn}(y_i + \alpha)|y_i + \alpha|^\lambda - 1}{\lambda (\operatorname{GM}(y+\alpha))^{\lambda - 1}} & \text{if } \lambda\neq 0, \\ \\ \operatorname{GM}(y+\alpha)\operatorname{sgn}(y+\alpha)\ln(y_i + \alpha)& \text{if } \lambda=0,\end{cases} }[/math]
where sgn(.) is the sign function. This change in definition has little practical import as long as [math]\displaystyle{ \alpha }[/math] is less than [math]\displaystyle{ \operatorname{min}(y_i) }[/math], which it usually is.[5]
Bickel and Doksum also proved that the parameter estimates are consistent and asymptotically normal under appropriate regularity conditions, though the standard Cramér–Rao lower bound can substantially underestimate the variance when parameter values are small relative to the noise variance.[5] However, this problem of underestimating the variance may not be a substantive problem in many applications.[6][7]
Box–Cox transformation
The one-parameter Box–Cox transformations are defined as
- [math]\displaystyle{ y_i^{(\lambda)} = \begin{cases} \dfrac{y_i^\lambda - 1}{\lambda} & \text{if } \lambda \neq 0, \\ \ln y_i & \text{if } \lambda = 0, \end{cases} }[/math]
and the two-parameter Box–Cox transformations as
- [math]\displaystyle{ y_i^{(\boldsymbol{\lambda})} = \begin{cases} \dfrac{(y_i + \lambda_2)^{\lambda_1} - 1}{\lambda_1} & \text{if } \lambda_1 \neq 0, \\ \ln (y_i + \lambda_2) & \text{if } \lambda_1 = 0, \end{cases} }[/math]
as described in the original article.[8][9] Moreover, the first transformations hold for [math]\displaystyle{ y_i \gt 0 }[/math], and the second for [math]\displaystyle{ y_i \gt -\lambda_2 }[/math].[8]
The parameter [math]\displaystyle{ \lambda }[/math] is estimated using the profile likelihood function and using goodness-of-fit tests.[10]
Confidence interval
Confidence interval for the Box–Cox transformation can be asymptotically constructed using Wilks's theorem on the profile likelihood function to find all the possible values of [math]\displaystyle{ \lambda }[/math] that fulfill the following restriction:[11]
- [math]\displaystyle{ \ln \big(L(\lambda)\big) \ge \ln \big(L(\hat\lambda)\big) - \frac{1}{2} {\chi^2}_{1,1 - \alpha}. }[/math]
Example
The BUPA liver data set[12] contains data on liver enzymes ALT and γGT. Suppose we are interested in using log(γGT) to predict ALT. A plot of the data appears in panel (a) of the figure. There appears to be non-constant variance, and a Box–Cox transformation might help.
image:BUPA BoxCox.JPG
The log-likelihood of the power parameter appears in panel (b). The horizontal reference line is at a distance of χ12/2 from the maximum and can be used to read off an approximate 95% confidence interval for λ. It appears as though a value close to zero would be good, so we take logs.
Possibly, the transformation could be improved by adding a shift parameter to the log transformation. Panel (c) of the figure shows the log-likelihood. In this case, the maximum of the likelihood is close to zero suggesting that a shift parameter is not needed. The final panel shows the transformed data with a superimposed regression line.
Note that although Box–Cox transformations can make big improvements in model fit, there are some issues that the transformation cannot help with. In the current example, the data are rather heavy-tailed so that the assumption of normality is not realistic and a robust regression approach leads to a more precise model.
Econometric application
Economists often characterize production relationships by some variant of the Box–Cox transformation.[13]
Consider a common representation of production Q as dependent on services provided by a capital stock K and by labor hours N:
- [math]\displaystyle{ \tau(Q)=\alpha \tau(K)+ (1-\alpha)\tau(N).\, }[/math]
Solving for Q by inverting the Box–Cox transformation we find
- [math]\displaystyle{ Q=\big(\alpha K^\lambda + (1-\alpha) N^\lambda\big)^{1/\lambda},\, }[/math]
which is known as the constant elasticity of substitution (CES) production function.
The CES production function is a homogeneous function of degree one.
When λ = 1, this produces the linear production function:
- [math]\displaystyle{ Q=\alpha K + (1-\alpha)N.\, }[/math]
When λ → 0 this produces the famous Cobb–Douglas production function:
- [math]\displaystyle{ Q=K^\alpha N^{1-\alpha}.\, }[/math]
Activities and demonstrations
The SOCR resource pages contain a number of hands-on interactive activities[14] demonstrating the Box–Cox (power) transformation using Java applets and charts. These directly illustrate the effects of this transform on Q–Q plots, X–Y scatterplots, time-series plots and histograms.
Yeo–Johnson transformation
The Yeo–Johnson transformation[15] allows also for zero and negative values of [math]\displaystyle{ y }[/math]. [math]\displaystyle{ \lambda }[/math] can be any real number, where [math]\displaystyle{ \lambda = 1 }[/math] produces the identity transformation. The transformation law reads:
- [math]\displaystyle{ y_i^{(\lambda)} = \begin{cases} ((y_i+1)^\lambda-1)/\lambda & \text{if }\lambda \neq 0, y \geq 0 \\[4pt] \ln(y_i + 1) & \text{if }\lambda = 0, y \geq 0 \\[4pt] -((-y_i + 1)^{(2-\lambda)} - 1) / (2 - \lambda) & \text{if }\lambda \neq 2, y \lt 0 \\[4pt] -\ln(-y_i + 1) & \text{if }\lambda = 2, y \lt 0 \end{cases} }[/math]
Notes
- ↑ Gao, Peisheng; Wu, Weilin (2006). "Power Quality Disturbances Classification using Wavelet and Support Vector Machines". Sixth International Conference on Intelligent Systems Design and Applications. ISDA '06. 1. Washington, DC, USA: IEEE Computer Society. pp. 201–206. doi:10.1109/ISDA.2006.217. ISBN 9780769525280.
- ↑ Gluzman, S.; Yukalov, V. I. (2006-01-01). "Self-similar power transforms in extrapolation problems" (in en). Journal of Mathematical Chemistry 39 (1): 47–56. doi:10.1007/s10910-005-9003-7. ISSN 1572-8897. Bibcode: 2006cond.mat..6104G.
- ↑ Howarth, R. J.; Earle, S. A. M. (1979-02-01). "Application of a generalized power transformation to geochemical data" (in en). Journal of the International Association for Mathematical Geology 11 (1): 45–62. doi:10.1007/BF01043245. ISSN 1573-8868.
- ↑ Peters, J. L.; Rushton, L.; Sutton, A. J.; Jones, D. R.; Abrams, K. R.; Mugglestone, M. A. (2005). "Bayesian methods for the cross-design synthesis of epidemiological and toxicological evidence". Journal of the Royal Statistical Society, Series C 54: 159–172. doi:10.1111/j.1467-9876.2005.00476.x.
- ↑ 5.0 5.1 Bickel, Peter J.; Doksum, Kjell A. (June 1981). "An analysis of transformations revisited". Journal of the American Statistical Association 76 (374): 296–311. doi:10.1080/01621459.1981.10477649.
- ↑ Sakia, R. M. (1992), "The Box–Cox transformation technique: a review", The Statistician 41 (2): 169–178, doi:10.2307/2348250
- ↑ Li, Fengfei (April 11, 2005), Box–Cox Transformations: An Overview, Sao Paulo, Brazil: University of Sao Paulo, Brazil, http://www.ime.usp.br/~abe/lista/pdfm9cJKUmFZp.pdf, retrieved 2014-11-02
- ↑ 8.0 8.1 Box, George E. P.; Cox, D. R. (1964). "An analysis of transformations". Journal of the Royal Statistical Society, Series B 26 (2): 211–252.
- ↑ Johnston, J. (1984). Econometric Methods (Third ed.). New York: McGraw-Hill. pp. 61–74. ISBN 978-0-07-032685-9.
- ↑ Asar, O.; Ilk, O.; Dag, O. (2017). "Estimating Box-Cox power transformation parameter via goodness-of-fit tests". Communications in Statistics - Simulation and Computation 46 (1): 91–105. doi:10.1080/03610918.2014.957839.
- ↑ Abramovich, Felix; Ritov, Ya'acov (2013). Statistical Theory: A Concise Introduction. CRC Press. pp. 121–122. ISBN 978-1-4398-5184-5.
- ↑ BUPA liver disorder dataset
- ↑ Zarembka, P. (1974). "Transformation of Variables in Econometrics". Frontiers in Econometrics. New York: Academic Press. pp. 81–104. ISBN 0-12-776150-0.
- ↑ Power Transform Family Graphs, SOCR webpages
- ↑ Yeo, In-Kwon; Johnson, Richard A. (2000). "A New Family of Power Transformations to Improve Normality or Symmetry". Biometrika 87 (4): 954–959. doi:10.1093/biomet/87.4.954.
References
- "An analysis of transformations". Journal of the Royal Statistical Society, Series B 26 (2): 211–252. 1964.
- "On prediction and the power transformation family". Biometrika 68 (3): 609–615. 1981. doi:10.1093/biomet/68.3.609. http://wiki.stat.ucla.edu/socr/uploads/b/b8/PowerTransformFamily_Biometrica609.pdf.
- "A Conversation with George Box". Statistical Science 2 (3): 239–258. 1987. doi:10.1214/ss/1177013223.
- "Optimal Power Transformations for Analysis of Sperm Concentration and Other Semen Variables". Journal of Andrology 23 (5). 2002.
- "Self-similar power transforms in extrapolation problems". Journal of Mathematical Chemistry 39 (1): 47–56. 2006. doi:10.1007/s10910-005-9003-7. Bibcode: 2006cond.mat..6104G.
- "Application of a generalized power transformation to geochemical data". Journal of the International Association for Mathematical Geology 11 (1): 45–62. 1979. doi:10.1007/BF01043245.
External links
- Hazewinkel, Michiel, ed. (2001), "Box–Cox transformation", Encyclopedia of Mathematics, Springer Science+Business Media B.V. / Kluwer Academic Publishers, ISBN 978-1-55608-010-4, https://www.encyclopediaofmath.org/index.php?title=B/b110790 (fixed link)
- Sanford Weisberg, Yeo-Johnson Power Transformations
![]() | Original source: https://en.wikipedia.org/wiki/Power transform.
Read more |