Response modeling methodology

From HandWiki
Revision as of 14:07, 6 February 2024 by LinXED (talk | contribs) (over-write)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

Response modeling methodology (RMM) is a general platform for statistical modeling of a linear/nonlinear relationship between a response variable (dependent variable) and a linear predictor (a linear combination of predictors/effects/factors/independent variables), often denoted the linear predictor function. It is generally assumed that the modeled relationship is monotone convex (delivering monotone convex function) or monotone concave (delivering monotone concave function). However, many non-monotone functions, like the quadratic equation, are special cases of the general model.

RMM was initially developed as a series of extensions to the original inverse Box–Cox transformation:[math]\displaystyle{ y={{(1+\lambda z)}^{1/\lambda }}, }[/math] where y is a percentile of the modeled response, Y (the modeled random variable), z is the respective percentile of a normal variate and λ is the Box–Cox parameter. As λ goes to zero, the inverse Box–Cox transformation becomes: [math]\displaystyle{ y=e^z, }[/math] an exponential model. Therefore, the original inverse Box-Cox transformation contains a trio of models: linear (λ = 1), power (λ ≠ 1, λ ≠ 0) and exponential (λ = 0). This implies that on estimating λ, using sample data, the final model is not determined in advance (prior to estimation) but rather as a result of estimating. In other words, data alone determine the final model.

Extensions to the inverse Box–Cox transformation were developed by Shore (2001a[1]) and were denoted Inverse Normalizing Transformations (INTs). They had been applied to model monotone convex relationships in various engineering areas, mostly to model physical properties of chemical compounds (Shore et al., 2001a,[1] and references therein). Once it had been realized that INT models may be perceived as special cases of a much broader general approach for modeling non-linear monotone convex relationships, the new Response Modeling Methodology had been initiated and developed (Shore, 2005a,[2] 2011[3] and references therein).

The RMM model expresses the relationship between a response, Y (the modeled random variable), and two components that deliver variation to Y:

  • The linear predictor function, LP (denoted η):[math]\displaystyle{ \eta =\beta_0 +\beta_1 X_1+ \cdots + \beta_k X_k, }[/math] where {X1,...,Xk} are regressor-variables (“affecting factors”) that deliver systematic variation to the response;
  • Normal errors, delivering random variation to the response.

The basic RMM model describes Y in terms of the LP, two possibly correlated zero-mean normal errors, ε1 and ε2 (with correlation ρ and standard deviations σε1 and σε2, respectively) and a vector of parameters {α,λ,μ} (Shore, 2005a,[2] 2011[3]):

[math]\displaystyle{ W=\log (Y)=\mu + \left( \frac \alpha \lambda \right) [(\eta + \varepsilon_1)^\lambda-1] + \varepsilon_2,\, }[/math]

and ε1 represents uncertainty (measurement imprecision or otherwise) in the explanatory variables (included in the LP). This is in addition to uncertainty associated with the response (ε2). Expressing ε1 and ε2 in terms of standard normal variates, Z1 and Z2, respectively, having correlation ρ, and conditioning Z2 | Z1 = z1 (Z2 given that Z1 is equal to a given value z1), we may write in terms of a single error, ε:

[math]\displaystyle{ \begin{align} \varepsilon_1 & = \sigma_{\varepsilon_1} Z_1\,\,;\,\,\varepsilon_2 = \sigma_{\varepsilon_2} Z_2; \\[4pt] \varepsilon_2 & = \sigma_{\varepsilon_2} \rho z_1 + (1-\rho^2)^{(1/2)}\sigma_{\varepsilon_2} Z = dz_1 + \varepsilon, \\ \end{align} }[/math]

where Z is a standard normal variate, independent of both Z1 and Z2, ε is a zero-mean error and d is a parameter. From these relationships, the associated RMM quantile function is (Shore, 2011[3]):

[math]\displaystyle{ w=\log (y)=\mu +\left( \frac \alpha \lambda \right)[(\eta +cz)^\lambda-1]+(d)z+\varepsilon, }[/math]

or, after re-parameterization:

[math]\displaystyle{ w=\log (y)=\log (M_Y)+\left( \frac{a\eta^b} b \right)\left\{ \left[ 1+\left( \frac c \eta \right) z \right]^b - 1 \right\} + (d)z+\varepsilon , }[/math]

where y is the percentile of the response (Y), z is the respective standard normal percentile, ε is the model's zero-mean normal error with constant variance, σ, {a,b,c,d} are parameters and MY is the response median (z = 0), dependent on values of the parameters and the value of the LP, η:

[math]\displaystyle{ \log (M_Y)=\mu +\left( \frac a b \right) [\eta^b-1]=\log (m)+\left( \frac a b \right)[\eta^b-1], }[/math]

where μ (or m) is an additional parameter.

If it may be assumed that cz<<η, the above model for RMM quantile function can be approximated by:

[math]\displaystyle{ w=\log (y)=\log (M_Y)+\left( \frac{a\eta^b} b \right) \left[\exp \left( \frac{bcz} \eta \right)-1\right]+(d)z+\varepsilon . }[/math]

The parameter “c” cannot be “absorbed” into the parameters of the LP (η) since “c” and LP are estimated in two separate stages (as expounded below).

If the response data used to estimate the model contain values that change sign, or if the lowest response value is far from zero (for example, when data are left-truncated), a location parameter, L, may be added to the response so that the expressions for the quantile function and for the median become, respectively:

[math]\displaystyle{ w=\log (y-L)=\log (M_Y-L)+\left( \frac{a\eta^b} b \right)\left\{ \left[ 1+\left( \frac c \eta \right)z \right]^b -1 \right\}+(d)z+\varepsilon \,; }[/math]
[math]\displaystyle{ \log (M_Y-L)=\mu +\left( \frac a b \right)[\eta^b-1]. }[/math]

Continuous monotonic convexity

As shown earlier, the inverse Box–Cox transformation depends on a single parameter, λ, which determines the final form of the model (whether linear, power or exponential). All three models thus constitute mere points on a continuous spectrum of monotonic convexity, spanned by λ. This property, where different known models become mere points on a continuous spectrum, spanned by the model's parameters, is denoted the Continuous Monotonic Convexity (CMC) property. The latter characterizes all RMM models, and it allows the basic “linear-power-exponential” cycle (underlying the inverse Box–Cox transformation) to be repeated ad infinitum, allowing for ever more convex models to be derived. Examples for such models are an exponential-power model or an exponential-exponential-power model (see explicit models expounded further on). Since the final form of the model is determined by the values of RMM parameters, this implies that the data, used to estimate the parameters, determine the final form of the estimated RMM model (as with the Box–Cox inverse transformation). The CMC property thus grant RMM models high flexibility in accommodating the data used to estimate the parameters. References given below display published results of comparisons between RMM models and existing models. These comparisons demonstrate the effectiveness of the CMC property.

Examples of RMM models

Ignoring RMM errors (ignore the terms cz, dz, and e in the percentile model), we obtain the following RMM models, presented in an increasing order of monotone convexity:

[math]\displaystyle{ \begin{align} & \text{linear: } y = \eta & & (\alpha = 1, \lambda=0); \\[5pt] & \text{power: } y = \eta^\alpha, & & (\alpha\ne1, \lambda=0); \\[5pt] & \text{exponential-linear: } y = k \exp(\eta), & & (\alpha\ne1,\lambda=1); \\[5pt] & \text{exponential-power: } y=k\exp (\eta^\lambda), & & (\alpha \ne 1, \lambda\ne 1; k \text{ is a non-negative parameter}.) \end{align} }[/math]

Adding two new parameters by introducing for η (in the percentile model): [math]\displaystyle{ \exp \left[\left( \frac \beta \kappa \right) (\eta^\kappa-1)\right] }[/math], a new cycle of “linear-power-exponential” is iterated to produce models with stronger monotone convexity (Shore, 2005a,[2] 2011,[3] 2012[4]):

[math]\displaystyle{ \begin{align} & \text{exponential-power: } y = k \exp (\eta^\lambda), & & (\alpha\neq, \lambda\ne1, \beta=1, \kappa =0, \\ & & & \text{ restoring the former model}); \\[6pt] & \text{exponential-exponential-linear: } y = k_1 \exp [k_2\exp (\eta)], & & (\alpha\ne1, \lambda\ne1, \beta=1, \kappa=1); \\[6pt] & \text{exponential-exponential-power: } y = k_1 \exp [k_2\exp (\eta^\kappa)], & & (\alpha\ne1, \lambda\ne1, \beta=1, \kappa\ne 1). \end{align} }[/math]

It is realized that this series of monotonic convex models, presented as they appear in a hierarchical order on the “Ladder of Monotonic Convex Functions” (Shore, 2011[3]), is unlimited from above. However, all models are mere points on a continuous spectrum, spanned by RMM parameters. Also note that numerous growth models, like the Gompertz function, are exact special cases of the RMM model.

Moments

The k-th non-central moment of Y is (assuming L = 0; Shore, 2005a,[2] 2011[3]):

[math]\displaystyle{ \operatorname{E}(Y^k) = (M_Y)^k \operatorname{E} \left\{ \exp \left\{ \left( \frac{k\alpha} \lambda \right) [(\eta +cZ)^\lambda - 1] + (kd)Z \right\} \right\}. }[/math]

Expanding Yk, as given on the right-hand-side, into a Taylor series around zero, in terms of powers of Z (the standard normal variate), and then taking expectation on both sides, assuming that cZ ≪ η so that η + cZ ≈ η, an approximate simple expression for the k-th non-central moment, based on the first six terms in the expansion, is:

[math]\displaystyle{ \operatorname{E} (Y)^k \cong (M_Y)^k e^{\alpha k \left( \eta^\lambda-1 \right)/\lambda} \left\{ 1+\frac 1 2 (kd)^2 + \frac 1 8 (kd)^4 \right\}. }[/math]

An analogous expression may be derived without assuming cZ ≪ η. This would result in a more accurate (however lengthy and cumbersome) expression. Once cZ in the above expression is neglected, Y becomes a log-normal random variable (with parameters that depend on η).

Fitting and estimation

RMM models may be used to model random variation (as a general platform for distribution fitting) or to model systematic variation (analogously to generalized linear models, GLM).

In the former case (no systematic variation, namely, η = constant), RMM Quantile function is fitted to known distributions. If the underlying distribution is unknown, the RMM quantile function is estimated using available sample data. Modeling random variation with RMM is addressed and demonstrated in Shore (2011[3] and references therein).

In the latter case (modeling systematic variation), RMM models are estimated assuming that variation in the linear predictor (generated via variation in the regressor-variables) contribute to the overall variation of the modeled response variable (Y). This case is addressed and demonstrated in Shore (2005a,[2] 2012[4] and relevant references therein). Estimation is conducted in two stages. First the median is estimated by minimizing the sum of absolute deviations (of fitted model from sample data points). In the second stage, the remaining two parameters (not estimated in the first stage, namely, {c,d}), are estimated. Three estimation approaches are presented in Shore (2012[4]): maximum likelihood, moment matching and nonlinear quantile regression.

Literature review

AS of 2021, RMM literature addresses three areas:

(1) Developing INTs and later the RMM approach, with allied estimation methods;

(2) Exploring the properties of RMM and comparing RMM effectiveness to other current modelling approaches (for distribution fitting or for modelling systematic variation);

(3) Applications.

Shore (2003a[5]) developed Inverse Normalizing Transformations (INTs) in the first years of the 21st century and has applied them to various engineering disciplines like statistical process control (Shore, 2000a,[1] b,[6] 2001a,[7] b,[8] 2002a[9]) and chemical engineering (Shore at al., 2002[10]). Subsequently, as the new Response Modeling Methodology (RMM) had been emerging and developing into a full-fledged platform for modeling monotone convex relationships (ultimately presented in a book, Shore, 2005a[2]), RMM properties were explored (Shore, 2002b,[11] 2004a,[12] b,[13] 2008a,[14] 2011[3]), estimation procedures developed (Shore, 2005a,[2] b,[15] 2012[4]) and the new modeling methodology compared to other approaches, for modeling random variation (Shore 2005c,[16] 2007,[17] 2010;[18] Shore and A’wad 2010[19]), and for modeling systematic variation (Shore, 2008b[20]).

Concurrently, RMM had been applied to various scientific and engineering disciplines and compared to current models and modeling approaches practiced therein. For example, chemical engineering (Shore, 2003b;[21] Benson-Karhi et al., 2007;[22] Shacham et al., 2008;[23] Shore and Benson-Karhi, 2010[24]), statistical process control (Shore, 2014;[25] Shore et al., 2014;[26] Danoch and Shore, 2016[27]), reliability engineering (Shore, 2004c;[28] Ladany and Shore, 2007[29]), forecasting (Shore and Benson-Karhi, 2007[30]), ecology (Shore, 2014[25]), and the medical profession (Shore et al., 2014;[26] Benson-Karhi et al., 2017[31]).

References

  1. ↑ 1.0 1.1 1.2 Shore, Haim (2000-12-01). "Three Approaches to Analyze Quality Data Originating in Non-Normal Populations". Quality Engineering 13 (2): 277–291. doi:10.1080/08982110108918651. ISSN 0898-2112. 
  2. ↑ 2.0 2.1 2.2 2.3 2.4 2.5 2.6 Haim., Shore (2006-01-01). Response modeling methodology : empirical modeling for engineering and science. World Scientific. ISBN 978-9812561022. OCLC 949697181. 
  3. ↑ 3.0 3.1 3.2 3.3 3.4 3.5 3.6 3.7 Shore, Haim (2011). "Response Modeling Methodology". WIREs Comput Stat 3 (4): 357–372. doi:10.1002/wics.151. 
  4. ↑ 4.0 4.1 4.2 4.3 Shore, Haim (2012). "Estimating Response Modeling Methodology models". WIREs Comput Stat 4 (3): 323–333. doi:10.1002/wics.1199. 
  5. ↑ Shore, Haim (2003-04-24). "Inverse Normalizing Transformations and an Extended Normalizing Transformation". Advances on Theoretical and Methodological Aspects of Probability and Statistics. CRC Press. pp. 131–145. doi:10.1201/9780203493205.ch9. ISBN 9781560329817. 
  6. ↑ Shore, Haim (2000-05-01). "General control charts for variables". International Journal of Production Research 38 (8): 1875–1897. doi:10.1080/002075400188645. ISSN 0020-7543. 
  7. ↑ Shore, Haim (2001-01-01). "Process Control for Non-Normal Populations Based on an Inverse Normalizing Transformation" (in en). Frontiers in Statistical Quality Control 6. Physica, Heidelberg. pp. 194–206. doi:10.1007/978-3-642-57590-7_12. ISBN 978-3-7908-1374-6. 
  8. ↑ Shore, H. (2001-01-01). "Modelling a non-normal response for quality improvement". International Journal of Production Research 39 (17): 4049–4063. doi:10.1080/00207540110072245. ISSN 0020-7543. 
  9. ↑ Shore, Haim (2002-06-18). "Modeling a Response with Self-Generated and Externally Generated Sources of Variation". Quality Engineering 14 (4): 563–578. doi:10.1081/QEN-120003559. ISSN 0898-2112. 
  10. ↑ Shore, Haim; Brauner, Neima; Shacham, Mordechai (2002-02-01). "Modeling Physical and Thermodynamic Properties via Inverse Normalizing Transformations". Industrial & Engineering Chemistry Research 41 (3): 651–656. doi:10.1021/ie010039s. ISSN 0888-5885. 
  11. ↑ Shore, Haim (2002-12-31). "Response Modeling Methodology (rmm)—Exploring the Properties of the Implied Error Distribution". Communications in Statistics - Theory and Methods 31 (12): 2225–2249. doi:10.1081/STA-120017223. ISSN 0361-0926. 
  12. ↑ Shore, Haim (2004). "Response Modeling Methodology (RMM) - Current distributions, transformations, and approximations as special cases of the RMM error distribution". Communications in Statistics - Theory and Methods 33 (7): 1491–1510. doi:10.1081/STA-120017223. https://www.researchgate.net/publication/233918439. 
  13. ↑ Shore, Haim (2004). "Response Modeling Methodology Validating Evidence from Engineering and the Sciences". Qual. Reliab. Eng. Int. 20: 61–79. doi:10.1002/qre.547. 
  14. ↑ Shore, Haim (2008-01-01). "Distribution Fitting with Response Modeling Methodology (RMM) — Some Recent Results". American Journal of Mathematical and Management Sciences 28 (1–2): 3–18. doi:10.1080/01966324.2008.10737714. ISSN 0196-6324. 
  15. ↑ Shore, Haim (2005-06-15). "Response modeling methodology (RMM)—maximum likelihood estimation procedures". Computational Statistics & Data Analysis 49 (4): 1148–1172. doi:10.1016/j.csda.2004.07.006. 
  16. ↑ Shore, Haim (2005-03-01). "Accurate RMM-Based Approximations for the CDF of the Normal Distribution". Communications in Statistics - Theory and Methods 34 (3): 507–513. doi:10.1081/STA-200052102. ISSN 0361-0926. 
  17. ↑ Shore, Haim (2007-11-09). "Comparison of Generalized Lambda Distribution (GLD) and Response Modeling Methodology (RMM) as General Platforms for Distribution Fitting". Communications in Statistics - Theory and Methods 36 (15): 2805–2819. doi:10.1080/03610920701386885. ISSN 0361-0926. 
  18. ↑ Shore, Haim (2010-10-01). "Distribution Fitting with the Quantile Function of Response Modeling Methodology (RMM)". Handbook of Fitting Statistical Distributions with R. Chapman and Hall/CRC. pp. 537–556. doi:10.1201/b10159-17. ISBN 9781584887119. 
  19. ↑ Shore, Haim; A'wad, Fatina (2010-05-12). "Statistical Comparison of the Goodness of Fit Delivered by Five Families of Distributions Used in Distribution Fitting". Communications in Statistics - Theory and Methods 39 (10): 1707–1728. doi:10.1080/03610920902887707. ISSN 0361-0926. 
  20. ↑ Shore, Haim (2008). "Comparison of linear predictors obtained by data transformation, generalized linear models (GLM) and response modeling methodology (RMM)". Qual. Reliab. Eng. Int. 24 (4): 389–399. doi:10.1002/qre.898. 
  21. ↑ Shore, Haim (2003-05-15). "Response modeling methodology (RMM)—a new approach to model a chemo-response for a monotone convex/concave relationship". Computers & Chemical Engineering 27 (5): 715–726. doi:10.1016/S0098-1354(02)00255-7. 
  22. ↑ Benson-Karhi, Diamanta; Shore, Haim; Shacham, Mordechai (2007-05-01). "Modeling Temperature-Dependent Properties of Water via Response Modeling Methodology (RMM) and Comparison with Acceptable Models". Industrial & Engineering Chemistry Research 46 (10): 3446–3463. doi:10.1021/ie061252x. ISSN 0888-5885. 
  23. ↑ Shacham, Mordechai; Brauner, Neima; Shore, Haim; Benson-Karhi, Diamanta (2008-07-01). "Predicting Temperature-Dependent Properties by Correlations Based on Similarities of Molecular Structures: Application to Liquid Density". Industrial & Engineering Chemistry Research 47 (13): 4496–4504. doi:10.1021/ie701766m. ISSN 0888-5885. 
  24. ↑ Shore, Haim; Benson-Karhi, Diamanta (2010-10-06). "Modeling Temperature-Dependent Properties of Oxygen, Argon, and Nitrogen via Response Modeling Methodology (RMM) and Comparison with Acceptable Models". Industrial & Engineering Chemistry Research 49 (19): 9469–9485. doi:10.1021/ie100981y. ISSN 0888-5885. 
  25. ↑ 25.0 25.1 Shore, Haim (2014). "Modeling and monitoring ecological systems — a statistical process control approach". Quality and Reliability Engineering International 30 (8): 1233–1248. doi:10.1002/qre.1544. 
  26. ↑ 26.0 26.1 Shore, Haim; Benson-Karhi, Diamanta; Malamud, Maya; Bashiri, Asher (2014-07-03). "Customized Fetal Growth Modeling and Monitoring—A Statistical Process Control Approach". Quality Engineering 26 (3): 290–310. doi:10.1080/08982112.2013.830742. ISSN 0898-2112. 
  27. ↑ Danoch, Revital; Shore, Haim (2016). "SPC scheme to monitor linear predictors embedded in nonlinear profiles". Qual. Reliab. Eng. Int. 32 (4): 1453–1466. doi:10.1002/qre.1856. 
  28. ↑ "Letter to the Editor". Communications in Statistics - Simulation and Computation 33 (2): 537–539. 2004-01-02. doi:10.1081/SAC-120037902. ISSN 0361-0918. 
  29. ↑ Ladany, Shaul; Shore, Haim (2007). "Profit Maximizing Warranty Period with Sales Expressed by a Demand Function". Qual. Reliab. Eng. Int. 23 (3): 291–301. doi:10.1002/qre.790. 
  30. ↑ Shore, H.; Benson-Karhi, D. (2007-06-01). "Forecasting S-shaped diffusion processes via response modelling methodology" (in en). Journal of the Operational Research Society 58 (6): 720–728. doi:10.1057/palgrave.jors.2602187. ISSN 0160-5682. 
  31. ↑ Benson-Karhi, Diamanta; Shore, Haim; Malamud, Maya (2017-01-23). "Modeling fetal-growth biometry with response modeling methodology (RMM) and comparison to current models". Communications in Statistics - Simulation and Computation 47: 129–142. doi:10.1080/03610918.2017.1280160. ISSN 0361-0918. 



Retrieved from "https://handwiki.org/wiki/index.php?title=Response_modeling_methodology&oldid=3358508"