Congeneric reliability

From HandWiki

In statistical models applied to psychometrics, congeneric reliability [math]\displaystyle{ \rho_C }[/math] ("rho C")[1] a single-administration test score reliability (i.e., the reliability of persons over items holding occasion fixed[2]) coefficient, commonly referred to as composite reliability, construct reliability, and coefficient omega. [math]\displaystyle{ \rho_C }[/math] is a structural equation model (SEM)-based reliability coefficients and is obtained from on a unidimensional model. [math]\displaystyle{ \rho_C }[/math] is the second most commonly used reliability factor after tau-equivalent reliability([math]\displaystyle{ \rho_T }[/math]; also known as Cronbach's alpha), and is often recommended as its alternative.

Formula and calculation

Systematic and conventional formula

Congeneric measurement model

Let [math]\displaystyle{ X_i }[/math] denote the observed score of item [math]\displaystyle{ i }[/math] and [math]\displaystyle{ X(=X_1 + X_2 + \cdots + X_k) }[/math] denote the sum of all items in a test consisting of [math]\displaystyle{ k }[/math] items. It is assumed that each item's (observation) score consists of the item's (unobserved) true score and the item's error (i.e., [math]\displaystyle{ X_i=T_i+e_i }[/math]). The congeneric model assumes that each item's true score is a linear combination of a common factor ([math]\displaystyle{ F }[/math]) (i.e., [math]\displaystyle{ T_i=\mu_i+\lambda_i F }[/math]). [math]\displaystyle{ \lambda_i }[/math] is often referred to as a factor loading of item [math]\displaystyle{ i }[/math]. [math]\displaystyle{ \sigma^2_X }[/math] is the sum of all the elements of the fitted/implied covariance matrix of [math]\displaystyle{ X }[/math] obtained from estimates of [math]\displaystyle{ \lambda_i }[/math]'s and [math]\displaystyle{ \sigma_{e_i} }[/math]'s.

[math]\displaystyle{ \rho_C }[/math]'s "systematic formula"[1] is:

[math]\displaystyle{ \rho_C = \frac{\left( \sum_{i=1}^k \lambda_i \right)^2}{\sigma^2_X} }[/math]

Its conventional (i.e., more often used) formula is:

[math]\displaystyle{ \rho_C = \frac{ \left( \sum_{i=1}^k \lambda_i \right)^2 }{ \left( \sum_{i=1}^k \lambda_i \right)^2 + \sum_{i=1}^k \sigma^2_{e_i} } }[/math]

Example

Fitted/implied covariance matrix
[math]\displaystyle{ X_1 }[/math] [math]\displaystyle{ X_2 }[/math] [math]\displaystyle{ X_3 }[/math] [math]\displaystyle{ X_4 }[/math]
[math]\displaystyle{ X_1 }[/math] [math]\displaystyle{ 10.00 }[/math]
[math]\displaystyle{ X_2 }[/math] [math]\displaystyle{ 4.42 }[/math] [math]\displaystyle{ 11.00 }[/math]
[math]\displaystyle{ X_3 }[/math] [math]\displaystyle{ 4.98 }[/math] [math]\displaystyle{ 5.71 }[/math] [math]\displaystyle{ 12.00 }[/math]
[math]\displaystyle{ X_4 }[/math] [math]\displaystyle{ 6.98 }[/math] [math]\displaystyle{ 7.99 }[/math] [math]\displaystyle{ 9.01 }[/math] [math]\displaystyle{ 13.00 }[/math]
[math]\displaystyle{ \Sigma }[/math] [math]\displaystyle{ 124.23 = \Sigma_{diagonal} + 2 \times \Sigma_{subdiagonal} }[/math]

These are the estimates of the factor loadings and errors:

Factor loadings and errors
[math]\displaystyle{ \hat{\lambda}_i }[/math] [math]\displaystyle{ \hat{\sigma}^{2}_{e_i} }[/math]
[math]\displaystyle{ X_1 }[/math] [math]\displaystyle{ 1.96 }[/math] [math]\displaystyle{ 6.13 }[/math]
[math]\displaystyle{ X_2 }[/math] [math]\displaystyle{ 2.25 }[/math] [math]\displaystyle{ 5.92 }[/math]
[math]\displaystyle{ X_3 }[/math] [math]\displaystyle{ 2.53 }[/math] [math]\displaystyle{ 5.56 }[/math]
[math]\displaystyle{ X_4 }[/math] [math]\displaystyle{ 3.55 }[/math] [math]\displaystyle{ .37 }[/math]
[math]\displaystyle{ \Sigma }[/math] [math]\displaystyle{ 10.30 }[/math] [math]\displaystyle{ 18.01 }[/math]
[math]\displaystyle{ \Sigma^2 }[/math] [math]\displaystyle{ 106.22 }[/math]
[math]\displaystyle{ \hat{\rho}_{C} = \frac{ \left( \sum_{i=1}^k \hat{\lambda}_i \right)^2 }{ \hat{\sigma}^{2}_{X} } = \frac{ 106.22 }{ 124.23 } = .8550 }[/math]
[math]\displaystyle{ \hat{\rho}_{C} = \frac{ \left( \sum_{i=1}^k \hat{\lambda}_i \right)^2 }{ \left( \sum_{i=1}^k \hat{\lambda}_i \right)^2 + \sum_{i=1}^k \hat{\sigma}^{2}_{e_i} } = \frac{ 106.22 }{ 106.22 + 18.01 } = .8550 }[/math]

Compare this value with the value of applying tau-equivalent reliability to the same data.

History

[math]\displaystyle{ {\rho}_{C} }[/math]'s formula was first introduced by Jöreskog (1971) in a matrix notation.[2] Its conventional formula first appeared in Werts et al. (1974).[3] They did not give the formula a special name and just referred to it as "reliability".[1] In other words, this formula has no official name, and this absence causes various versions of the name to be created.

Names of congeneric reliability

[math]\displaystyle{ {\rho}_{C} }[/math] has been referred to by various names between applied researchers and between reliability researchers. In addition, the names used by applied researchers differ from the names used by reliability researchers. This diversity and difference create confusion and inaccuracies in communication.

Composite reliability

Source:[1]

The term composite reliability is short for the 'reliability of composite scores'. Unless measured by a single item, all reliability coefficients are composite reliability. Therefore, this name is not suitable as a specific formula name. The name composite reliability gives the impression that this reliability coefficient is complex, or that it has been synthesized from other reliability coefficients.

Werts et al. (1978)[4] also called this formula "reliability". However, they used the expression "the composite reliability" once as an abbreviation of the reliability of a composite score to distinguish the reliability of a single item. Since then, this unintended name has been used as the name of this formula.

Applied researchers most often use this name when referring to [math]\displaystyle{ \rho_{C} }[/math].[5] Researchers who publish papers on reliability rarely use this name.

Construct reliability

Construct reliability is short for the 'reliability of a construct'. Construct is synonymous with concept.[6] A construct is a theoretical and abstract entity and is embodied through measurement.[7] We can estimate the reliability of a measurement, but not the reliability of a construct. For example, you can say the reliability of "a measure of height", but not the reliability of the concept of "height". Construct reliability is a term that is not logically established.

Let's say that this term makes sense. All other reliability coefficients also originate from the measurement of a construct and should be called construct reliability. Construct reliability is not suitable as a term referring to a specific reliability coefficient.

The term has been used in the books of Hair and his colleagues,[8][9][10][11][12] the world's best sellers for practical statistical analysis.

Applied researchers use the term construct reliability at a frequency of 1/3 of composite reliability.[13] Researchers who publish papers on reliability rarely use this name.

Coefficient omega

Source:[14]

Various SEM-based reliability coefficients are referred to as [math]\displaystyle{ \omega }[/math], typically without a definition. Therefore, it is difficult for readers to know exactly what the name [math]\displaystyle{ \omega }[/math] refers to. This practice reduces the accuracy of communication. If we need a generic name to refer to a variety of reliability coefficients, using [math]\displaystyle{ \rho }[/math] rather than [math]\displaystyle{ \omega }[/math] is more traditional.

The name coefficient [math]\displaystyle{ \omega }[/math] is based on McDonald's (1985, 1999)[15][16] claim that McDonald (1970)[17] first developed [math]\displaystyle{ \rho_{C} }[/math]. In his paper on exploratory factor analysis (EFA), McDonald (1970)[17] presents a reliability formula using the [math]\displaystyle{ \theta }[/math] symbol. This formula was included in the footnote of the article without any explanation. McDonald (1985)[15] refers to a formula algebraically equivalent to [math]\displaystyle{ \rho_{C} }[/math] as [math]\displaystyle{ \omega }[/math] in his book. He also says that the [math]\displaystyle{ \theta }[/math] presented by McDonald (1970) is renamed [math]\displaystyle{ \omega }[/math]. McDonald (1999)[16] describes various types of reliability coefficients (e.g., unidimensional and multi-dimensional models) as [math]\displaystyle{ \omega }[/math]. He explicitly declares that he first proposed [math]\displaystyle{ \rho_{C} }[/math]. McDonald (1985, 1999)[15])[16] does not cite Jöreskog (1971) or Werts et al. (1974).

The following objections were made. First, the formula proposed by McDonald (1970) was not new. If this formula were of high academic value at that time, it would not have been presented without explanation in the footnotes. In the context of EFA, there are studies suggesting similar reliability formulas.[18][19] Second, McDonald (1970)'s [math]\displaystyle{ \theta }[/math] differs from [math]\displaystyle{ \rho_{C} }[/math]. The denominator of the formula given by McDonald (1970) is observed covariances, and the denominator of [math]\displaystyle{ \rho_{C} }[/math] is fitted covariances. Third, McDonald (1970) did not discuss how to actually obtain this coefficient. While it is easy to derive a reliability formula, the more important barrier at that time was how to obtain estimates of each parameter. Jöreskog has addressed this issue across studies.[20][21][2] Fourth, it was Jöreskog (1971) that actually influenced users. McDonald (1970) was occasionally cited in EFA literature, but rarely cited in reliability literature. The expression coefficients [math]\displaystyle{ \omega }[/math] was rarely used before 2009.

Applied researchers rarely use this name. Researchers who publish papers on reliability often use this name recently.[22][23][24][25][26][27][28]

Congeneric reliability

Unlike other names that give no information about the characteristics of the coefficients, the name congeneric reliability contains information about when this coefficient should be used.

Jöreskog (1971) did not propose a name for [math]\displaystyle{ \rho_{C} }[/math], but referred to the measurement model from which [math]\displaystyle{ \rho_{C} }[/math] was derived as a congeneric model. The name congeneric reliability has been used occasionally in reliability literature since then.[29][30] Cho (2016)[1] proposed that this coefficient be referred to as [math]\displaystyle{ \rho_{C} }[/math] for a consistent system with other reliability coefficients.

Related coefficients

A related coefficient is average variance extracted.

References

  1. ↑ 1.0 1.1 1.2 1.3 1.4 Cho, E. (2016). Making reliability reliable: A systematic approach to reliability coefficients. Organizational Research Methods, 19(4), 651–682. https://doi.org/10.1177/1094428116656239
  2. ↑ 2.0 2.1 Jöreskog, K. G. (1971). Statistical analysis of sets of congeneric tests. Psychometrika, 36(2), 109–133. https://doi.org/10.1007/BF02291393
  3. ↑ Werts, C. E., Linn, R. L., & Jöreskog, K. G. (1974). Intraclass reliability estimates: Testing structural assumptions. Educational and Psychological Measurement, 34, 25–33. https://doi.org/10.1177/001316447403400104
  4. ↑ Werts, C. E., Rock, D. R., Linn, R. L., & Jöreskog, K. G. (1978). A general method of estimating the reliability of a composite. Educational and Psychological Measurement, 38(4), 933–938. https://doi.org/10.1177/001316447803800412
  5. ↑ Google search in April 2020 found 462,000 web documents which included the term "composite reliability".
  6. ↑ Podsakoff, P. M., MacKenzie, S. B., & Podsakoff, N. P. (2016). Recommendations for creating better concept definitions in the organizational, behavioral, and social sciences. Organizational Research Methods, 19(2), 159–203. https://doi.org/10.1177/1094428115624965
  7. ↑ Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), 281–302. https://doi.org/10.1037/h0040957
  8. ↑ Hair, J. F., Anderson, R. E., Tatham, R. L., & Black, W. C. (1995). Multivariate data analysis with readings (4th ed.). Prentice Hall.
  9. ↑ Hair, J. F., Babin, B. J., Anderson, R. E., & Black, W. C. (2018). Multivariate data analysis (8th ed.). Cengage.
  10. ↑ Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2009). Multivariate data analysis (7th ed.). Pearson.
  11. ↑ Hair, J. F., Black, W. C., Babin, B. J., Anderson, R. E., & Tatham, R. L. (2005). Mulltivariate data analysis (6th ed.). Pearson.
  12. ↑ Hair, J. F., Tatham, R. L., Anderson, R. E., & Black, W. C. (1998). Multivariate data analysis (5th ed.). Prentice Hall.
  13. ↑ Google search in April 2020 found 149,000 web documents which included the term "construct reliability".
  14. ↑ Cho, E. and Chun, S. (2018), Fixing a broken clock: A historical review of the originators of reliability coefficients including Cronbach’s alpha. Survey Research, 19(2), 23–54.
  15. ↑ 15.0 15.1 15.2 McDonald, R. P. (1985). Factor analysis and related methods. Hillsdale, NJ: Lawrence Erlbaum.
  16. ↑ 16.0 16.1 16.2 McDonald, R. P. (1999). Test theory: A unified treatment. Mahwah, NJ: Lawrence Erlbaum.
  17. ↑ 17.0 17.1 McDonald, R. P. (1970). Theoretical canonical foundations of principal factor analysis, canonical factor analysis, and alpha factor analysis. British Journal of Mathematical and Statistical Psychology, 23, 1-21. doi:10.1111/j.2044-8317.1970.tb00432.x.
  18. ↑ Bentler, P. M. (1968). Alpha-maximized factor analysis (alphamax): Its relation to alpha and canonical factor analysis. Psychometrika, 33(3), 335–345. https://doi.org/10.1007/BF02289328
  19. ↑ Heise, D. R., & Bohrnstedt, G. W. (1970). Validity, invalidity, and reliability. Sociological Methodology, 2, 104-129. doi:10.2307/270785.
  20. ↑ Jöreskog, K. G. (1969). A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 34(2), 183–202. https://doi.org/10.1007/BF02289343
  21. ↑ Jöreskog, K. G. (1970). A general method for analysis of covariance structures. Biometrika, 57, 239-251. doi:10.1093/biomet/57.2.239.
  22. ↑ Brunner, M., Nagy, G., & Wilhelm, O. (2012). A Tutorial on Hierarchically Structured Constructs. Journal of Personality. https://doi.org/10.1111/j.1467-6494.2011.00749.x
  23. ↑ Deng, L., & Chan, W. (2017). Testing the difference between reliability coefficients alpha and omega. Educational and Psychological Measurement, 77(2), 185–203. https://doi.org/10.1177/0013164416658325
  24. ↑ Dunn, T. J., Baguley, T., & Brunsden, V. (2014). From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. British Journal of Psychology, 105(3), 399–412. https://doi.org/10.1111/bjop.12046
  25. ↑ Green, S. B., & Yang, Y. (2015). Evaluation of Dimensionality in the Assessment of Internal Consistency Reliability: Coefficient Alpha and Omega Coefficients. Educational Measurement: Issues and Practice, 34(4), 14–20. https://doi.org/10.1111/emip.12100
  26. ↑ Padilla, M. (2019). A Primer on Reliability via Coefficient Alpha and Omega. Archives of Psychology, 3(8), Article 8. https://doi.org/10.31296/aop.v3i8.125
  27. ↑ Padilla, M. A., & Divers, J. (2016). A Comparison of Composite Reliability Estimators: Coefficient Omega Confidence Intervals in the Current Literature. Educational and Psychological Measurement, 76(3), 436–453. https://doi.org/10.1177/0013164415593776
  28. ↑ Revelle, W., & Zinbarg, R. E. (2009). Coefficients alpha, beta, omega, and the glb: Comments on Sijtsma. Psychometrika, 74(1), 145–154. https://doi.org/10.1007/s11336-008-9102-z
  29. ↑ Graham, J. M. (2006). Congeneric and (Essentially) Tau-Equivalent Estimates of Score Reliability What They Are and How to Use Them. Educational and Psychological Measurement, 66(6), 930–944. https://doi.org/10.1177/0013164406288165
  30. ↑ Lucke, J. F. (2005). “Rassling the Hog”: The Influence of Correlated Item Error on Internal Consistency, Classical Reliability, and Congeneric Reliability. Applied Psychological Measurement, 29(2), 106–125. https://doi.org/10.1177/0146621604272739

External links

  • RelCalc, tools to calculate congeneric reliability and other coefficients.
  • Handbook of Management Scales, Wikibook that contains management related measurement models, their indicators and often congeneric reliability.



Retrieved from "https://handwiki.org/wiki/index.php?title=Congeneric_reliability&oldid=3387703"