Social:Person-fit analysis

Short description: Psychometric technique for assessing validity of test scores.

The person-fit analysis is a technique for determining if a person's results on a given test are valid, meaning they are a result of the trait being tested, and not some external factor such as cheating, falling asleep in the middle of the test or otherwise.

An item-score vector is a list of "scores" that a person gets on the items of a test, where "1" is correct and "0" is incorrect. For example, if a person took a ten-item quiz and answered only the first five questions correctly, the vector would be {1, 1, 1, 1, 1, 0, 0, 0, 0, 0}. The analysis can determine how unlikely an item-score vector is compared to a hypothesized test theory model such as item response theory, or compared with the majority of item-score vectors in the sample.

In individual decision-making in fields such as education, psychology, and personnel selection, it is important that test users have confidence in the test scores used. The validity of individual test scores may be threatened when the examinee's answers are governed by factors other than the psychological trait of interest—factors that can range from something as benign as the examinee dozing off to concerted fraud efforts. Person-fit methods are used to detect item-score vectors where such external factors may be relevant, and as a result, indicate invalid measurement.

Unfortunately, person-fit statistics can only tell if the set of responses is likely or unlikely to be valid, and does not prove anything. The results of the analysis might look like an examinee cheated, but the ability to prove it by returning to when the test was administered is not possible. This limits its practical applicability on an individual scale. However, it might be useful on a larger scale; if most examinees at a certain test site or with a certain proctor have unlikely responses, an investigation might be warranted.

References

• Emons, W.H.M., Sijtsma, K., & Meijer, R.R. (2005). Global, local, and graphical person-fit analysis using person response functions. Psychological Methods, 10(1), 101-119.
• Emons, W.H.M., Glas, C.A.W., Meijer, R.R., & Sijtsma, K. (2003). Person fit in order-restricted latent class models. Applied Psychological Measurement, 27(6), 459-478.
• Meijer, R. R., & Sijtsma, K. (2001). Methodology review: Evaluating person-fit. Applied Psychological Measurement, 25, 107-135.