×

Analysing model fit of psychometric process models: an overview, a new test and an application to the diffusion model. (English) Zbl 1406.91377

Summary: Cognitive psychometric models embed cognitive process models into a latent trait framework in order to allow for individual differences. Due to their close relationship to the response process the models allow for profound conclusions about the test takers. However, before such a model can be used its fit has to be checked carefully. In this manuscript we give an overview over existing tests of model fit and show their relation to the generalized moment test of W. K. Newey [Econometrica 53, 1047–1070 (1985; Zbl 0629.62107)] and G. Tauchen [J. Econom. 30, 415–443 (1985; Zbl 0591.62094)]. We also present a new test, the Hausman test of misspecification [J. A. Hausman, Econometrica 46, 1251–1271 (1978; Zbl 0397.62043)]. The Hausman test consists of a comparison of two estimates of the same item parameters which should be similar if the model holds. The performance of the Hausman test is evaluated in a simulation study. In this study we illustrate its application to two popular models in cognitive psychometrics, the Q-diffusion model and the D-diffusion model [H. von der Maas et al., “Cognitive psychology meets psychometric theory: on the relation between process models for decision making and latent variable models for individual differences”, Psychol. Rev. 118, No. 2, 339–356 (2011; doi:10.1037/a0022749)]; D. Molenaar et al., “Fitting diffusion item response theory models for responses and response times using the R package diffIRT”, J. Stat. Softw. 66, Paper No. 4, 34 p. (2015; doi:10.18637/jss.v066.i04)]. We also compare the performance of the test to four alternative tests of model fit, namely the \(M_2\) test [Molenaar et al., loc. cit.], the moment test [J. Ranger et al., “Limited information estimation of the diffusion-based item response theory model for responses and response times”, Br. J. Math. Stat. Psychol. 69, No. 2, 122-138 (2016; doi:10.1111/bmsp.12064)] and the test for binned time [J. Ranger and J.-T. Kuhn, “Testing fit of latent trait models for responses and response times in tests”, Psychol. Test. Asess. Model. 56, No. 4, 382–404 (2014)]. The simulation study indicates that the Hausman test is superior to the latter tests. The test closely adheres to the nominal Type I error rate and has higher power in most simulation conditions.

MSC:

91E45 Measurement and performance in psychology
91E10 Cognitive psychology

Software:

R; diffIRT
Full Text: DOI