TAM — tam.modelfit" />
tam.modelfit.RdThe function tam.modelfit computes several model fit statistics.
It includes the Q3 statistic (Yen, 1984) and an
adjusted variant of it (see Details). Effect sizes of model fit
(MADaQ3, \(MADRESIDCOV\),
\(SRMR\)) are also available.
The function IRT.modelfit is a wrapper to tam.modelfit,
but allows convenient model comparisons by using the
CDM::IRT.compareModels function.
The tam.modelfit function can also be used for fitted
models outside the TAM package by applying
tam.modelfit.IRT or tam.modelfit.args.
The function tam.Q3 computes the \(Q_3\) statistic based on
weighted likelihood estimates (see tam.wle).
tam.modelfit(tamobj, progress=TRUE) # S3 method for tam.modelfit summary(object,...) # S3 method for tam.mml IRT.modelfit(object, ...) # S3 method for tam.mml.3pl IRT.modelfit(object, ...) # S3 method for tamaan IRT.modelfit(object, ...) # S3 method for IRT.modelfit.tam.mml summary(object, ...) # S3 method for IRT.modelfit.tam.mml.3pl summary(object, ...) # S3 method for IRT.modelfit.tamaan summary(object, ...) tam.modelfit.IRT( object, progress=TRUE ) tam.modelfit.args( resp, probs, theta, post, progress=TRUE ) tam.Q3(tamobj, ... ) # S3 method for tam.Q3 summary(object,...)
| tamobj | Object of class |
|---|---|
| progress | An optional logical indicating whether progress should be displayed |
| object | Object of class |
| resp | Dataset with item responses |
| probs | Array with item response functions evaluated at |
| theta | Matrix with used \(\bold{\theta}\) grid |
| post | Individual posterior distribution |
| ... | Further arguments to be passed |
For each item \(i\) and each person \(n\), residuals \(e_{ni}=X_{ni}-E(X_{ni})\) are computed. The expected value \(E(X_{ni})\) is obtained by integrating the individual posterior distribution.
The Q3 statistic of item pairs \(i\) and \(j\) is defined as the
correlation \(Q3_{ij}=Cor( e_{ni}, e_{nj} )\). The residuals in
tam.modelfit are
calculated by integrating values of the individual posterior distribution.
Residuals in tam.Q3 are calculated by using weighted likelihood
estimates (WLEs) from tam.wle.
It is known that under local independence, the expected value of \(Q_3\)
is slightly smaller than zero. Therefore,
an adjusted Q3 statistic (aQ3; \(aQ3_{ij}\))
is computed by subtracting the average of all Q3 statistics from
Q3. To control for multiple testing,
a p value adjustment by the method of
Holm (p.holm) is employed (see Chen, de la Torre & Zhang, 2013).
An effect size of model fit (MADaQ3) is defined as
the average of absolute values of \(aQ3\) statistics. An equivalent
statistic based on the \(Q_3\) statistic is similar to the
standardized generalized dimensionality discrepancy measure (SGDDM; Levy,
Xu, Yel & Svetina, 2015).
The SRMSR (standardized root mean square root of squared residuals, Maydeu-Olivaras, 2013) is based on comparing residual correlations of item pairs $$ SRMSR=\sqrt{ \frac{1}{ J(J-1)/2 } \sum_{i < j} ( r_{ij} - \hat{r}_{ij} )^2 } $$ Additionally, the SRMR is computed as $$ SRMR=\frac{1}{ J(J-1)/2 } \sum_{i < j} | r_{ij} - \hat{r}_{ij} | $$
The \(MADRESIDCOV\) statistic (McDonald & Mok, 1995) is based on comparing residual covariances of item pairs $$ MADRESIDCOV=\frac{1}{ J(J-1)/2 } \sum_{i < j} | c_{ij} - \hat{c}_{ij} | $$ This statistic is just multiplied by 100 in the output of this function.
A list with following entries
Global fit statistic MADaQ3 and
global model test with p value obtained
by Holm adjustment
Data frame with chi square tests of conditional independence for every item pair (Chen & Thissen, 1997)
Model fit statistics \(100 \cdot MADRESIDCOV\), \(SRMR\) and \(SRMSR\)
Test statistic of global fit based on multiple testing correction of \(\chi^2\) statistics
Q3 and adjusted Q3 statistic for all item pairs
Residuals
Matrix of \(Q_3\) statistics
Matrix of adjusted \(Q_3\) statistics
Summary of \(Q_3\) statistics
Sample size for each item pair
Chen, J., de la Torre, J., & Zhang, Z. (2013). Relative and absolute fit evaluation in cognitive diagnosis modeling. Journal of Educational Measurement, 50, 123-140. doi: 10.1111/j.1745-3984.2012.00185.x
Chen, W., & Thissen, D. (1997). Local dependence indexes for item pairs using item response theory. Journal of Educational and Behavioral Statistics, 22, 265-289.
Levy, R., Xu, Y., Yel, N., & Svetina, D. (2015). A standardized generalized dimensionality discrepancy measure and a standardized model-based covariance for dimensionality assessment for multidimensional models. Journal of Educational Measurement, 52(2), 144--158. doi: 10.1111/jedm.12070
Maydeu-Olivares, A. (2013). Goodness-of-fit assessment of item response theory models (with discussion). Measurement: Interdisciplinary Research and Perspectives, 11, 71-137. doi: 10.1080/15366367.2013.831680
McDonald, R. P., & Mok, M. M.-C. (1995). Goodness of fit in item response models. Multivariate Behavioral Research, 30, 23-40. doi: 10.1207/s15327906mbr3001_2
Yen, W. M. (1984). Effects of local item dependence on the fit and equating performance of the three-parameter logistic model. Applied Psychological Measurement, 8, 125-145. doi: 10.1177/014662168400800201
############################################################################# # EXAMPLE 1: data.cqc01 ############################################################################# data(data.cqc01) dat <- data.cqc01 #***************************************************** #*** Model 1: Rasch model mod1 <- TAM::tam.mml( dat ) # assess model fit res1 <- TAM::tam.modelfit( tamobj=mod1 ) summary(res1) # display item pairs with five largest adjusted Q3 statistics res1$stat.itempair[1:5,c("item1","item2","aQ3","p","p.holm")] if (FALSE) { # IRT.modelfit fmod1 <- IRT.modelfit(mod1) summary(fmod1) #***************************************************** #*** Model 2: 2PL model mod2 <- TAM::tam.mml.2pl( dat ) # IRT.modelfit fmod2 <- IRT.modelfit(mod2) summary(fmod2) # model comparison IRT.compareModels(fmod1, fmod2 ) ############################################################################# # SIMULATED EXAMPLE 2: Rasch model ############################################################################# set.seed(8766) N <- 1000 # number of persons I <- 20 # number of items # simulate responses library(sirt) dat <- sirt::sim.raschtype( stats::rnorm(N), b=seq(-1.5,1.5,len=I) ) #*** estimation mod1 <- TAM::tam.mml( dat ) summary(dat) #*** model fit res1 <- TAM::tam.modelfit( tamobj=mod1) summary(res1) ############################################################################# # EXAMPLE 3: Model fit data.gpcm | Partial credit model ############################################################################# data(data.gpcm) dat <- data.gpcm # estimate partial credit model mod1 <- TAM::tam.mml( dat) summary(mod1) # assess model fit tmod1 <- TAM::tam.modelfit( mod1 ) summary(tmod1) ############################################################################# # EXAMPLE 4: data.read | Comparison Q3 statistic ############################################################################# library(sirt) data(data.read, package="sirt") dat <- data.read #**** Model 1: 1PL model mod1 <- TAM::tam.mml( dat ) summary(mod1) #**** Model 2: 2PL model mod2 <- TAM::tam.mml.2pl( dat ) summary(mod2) #**** assess model fits # Q3 based on posterior fmod1 <- TAM::tam.modelfit(mod1) fmod2 <- TAM::tam.modelfit(mod2) # Q3 based on WLEs q3_mod1 <- TAM::tam.Q3(mod1) q3_mod2 <- TAM::tam.Q3(mod2) summary(fmod1) summary(fmod2) summary(q3_mod1) summary(q3_mod2) }