\documentclass[nojss]{jss}

\usepackage[utf8]{inputenc}
%% need no \usepackage{Sweave}
\SweaveOpts{engine = R, keep.source=TRUE, eps = FALSE, echo = TRUE}

%\VignetteIndexEntry{Using the raschtree Function for Detecting Differential Item Functioning in the Rasch Model}
%\VignetteDepends{psychotree, colorspace}
%\VignetteKeywords{item response theory, IRT, Rasch model, differential item functioning, DIF, structural change, multidimensionality}
%\VignettePackage{psychotree}

\renewcommand{\rm}[0]{Rasch model}
%\newcommand{\dif}[0]{differential item functioning}
\newcommand{\dif}[0]{DIF}
\newcommand{\ip}[0]{item parameter}
\newcommand{\mob}[0]{model-based recursive partitioning}
\newcommand{\rt}[0]{Rasch tree}

%% math commands
\newcommand{\argmin}{\operatorname{argmin}\displaylimits}
\newcommand{\argmax}{\operatorname{argmax}\displaylimits}
\newcommand{\indic}{I}
\newcommand{\ui}{\underline{i}}
\newcommand{\oi}{\overline{\imath}}
\newcommand{\bs}[1]{\boldsymbol{#1}}

\newcommand{\fixme}[1]{\textcolor{red}{#1}}


\title{Using the \code{raschtree} function for detecting differential item functioning in the Rasch model}
\Plaintitle{Using the raschtree function for detecting differential item functioning in the Rasch model}
\Shorttitle{Using \texttt{raschtree} for detecting DIF in the Rasch model}
\author{Carolin Strobl\\Universit\"at Z\"urich
\And Julia Kopf\\Ludwig-Maximilians-\\Universit\"at M\"unchen
\And Achim Zeileis\\Universit\"at Innsbruck}
\Plainauthor{Carolin Strobl, Julia Kopf, Achim Zeileis}


\Abstract{
The \pkg{psychotree} package contains the function \code{raschtree}, that can be used to detect differential item functioning (\dif ) in the Rasch model.  
The \dif\ detection method implemented in \code{raschtree} is based on the
\mob\ framework of \citet{Zeietal:2008} and employs generalized M-fluctuation tests \citep{ZeiHor:2007} for detecting differences in the item parameters between different groups of subjects. The statistical methodology behind \code{raschtree} is described in detail in \citet{Stretal:2010:raschtree:techreport}.
The main advantage of this approach is that it allows to detect groups of subjects exhibiting \dif , that are not pre-specified, but are detected automatically from combinations of covariates. 
In this vignette, the practical usage of \code{raschtree} is illustrated.
}

\Keywords{Item response theory, IRT, \rm, differential item functioning, DIF, structural change, multidimensionality}

\Address{
  Carolin Strobl\\
  Department of Psychology\\
  Universit\"at Z\"urich\\
  Binzm\"uhlestr.~14\\
  8050 Z\"urich, Switzerland\\
  E-mail: \email{Carolin.Strobl@psychologie.uzh.ch}\\
  URL: \url{http://www.psychologie.uzh.ch/fachrichtungen/methoden.html}\\

  Julia Kopf\\
  Department of Statistics\\
  Ludwig-Maximilians-Universit\"at M\"unchen\\
  Ludwigstra{\ss}e 33\\
  80539 M\"unchen, Germany\\
  E-mail: \email{Julia.Kopf@stat.uni-muenchen.de}\\

  Achim Zeileis\\
  Department of Statistics\\
  Faculty of Economics and Statistics\\
  Universit\"at Innsbruck\\
  Universit\"atsstr.~15\\
  6020 Innsbruck, Austria\\
  E-mail: \email{Achim.Zeileis@R-project.org}\\
}

\begin{document}

\section{Differential item functioning in the Rasch model}

A key assumption of the \rm\ is that the item parameter estimates should not depend on the person sample (and vice versa). This assumption may be violated if certain items are easier or harder to solve for certain groups of subjects -- regardless of their true ability -- in which case we speak of differential item functioning (\dif ).

In order to detect \dif\ with the \code{raschtree} function, the item responses and all covariates that should be tested for \dif\ need to be handed over to
the method, as described below. Then the following steps are conducted:
%
\begin{enumerate}
  \item At first, one joint \rm\ is fit for all subjects.
  \item Then it is tested statistically whether the item parameters differ along any of the covariates.
  \item In that case the sample is split along that covariate and two separate \rm s are estimated.
  \item This process is repeated as long as there is further \dif\ (and the subsample is still large enough).
\end{enumerate}

For details on the underlying statistical framework implemented in \code{raschtree} see \citet{Stretal:2010:raschtree:techreport}. 

The main advantage of the \rt\ approach is that \dif\ can be detected between groups of subjects created by more than one covariate. For example, certain items may be easier for male subjects over the age of 40 as opposed to all other subjects. In this case \dif\ is associated with an interaction of the variables gender and age, rather than any one variable alone. 

Moreover, with this approach it is not necessary to pre-define cutpoints in continuous variables, as would be the standard approach when using, e.g., a
likelihood ratio or Wald test: Usually, age groups are pre-specified, for example by means of splitting at the median. However, the median may not be where the actual parameter change occurs -- it could be that only very young or very old subjects find certain items particularly easy or hard. By splitting at the median this effect may be disguised.
Therefore, the Rasch tree method searches for the value corresponding to the strongest parameter change and splits the sample at that value. Certain statistical techniques are necessary for doing this in a statistically sound way, as described in detail in \citet{Stretal:2010:raschtree:techreport}.
 
Now the practical application of \code{raschtree} is outlined, starting with the data preparation. 

\section{Data preparation}

When using \code{raschtree} for the first time, the \pkg{psychotree} package needs to be installed first:
%
<<prep_install, eval=FALSE>>=
install.packages("psychotree")
@
%
After this, the package is permanently installed on the computer, but needs to be made available at the start of every new \proglang{R} session:
%
<<prep_library>>=
library("psychotree")
@
%
The package contains a data example for illustrating the \rt s, that can be loaded with: 
%
<<prep_data>>=
data("SPISA", package = "psychotree")
@
%
The data set \code{SPISA}
consists of the item responses and covariate values of \Sexpr{nrow(SPISA)} subjects. 
It is a subsample of a larger data set from an online quiz, that was carried out by the German weekly news magazine SPIEGEL in 2009
via the online version of the magazine SPIEGEL Online (SPON). The quiz was designed for testing one's general knowledge and consisted of a total of 45~items from five different topics: politics, history, economy, culture and natural sciences. A thorough analysis and discussion of the original data set is provided in \citet{SPISA:book}.

The data are structured in the following way: The variable \code{spisa} contains the 0/1-responses of all subjects to all test items
(i.e., \code{spisa} is only a single variable but contains a matrix of responses).
In addition to that, covariates like age and gender are available for each subject: 


\begin{center}
\begin{tabular}{|ccccccccccc|ccccc|}
\hline
\multicolumn{11}{|c|}{Item reponses} &  \multicolumn{5}{c|}{Covariates}\\ 
\multicolumn{11}{|c|}{\code{\Sexpr{names(SPISA)[1]}}} 
     & \code{\Sexpr{names(SPISA)[2]}} & \code{\Sexpr{names(SPISA)[3]}} & \code{\Sexpr{names(SPISA)[4]}} 
     & \code{\Sexpr{names(SPISA)[5]}} & \code{\Sexpr{names(SPISA)[6]}}\\
\hline
1 & 0 & 0 & 1 & 1 & $\cdots$ & 0 & 1 & 1 & 1 & 1 & female & 21 & 3 & no & 1--3/month\\
0 & 1 & 0 & 1 & 1 & $\cdots$ & 1 & 1 & 1 & 1 & 1 & male & 20 & 1 & no & 4--5/week\\
0 & 0 & 0 & 1 & 0 & $\cdots$ & 0 & 1 & 1 & 1 & 1 & female & 25 & 9 & no & 1--3/month\\
0 & 0 & 1 & 1 & 1 & $\cdots$ & 1 & 1 & 0 & 1 & 1 & male & 27 & 10 & no & never\\
1 & 1 & 1 & 1 & 1 & $\cdots$ & 0 & 0 & 1 & 1 & 1 & male & 24 & 8 & no & 1/week\\
1 & 0 & 0 & 1 & 0 & $\cdots$ & 1 & 1 & 0 & 1 & 1 & male & 20 & 1 & yes & 1--3/month\\
  &   &   &   &   & $\vdots$ &   &   &   &   &   & $\vdots$ & $\vdots$ & $\vdots$ & $\vdots$ & $\vdots$ \\
\hline 
\end{tabular}
\end{center}

\medskip

If your own data set, termed for example \code{mydata}, is in a different format, it is easy to change it into the right format for \code{raschtree}. For example, if the item responses are coded as individual variables like this:

\begin{center}
\begin{tabular}{|ccccc|ccc|}
\hline
\multicolumn{5}{|c|}{Item reponses} &  \multicolumn{3}{c|}{Covariates}\\ 
\code{item1} & \code{item2} & \code{item3} & \code{item4} & \code{item5} & \code{gender} & \code{age} & \code{semester} \\
\hline
1  &  0  &  0  &  1  &  1   & female & 21 & 3 \\
0  &  1  &  0  &  1  &  1   & male & 20 & 1 \\
0  &  0  &  0  &  1  &  0   & female & 25 & 9 \\
0  &  0  &  1  &  1  &  1   & male & 27 & 10 \\
1  &  1  &  1  &  1  &  1   & male & 24 & 8 \\
\hline 
\end{tabular}
\end{center}

\medskip

You can bring them into more convenient format by first defining a new variable \code{resp} that contains the matrix of item responses
(i.e., first five columns of \code{mydata}):
%
<<recode, eval=FALSE>>=
mydata$resp <- as.matrix(mydata[ , 1:5])
@
%
Then you can omit the original separate item response variables from the data set
%
<<recode2, eval=FALSE>>=
mydata <- mydata[ , -(1:5)]
@
%
The data set then contains both the complete matrix of item responses -- termed \code{resp} -- and the covariates as individual columns, so that later it is easier to address the complete matrix of item responses in the function call.  

If the item responses include cases where all observed item responses are 0 or all observed item responses are 1, these cases need to be excluded prior to
model fitting. (These cases do not contribute to the Rasch model anyway, because they do not contain any information on which items are easier or harder to solve, and the corresponding person parameters are not identified.)
For example, the cases
indicated by arrows in the example below would need to be excluded, because all observed item responses are either 0 or 1:


\begin{center}
\begin{tabular}{c|cccccccccc|}
\cline{2-11}
&\multicolumn{10}{|c|}{Item reponses} \\ 
&\multicolumn{10}{|c|}{\code{resp}} \\
\cline{2-11}
             & 1  & NA  &  0  &  1  &  1  &  1  &  0  &  1  &  1  &  1  \\ 
$\Rightarrow$& 1  &  1  &  1  &  1  &  NA  &  1  &  1  &  1  &  1 &  1  \\
$\Rightarrow$& 0  &  0  &  NA  &  0  &  0  &  0  &  0  &  0  &  NA  &  0  \\
             & 0  &  0  &  1  &  1  &  1  &  0  &  1  &  0  &  1  &  0  \\
$\Rightarrow$& 1  &  1  &  1  &  1  &  1  &  1  &  1  &  1  &  1  &  1  \\
\cline{2-11}
\end{tabular}
\end{center}

\medskip

To exclude rows where all observed item responses are either 0 or 1, we select only the subset of cases for which the proportion of correct item responses is strictly between 0 and 1 for futher analysis.
%
<<delete_cases, eval=FALSE>>=
mydata <- subset(mydata, rowMeans(resp, na.rm = TRUE) > 0 &
  rowMeans(resp, na.rm = TRUE) < 1)
@
%
Now the data preparation is done and we can fit a \rt .

\section{Model fitting, plotting and extraction of parameter values}

The idea of \rt s is to model differences in the \rm\ for the item responses by means of the covariates. This idea translates intuitively into the formula
interface that is commonly used in \proglang{R} functions, such as \code{lm} for linear models: In a linear model, where the response variable \code{y} is
modeled by the covariates \code{x1} and \code{x2}, the formula in \proglang{R} looks like this:
%
\begin{center}
\code{y ~ x1 + x2}
\end{center}
%
Very similarly, in the \rt\ for our \code{SPISA} data, where the item responses \code{spisa} are modeled by the covariates \code{age}, \code{gender}, \code{semester}, \code{elite} and \code{spon}, the formula used in \code{raschtree} looks like this:
%
\begin{center}
\code{spisa ~ age + gender + semester + elite + spon}
\end{center}
%
The complete call is
%
<<fit_raschtree, eval=FALSE>>=
my_first_raschtree <- raschtree(spisa ~ age + gender +
  semester + elite + spon, data = SPISA)
@
%
<<fit_raschtree_cache, echo=FALSE, results=hide>>=
if(file.exists("raschtree-spisa.rda")) load("raschtree-spisa.rda") else {
<<fit_raschtree>>
save(my_first_raschtree, file = "raschtree-spisa.rda")
}
file.remove("raschtree-spisa.rda")
@
%
Note that the model is not only fitted, but also saved under the name \code{my_first_raschtree}, so that we can later extract information from the fitted model object and plot the \rt .

As a shortcut, when all other variables in the data set are to be used as covariates, as in our example, the covariates do not have to be listed explicitly
in the formula but can replaced by a dot, as in 
\code{raschtree(spisa ~ ., data = SPISA)}
(leading to equivalent output as the call above).
Moreover, if you want to see the process of the \rt\ fitting, including the computation of the $p$-values and corresponding split decisions in each step, you can use the \code{verbose} option, as in
\code{raschtree(spisa ~ ., data = SPISA, verbose = TRUE)}.
The \code{verbose} option also has the advantage that you can see something happening on your screen when \code{raschtree} takes a while to complete -- which may be the case if there are many variables with \dif\ and if these variables offer many possible cutpoints, like continuous variables and factors with many categories.

The resulting \rt\ can then be plotted with the generic \code{plot} call:
%
\setkeys{Gin}{width=\textwidth}
\begin{center}
<<plot_raschtree, eval=TRUE, eps=FALSE, fig=TRUE, height=6, width=14>>=
plot(my_first_raschtree)
@
\end{center}

The plot function also accepts many options for standard plot functions, including coloring. Here, a set of ``rainbow''
colors \citep[from package \pkg{colorspace}, see][]{Zeietal:2009} is employed to indicate the blocks of nine items
from each of the five different topics covered in the quiz: politics, history, economy, culture and natural sciences:
%
\begin{center}
<<plot_raschtree_col, eval=TRUE, eps=FALSE, fig=TRUE, height=6, width=14>>=
library("colorspace")
plot(my_first_raschtree, 
      col = rep(rainbow_hcl(5, c = 65, l = 65), each = 9))
@
\end{center}


For extracting the estimated item parameters for each group, there are two different calls corresponding to the two different ways to scale the item parameters: The parameters of a \rm\ are unique only up to linear transformations. In particular, the origin of the scale is not fixed but chosen arbitrarily. There are two common ways to choose the origin: setting one item parameter to zero or setting the sum of all item parameters to zero. 
Accordingly, there are two calls to extract the item parameters from \code{raschtree} one way or the other:

\newpage %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% VORSICHT kuenstlicher Seitenwechsel! %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

<<coef_raschtree>>=
coef(my_first_raschtree, node = 4)
@

where the parameter for the first item is set to zero and therefore not displayed (the call is termed \code{coef}, because that is the name of the call extracting the estimated parameters, or coefficients, from standard regression models generated, e.g., with the \code{lm} function) and 

<<itempar_raschtree>>=
itempar(my_first_raschtree, node = 4)
@

where the item parameters by default sum to zero (other restrictions can be specified as well). 

Here the item parameters have been displayed only for the subjects in node number 4 (representing female students who access the online magazine more than once per week) to save space. The item parameters for all groups can be displayed by omitting the \code{node} argument.


\section{Interpretation}

Ideally, if none of the items showed \dif , we would find a tree with only one single node. In this case, one joint, unidimensional \rm\ would be appropriate to describe the entire data set. 

If however, the \rt\ shows at least one split, \dif\ is present and it is not appropriate to compare the different groups of subjects with the test. 
The \dif\ may be caused by certain characteristics of the items, such as their wording, but may also be an indicator of multidimensionality: 

If, for example, certain groups of subjects are disadvantaged by the wording or content of certain items, it would be unfair to compare the different groups with the test including those items. 
In practice, items showing \dif\ will then be excluded from the test before rating the subjects' performance. Sometimes it is also possible to rephrase the items, for example when \dif\ is present only for subjects who are not native speakers of the test language.

If, however, in a general knowledge quiz, e.g., all history items are easier for a particular group of subjects, this may indicate that history knowledge should be considered as a sub-dimension of general knowledge (in which the particular group happens to outperform the others). In this case, a multidimensional \rm\ would be called for (that is unfortunately not  available in \proglang{R} yet). 

Note in particular that when one joint, unidimensional \rm\ is not appropriate to describe the test, this also means that a ranking of the subjects based on the raw scores (i.e., the number of items that each subject answered correctly) is not appropriate either, because this would also assume that the test is unidimensional. 

\section{Outlook}

We are currently working on functionality for facilitating the interpretation of the \rt s by means of summarizing in tables which items show the strongest
\dif\ with respect to which groups, and on generalizations of the method to, e.g., the partial credit model for items with more than two response categories. 

\section*{Acknowledgements}
The work of Carolin Strobl was supported by grant STR1142/1-1 (``Methods to Account for Subject-Co\-vari\-ates
in IRT-Models'') from the German Research Foundation (Deutsche Forschungsgemeinschaft). The authors would like to thank Reinhold Hatzinger for important
insights stimulated by conversations and the \proglang{R}~package \pkg{eRm} 
\citep{MaiHat:2007,eRm:pkg}. 


\bibliography{psychotree}

\end{document}

