In their 1986 book on test theory, crocker and algina defined a random. To provide comparisons and a worked example of item and scalelevel evaluations based on three psychometric methods used in patientreported outcome developmentclassical test theory ctt, item response theory irt, and rasch measurement theory rmtin an analysis of the national eye institute visual functioning questionnaire vfq25. For example, in modeling especially with regards to item what is often referred to as the classical test model, development. Item response theory irt has a number of potential advantages over classical test theory in assessing selfreported health outcomes.
Applying item response theory modeling in educational research. Item response theory irt, also known as latent trait theory or modern mental test theory. Jul 15, 2015 classical test theory and item response theory 1. The ctt and irt were compared across two samples and two forms of test on their item difficulty, internal consistency, and measurement errors. Classical test theory and item response theory lrt will be described in relation to approaches to measure the validity and reliability. In this sense, classical test theory ctt has been extensively serving the testing field for about 100 years. The purposes of this instructional module are a to focus attention on the similarities and differences between classical test theory and item response theory and related. Request pdf an empirical comparison of item response theory and classical test theory itemperson statistics in the theory of measurement.
Both classical test theory sum scores and item response theory estimates measure the same underlying dimension, but differences in the two scales may lead one to be more preferential than the other in interpreting data. Applying item response theory modeling in educational research daitrang le iowa state university follow this and additional works at. Comparisons between classical test theory and item response. Classical test theory ctt and itemresponse theory irt are testing item assessment approaches. Here is two empirical example of comparison between these two methods.
Item response theory irt looks at the examinees performance by using item as the unit of assessment. To provide comparisons and a worked example of item and scalelevel evaluations based on. Jun 28, 2009 the present report demonstrates the difference between classical test theory ctt and item response theory irt approach using an actual test data for chemistry junior high school students. Embretsons new rules of measurement, for example, is a nice book that i. Mar 25, 2010 patientsreported outcomes pro are increasingly used in clinical and epidemiological research. In keeping with the tenets of this book, the goal of this chapter is to induce broad and. Nov 30, 2010 this study compares the psychometric utility of classical test theory ctt and item response theory irt for scale construction with data from higher education student surveys.
An ncme instructional module on comparison of classical. This study examined the psychometric properties of the model of human occupation screening tool, using both item response theory and classical test theory. Using 2008 your first college year yfcy survey data from the cooperative institutional research program at the higher education research institute at ucla, two scales are built and testedone measuring social. However, whether irt or ctt would be the most appropriate method to analyse pro data remains unknown. Item analysis classical latent trait models rasch item response theory irt1 irt2 irt3 irt4 classical test theory classical analysis is the easiest and most widely used form of analysis. Irt models yield invariant item and latent trait estimates within a linear transformation, standard errors conditional on trait level, and trait estimates anchored to item content. Comparing classical test theory and item response theory. However, this is only partially reflected in the psychometric practice. Educational and psychological measurem june 1998 v58 n3. Students ranking, based on their abilities on objective type. Comparisons between classical test theory and item. Item selection using ctt and irt with unrepresentative samples. Basics of classical test theory theory and assumptions types of reliability example classical test theory classical test theory ctt often called the true score model called classic relative to item response theory irt which is a more modern approach ctt describes a set of psychometric procedures used to test items and scales. An ncme instructional module on educational measurement.
Classical test theory vs item response theory by chris allred. Item response theory complements and contrasts classical test theory ctt, which is the predominant psychometric theory taught in undergraduate and graduate programs. A comparative study of classical theory ct and item. Trait true score observed score classical test theory. Classical test theory differs from irt in several ways that will be discussed throughout this entry. This study empirically examined the behaviors of the item and person statistics derived. Two main types of analytical strategies can be found for these data. Patientsreported outcomes pro are increasingly used in clinical and epidemiological research. Comparison of classical test theory and item response theory and their applications to test development. An empirical comparison of item response theory and classical test. Educational and psychological measurem june 1998 v58 n3 p357. Irt arose out of the acclaimed limitations of classical test theory ctt. Theories of measurement help to explain measurement results i. Whereas classical test theory focuses on the test as a whole, item response theory shifts its focus to the individual items questions themselves.
In part, this may be due to advertising about the advantages of irt over ctt. This study compared classical test theory ctt and item response theory irt. Irt is an example of what psychologists call a latent trait. The conceptual foundations, assumptions, and extensions of the basic premises of ctt have allowed for the development of some excellent psychometrically sound scales. One hundred and one people with mental health problems, aged 1865 years, were recruited. It is somewhat surprising that empirical studies examining andor comparing the. Comparison of classical test theory and item response theory and their applications to. An empirical comparison of item response theory and. May 31, 2015 classical test theory ctt and item response theory irt are testing item assessment approaches.
Kline 2005 suggests ctt is known for development of some excellent psychometrically sound. Comparison of classical test theory and item response. True t or f cross cultural fairness in testing has always been a critical factor in the development of tests. Irt may be regarded as roughly synonymous with latent trait theory. Classical test theory ctt and itemresponse theory irt classical test theory ctt and itemresponse theory irt are testing item assessment approaches. It is understood that in the ctt framework, person and item statistics are test and sampledependent.
Item response theory and health outcomes measurement in the. Part of theinstructional media design commons, and thestatistics and probability commons. Classical test theory assumptions, equations, limitations, and item analyses c lassical test theory ctt has been the foundation for measurement theory for over 80 years. Classical test theory ctt is a measurement theory used primarily in psychology, education, and related fields. Within that theoretical framework, models of theory ctt and item response theory irt various forms have been formulated. Professor of education and psychology at the university of massachusetts, hills south, room 152, amherst, ma 01003. Eric ed466779 classical test theory and item response. An application of item response theory to psychological test. An ncme instructional module on comparison of classical test. Kline 2005 suggests ctt is known for development of some excellent psychometrically sound scales, founded by charles spearman around 1904. Item response theory provides powerful analytical tools that, even in their most basic applications, can be a valuable. His specializations are item response theory and applications and measurementpractices.
Classical test theory is an influential theory of test scores in the social sciences. These properties of irt are also the main theoretical advantages of. Item response theory irt vs classical test theory ctt. Using classical test theory, item response theory, and rasch. Methodological issues regarding power of classical test. Relationships among classical test theory and item response. The statistics produced under ctt include measures of item difficulty. A brief introduction to classical test theory, generalisability theory and item response theory classical test theory is the most common measurement theory used and dates back to work done by charles spearman 1904a, 1904b, 1927 at the turn of the last century. In ctt, the item difficulty index p p value, the proportion of examinees passing an item, expresses item difficulty on an ordinal scale not on an interval scale. Overview of classical test theory and item response theory. Item response theory irt is, for some researchers, the answer to the limitations of classical test theory as stated by courville 2004, p. Although the 3rd edition was ed in 2008, there have been no revisions to the text since the 1980s. The statistics can be computed by generic statistical packages or at a push by hand and need no specialist software.
Item reponses theory ctt testoriented indices like reliability are groupspecific scores are testspecific contribution of item measured using other items e. Item response theory is a newer theory with a focus on test items that adds more tools for solving measurement problems in psychology test bias adaptive testing item selection ctt focuses more on the total score of a scale or subscale. The intent of this module is to provide a comparison of classical theory and item response theory. Basics of classical test theory california state university. Using 2008 your first college year yfcy survey data from the cooperative institutional research program at the higher education research institute at ucla, two scales are built and testedone measuring. In psychometrics, the theory has been superseded by the more sophisticated models in item response theory irt and generalizability theory g theory. An empirical comparison of item response theory and classical test theory spela progar1 and gregor socan2 1mirna pec, slovenija 2university of ljubljana, department of psychology, ljubljana, slovenia abstract. Comparison of classical test theory and item response theory. This study compares the psychometric utility of classical test theory ctt and item response theory irt for scale construction with data from higher education student surveys. Item response theory, graded response model, psychological assessment, affects background valid and reliable measures are essential to the field of psychology, as well as, to the study of abilities, aptitudes, and attitudes. The present report demonstrates the difference between classical test theory ctt and item response theory irt approach using an actual test data for chemistry junior high school students.
Part ii, comparison between item analysis based on irt and ctt, is a. It is usually represented by the following formula. The only downside is that the text is a little dated. In psychometrics, item response theory irt also known as latent trait theory, strong true score theory, or modern mental test theory is a paradigm for the design, analysis, and scoring of tests, questionnaires, and similar instruments measuring abilities, attitudes, or other variables. In psychometrics, item response theory irt also known as latent trait theory, strong true score theory, or modern mental test theory, is a paradigm for the design, analysis, and scoring of tests, questionnaires, and similar instruments measuring abilities, attitudes, or other variables. Itemresponse theory irt appears to be the currently prevailing paradigm within the psychometric theory.
See how well you understand the theories psychologists use to create tests, the item response theory and the classical test theory, with this. Test construction using ctt and irt with unrepresentative samples item response theory irt has clearly gained mindshare among io psychologists and researchers, as well as psychometricians. Both classical test theory sum scores and item response theory estimates measure the same underlying dimension, but differences in the two scales may lead one to be more preferential than the. In psychometrics, the theory has been superseded by the more sophisticated models in item response theory irt and generalizability theory gtheory. Despite theoretical differences between item response theory irt and classical test theory ctt, there is a lack of empirical knowledge about how, and to what extent, the irt and cttbased item and person statistics behave differently. This isnt a big problem on the classical test theory chapters, but more modern chapters such as the item response theory chapter need updating. Demonstrating the difference between classical test theory. An empirical comparison of item response theory and classical. T or f item response theory has the advantage over classical test theory in that it provides more detailed information regarding each item on a test. Comparisons between classical test theory and item response theory in automated assembly of parallel test forms the journal of technology, learning, and assessment volume 6, number 8 april 2008 a publication of the technology and assessment study collaborative caroline a. Classical test theory ctt, also known as the true score theory, refers to the analysis of test results based on test scores. Irt has been vigorously researched by psychometricians, and numerous books and. Measurement is the process of quantifying the characteristics of a person or object.
Another branch of psychometric theory is the item response theory irt. Based on nonlinear models between the measured latent variable and the item response, item response theory irt enables independent. Item response theory an evaluation of the theory test in the swedish drivinglicense test marie wiberg abstract the swedish drivinglicense test consists of a theory test and a practical road test. Mismatch between individual ability and test difficulty can further. Classical psychometric test theory ctt aims at studying the reliability of a realvalued test score variable measurement, test that maps a crucial aspect of qualitative or quantitative observations into the set of real numbers. The item parameter indices from classical test theory, item response theory, and generalized linear model behaved very similarly, and the correlations were high ranging from. We propose here that item response theory analyses complements the basic ctt techniques presented in janssen and meier 20. It is sometimes referred to as the strong true score theory or modern mental test theory because irt is a more recent body of theory and makes stronger assumptions as compared to classical test theory. The behavior of the item and person statistics derived from these two measurement frameworks was examined analytically and empirically using a data set obtained from bilog r. Classical test theory ctt comprises a set of concepts and methods that provide a basis for many of the measurement tools currently used in health research. Aside from determining the reliability of a test score variable itself ctt allows answering questions such as. Item response theory irt is all about your performance on an exam, and how it relates to individual items or questions on a test. Classical test theory ctt and item response theory irt ctt and its use in test analysis as the name would imply, classical test theory ctt is one traditional way of understanding test scores. There are welldefined theoretical differences between the classical test theory ctt and item response theory irt frameworks.
1218 493 648 1403 159 1276 1326 1379 1545 108 479 588 374 25 432 1413 268 254 204 8 1251 1550 641 1140 1206 984 1421 1273 745 93 1051 1444 157 1073 1119 969 753 10 842 1362 149 1059 726 1128