“Classical” or “Traditional” Test Theory
Basic concepts:
xis the observed summed-score
tis the “true score”
eis the “error” x = t + e
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Guidelines and tips
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Community
Ask the community
Ask the community for help and clear up your study doubts
University Rankings
Discover the best universities in your country according to Docsity users
Free resources
Our save-the-student-ebooks!
Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors
Lecture Notes for Classical Test Theory - Tests and Measurement | PSYC 531, Study notes of Psychology
Material Type: Notes; Class: Tests and Measurement; Subject: PSYCHOLOGY; University: University of North Carolina - Chapel Hill; Term: Spring 2009;
Typology: Study notes
Pre 2010
1 / 22
This page cannot be seen from the preview
Don't miss anything!
Related documents
Partial preview of the text
Download Lecture Notes for Classical Test Theory - Tests and Measurement | PSYC 531 and more Study notes Psychology in PDF only on Docsity!
“Classical” or “Traditional” Test Theory
Basic concepts:
x is the observed summed-score
t is the “true score”
e is the “error”
x = t + e
The “true score” is a hypothetical entity with
many definitions. The most commonly-
used definition is that it is E ( x ); that is, it is
the expected value of x , or the mean of all
the x s that would be obtained in
(hypothetically) infinitely many retestings.
The error component is defined by
subtraction:
e = x – t
Reliability
rx = Var ( t )/ Var ( x )
(by definition), and
Var ( t ) = Var ( x ) rx
For two parallel measurements, x and x' :
rxx' = σ xx' /σ x σ x'
σ( t + e )( t + e' )
σ x σ x'
t +^ σ te^ +^ σ te'^ +^ σ ee'
σ x σ x'
t /^ σ
x
Thus,
rx = σ
t /^ σ
x =^ rxx'
Assessing reliability (or generalizability)
Test-retest (or generalizability over time):
Test twice, and rxx' is the estimate. This is
the most intuitively appealing way to think
about reliability, until you think about the
psychology of testing twice. Then it almost
never works very well.
Alternate forms (or generalizability over forms
Test twice, using two different forms,
essentially at the same time and rxx' is the
estimate. It is frequently difficult to
generate parallel alternate forms, except in
major testing programs (where it is
routine… and expensive).
Many years ago, Spearman (1910) and
Brown (1910) did the algebra (dignified as
the “Spearman Brown prophecy formula”)
explaining that, under the assumptions of
the traditional test theory plus the
assumption that all of the items are equally
correlated with each other,
rxx* =
2 rxx'
1 + rxx'
In general, the reliability of a test N times
longer than the one giving the score x is
rxx* =
Nrxx'
1 + ( N – 1) rxx'
Tables and graphs are available.
Coefficient α
There are other derivations of this same
number; and it is almost always computed
using some computer program (it is not
computed by splitting the test into all
possible halves!). But the most informative
formula is
N ¯ r
1 + ( N – 1)¯ r
where N is the number of items and ¯ r is an
average interitem correlation.
Coefficient α is just the Spearman-Brown
formula taking the average of the interitem
correlations as the reliability of little one-
item tests.
Also note that as long as the average
interitem correlation stays positive , the
longer you make the test the more reliable it
is. So what?
The Standard Error of Measurement:
SEM = SD 1 – rxx' ;
The standard error of the difference
between two scores is
SEdiff = (SEM 1 )^2 + (SEM 2 )^2 ' ;
E.g., rough numbers for the CEEB scales
are
SEM = 100 1 – 0.92 = 30
SEdiff = (30)^2 + (30)^2 ' = 40
Scatterplot of scores for two parallel forms; the reliability of each is
J
J
J
J
J
J
J
J
J
J J
J
J
J
J
J
J
J
J
J
J J
J
J
J
J
J
J
J
J
J
J
J
J
J J
J
J
J J J
J
J
J
J
J
J
J
J
J
JJJ
J
J J J
J
J
J
J
J
J
J
J
J J
J
J
J
J J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J J
J
J
J
J
J J
J
J
J
J J
J J
J
J
J
J
J
J
J
J J
J
J
J
J
J
J
J
J
J
J
J
J J
J
J J
J
J
J
J
J
JJ^ J J
J
J
J
J
J
J
J
J
J
J
J
JJ
J
J
J J
J
J
J
J J
J
J
J
J
J
J
J J
J
J
J
J
J
J
J
J
J J
J J
J
J
J
J
J
J J
J
J
J
J
J
J J
J
J
J
J
J J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J J
J J^ J
J
JJ
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
JJ
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J J J
J
J
J
J
J
J
J J J
J J
J
J
J
J
J
J
J
J
J
J J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J J
J
J
J
J J
J
J
J
J J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J J
J
J
J J
J
J
J
J
J
J
J
J
J
J
J
J J
J
J J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J J J
J J
J
J
J
J
J
J
J
J
J
J J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J J
J J
J
J
J
J
J
J
J
J
J
J
JJ
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J J
J
J
J
J
J
J
J
J
J
J
J
JJ
J
J
J
J
J
J J
J
J
J
J
J
J J
J
J
J
J
J
J
J J
J
J
J
J
J
J
J
J
J J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J J
J
J
J
J
J J
J
J
J
J
J
J
J
J
J
J
J
J
J J J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
JJ J
J
J
J
J
J
J J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J J
J
J
J
J
J
J J
J
J
J
J
J
J
J
J
J J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
JJJ
J J
J J J J
J J
J
J J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J J
J
J
J
J
J
J
J
JJ
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J J^ J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J J
J
J
JJ
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J
J J
J
J
J J J
J
J
J
J
J
J
JJ
J
J J
J
J
J J
J
J
J
J J
J
J
J
J
J
J
J J
J
J
J
J
J
J
J
J J
J
J
J
J
J
J
J
J
J
J
J
JJ
J
J
J
J
J
J
J
J
J
J J
J
J
J
J
J
J
J
J J
J
J
J
J J
J
J
J
J
J
J
J
J J
J J
J
J
J
J
200
400
600
800
200 400 600 800 First Test