Module 3: Reliability (screen 3 of 4)
Reliability and Test Selection

What does reliability have to do with the practical aspects of selecting and administering tests?  The chart below gives you several different ways to think about reliability and how it relates to selecting and using tests.  Click here to return to the previous chart.

Reliability of student performance

One student should be able to take a test on Monday and then again on Tuesday and get very similar results each day.

 
Knowing a test is reliable, you can explain to a student disappointed with her scores why taking the test again two days after she took it the first time will not improve her score in a meaningful way. 

Selecting a test that gives a reliable picture of student performance is important because then you can be confident of the results and the decisions you make with those results.

Reliability of scoring

The results should be the same no matter who scores the test and when they score it.

 
You can check to see if you are scoring reliably by re-scoring a test and comparing with the original results.  You can also have two different people score a test and then compare the results to see if they are the same. 

What does this have to do with selecting a test?  If the test has many constructed responses, the scorers will need to be trained to score the tests consistently.  This, of course, involves resources such as time and money.

Reliability between different forms of the same test

Different forms of a test have slightly different questions in a slightly different order.  However, the content and difficulty level are the same and a student should perform comparably on both tests. 

Sometimes teachers find it necessary to administer different versions of the same test to different classes or to the same student at different times.  If you give forms that elicit consistent performance, you can assure your students that they have an equal chance of doing well on either form. 

If it would be useful to have more than one version of a test, you might want to find out if parallel forms have been developed while you are collecting information about possible tests to use.

 

Before you select a test, you’ll want to think about your program, your needs, and the resources you have.  Is this a high-stakes test for the student?  If so, it is especially important that the test give consistent results each time.  Do students often contest their results?  Then it may be especially important to find a test whose reliability information has been published in the test manual or professional journal articles.  Do you have very few staff for scoring or even double-scoring the same test?  Then you may want to choose a test with objective responses that can be scored quickly and reliably by a person or machine.