Computers are a promising technological tool for the next generation of assessments (Plank, Norton, Arraez, & Washington, 2011); not only do they provide an expedient way to disseminate and score assessments, but students have also been shown to respond favorably to them (Abedi, 2010). ONPAR is an innovative computerized assessment that utilizes numerous computer capabilities such as color graphics, animation, and sound. It has been designed with English learners (ELs) in mind, incorporating help features to make content comprehensible.
ONPAR’s development process incorporated cognitive interviewing (Ericsson & Simon, 1993; Almond, et al., 2009) to investigate how students understood and interacted with novel items. A total of 57 interviews were conducted in Virginia in spring, 2010 with high school biology and chemistry students.
In this presentation, we discuss the relative success of various computerized features, how students interacted with graphics, language, and sound, and whether students understood and reasoned through the items as intended by measurement goals. We provide recommendations for making computerized assessments comprehensible to all students, and areas which may need further research, especially for ELs.