Refining Item Specifications for Innovative Item Types in a Large-Scale English Language Proficiency Assessment

8:00am - 5:30pm EDT

Please consult the event program for time and location.

Presented at: ECOLT 2015

Mainstream use of technology in education has created opportunities for innovative ways to measure K–12 English language learner (ELL) proficiency. As English language proficiency (ELP) assessments move from paper to computer delivery, it is critical to: (a) ensure the most effective testing environment for all students, particularly the youngest learners; (b) increase student engagement; and (c) effectively measure the intended construct, without introducing construct-irrelevant variance. As previous studies have shown, innovative item types, when implemented well, have the potential to gather more information than multiple choice items about students at all ability levels (Jodoin, 2003). In addition, innovative item types “can greatly enhance the validity of interpretations made on the basis of test scores” (Sireci & Zenisky, 2006, p.329).

This study highlights the development of drag and drop and hotspot item types, which are replacing traditional multiple choice items as an avenue for achieving these goals on a large-scale ELP test. These item types contextualize text and graphic stimuli in more authentic, engaging presentation formats. As part of the research and development phase of test development for a computer-delivered large scale K – 12 ELP test, cognitive labs are carried out with the target population to inform the student directions and practice, test item prototype refinements, and the development of specifications. Based on this work, this poster will (a) discuss key questions and findings from item development and cognitive interview studies on these new item types; (b)  share insights into how these innovative item types are being designed and refined to enhance K – 12 ELP assessments; and (c) describe implications for item specifications.