East Coast Organization of Language Testers 2011 Annual Conference
October 28 - 29, 2011
View the conference website.
The Development of Specifications for an Oral Proficiency Test: Contributions to Validity Claims
This paper describes the development of specifications for a computer-delivered oral proficiency test for language learners of high school age and above that is aligned to the ACTFL Proficiency Guidelines—Speaking. The paper presents methodology used to develop specifications consistent with Popham’s test specification model (1978), including procedures for task writing, review, and banking. Presenters will emphasize how the iterative process of test specification development contributes to validity claims. Presenters will also discuss the issue of specificity in view of lessons learned during task writing and implications for operational development. The iterative methodology presented may be replicated by other language testers to build a validity argument as well as a foundation to support efficient test development.
Presenters include: Francesca Di Silvio, Anne Donovan
Bridging the Gap: How Language Testers Can Build Assessment Literacy in Practitioners of Other Fields (poster session)
This poster presents the initial results of a multi-year project that examines the assessment literacy needs of two groups: Second Language Acquisition (SLA) researchers and professionals who work in Language Teacher Education (LTE). The project investigates the current assessment knowledge base of both groups to identify what resources should be developed to meet the needs of these audiences. This poster presents results from four focus groups conducted with SLA and LTE researchers and practitioners. Focus groups were recorded, transcribed, and analyzed for major themes related to current practices, challenges, and resources needed in language testing. The results of the focus groups will inform the development of future phases of the project, including surveys of both groupsí assessment needs. In addition to presenting the focus group results, the poster will discuss implications for collaboration between language testers and other language professionals, and will raise questions about the responsibility of the language testing community to make testing research accessible to colleagues in related fields.
Presenters: Anne Donovan, Margaret Malone, Francesca Di Silvio, Megan Montee
Investigating Assessment Literacy Through Teacher Professional Development (poster session)
This poster will present findings of a study of assessment literacy of teachers of less commonly taught languages; it describes the changes in instructor knowledge during a blended learning assessment course that combines online and face-to-face formats. This study provides insight into how to cultivate assessment literacy by tracking the progress and depth of understanding of course participants over time by analyzing data from various sources. Data were collected through pre- and post-course surveys, assessment tasks developed during the course, discussion board posts, and observations. These data also explore what LCTL teachers know about assessment, how their knowledge emerges through assessment practices, and what opportunities exist for providing teacher professional development in order to strengthen their use of assessments. Since this study explores the evolution of LCTL teachersí assessment knowledge, beliefs, and practices, it provides an important and often-overlooked foundation for the development of assessment literacy training.
Presenters: Aileen Bach, Anne Donovan, Abbe Spokane
Best Practices in Pilot Testing a High-stakes English Language Proficiency Test
Pilot testing—pre-operational research that is often small-scale and qualitative, and is intended to identify necessary revisions in items—is a widespread practice but has little documentation in literature. The proposed poster aims to document and disseminate pilot testing methods for a high-stakes, large-scale assessment of English language proficiency using the framework of Bachman’s (2005) Assessment Use Argument (AUA).
Presenters: Abbe Spokane, Tiffany Yanosky
Assessing Learning Outcomes in Short-term Foreign Language Programs: Validation Results of a Triangulated Assessment System
The current paper describes the results of a national, two-year study (N=700) of three different assessment tools piloted for use to inform the development of a triangulated assessment system. The purpose of the assessment system was to examine the reliability, consistency across language programs and convergent validity of three instruments developed as indicators of learning outcomes.
Presenters include: Margaret E. Malone
Return to CAL's list of past presentations.