Research on the literacy and language development of Spanish-speaking English language learners
VIAS Subproject 4
Enhancing Vocabulary through Cognate Awareness (EVOCA)
This study investigates the effects of two cognate-based interventions on the development of literacy skills among Spanish-speaking adolescent students. The two approaches both focus on the same target words and employ the same research-based pedagogical strategies, but one approach (monolingual) is delivered solely in English and does not make any reference to the fact that all target words are cognates, while the second approach (cross-linguistic) makes this fact explicit and introduces the Spanish counterparts of target words as well as related roots and affixes. Having these two experimental conditions allows for investigation of the differential effects of the two approaches on participants’ vocabulary and reading comprehension skills and the extent to which these effects may vary as a function of instructional history, home language and literacy exposure, and/or students’ current levels of English and Spanish oral proficiency and literacy ability as measured by pretests. Preliminary findings indicate that most students in the monolingual condition do not make cognate connections on their own, but students in both groups make comparable gains in their mastery of taught words.
Specific Aims
The Enhancing Vocabulary through Cognate Awareness (EVoCA) Study consists of an intervention designed to improve the vocabulary and reading comprehension skills of Spanish-speaking middle school students. The first three years of work have focused on designing, developing, and piloting the intervention program, which consists of two conditions, a monolingual and a cross-linguistic curriculum. The two approaches focus on the same target words and employ the same research-based pedagogical strategies, but one approach (monolingual) is delivered solely in English and does not make any reference to the fact that all target words are cognates, while the second approach (cross-linguistic) makes this fact explicit and introduces the Spanish counterparts of target words as well as related roots and affixes. Having these two experimental conditions allows for investigation of the differential effects of the two approaches on participants’ vocabulary and reading comprehension skills and the extent to which these effects may vary as a function of instructional history, home language and literacy exposure, and/or students’ current levels of English and Spanish oral proficiency and literacy ability as measured by pretests. During the fourth year, a full-scale study will be implemented. The design of the intervention is quasi-experimental with assignment at the classroom level.
Activities and Results
The project was initiated in June 2007. Since this time, the Principal Investigators and the research team have designed and developed an intervention program that is currently being pilot-tested in sixth- through eighth-grade classrooms in a public school in Connecticut. The intervention curricula draw partly on the Word Generation curriculum. Word Generation, an Initiative of the Boston Public Schools and the Strategic Education Research Partnership Institute (SERP) directed by Catherine Snow, is a research-based vocabulary program for middle school students designed to teach words through different content areas. The program consists of weekly units that introduce a small number of high-utility target words through brief passages about contemporary issues of interest to this age group. The EVoCA curriculum consists of seven units, all based on school-centered topics, such as the purpose of schooling, bullying, and incentive programs. Each unit is delivered during six, 50-minute language arts class periods. There are ten target words per unit, all of which are cognates and can be classified as general academic vocabulary. There are also two target affixes and two target roots per unit. The units integrate listening, speaking, reading, and writing activities and are aligned with state and national standards for English language arts and the education of English language learners (ELLs).
The sequencing of the unit activities is as follows:
- Day 1: building background and introducing the passage and the vocabulary
- Day 2: deepening word knowledge
- Day 3: using words in oral language and writing
- Day 4: word study, part 1: roots
- Day 5: word study, part 2: affixes
- Day 6: review and quiz
To date we have developed the first five units of the curriculum and two study-specific measures to evaluate the efficacy of the interventions on vocabulary growth, namely, the Spanish Cognate Test and the English Vocabulary Test. The first three curriculum units were field-tested during the Development phase of the study, in the spring of 2009. During that phase, we worked closely with four veteran teachers (two ESL and two literacy specialists) in a single district with a high percentage of ELLs. The teachers delivered the first three units of the curriculum and provided feedback on the student materials and guidance about teacher-training and teacher support materials. We also conducted classroom observations in order to learn more about student participation and engagement with the activities, as well as implementation issues faced by the teachers. Finally, we informally assessed student performance on mastery of target vocabulary through unit vocabulary assessments and their use of the target words in a persuasive writing task. We used all of this information to revise the curricular materials, teacher support materials, and teacher training activities developed thus far. For example, the glossary now includes pictures along with sample sentences in order to provide greater context for word learning; the word-study component of the curriculum now includes root study as well as affix study, and instructional activities for both now incorporate more interactive features to make instruction more engaging to students. Finally, each unit includes a day of review that cycles back through previously taught words, roots, and affixes as well as those focused on in the current unit.
The two vocabulary assessments, the English Vocabulary Test and the Spanish Cognate Test, were field-tested in June 2009 with 350 middle school students in a single school district in Connecticut. The purpose of the field test was the validation and optimization of the measures. The English Vocabulary Test is a group-administered, multiple-choice assessment designed to measure growth on cognate words that were taught as part of the interventions and other words that were not taught as part of the interventions (both cognates and noncognates). Including both words taught and words not taught (at the same frequency level) on all assessments will enable the researchers to accomplish multiple goals. Performance on the taught words and on the measure as a whole will provide indicators of the relative effectiveness of each intervention, while performance on the nontaught cognates versus nontaught other words will provide insight into issues of transfer in each of the domains measured.
The Spanish Cognate Test is also a group-administered, multiple-choice assessment designed to measure prior knowledge of Spanish equivalents of the English cognate words targeted in the intervention. The purpose of this assessment is to determine the extent to which the Spanish-speaking participants in the study are in fact already familiar with the Spanish equivalents of the English words targeted in the intervention, in order to help predict the extent to which they may be able to capitalize on their knowledge of Spanish to unlock the meanings of the English target words.
Item analyses conducted over the summer of 2009 provided information on the validity and reliability of the tests as well as on the fitness of the individual test items. Misfitting items were either eliminated or modified based on the Rasch analyses findings. Additionally, information on the difficulty level of the items (i.e., p value) was used to eliminate fitting items that contained target words that were known by at least 60% of the participants. Revised versions of the assessments were created based on findings from the item analyses.
During the current school year, 2009-2010, we are conducting a small-scale, quasi-experimental pilot study in one middle school to test the efficacy of the curriculum, to get an approximate idea of the likely magnitude of the effect size for the study, and to help validate the revised researcher-developed measures of word knowledge. We are piloting 5 units of the intervention in 21 classrooms (7 per grade level) in a single middle school (n=350). Approximately two thirds of the students are native Spanish speakers and one third native English speakers. Classrooms were assigned to the monolingual English intervention (n=6, two per grade level), the cross-linguistic intervention (n=6, two per grade level), or the no-treatment control group (n=9, three per grade level). Two trained research assistants are currently delivering the interventions, each being responsible for teaching either the cross-linguistic or the monolingual curriculum to all six classes assigned to each condition.
Before the interventions began, pretests were administered to all students to determine their baseline abilities in English and Spanish vocabulary and reading. In addition to the two study-specific assessments, standardized assessments of vocabulary and reading comprehension (the Gates-MacGinitie Reading Tests) were administered to all participating students. Post-tests will be administered at the end of the pilot study period to evaluate the effects of the two interventions relative to each other and to the no-treatment control group, using a multilevel approach. During this phase, project researchers are also collecting background data on participating students and conducting fidelity of implementation observations in the treatment classrooms.
The Principal Investigators and project team were invited to present on the project at Wordstock Spring 2010, a meeting on February 26 at Harvard University for researchers doing vocabulary intervention work, and at the 44th Annual TESOL Convention in Boston on March 26 alongside Josh Lawrence and Claire White of the Word Generation project. In June 2010, the Principal Investigators will be presenting on the project at the IES Research Annual meeting in Washington, DC.
Significance
This study contributes to the body of research on effective vocabulary instruction for adolescent Spanish-speaking students. The two conditions, monolingual versus cross-linguistic, will shed light regarding the extent to which making connections between English and Spanish is a naturally occurring phenomenon among Spanish-speaking middle school students (monolingual) or whether these connections need to be made explicit (cross-linguistic), and the extent to which the effects of the two conditions may vary as a function of instructional history, home language and literacy exposure, and/or students’ current levels of English and Spanish oral proficiency and literacy ability as measured by pretests.
Plans
During the summer of 2010, the research team will be working on completing the pilot data analyses and will use findings from these analyses as well as feedback from the research assistants and the English language arts teachers who implemented the pilot interventions to revise the curriculum and the researcher-developed instruments.
During the school year 2010-2011, we will take the intervention to scale by conducting a quasi-experimental study that will employ a multisite cluster randomized trial design with fixed effects at level 3 (the teacher level) in order to maximize the likelihood of determining causal relationships between each intervention and students’ literacy outcomes (Cook & Campbell, 1979; Pedhazur & Schmelkin, 1991). Teachers, who are nested within schools, will be randomly assigned to one of three conditions (cross-linguistic, monolingual, or no treatment control). Participating teachers will receive prior professional development and ongoing support and will be observed on a regular basis to ensure fidelity of implementation across classrooms. Pretests of Spanish and English literacy ability will be administered to all participating students before the intervention begins and will be followed up by post-tests to assess intervention effects. Finally, multilevel analyses will be carried out with the collected data in order to answer the research questions. Specifically, multilevel modeling will be used to determine the potential effects of the interventions, taking into consideration student factors (e.g., home language and literacy exposure and language of prior schooling) and treatment condition, as well as fixed effects for teachers and schools.
References
Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Boston: Houghton Mifflin.
Pedhazur, E. J., & Schmelkin, L. P. (1991). Measurement, design, and analysis: An integrated approach. Hillsdale, NJ: Erlbaum.
