Ask CAELA answers appear in reverse chronological order.
- What are factors to consider when planning for, setting up, and evaluating a workplace program for immigrant workers? (December 2006)
- Literacy-level adult English language learners (June 2006)
- Vocabulary acquisition and adult English language learners (March 2006)
- Professional development communities and adult ESL (November 2005)
- Challenges in testing language proficiency for adult English language learners (October 2005)
- Student Performance Levels (SPLs) and the National Reporting System (NRS) (September 2005)
What are factors to consider when planning for, setting up, and evaluating a workplace program for immigrant workers?
(Read an interview with CAELA Associate Director Miriam Burt below on workplace on current issues in workplace English education)
Although the terminology (e.g., VESL. workplace, workforce), venues, and funding sources may change, the need for effective classes and programs targeted to meet the needs of immigrant workers and their employers has remained the same since the early 1980’s. As immigrants join the workforce, especially in areas experiencing new and rapid growth in immigrant population such as Georgia, Nebraska, South Carolina, administrators, teachers, and employers are asking for help. At least some of the answers can be found by searching the archives from earlier workplace initiatives. The following advice has been adapted from Grognet, A., (1998). Planning, Implementing, and Evaluating Workplace ESL Programs
To maximize effectiveness –both from the point of view of workers and employers--employment-related English as a second language (ESL) programs, whether conducted on the job or as pre-employment training, should be the result of five interrelated steps:
- Conducting a needs analysis of the language and culture needed to perform successfully in a specific workplace or occupation. The needs analysis leads to the development of objectives for the program.
- Developing a curriculum, based on the objectives, that identifies tasks and skills for verbal interaction on the job, and tasks and skills for reading and writing on the job. The curriculum should also prioritize these tasks and skills.
- Planning instruction by gathering text material and realia, determining classroom activities, and identifying opportunities for learners to put their skills in practice outside the classroom.
- Determining instructional strategies that include a variety of activities that focus on the objectives, keep the class learner-centered, and include as much paired and group work as possible. Strategies for assessment should also be determined when planning instruction.
- Evaluating the program on both a formative and summative basis.
These steps (below) are from the point of view of what the instructor needs to consider in planning, implementing, and evaluating a program. However, throughout the process, the instructor must remember that the "buy-in" of the business partner, especially at the level of the frontline supervisor, is indispensable to the success of any workplace ESL program
Miriam Burt has planned, developed, and evaluated workplace programs for native and non-native speakers of English as well as consulted on workplace issues for U. S. Departments of Education and Labor. She has written briefs, articles, and books on designing effective workplace education programs for immigrant workers. CAELA staff interviewed Miriam about current initiatives, issues, and concerns in workplace ESL education.
CAELA: Thanks for taking the time to update us on workplace education issues, Miriam. Do you think the five steps (for planning and evaluating, above) are still necessary and effective for current workplace classes?
Miriam: You’re welcome. In this era of accountability, I think the steps for planning and evaluating workplace programs are more important than ever. Employers, immigrant workers, and other stakeholders (such as unions and outside funders) need to come together in the planning stages so they can decide on program objectives. If all stakeholders are on the same page about what outcomes can be reasonably expected from the workplace program, its easier for everyone to work together for a common, realistic purpose. Of course, you can see that a great deal of groundwork needs to be laid to support an effective program.
CAELA: You seem to be saying that a good needs analysis is important to the success of a workplace program. Can you share some insights from needs analyses you’ve conducted and why they were so important to the success of the program?
Miriam: Yes, I think that a thorough needs analysis is a key element in developing and maintaining a good program. And when I say needs analysis—I mean the needs of everyone need to be analyzed: supervisors of immigrant workers and the workers themselves, but also higher-ups in the company, human resources or personnel, union representatives, co-workers who are native speakers of English. Instructors’ needs should be included, and, in the service sector, knowing what the customer wants and needs is also important. Gathering all this information involves more than passing out questionnaires—the program developer needs to get right into the workplace. I remember when I was doing the needs analysis for Skills Enhancement Program (S.E.T.) for Local #32 of the Food and Beverage Workers here in Washington, D.C. I spent days observing the workplaces that program students would come from. I went into the kitchen, listened to workers, supervisors, union representatives, customers, and I even sat in on workers’ breaks to hear what they had to say. I also ate at the cafeterias to get a full sense of the work, the workers, and the products. This needs analysis took a long time and a lot of effort, but I think that was one of the reasons that program was so successful. This project was recognized by the U. S. Department of Education as being one of 10 model workplace projects in 1993.
CAELA: Workplace instructors, programs, or businesses sometimes search for workplace curricula for a particular job or industry—perhaps to avoid the hard work, time, and cost involved in creating a workplace-specific needs analysis and plan. What’s your advice about using previously developed curricula?
Miriam: I think it’s a good idea to see what has already been developed—no one needs to reinvent curricula from scratch. It’s sort of like doing the “literature review” before beginning a research study. Having said that though, I want to say that any curricula or materials need to be adapted to the individual workplace, the needs determined by the needs analysis, and the hoped-for outcomes. And to add to that, things often change over time because of the economy, because goals change, or even because workers come and go. What I have been saying, I guess, is that developing an effective workplace education program takes a great deal of time, effort, support—financial and emotional—you just can’t take a program off the shelf and use it in every new situation.
CAELA: What’s been your experience in getting workers, employers, and instructors all on the same page—so everyone knows what outcomes to expect from the class?
Miriam: One way to insure that all stake holders are on the same page is to conduct the needs analysis and not just shove it in a file, but keep the data you gathered right on the table. Another way is to have meetings—with the individual stakeholder groups, with employers and employees together, labor and management, human resources and workplace instructors. These meetings happen during the planning stages and should include worker focus groups. Now, this is not only labor intensive, it isn’t always easy. At the S.E.T. program, I was able to find my way into the confidence of the workers because I partnered with a union representative who had the trust and respect of the workers and of employers. And thinking about how to prepare workplace teachers—all the teachers who worked in the program were also required to observe the workplace—to step behind the steam table, to hear workers’ describe their jobs, read workplace safety signs, and to get a sense of interaction between supervisors, workers, and customers. Time consuming, yes, but it helped cement everyone’s understanding of the realities of the workplace.
CAELA: Instructors used to talk about the importance of learner-centered instruction, especially in workplace contexts. Is that still an important element of workplace classes? Why or why not?
Miriam: Yes, of course learner-centered instruction is important in all classes. Whatever curriculum or content or language skill being taught should be adjusted to the realities of that workplace and the learners in the class. This is important too, because workplace and many other adult ESL classes are usually of limited intensity and short duration. It’s not like K-12 where the students have 12 years of full days to learn—many workplace classes are only 60 hours total. So, teachers have to focus. An important thing for teachers to remember is to always be deliberate and explicit about why students are asked to complete a certain activity. Even when a teacher focuses on a certain grammar or vocabulary point, she should highlight the connection with the workplace. For example, a teacher might offer a lesson on the use of the past continuous tense because one of the workers in the class had complained about having had to work washing the pots and pans all morning. So when modeling something like “I was working all morning at the pot sink” the teacher makes the connection between English tenses and a work situation.
CAELA: The word evaluation can cover a lot of territory—from standardized pre- and post-assessments and learner, instructor, and business self-assessment to real-life outcomes that effect workers’ wages, job security, and upward mobility to employers’ abilities to stay solvent and make profit in the global economy. What’s your advice about where (or how) to start thinking about evaluation, assessment, and workplace ESL?
Miriam: Well one thing that we learned as a field is that evaluation needs to be multifaceted. Different stakeholders need different measures: employers, immigrant workers, customers, funders, unions, and others have ideas about how to look at the success of a workplace education program. I think it all goes back to what I said at the beginning of the interview—it's imperative for everyone to be on the same page about the expected outcomes. I know that is not easy; adult ESL professionals know that it takes a long time to learn English. It’s even hard to know what “learn English” means. Some employers may be talking about accent reduction, some might be focusing on having service staff be able to appropriately greet customers, and others might mean having immigrant staff be able to read and write complex, technical documents. Pre- and post-standardized assessments can give some information about workers’ English proficiency in certain skills. Beyond that though, performance-based assessment makes sense for both formative and summative evaluation. Both the employers and workers want to know whether they have the English ability to perform certain tasks at work. This is particular important in such areas as health and safety compliance and as workers seek to move up in a company.
CAELA: Thank you again for taking the time to talk to CAELA about current issues in immigrant workplace education.
Note: To read more of Miriam’s comments on workplace education programs, read Issues in Improving Immigrant Workers' English Language Skills. Also see www.cal.org/caela/tools/program_development/workplace_programs.html for more resources related to immigrants in the workplace.
ABC Canada. (1999). Success stories in workplace basic education for small business. Don Mills, Ontario, Canada: Author. www.abc-canada.org/workplace_education/success.asp
Alamprese, J. A., & Kay, A. (1993). Literacy on the cafeteria line: Evaluation of the Skills Enhancement Training Program. Washington, DC: U.S. Department of Education. (ERIC No. ED368933)
Burt, M. (2003). Issues in Improving Immigrant Workers’ English Language Skills. Washington, DC: National Center for ESL Literacy Education.
Burt, M. (1997). Workplace ESL instruction: Interviews from the field. Washington, DC: National Center for ESL Literacy Education. www.literacynet.org/eslwp/home.html
Mikulecky, L. (1992). Workplace literacy programs: Variations of approach and limits of impact. San Antonio, TX: National Reading Conference. (ERIC No. ED353 461)
Mikulecky, L. (1997). Too little time and too many goals. Focus on Basics, I(D) 10-13. www.ncsall.net/?id=432
Who are literacy-level adult English language learners?
Literacy learners adult English language learners are generally those with six or fewer years of education in their native countries who need focused instruction on learning to read and write English. The population participating in literacy-level classes is diverse: These classes may include men and women with different native languages, ages, length of time in country, life and language learning goals, and access to previous education (Centre for Canadian Language Benchmarks, 2000; Shank & Terrill, 1997). Literacy learners also have a wide range of oral skills in English. (For a more detailed description of the varieties of first language literacy and effects on second language literacy, see Burt, Peyton, & Adams, 2003.) The learners are further differentiated by their experiences. Many have experienced trauma related to events in their native countries and to resettlement in the United States, and this trauma may affect the speed and facility with which they learn English (Adkins, Sample, & Birman, 1999). The following learners might attend the same literacy class:
Preliterate (The native language does not yet have a writing system.) Wanankhucha, a Bantu from Somalia, entered the class as a recent refugee. She knows her native Af-Maay only orally, as a written form of the language is just now being developed. Furthermore, as a refugee, Wanankhucha shows evidence of trauma.
Nonliterate (The native language has a written form, but the learner has no literacy.) Trang is a young, single mother from rural Vietnam who grew up without access to education. Here in the United States, she lacks many of the educational and cultural supports earlier Vietnamese refugees enjoyed.
Semiliterate (The learner has minimal literacy in native language.) Roberto attended a rural school in El Salvador for 3 years. Although he wanted to continue, his family needed him to work on the family farm.
Nonalphabet literate (The learner is literate in a language that is not alphabetic.) Xian is a retired minor bureaucrat from China. He is highly literate in the Mandarin script, but he is unfamiliar with any alphabet, including Roman.
Non-Roman alphabet literate (The learner is literate in an alphabtic language other than Roman.) Khalil comes from Jordan. He completed 2 years of secondary school and is literate in Arabic.
Roman-alphabet literate (The learner is literate in a language that is written in the Roman alphabet). Alex is a senior from Russia. As a young man, he studied French. Even though he was a highly educated professional (engineer) in his own country, he does not want to move to amore advanced level class.
Others who may benefit from a literacy-level class are individuals with learning disabilities or individuals who, because of age, physical or mental health issues, or family situations, find that the slow and repetitive pace of such a class better meets their needs and goals (Holt, 1995).
This article is excerpted from Working with Literacy-Level Adult English Language Learners
Adkins, M. A., Sample, B., & Birman, D. (1999). Mental health and the adult refugee: The role of the ESL teacher. Washington, DC: National Center for ESL Literacy Education.
Burt, M., Peyton, J. K., & Adams, R. (2003). Reading and adult English language learners: A review of the research. Washington, DC: Center for Applied Linguistics (CAL).
Shank, C., & Terrill, L. (1997). Multilevel literacy planning and practice. Focus on Basics 1(c), 18-22.Resources
Brod, S. (1999). What non-readers or beginning readers need to know: Performance-based ESL adult literacy. Denver, CO: Spring Institute for International Studies.
Haller, L. (2000) Modeling class activities for low-level literacy learners. Retrieved June 15, 2006, from http://www.sabes.org/resources/fieldnotes/vol10/f02halle.htm
Hvitfeldt, C. (1985). Picture perception and interpretation among preliterate adults. Passage: A Journal of Refugee Education 1(1), 27-30.
Isserlis, J. ( 2000). Trauma and the adult English language learner. Washington, DC: National Center for ESL Literacy Education.
Silliman, A., & Tom, A. (2000). Practical resources for adult ESL. Burlingame, CA: ALTA Book Centers.
Tom, A., Tiller, C., & Bigelow, A. (1998, September-October). So, they gave you the beginning class. Hands-on English, 8(3), 6-7.
Vocabulary has emerged as a major factor in the acquisition of reading skills by English language learners. What are the some of the issues involved in vocabulary acquisition for adult English language learners?
Keith S. Folse (2004) reviewed the research on teaching vocabulary in semantic sets (e.g. colors, foods, furniture, days of the week) and found that grouping words in this way can actually impede the learning of vocabulary. This is because if similar new words are presented together, such as a set of colors or the days of the week, the learner is likely to confuse the words. The same is true if antonym pairs such as hot/cold, fat/thin, right/left are presented together. Folse suggests grouping new vocabulary around looser themes such as going out to eat, planning a trip, or celebrating an anniversary. Nation (2000, 2005) recommends teaching high-frequency vocabulary first. For example, rather than presenting red, yellow, blue, black, white, etc. at one time, he suggests beginning with one color, and starting with one that is more frequently used, or salient. In this way red, which is used more frequently than orange, would be taught before orange. Tuesday, which is used more frequently than Thursday, would be taught before Thursday (Nation, 2000). This separation of Tuesday and Thursday would also avoid the confusions that surface between these two words, which are similar phonologically and in spelling (Folse, 2004).
Acquiring the meaning of a vocabulary item through context clues – a strategy often taught by ABE teachers – is difficult for learners of English as a second language, because they often do not have the vocabulary in English that native speakers have (Eskey, 2005). For example, while fluent English speakers possess a written English vocabulary of 10,000-100,000 words, second language learners generally know only 2,000-7,000 English words when they begin their academic studies (Hadley, 1993). This gap can impede success in listening to lectures, reading academic material, or writing essays. Using context to understand new vocabulary requires an understanding of more than 98% of the words of a passage (Nation, 2005). Furthermore, even if the meaning of a word can be guessed from context, knowledge of the word may be superficial. Truly knowing a word includes knowing its pronunciation, spelling, morphological and syntactic properties (e.g., part of speech, prefixes and suffixes it has), and multiple meanings; the contexts in which it can be used; the frequency with which it is used; and its collocates, or how it combines with other words (e.g., the word squander is often paired with resources, time, or money; Folse, 2004). For these reasons, vocabulary teaching needs to be planned and deliberate with English language learners.
Suggestions for teaching adult English language learners
Because of the need for English language learners to acquire more English vocabulary for all aspects of their lives, Birch (2002), Eskey (2005), Folse (2004), and Nation (2000, 2005) suggest the following:
- Pre-teach the vocabulary in a reading passage.
- To limit the number of vocabulary items that must be pre-taught, select reading passages that are only slightly above what learners can read independently.
Teach high-frequency words first.
- Provide learners with multiple exposures to specific words in multiple contexts.
- Provide learners with lists of words for intentional learning.
- Avoid presenting synonyms, antonyms, or words in the same semantic set together.
- Teach learners to use both monolingual and bilingual dictionaries. Because even English dictionaries designed specifically for learners contain about 2,000 words (Nation, 2005) and the definitions and examples are in English, learners at basic reading levels may not understand the definitions and explanations. They will need to use bilingual dictionaries.
- Encourage learners to use word cards –notes cards with the English words on one side and the translation on the back-- and to study them frequently.
- Encourage vocabulary learning through regular tests where students can prove receptive knowledge of words through matching words to definitions or multiple choice exercises.
Adams, R. & Burt, M. (2002). Research on reading Development of adult
English language learners: An annotated bibliography. Washington, DC: Center for Applied Linguistics.
Birch, B. M. (2002). English L2 reading: Getting to the bottom. Mahwah, NJ: Lawrence Erlbaum Associates.
Burt, M., Peyton, J.K., & Van Duzer, C. (2005). How should adult ESL reading instruction differ from ABE reading instruction? Washington, DC: Center for Adult English Language Acquisition. www.cal.org/caela/esl_resources/briefs/readingdif.html
Burt, M., Peyton, J. K., & Adams, R. (2003). Reading and adult English language learners: A review of the research. Washington, DC: Center for Applied Linguistics.
Chiswick, B. R., & Miller, P. W. (2002). Immigrant earnings: Language skills, linguistic concentrations, and the business cycle. Journal of Popular Economics, 15, 31-57.
Eskey, D. (2005). Reading in a second language. In E. Hinkel, Ed., Handbook of research in second language teaching and learning (pp. 563-580). Mahwah, NJ: Erlbaum.
Folse, K.S. (2004). Vocabulary myths: Applying second language research to classroom teaching. Ann Arbor, MI: University of Michigan
Hadley, A.O. (1993). Teaching language in context. Boston, MA: Heinle & Heinle.
Nation, I. M. P. (2000). Learning vocabulary in lexical sets: Dangers and guidelines. TESOL Journal, 9(2), 6-10.
Nation, I. M. P. (2005). Teaching and learning vocabulary. In E. Hinkel, Ed., Handbook of research in second language teaching and learning (pp. 581-595). Mahwah, NJ: Erlbaum.
What is a professional learning community and why might adult ESL practitioners want to participate in one?
(Read an interview with Arizona adult educator Jessica Dilworth below)
A professional learning community is a group of educators, teachers, and administrators who “continuously seek and share learning, and act on their learning. The goal of their actions is to enhance their effectiveness as professionals for the students’ benefit . . .” (Hord, 1997). Professional learning communities adopt many different formats, but all have as their goal enhanced student learning. These communities are more than opportunities to have teachers and administrators sit together and study. After study, teachers help each other implement what they have learned through peer observation, conferencing, and other methods of giving and receiving feedback.
The concept of a professional learning community came to education from business. In his seminal work, The Fifth Discipline: The Art and Practice of the Learning Organization, Peter Senge describes “sets of practices” that businesses can implement to make a business a true learning organization and help workers achieve their highest aspirations (Galagan, 1991).
There are two types of professional learning communities:
1. A cohort-based community
- Formed within a particular cohort of faculty and staff
- Sometimes this group has been affected by isolation, fragmentation, stress, or neglect (FIPSE Project on Faculty Learning Communities)
- The community is formed to bring the cohort together to study a broad range of teaching and learning areas and mutual topics of interest
- The Teaching Scholars Community for junior faculty at Miami University in Ohio is one example of this type of community
2. A topic-based community
- Formed to address a specific topic, interest, need, or opportunity
- Members come from many different cohorts across the institution
- Some of the topic-based communities at Miami University in Ohio are centered on team-teaching, problem-based learning, teaching portfolio development, and similar topics
- At Louisa County High School in Louisa, Virginia, the staff of 120 have formed 21 different learning communities, each with its own purpose to improve student learning
Senge advises that a professional learning community is best started with a small group of teachers and administrators. They decide what they want to study and plan how to study and implement their ideas. They, then, need to open their group up to all in the school who wish to be involved (Newcomb, 2003). Administrative support is extremely important. However, administrators need to be seen as members of the group, not as directors of the group (Hord, 1997). All members of the professional learning community need to commit their own time and determine a regular time and place to meet. Members of the community also commit time when doing classroom observations and giving peer feedback.
Even though participating in a professional learning community takes time and commitment, the results are worth the time and effort.
- Hord (1997) reports that results for teachers include a reduced sense of isolation, an increased commitment to the mission and goals of the program, a sense of shared responsibility for the development of students, learning that exemplifies good teaching and classroom practice, and the sense that “teachers will be well informed, professionally renewed and inspired to inspire their students.”
- Hord also reports that students benefit when their teachers and program administrators are part of professional learning communities and this shows in decreased dropout rates and fewer absences, increased learning, larger academic gains in math, science, history and reading, and smaller achievement gaps between students from different backgrounds (1997).
Professional learning communities are an excellent way to ensure that the professional development activities of teachers and administrators result in enhanced student learning.
FIPSE Project on Faculty Learning Communities. (n.d.). Website for developing faculty and professional learning communities (FLCs) to transform campus culture for learning. Retrieved October 18, 2005, from www.units.muohio.edu/flc/what.shtml
Galagan, P. (1991, October). [Review of the book The Fifth Discipline: The Art and Practice of the Learning Organization]. Training and Development, 41.
Hord, S. (1997). Professional learning communities: Communities of continuous inquiry and improvement. Retrieved November 4, 2005, from www.sedl.org/pubs/change34/welcome.html
Newcomb, A. (2003). Peter Senge on organizational learning [Electronic version]. School Administrator 60(5).
Professional Learning Communities: An Interview with Jessica Dilworth
Arizona is a state participating in CAELA activities and its professional development plan includes the development of professional learning communities. Jessica Dilworth is the adult education director at Cochise College in Douglas, Arizona. A professional learning community was recently started at Cochise College and CAELA staff interviewed her about how it was started and how adult educators are participating in it.
CAELA: Jessica, thanks for taking part in this. Could you first tell us who is in the learning community at Cochise?
Jessica: We established a learning community, or LC, for the 6 ELA (English language acquisition) instructors at an adult learning center in Sierra Vista. We embedded one hour each week into all the ELA teachers' schedules. Those who are part-time get paid an extra hour for their LC time. If there is work outside of the meeting they commit to, they can get paid for that as well. I attend the LC meeting as often as I can. We have also invited the data coordinator to a few meetings, when her input was important.
CAELA: What are you studying?
Jessica: To start the year, we read some articles on learning communities, established norms, talked about purpose, and set up meeting protocols. We rotate note-taker and facilitator weekly. The note-taker becomes the facilitator the following week. We have reflection sheets to capture how everyone is feeling about being in a learning community. During the check-in, some Instructors read from their reflection sheets. The first thing we did was to read the NCSALL-produced handbook, Creating Authentic Materials for the Adult Literacy Classroom and used a study group protocol. We chose this because we are a project-based program and this approach was a good fit with what we do. We also liked the idea that it was a handbook based on a research study. We read approximately one chapter a week, discussing the content, and thinking about applications to our students, classrooms, and program.
Next, we analyzed the data we had from our classes, thought about implications for program improvement, and made a list of the data (student learning, teacher assessment, perception, demographic, classroom instruction, and process data) we needed to get a better picture of what was happening in our ELA program (Bernhardt, 2000). Now we are creating tools to give us the data we are missing. We would like to add teacher background (using the CAELA Adult ESL Instructor Needs Assessment) and we are working on authentic student learning data and teacher performance data tools. By the end of the year (June 30, 2006) we will be able to make some recommendations for program improvement using a data driven decision making process ( Easton, 2004), so we will be making programmatic decisions based on the data we have gathered about our program.
CAELA: Are you using any special materials?
Jessica: Yes, we are using the chapter "What is a Learning Community" from On Common Ground, the NCSALL-produced handbook, Creating Authentic Materials for the Adult Literacy Classroom and our program data.
CAELA: How are the members of the learning community helping each other implement what they have learned?
Jessica: Everyone is faithful to the weekly meeting. As application ideas come up, instructors talk about what they have done and will do in their classes; sometimes they bring in actual things they have done. After we have designed the new tools (probably by February), we are hoping to have about 30 minutes each session dedicated to one instructor who will bring in the classroom work, talk about what happened in class, and get feedback from others.
CAELA: Thank you for the time you took to share this with us and others.
If you would like more information, you can contact Jessica:
Adult Education Director
4190 W. Hwy 80
Douglas, Arizona 85607
Bernhardt, V. (2000). Intersections. At issue. (Winter), 33-36. Retrieved November 22, 2005, from www.nsdc.org/library/publications/jsd/bernhardt211.pdf
Dufour, R. (2005). On common ground. Bloomington, Indiana: National Educational Service.
Easton , L. (2004) Powerful designs for professional development. Oxford, Ohio: National Staff Development Council.
Jacobson, E., Degener, S., Purcell-Gates, V. (2003, April). Creating authentic materials and activities for the adult literacy classroom. Retrieved November 21, 2005, from www.ncsall.net/fileadmin/resources/teach/jacobson.pdf
What are the challenges in testing language proficiency level gains for adult English language learners?
There are many definitions of language proficiency. Because language has so many facets and so many uses, different tests approach different aspects of language proficiency. Over the years, proficiency testing has reflected changes in our understanding of language theory. It has moved from a structural view (e.g., discrete point tests of grammar, phonology, or other components of language), through a sociolinguistic view (e.g., integrative tests such as cloze and dictation), to a communicative view (e.g., oral interviews that assess the learner’s ability to use language to carry out communicative tasks). Today, given the focus on real-life practical content in adult ESL instruction and the goals of the learners—as well as in the educational functioning level descriptors of the National Reporting System (NRS)—a test should, in some way, look at language as communication.
Nevertheless, most items in most tests do not related directly to either theoretically or empirically derived understandings of adult English language proficiency development. It is not enough to assume that if a test is constructed in English and requires responses in English, then higher scores will automatically correspond to higher levels of English proficiency.
At the same time, a number of tests designed for English language learners relate scores to some broadly defined scale of proficiency levels that are described in very global terms. The actual items in the test, however, may be particular to a competency area and may sample very narrowly from the broad ranges of behavior described by the proficiency levels. The generalizability of from performance on the test items back to all situations consistent with a particular level description may be very difficult to establish.
The use of a single test form to assess the full spectrum of proficiency levels also means that most items will not match any particular test taker’s current level of functioning—that is, there are too few items at any one level of proficiency. Adaptive tests or tests with multiple levels usually make more accurate assessments of functioning levels.
Therefore, it is important for program and instructional staff to read (and ask for) the supporting test development documentation, not just the administration and scoring guidelines, for commercially available tests. Look for information about the population for whom the test is designed, the theoretical underpinnings upon which it was constructed, and the reliability and validity studies that support what the test claims it can do. Evaluate the test’s usefulness for the population and the purposes for which it will be used by your program.
What are SPLs and how are they related to the National Reporting System (NRS)?
The Student Performance Levels (SPLs) are descriptions of English language proficiency levels for adult non-native speakers of English. The SPLs were developed in the mid-1980s as part of the Mainstream English Language Training (MELT) project under the Office of Refugee Resettlement of the U.S. Department of Health and Human Services.
The SPLs were developed so that teachers in the refugees camps such as the Philippines or Thailand and programs in the United States (where the learners would eventually go) could communicate effectively about adult learners' English skills. The SPLs described general language ability as well as the four skills; listening comprehension, oral communication, reading, and writing. That is, the SPLs were developed so that a refugee program in Arlington, Virginia or Denver Colorado, or Bataan, Philippines, could all understand--generally--what it meant for a learner to be a SPL 3 in listening (for example).
With input from adult ESL practitioners around the United States, staff members at the Spring Institute for Intercultural Learning www.spring-institute.org/ updated the speaking and listening descriptors in 1998. Reading and writing SPLs were reviewed and updated in 2003.
While some programs that served refugees continued to understand and use SPLs, many others were not aware of them until the Workforce Investment Act (WIA) of 1998 (H.R. 1385, Pub. L. No. 105-220) was enacted. Title II, also known as the Adult Education and Family Literacy Act (AEFLA) required adult education agencies to establish core indicators of program quality and of learner performance related to educational gain, placement and retention in employment, participation in postsecondary education or training, and high school completion. From this legislation came the National Reporting System (NRS) www.nrsweb.org/. The SPL document was one of the sets of proficiency descriptors that informed the development of the NRS levels. So now, SPLS are used as examples of benchmarks that help define the NRS levels.
For more information:
Measures and at Methods for the National Reporting System for Adult Education: implementation Guidelines (March 2001) at www.nrsweb.org/reports/implement.pdf
NRS online training at www.oei-tech.com/nrs/
Student Performance Level (SPL) Descriptors for Listening Comprehension and Oral Communication at www.cal.org/caela/esl_resources/slspls.html
Reading and Writing SPLs are available at: www.cal.org/caela/esl_resources/rwspls.html