Resources

CAELA Network Briefs

Using Data to Plan Professional Development for Practitioners Working With Adult English Language Learners

Sarah Young and Joy Kreeft Peyton
Center for Applied Linguistics
September 2008

Background on Adult Learners

Adult education programs serve both native English speakers and learners whose first, or native, language is not English. Native English speakers attend adult basic education (ABE) classes to learn basic skills needed to improve their literacy levels; they attend adult secondary education (ASE) classes to earn high school equivalency certificates. Both ABE and ASE instruction help learners achieve other goals related to job, family, or further education. English language learners attend English as a second language (ESL), ABE, or workforce preparation classes to improve their oral and literacy skills in English and to achieve goals similar to those of native English speakers.

Audience for This Brief

This brief is written for state adult education staff, program administrators, professional development staff and teacher trainers, teachers, education researchers, and policy makers. It describes a process for adult education staff to use data to plan professional development for practitioners who work with adults learning English.

Introduction

Professional development for educators who work with adult English language learners is critical for ensuring that programs are strong and that learners achieve their goals. The complex process of planning professional development for these educators must be carried out in a systematic way, take into account the characteristics and needs of multiple stakeholder groups, and reflect standards for quality (Center for Applied Linguistics, 2008; Schaetzel, Peyton, & Burt, 2007; Sherman, Safford-Ramus, Hector-Mason, Condelli, Olinger, & Jani, 2006). This brief describes a professional development planning process that can help adult education staff at the local, regional, and state levels make data-driven, systematic decisions to respond to the needs of adult learners, practitioners, administrators, and funders and provide a foundation upon which to evaluate the outcomes of professional development efforts.

Professional development planning begins with the establishment of a professional development planning team and a data collection and management system. Once these foundations are in place, the team can begin the actual planning process, which includes the following steps:

Collect baseline data
Analyze the data to identify specific professional development needs
Prioritize professional development needs
Write a professional development plan

This brief describes the process from the establishment of a planning team through the writing of a professional development plan. It does not discuss implementation of the plan or evaluation of outcomes, but it directs readers to relevant sources of information about these steps.

Professional Development Planning Team

A team approach to planning helps to ensure that the needs of multiple stakeholders are understood and addressed. In the case of professional development for educators working with adults learning English, the planning team should include an administrator capable of making decisions based on personnel and funding, teachers who represent various practitioner groups (e.g., levels of experience teaching adult ESL, student populations taught, areas of expertise), professional developers, assessment specialists, and others involved in providing instruction and services in the program (e.g., volunteer tutors, data management specialists). One person may appropriately represent more than one of these groups. The planning team engages in a systematic process for collecting and analyzing data; determining and prioritizing the professional development needs of particular practitioner groups based on these data; selecting resources and personnel to support the professional development effort; and writing, implementing, and evaluating a coherent professional development plan. The specifics of this work depend on whether the team is working at the local, regional, or state level and on the scope of responsibility and capacity of the team. The guidelines provided in this brief should be adapted by the planning team to fit their specific context.

Data Collection and Management System

Each state must put in place a data collection and management system to gather accurate, complete, recent, and historical data on all adult learners, including English language learners. Federally funded adult education programs are required to collect and report data to the National Reporting System (NRS)¹ on their students’ enrollment, attendance, persistence, and progress (U.S. Department of Education, 2007). Thus, data collected for NRS reporting may be readily available for analysis. Additional types of data on students, teachers, and programs are also important. Processes may need to be implemented for program administrators, data entry specialists, assessment specialists, and other stakeholders to collect, enter, report, and analyze additional data for professional development planning.

The Professional Development Planning Process

With a data collection and management system in place, a state or local team can begin the professional development planning process following the four steps outlined here.

Step 1: Collect Baseline Data

The planning team begins by collecting data. When appropriate and meaningful data are collected and analyzed, the resulting professional development is likely to be relevant and to produce positive teacher change (Gonzalez & Darling-Hammond, 1997; Sherman, Kutner, Tibbetts, & Weidler, 2000; Smith, Hofer, Gillespie, Solomon, & Rowe, 2003). Table 1 shows four types of data to consider collecting.

Table 1: Suggested Data to Collect and Analyze for Professional Development Planning

Student data	- Local immigrant population data (including U.S. Census data on the foreign-born) - Personal background data (education, native language[s], literacy levels in native language and English, ethnicity, time in the United States, occupation in the native country and in the United States) - Ethnicity, age, sex, enrollment, attendance, persistence, and educational gain as recorded in NRS Tables 1, 3, 4, and 4b - Educational goals - Feedback about services received
Practitioner data (administrators and teachers)	- Background data (education, adult ESL teaching experience and training, teaching credentials, non-English languages spoken) - Geographical areas and types of programs in which they work and levels they teach - Professional development needs (preferred topics and formats, feedback from previous trainings) - Observations of classroom instruction - Individual professional development plans
Program data	- Program characteristics and growth; curriculum and course offerings; student recruitment, orientation, intake, and assessment processes; instructional approaches and materials; uses of technology for instruction and staff development; teacher and administrator recruitment; inservice training for teachers and administrators; special projects - Program effectiveness; alignment with program quality standards; program monitoring reviews
Professional development planning data	- Current professional development planning process

Collecting and analyzing all four types of data reveals patterns and trends from a variety of perspectives and allows needs to be addressed in a number of different ways. That is, while planning team members can examine NRS data on students on a regular basis to identify areas that may be addressed through teacher training and other methods of professional development, they will also want to consider data on the population of students in the state and local areas and their specific goals and needs; data on the practitioners who work with these students—their backgrounds, knowledge, and instructional approaches; data on program qualities that may affect practitioner effectiveness and student performance; and data on current and planned professional development offerings.

In gathering information about the professional development that is already offered, it is helpful to note any professional development activities that were planned but not carried out. If a review of the data shows that these activities are still needed, they can be planned for the following year. Discussion of the following questions can help the planning team understand the current planning process and policies for professional development and identify areas where improvements may be needed.

Who are the practitioners involved in the instruction of adults learning English, and who is providing professional development for these practitioners?
Is information collected from practitioners to help determine the professional development activities that will be conducted? If so, what types of information are collected, in what forms (e.g., survey), by whom, and how frequently? What other data are consulted?
What is the process for determining the types of professional development activities for these practitioners? Who determines which activities will be conducted? When and how are decisions made?
What other factors influence decisions about professional development (e.g., availability of annual state leadership funds or special grants, agency or state initiatives)?
Does the program prepare a formal plan for professional development? If so, who prepares the plan, who provides input, and what types of information are in the plan? Who approves the plan?
What professional development activities have been planned in the past but not carried out? How does this inform future planning?

Step 2: Analyze the Data to Identify Specific Professional Development Needs

Once the data have been collected, the planning team can begin to analyze data from each source (student, practitioner, program, and professional development planning) to identify emerging professional development needs. For example, while analyzing student data, the team might note a significant number of learners in a certain age group (e.g., youth or seniors) and decide that professional development may be needed for those working with that specific population. Within or across a particular NRS educational functioning level, high numbers of enrollment, low average hours of attendance, low rates of persistence, low percentages of gain from one level to the next, or low rates of posttest completion may indicate a need for teachers working with students at that level to improve instruction in order to increase attendance, persistence, and level gain. This information can be compiled in NRS table format using the templates found on the NRS Web site (see Table 2).

Table 2: Relevant NRS Tables for Analysis of Student Data

NRS table	Information included	Focus of professional development
Table 1: Participants by Entering Educational Functioning Level, Ethnicity, and Sex	Number of students enrolled in ESL programs by educational functioning level, ethnicity, and sex	Practitioners working with specific ethnic or gender groups of ESL students at specific educational functioning levels
Table 3: Participants by Program Type and Age	Number of students enrolled in ESL programs by age	Practitioners working with large groups of ESL students at specific ages, such as youth or seniors
Table 4: Educational Gains and Attendance by Educational Functioning Level	a. Number of students enrolled at each NRS ESL educational functioning level b. Average hours of attendance at each level c. Percentage of students completing each level (as indicated by pretest-posttest assessment data) d. Number and percentage of students pre- and posttested at each level (showing student persistence) e. Percentage of students stopping out before completing a level (i.e., student separation) f. NRS levels in which students met or did not meet negotiated performance targets	Practitioners working with student groups with the a. largest enrollment at a given level b. lowest average attendance hours c. lowest percentage of level completion d. lowest percentage of persistence e. highest percentage of separation f. lowest percentage of students meeting negotiated performance targets
Table 4b: Educational Gains and Attendance for Pre- and Posttested Participants	a. Number of students enrolled at each NRS ESL educational functioning level who were both pretested and posttested b. Average hours of attendance at each level c. Number of students enrolled at each NRS ESL educational functioning level who advanced at least one level d. Percentage of students at each level who completed the level	Practitioners working with students who have been both pre- and posttested at each level and have the a. lowest rates of pretest and posttest completion b. largest enrollment c. lowest percentage of level advancement d. lowest percentage of level completion

Note. To see sample NRS tables go to www.nrsonline.org/reference/index.html?
chapter=7&section=0&topic=null&subtopic=null

Practitioner data (collected through surveys, observations, individual professional development plans, self-assessments, and evaluations of trainings) may yield results that supplement student data. For example, limited teaching experience and training among teachers and administrators may indicate a need for new teacher or program director orientation, or training for administrators in observing adult ESL classes and providing feedback to teachers. A finding that many practitioners work in rural or remote locations may indicate a need for trainings conducted online. Specific practitioner groups may express a need for professional development on specific topics or in specific formats. Participant evaluations of trainings that have been offered may indicate specific areas for follow-up, suggest new areas or topics to be offered, and provide feedback on the effectiveness of the trainers who provided the professional development.

Program data may indicate changes that need to be made in program offerings, in student assessment and the reporting of educational gains, or in the recruitment of students or teachers. They may also reveal a need for professional development to enable state and local initiatives, partnerships, and projects to be carried out effectively.

Professional development planning data will show how systematic and inclusive the planning process is, what information has been used to inform planning, and whether there are gaps in the planning process that need to be addressed.

Each type of data has particular strengths and limitations; these are summarized in Table 3. Awareness of the strengths and limitations of each type of data can result in more valid conclusions.

Table 3: Summary of Data Sources to Be Analyzed and Their Strengths and Limitations

Type of information	Data source	Data type	Analysis method	Strengths	Limitations
Student performance data	- NRS Table 4 - NRS Table 4b	Quantitative: – Numbers enrolled – Hours of attendance – Rates of completion, separation, and advancement	- Frequency counts -Percentages - Averages	- Standardized and quantifiable; various factors can be analyzed at once - Consulting NRS tables for a period of 2–3 years gives a picture of trends in enrollment and educational gain that may affect professional development	- Difficult to tie to instructional quality - Individual students not tracked at state or federal levels - Funding dependent on pretest-posttest scores and not other measures
Student demographics	- NRS Table 1 - NRS Table 3 - Local program survey of students upon enrollment	Quantitative data on students’ demographic characteristics: – Age – Ethnicity – Educational background – Native language	- Frequency counts - Percentages - Averages	- Standardized and quantifiable - Consulting student demographic data for a period of 2–3 years gives a picture of trends in enrollment and educational gain that may affect professional development	- Data may not be complete or accurate, because of dependence on self-reporting; For example, NRS Table 3 may not report all of the locally significant variables (e.g., student first language or country of origin)
Practitioner backgrounds and professional development needs	- Surveys - Interviews - Questionnaires	Qualitative data: – Anecdotes – Experiences – Professional development preferences Quantitative data: – Years of experience – Educational background	- Frequency counts - Percentages - Averages - Lists of preferences	- Background information on practitioners’ characteristics, work settings, and needs helps to identify areas to address	- Focus is often on practitioner preferences rather than on performance - Practitioners may not know or want what they need for professional development
Program performance	- Classroom observations - Program monitoring visits - Program goals Student feedback	- Reports - Observation forms	- Qualitative analysis of performance based on firsthand observations and conversations	- More complete picture of program structure, processes, strengths, and weaknesses	- Findings may be difficult to address simply through practitioner professional development
Professional development planning	- Planning team documents and plans	- Qualitative data - Quarterly and annual professional development offerings - List of data that inform planning - Activities planned but not offered Evaluations of activities conducted	- Qualitative review of team members and gaps in representation - Offerings each year and quarter - Data that inform offerings - Evaluations of activities	- Picture of team representation - Complete picture of what professional development is offered, what was planned but not offered, and why - Consideration of evaluations of offerings	- Process may not be clear or explicit enough to guide planning - Information may be decentralized and difficult for the planning team to access and compile

After the data have been collected and analyzed, the team looks across the various sources of data to identify the current and emerging needs of the adult ESL practitioners involved. For example, NRS Tables 4 and 4b might show that there are high enrollments at the Beginning ESL Literacy level but low rates of level gain; this may indicate a need for teachers to receive training on working with literacy-level learners. The teacher background survey may show that many new teachers are hired each year and that most do not have experience working with adults learning English, pointing to a need for teacher orientation and training on working in adult ESL contexts. Review of professional development planning processes may show that decisions are made about professional development offerings based on the latest topics discussed in the field rather than on actual data about teacher experiences and needs.

Step 3: Prioritize Professional Development Needs

After a list of professional development needs has been compiled, the planning team sets priorities based on the most pressing needs; on local capacity to address needs in light of factors such as funding, time, and expertise; and on the possible short- and long-term impact of various professional development options. That is, professional development efforts that are likely to have the greatest impact on teacher knowledge, classroom practice, and student progress and performance should be given top priority, while keeping the following questions in mind:

Which needs are supported by the greatest amount and diversity of data (student, practitioner, and program)?
Which needs reflect program and state priorities and initiatives?
Which needs are aligned with standards for high-quality professional development, such as those found in Framework for Quality Professional Development for Practitioners Working with Adult English Language Learners (Center for Applied Linguistics, 2008)?
Which needs are possible to address in the coming program year, and which needs might be addressed in subsequent years?
Which needs can be met with available personnel, resources, and technology?
Which needs require funding, personnel, time, and expertise that are not currently available?

Step 4: Write a Professional Development Plan

The culminating product of the planning process is a data-driven, systematic professional development plan that addresses the needs and priorities identified and describes the ways in which the outcomes of the professional development will be evaluated. The plan should include descriptions of the following:

The professional development needs identified and the priority of each
The types of professional development activities to be provided to address the needs
The practitioners who will be involved in each of the activities
The expected outcomes of each activity with each group of practitioners
The personnel and other resources needed for each activity
The measures to be used to determine if the desired outcomes have been achieved

The following types of activities might be included in the professional development plan:

Workshops and presentations (often used to present a body of new material)
Observation and feedback (often used as follow-up, in which teachers implement strategies introduced in workshops and are observed and given feedback by peers or mentors)
Inquiry and feedback (e.g., teachers explore issues in their own classrooms through reflective practice and action research; a study circle considers a topic or issue together over time)
Program and product development (teams work together to develop and improve their program and the products they use for instruction, professional resources, or training)

The CAELA Guide for Adult ESL Trainers (Center for Applied Linguistics, 2007, section II, pp. 29-39) provides information about the four types of professional development activities listed above and describes the benefits and limitations of each.

The professional development plan should also identify the expected outcomes of the activities offered. These outcomes might include the following:

Evidence of change in teacher knowledge or instructional behavior (through pre- and post-activity surveys, written and implemented lesson plans or classroom assessments, classroom observations, teacher reflections, and samples of student work)
Content outcomes (i.e., practitioners gain content knowledge or skills and strategies) measured through lesson plans that implement a new instructional strategy, teacher reflections on a new practice or process, mentor observation of a newly implemented strategy, teacher contributions to a professional discussion on the chosen topic, or a pretest-posttest knowledge assessment of new concepts and language learning theories
System outcomes (i.e., an aspect of the professional development system is added or changed) measured through a newly implemented, systematic professional development planning process, the creation of a mentor/master teacher match, or a review of intake procedures to improve teachers’ responses to student goal setting at the beginning of a course

Conclusion

Once a professional development plan has been developed, the planning team will oversee its implementation, collect appropriate data, evaluate the outcomes, and revise the plan as needed. These steps are the final pieces of professional development planning. This brief does not discuss the implementation and evaluation of professional development, but information about these processes can be found in Darling-Hammond (2005, 2006); Guskey (2002); Mitchem (2003); and Schaetzel, Peyton, and Burt (2007).

The successful creation of a data-driven, systematic professional development plan requires an understanding of the needs of the student and practitioner populations involved and other environmental factors. Collecting and analyzing data from a variety of sources paints a rich picture of the directions that professional development must take in order to serve the needs of those who work with adult English language learners. Careful prioritization of needs ensures that professional development activities are feasible, relevant, and effective. Evaluation of outcomes provides information for accountability and future planning. The formalization of this process results in a sustainable, recursive system to fulfill the calling of adult ESL practitioners to provide effective instructional and programmatic tools to improve the opportunities and achievement of adults learning English.

Note

¹The National Reporting System for Adult Education is the accountability system for federally funded adult basic education, adult secondary education, and English as a second language education in the United States. (See www.nrsweb.org for more information.)

References

Center for Applied Linguistics. (2007). CAELA guide for adult ESL trainers. Washington, DC: Author. Retrieved September 13, 2008, from http://www.cal.org/caela/scb/
guide.html

Center for Applied Linguistics. (2008). Framework for quality professional development for practitioners working with adult English language learners. Washington, DC: Author. Retrieved July 25, 2008, from http://www.cal.org/caelanetwork/
pd_resources/framework.html

Darling-Hammond, L. (2005). Teaching as a profession: Lessons in teacher preparation and professional development. Phi Delta Kappan, 87, 237-240.

Darling-Hammond, L. (2006). Assessing teacher education: The usefulness of multiple measures for assessing program outcomes. Journal of Teacher Education, 5(2), 120-138.

Gonzalez, J., & Darling-Hammond, L. (1997). New concepts for new challenges: Professional development for teachers of immigrant youth. Washington, DC, and McHenry, IL: Center for Applied Linguistics and Delta Systems.

Guskey, T. (2002). Does it make a difference? Evaluating professional development. Educational Leadership, 59(6), 45-51.

Mitchem, K. (2003, Winter). Data drives change: Linking professional development to improved outcomes. Rural Special Education Quarterly. Retrieved July 25, 2008, from http://findarticles.com/p/articles/mi_qa4052/is_200301/ai_n9223188/print

Schaetzel, K., Peyton, J. K., & Burt, M. (2007). Professional development for adult ESL practitioners: Building capacity. Washington, DC: Center for Applied Linguistics. Retrieved July 25, 2008, from ttp://www.cal.org/caela/esl_resources/
briefs/profdev.html

Sherman, R., Kutner, M., Tibbetts, J., & Weidler, D. (2000). Professional development resources supplement: Improving instruction, organization, and learner outcomes through professional development. Washington, DC: Building Professional Development Partnerships for Adult Educators Project. Retrieved July 25, 2008, from http://www.calpro-online.org/pubs/PDResSupp.pdf

Sherman, R., Safford-Ramus, K., Hector-Mason, A., Condelli, L., Olinger, A., & Jani, N. (2006). An environmental scan of adult numeracy professional development initiatives and practices. Washington, DC: American Institutes for Research.

Smith, C., Hofer, J., Gillespie, M., Solomon, M., & Rowe, K. (2003). How teachers change: A study of professional development in adult education (Rep. No. 25a). Cambridge, MA: National Center for the Study of Adult Learning and Literacy. Retrieved July 25, 2008, from http://www.ncsall.net/fileadmin/resources/research/report25a.pdf

U.S. Department of Education, Office of Vocational and Adult Education. (2007). Implementation guidelines: Measures and methods for the National Reporting System for Adult Education. Washington, DC: Author. Retrieved July 25, 2008, from http://www.nrsonline.org/reference/index.html

This document was produced at the Center for Applied Linguistics (4646 40th Street, NW, Washington, DC 20016 202-362-0700) with funding from the U.S. Department of Education (ED), Office of Vocational and Adult Education (OVAE), under Contract No. ED-07-CO-0084. The opinions expressed in this report do not necessarily reflect the positions or policies of ED. This document is in the public domain and may be reproduced without permission.