Does Adult Educator Professional Development Make a Difference?

Myths and Realities no. 28

by Sandra Kerka
2003

This project has been funded at least in part with Federal funds from the U.S. Department of Education under Contract No. ED-99-CO-0013. The content of this publication does not necessarily reflect the views or policies of the U.S. Department of Education nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government. ERIC/ACVE publications may be freely reproduced.

In today's evidence-based environment, adult education stakeholders want to know that investment in professional development is worthwhile. However, it is difficult to "prove": "An evidence-based connection between professional development of almost any kind and learner outcomes is tenuous at best, and elusive in practice" (Belzer and St. Clair 2003, p. 17). Yet, there is an intuitive assumption that professional development is linked to better teaching and learning outcomes (Belzer 2003). Part of the problem may be misconceptions about what professional development can and cannot accomplish. This publication examines research reports and other literature to address misconceptions about professional development (PD) in adult education in terms of models, methods, and impact.

Models and Methods

The field appears to be shifting away from one-shot, discrete events focused on transmission of knowledge and information to practitioner engagement in sustained knowledge construction and collaboration involving professionally meaningful questions (Belzer and St. Clair 2003; Jackson 2000). Does the method affect the outcome? Are there differences in the effectiveness of various types of professional development? (Due to space limitations, a limited number of methods are addressed here.)

Workshops are a common method of delivering staff development; they accommodate large numbers of participants and are often chosen as a preferred format in surveys (Sabatini et al. 2000; Sherman and Kutner 1998). However, location, time, and costs are often barriers to participation, particularly when practitioners are not provided released time, compensation, or incentives for attending (Sheckley n.d.; Smith et al. 2002). The relevance of content and follow-up support for transfer of learning are other hindrances (ibid.). Literacy practitioners in Washington felt that canned or prepackaged workshops did not meet their needs, particularly when they were focused on state mandates with little or no emphasis on issues related to learning (Sheckley n.d.). On the other hand, in a national survey of 423 adult literacy instructors (Sabatini et al. 2000), workshops by consultants or colleagues were ranked as two of the top three most useful PD formats.

Workshops may be most effective for certain learning styles, when sessions are based on learners' assessed needs, and when attention is given to such elements as modeling, practice, feedback, and coaching (Sherman and Kutner 1998). Single workshops may be a useful way to provide information and raise awareness of issues, but changes in behavior and practice require longer-term approaches (Kutner et al. 1997). For example, in the Pelavin three-phase process model, practitioners attend a traditional daylong workshop session, leaving with an assignment to accomplish or a "guiding question" to explore over a 6-week period. Then they return to discuss results they achieved and problems they encountered (Sheckley n.d.).

Collaborative practitioner inquiry/research approaches, supported by theories of constructivism and critical reflection, are based on the assumption that practitioners are active constructors of their own practice (Sherman and Kutner 1998). An evaluation of the Pennsylvania Adult Literacy Practitioner Inquiry Network (Belzer 1998) found that literacy instructors who participated in inquiry engaged in more reflection and problem solving, changed practices, and participated in a learning community for ongoing and in-depth discussion. The National Center for Adult Learning and Literacy's (NCSALL) Practitioner Dissemination and Research Network engaged adult basic education (ABE) teachers as practitioner leaders who organized research-based PD through study circles, conducted their own research, and helped others (Smith et al. 2002). Study circles created opportunities for practitioners to discuss research over an extended period of time. The direct change in the classrooms for those who conducted practitioner inquiry was "immediate."

One form of inquiry, action research, has been used extensively in PD. In an evaluation of the Pennsylvania Action Research Network over 5 years (Kuhne and Weirauch 2001), the majority of ABE supervisors felt that practitioner action research had resulted in changes in attitudes, programs, procedures, and courses. In a NCSALL study, ABE teachers and administrators in Tennessee, Kentucky, and Virginia undertook action research to learn how to document outcomes of ABE participation (Bingman et al. 2002). Participants reported ways their practice had changed. The process of identifying desired outcomes led to increased focus on meeting learner goals and achieving desired outcomes. In Britain, part-time literacy tutors undertook their own investigation of what constitutes effective professional support and development (Jackson 2000). The project gave a voice to these often marginalized tutors, helped them focus on the broader issues of teaching and learning, and increased learner-centered teaching, critical awareness, and reflectivity. Through recurring action research cycles, the teams evolved into support groups in which to discuss problems of practice. However, inquiry/research approaches are time consuming, especially for part-time staff, and require administrative support (ibid.). Key contextual factors influence their effectiveness: voluntary participation, an extended time-scale, small group size, embeddedness in issues that are professionally meaningful (Belzer 1998; Jackson 2000). Although intensity and continuity are enhancing features, shorter-term experiences (e.g., just-in-time inquiry on a meaningful immediate question) can also be beneficial (Belzer 1998).

Although inquiry-based professional development can result in significant changes in practice, large-scale implementation would require significant changes in the adult education system, and we still lack substantial evidence that it produces superior learner outcomes than any other model (Belzer and St. Clair 2003). Smith and Hofer's (2002) comparison of workshops, mentoring, and practitioner research found that the model of PD did not have as much effect as other factors: amount of time spent, quality of PD, personal motivation, teachers' working conditions, and program structure. All three models support change but are dependent on motivation, background, context, and quality.

To Cranton and King (2003), asking which PD method or model is more effective is the wrong question. Instrumental approaches such as how-to workshops neglect communicative and emancipatory kinds of knowledge about teaching. Professional development should be a transformative process of critical reflection that leads to changing one's frame of reference, discarding habits of mind, seeing alternatives, and acting differently. "Meaningful professional development must go far beyond learning to use a new piece of software or a new trick for increasing student participation. It must involve educators as whole persons—their values, beliefs, and assumptions about teaching and their ways of seeing the world" (ibid., p. 33).

The Impact of Professional Development

Funders, providers, and practitioners tend to agree that the ultimate goal of PD is improved outcomes for learners (Bingman et al. 2002; Guskey 2003; Sheckley n.d.). How to demonstrate the impact of professional development on learning outcomes is in dispute. A common method for evaluating PD activities is the participant satisfaction survey, but this does not provide information on desired changes in teaching and learning (Tolbert 2001). In a National Institute for Literacy survey, one-third of the states report that adult learner achievement is used to determine the impact of PD, typically by monitoring numerical indicators and gains in state programs (Tolbert 2001). However, Sherman and Kutner (1998) identify a number of problems with using learner achievement as a measure: teaching as one of many variables affecting learning, inconsistent attendance and turnover among adult learners, skewed results due to different learning styles, and heavy reliance on short-term changes, rather than long-term outcomes.

Guskey (2003) compared 13 lists of the characteristics of effective PD, finding that they were derived in very different ways, used different criteria to determine "effectiveness," and varied widely in characteristics identified. He cites other research-related problems: most lists mention sufficient time and resources for PD but some research shows time is unrelated to learning outcomes; PD time must be well organized, carefully structured, and purposefully directed to be effective. Most lists mention collegiality and collaboration but collaborative efforts must be structured and purposeful to improve learning. Most characteristics of effective PD are qualified with "yes, but." "The problem with such 'yes, but' statements is that they frustrate policy makers and practitioners who want simple answers to their questions about effective professional development" and may diminish the value of research evidence to those seeking unambiguous statements about best practices (p. 750).

Perhaps the problem is that there are no simple answers or unambiguous statements about the impact of PD. Professional development takes place in the real world where a "complex web" of factors (ibid.) influences the results, and "it is an elusive and costly task to document direct, traceable causal links between PD and learner outcomes" (Belzer 2003, p. 45). The intervening variables include the following (Belzer 2003; Belzer and St. Clair 2003; Sabatini et al. 2002):

• Practitioner characteristics—part-time status, amount of teacher training and experience, and motivation
• Learner characteristics—transience, diverse backgrounds and educational attainment of the adult student population; differences in what "outcomes" and "achievement" mean to adult students and to programs
• Program factors—lack of a standardized curriculum, teachers' working conditions, autonomy, decision-making input, access to resources and colleagues, administrative support for PD
• Professional development system characteristics—lack of alignment between standards for learners and for practitioners; degree of cooperation between state agencies and the field; accountability mandates; coherence, accessibility, and quality of offerings

For Belzer (2003), the way impact is defined matters. A systematic evaluation of Pennsylvania's PD system found that impact differs if PD is voluntary or mandatory, active or passive; and whether learner outcomes mean not just test scores but also recruitment, retention, or job or life change. She advocates broadening the definition of impact to include changes in classroom practice, ideas about teaching and learning, attitudes, programs, and the broader field. Different kinds of PD have different kinds of impact, so it is important to measure delivery stance/method (who/what is source of knowledge; process/content/outcome; conceptions of change; relationships of theory/research and practice; transmission or knowledge construction). Similarly, Guskey (2002) outlines five critical levels for evaluating professional development, noting that measurement becomes more complex at each successive level: (1) participant reactions; (2) participant learning; (3) organization support and change; (4) participant use of new knowledge and skills; and (5) student learning outcomes.

Does professional development make a difference in adult education? The answer depends on several issues:

• A inclusive definition of PD encompassing a wide range of activities for different contexts, with the goal of transformation (Cranton and King 2003; Sheckley n.d.)
• A broad conception of impact, with measurement focused on capturing outcomes over time, which values not just learner outcomes but practitioner, program, and field outcomes and accounts for the extent to which practitioners, learners, and programs do things differently as a result of PD (Belzer 2003)
• Administrative supports at national, state, and local levels, including time and funding for a broad range of activities such as practitioner research (Smith et al. 2002)
• Above all, development of a culture of professional development (ibid.) in which learning is embedded in ongoing, day-to-day processes and practitioners collaborate in continuous learning for the improvement of practice and the sake of adult learners

References

Belzer, A. The Pennsylvania Adult Literacy Practitioner Inquiry Network. Philadelphia: University of Pennsylvania, 1998. (ED 427 222)

Belzer, A. "Toward Broadening the Definition of Impact in Professional Development for ABE Practitioners." Adult Basic Education 13, no. 1 (Spring 2003): 44-59.

Belzer, A., and St. Clair, R. Opportunities and Limits: An Update on Adult Literacy Education. Columbus: Center on Education and Training for Employment, the Ohio State University, 2003.

Bingman, M. B. with Ebert, O., and Bell, B. Documenting Outcomes for Learners and Their Communities. Cambridge, MA: National Center for the Study of Adult Learning and Literacy, 2002. (ED 465 868) http://www.gse.harvard.edu/~ncsall/research/report20.pdf

Cranton, P., and King, K. P. "Transformative Learning as a Professional Development Goal." New Directions for Adult and Continuing Education no. 98 (Summer 2003): 31-37.

Guskey, T. R. "Does It Make a Difference?" Educational Leadership 59, no. 6 (March 2002): 45-51.

Guskey, T. R. "What Makes Professional Development Effective?" Phi Delta Kappan 84, no. 10 (June 2003): 748-750.

Jackson, A. "Action Technologies: Supporting Continuing Professional Development." Research in Post-Compulsory Education 5, no. 3 (2000): 361-370.

Kuhne, G. W., and Weirauch, D. "The Pennsylvania Action Research Network (PAARN): A Synopsis of Findings from Five Years of Practitioner Action Research in Pennsylvania." PAACE Journal of Adult Learning 10 (2001): 1-9.

Kutner, M.; Sherman, R.; Tibbetts, J.; and Condelli, L. Evaluating Professional Development: A Framework for Adult Education. Washington, DC: Pelavin Research Associates, 1997. http://www.nrsweb.org/reports/evalmon.pdf

Sabatini, J. P. et al. Teacher Perspectives on the Adult Education Profession: National Survey Findings about an Emerging Profession. Menlo Park, CA: SRI International; Philadelphia, PA: National Center on Adult Literacy, 2000. (ED 446 230) http://literacyonline.org/products/ncal/pdf/TR0002.pdf

Sheckley, B. G. "Professional Development in Adult Literacy Programs." Olympia: Washington State Board for Community and Technical Colleges, n.d. http://www.sbctc.ctc.edu/oal/docs/Staff%20dev%20report.pdf

Sherman, R.; and Kutner, M. Professional Development Resource Guide for Adult Educators. Washington, DC: American Institutes for Research, 1998. http://www.nrsweb.org/pubs.asp

Smith, C., and Hofer, J. "Pathways to Change: A Summary of Findings from NCSALL's Staff Development Study." Focus on Basics 5, issue D (June 2002): 1, 3-8. (ED 467 524) http://ncsall.gse.harvard.edu/fob/2002/fob_5id.pdf

Smith, C.; Bingman, M. B.; Hofer, J.; and Medina, P. Connecting Practitioners and Researchers: An Evaluation of NCSALL's Practitioner Dissemination and Research Network. Cambridge, MA: NCSALL, 2002. (ED 469 857) http://ncsall.gse.harvard.edu/research/report22.pdf

Tolbert, M. Professional Development for Adult Education Instructors. Washington, DC: National Institute for Literacy, 2001. (ED 465 079) http://www.nifl.gov/nifl/policy/development.pdf