A Little Less Evaluation, A Little More Feedback

˙ PREAL Blog

This post is also available in: Spanish

The following article by Alejandro Ganimian was originally written for WISE ed.review, a website that offers a daily selection of cutting-edge news, fresh insights and innovative trends in education. WISE has given The Inter-American Dialogue permission to reproduce the piece on the PREAL blog. To read the original article, click here.

Over the past decade, rigorous research has shown that teachers are the most important school-based determinant of student achievement. In fact, “effective teachers” (i.e., those whose students consistently make gains in standardized tests) can not only offset the learning deficits of disadvantaged students; they can also impact students’ long-term life outcomes, such as their propensity to avoid teenage pregnancy and criminal behavior and to be employed and earn a higher wage once they enter the labor market.

Some Latin American school systems—specifically, Chile, Colombia, Ecuador, Mexico, Peru and a number of Brazilian states—have taken this research seriously and have adopted (or are in the process of adopting) several policies seeking to improve the effectiveness of their teachers. One policy that has received considerable attention is teacher evaluation, not just upon entrance into the profession (to select candidates), but also during a probationary period (to award tenure) and once teachers are on the job (to reward effective teachers and/or to dismiss those who are chronically ineffective). In this blog post, I focus on the last of the three: the so-called “performance evaluations.”

Unfortunately, even if the political processes through which performance evaluations were adopted vary widely across the region, they have often resulted in confrontations between governments and teachers’ unions. In Chile, a considerable share of teachers resisted the evaluation when it was first adopted 10 years ago. In Mexico, unions took to the streets to halt reforms to the teachers’ statute by the Peña Nieto administration in 2013. In Colombia, unions went on strike during the last days of Santos’ recent reelection campaign to demand an overhaul of the teacher evaluation system, first enacted in 2007.

Yet, performance evaluations would enjoy a broader support across the region (from both teachers and policy-makers), and have greater chances of impacting teacher effectiveness, if they focused less on “grading” teachers or “weeding out” low performers and more on providing teachers with feedback on how to get better at classroom instruction.

The case for a greater focus on teacher feedback, rather than evaluation, is threefold. First, teachers both need and welcome such feedback. The evidence on this is admittedly scattered and incomplete, but consistent. A recent study by Barbara Bruns and Javier Luque at the World Bank shows that teachers in Brazil, Colombia, Honduras, Jamaica, Mexico and Peru spend less than 70 percent of lesson time actually teaching and devote up to 40 percent of their time to “off-task” activities, such as managing student behavior. Similarly, a recent international survey of teachers conducted by the Organization for Economic Cooperation and Development (OECD) in 34 countries indicates that only about 6 out of 10 teachers in Brazil, Chile and Mexico received pre-service training on the subject that they teach, how to teach it or had opportunities to practice doing so. In the same survey, more than 60 percent of teachers in these three countries reported that feedback on their practice made them feel more recognized, more than 85 percent that it improved their sense of self-efficacy and over 70 percent that it increased job satisfaction.

Second, evaluating and rewarding teachers based on how well their students perform on standardized tests might get them to attend school more often, and may get them to work harder, but it offers little guidance on how they can teach better. A World Bank study documented that teacher absenteeism remains a problem in several Latin American countries, including Ecuador and Peru, with teacher absenteeism rates of 14 and 11 percent respectively. In these countries, teacher evaluations (and incentives) tied to teacher attendance or students’ test scores studied seem to have positive effects. Yet, as Richard Murnane and I argue in a recent review of rigorous studies of educational interventions in developing countries, once this low-hanging fruit has been picked, the evidence shows that struggling teachers benefit considerably from “scaffolding” (clear, specific and actionable guidance and resources to teach lessons on a particular subject or topic). In fact, attaching stakes to measures that teachers do not know how to influence (e.g. test scores) can lead to dysfunctional behavior, such as student cheating and “teaching to the test”.

Finally, most Latin American countries have only begun to experiment with teacher evaluations; attaching stakes to these evaluations before they are shown to be reliable and predictive of good teaching may convey political resolve in the short term, but it is bound to create political backlash once the first round of results are made public. Perhaps one of the clearest examples of this occurred in 2008 in Peru: when the government adopted an entry exam for teachers, only 151 of the 183,118 who took the test passed it. Yet, as a recent OECD report indicates, even Chile, the country in the region with the longest-standing teacher performance evaluation, still has technical challenges to overcome.

Unfortunately, there is little experimentation on how to provide meaningful feedback to teachers in developing countries. The two experiments that have been rigorously studied (in India and Liberia) have mostly provided teachers with reports on students’ test scores, which is not much help if teachers do not know how to improve those scores.

For guidance on how to provide feedback to teachers, the region would be well-advised to learn from the Measures of Effective Teaching (MET) project, an initiative of the Bill and Melinda Gates Foundation in the United States to: (i) study the reliability and predictive power of multiple high-quality instruments to evaluate teachers (e.g., student achievement tests, student surveys, classroom observations, subject-specific pedagogical knowledge tests for teachers and principal ratings); (ii) document how to faithfully implement them; and (iii) partner with school districts to incorporate them into teacher evaluation systems. The final report of the MET project has several important lessons for the region about the comparative advantages of different measures, the payoffs from involving multiple actors in the evaluation process and the ways to combine these measures and actors to achieve different policy objectives.

Yet perhaps the most important lesson from MET is not in its findings, but its process. Its emphasis on how to use teacher evaluations as a tool for system improvement has made it trustworthy, the caliber of its researchers and the consideration of a wide array of instruments made its results widely accepted and its focus on how to continually improve these instruments enhanced teacher buy-in.

Latin American policy-makers may look at MET and think they cannot afford to undertake such an ambitious effort. Yet if they are interested in creating robust, reliable and useful teacher evaluations, I would argue they cannot afford not to do so.