What Impact Do Student Evaluations Have?Feb 16 2018
- Maria Oviedo
- Rafael de Hoyos
Blog & Topic
This post is also available in: Spanish
Standardized student evaluations are used around the world to measure learning and provide policymakers a sense of the quality of teaching in different schools. But what impact can sharing the results with teachers and administrators have on their teaching and management practices? A recent experiment in La Rioja, Argentina set out to answer this question. Rafael de Hoyos, the project’s lead researcher and lead economist at the World Bank, explains their results and what they mean for education policy in Latin America.
We know standardized student evaluations are critical tools for providing a sense of the quality of education in schools. Your recent study in Argentina explores the impact of student evaluation results on school practices. Can you tell us about the study and its findings?
Our main motivation when we designed this intervention more than 4 years ago was to prove that standardized tests do not need to be “neoliberal,” that is, create rankings or make funding dependent on results, to have a positive impact on student learning. We designed a randomized control trial to evaluate the impact of using the results of a standardized test to provide schools with user-friendly report cards and technical assistance to trigger improvements. The main conclusion of the evaluation is that providing schools with report cards and a minimum amount of technical assistance (to help them make sense of the data) results in positive impacts on student learning.
What were some of the behaviors that teachers and administrators adopted after the results of student assessments became public?
School directors used the standardized test results to set learning goals and monitor the performance of teachers and students based on those goals. Students in treatment schools reported that teachers devoted more time to explanations and made sure that all of the students understood the material. Students in treatment schools also reported that their teachers encouraged them to try their best and reflect more on what they read.
Tell us about the policy implications of these findings. Is information alone sufficient? What kind of activities can help to maximize the impact of public information on test results?
Perhaps the most important policy implication is that, regardless of the intrinsic limitations of standardized tests, there is a very high cost of not having a test. A second implication is that a standardized test does not have to be used for “accountability purposes” or to create rankings to have a positive impact on learning outcomes. In fact, the impacts observed in La Rioja and in a similar study in Colima, Mexico show that using standardized tests for “diagnostic purposes” is a very effective strategy to improve learning without the common political confrontation that surrounds accountability-based tests. Information plus a minimum amount of technical assistance to schools is enough to trigger improvements.
Do you think measuring student learning can set the basis for personalized instruction in the classroom? If so, what is needed to make this happen?
Yes, measuring student learning can set the basis for personalized instruction, but only if schools use evaluation tools with very different characteristics than conventional standardized tests. For example, it would have to be a test that could be administered by the schools at a low cost , and would ideally be administered at the beginning of the school year in order to identify the proficiency level of each student in the classroom, and then again at the end of the school year to track progress. The real challenge is how teachers can personalize instruction once they know the learning level of each student. I do not think this is feasible—at least not with the traditional “production function” of learning (one teacher to 15 to 30 students), However, the use of technology in the classroom makes personalized instruction significantly more plausible, as was shown in an article published last year by my colleague Alejandro Ganimian and others.
Some developed countries have mechanisms to reward high performance and punish low performance in student assessments – for instance, linking performance to teacher salaries and school funding. What is your take on this sort of accountability system? What do you think is most appropriate for the Latin American context?
Although in theory linking student learning with teacher salaries or school funding sounds like a sensible policy, in practice it is a bad idea because there are so many technical and political challenges in the region. Think about the following: if teacher salaries are to be determined by student learning, then you would have to have a standardized test for every grade and every year. In the case of secondary school teachers, you would need subject-specific tests every year, for all subjects. Assuming a country is willing to spend that amount of resources on testing, you would then need to have a reliable measure of the value-added of each teacher on student learning. This measure, however, would be highly subjective. Assuming experts could come to an agreement about the specification for this value-added function to define the contribution of teachers and link this to salaries (or school funding) then you would need to obtain the buy-in and support of the teachers, which, in Latin America, would be politically very difficult to achieve.
Rafael E. De Hoyos is a lead economist in the education unit for Latin America and the Caribbean of the World Bank.